Comparing human and AI emotional evaluations of images: GPT-4o performance across standard, persona, and language-specific prompting strategies.

backgroundLarge language models, such as GPT-4o, have demonstrated strong performance in emotion recognition tasks, yet documented cultural biases raise concerns about cross-cultural validity. Prompt-engineering strategies, including persona prompting and native-language prompting, have been proposed as lightweight approaches to cultural adaptation, but their effectiveness in image-based emotion recognition remains underexamined, particularly in non-Anglophone European contexts. The present study examines whether language choice (Polish vs. English) and persona prompting alter GPT-4o emotional evaluation when benchmarked against a mixed European reference sample from the Nencki Affective Picture System (NAPS).

methodsGPT-4o (v.2024-08-06) evaluated 1,276 NAPS images on three emotional dimensions (valence, arousal, approach-avoidance) using a nine-point scale under three conditions: standard GPT-4o, persona-type GPT (Polish university student persona), and Polish-language GPT. Each image received 55 independent evaluations per condition, matching the original NAPS dataset (N = 204 evaluators; 60% Polish nationals, 40% European exchange students). Human-artificial intelligence (AI) agreement was evaluated using Pearson correlations, intraclass correlation coefficients (ICC[2,1]), Bayesian one-way ANOVA, and Anderson-Darling distributional tests.

resultsAll AI conditions positively correlated with human ratings across the three dimensions (r = 0.525-0.809, all p < 0.001). Neither persona nor Polish-language prompting consistently improved agreement over standard GPT-4o. The Polish-language condition aligned more closely with human ratings for arousal but diverged more strongly in approach-avoidance (d = 0.47). Valence showed the strongest correspondence (r = 0.784-0.809; BF₀₁ = 36.55 favoring equivalence), whereas arousal and approach-avoidance exhibited reliable mean-level and distributional divergence. Landscape stimuli produced the highest agreement (r up to 0.933) and faces the lowest (r as low as 0.253). Distributional differences persisted across all nine human-AI comparisons.

conclusionsWithin the constraints of a mixed European reference sample, persona-based and native-language adaptation strategies did not yield consistent improvements in human-AI agreement. Effects varied across dimensions and analytical methods, indicating that such strategies require dimension-specific empirical validation. AI evaluation aligned acceptably with human ratings for valence and scenic stimuli but diverged for arousal, approach-avoidance, and facial stimuli. These findings underscore the importance of jointly examining rank-order agreement, central tendency, and distributional structure in cross-cultural AI emotion recognition.