Affective connotations according to LLMs

  • The affective connotations of words are central to meaning and important predictors of many social processes. As such, understanding the degree to which commercially-available generative language models (LLMs) replicate human judgements of affective connotations may help better understand human-model interactions. LLMs may also serve as useful tools for researchers seeking affective meaning estimates. We test the ability of three LLMs – GPT-4o, Mistral Large, and Llama 3.1 – to estimate human affective connotation ratings of words representing social identities, behaviours, modifiers, and settings in three language cultures: English (US), French (France), and German (Germany). We find that LLM ratings of terms correlate strongly with human ratings. However, their ratings tend to be overly extreme and patterns of correlations between meaning dimensions only loosely approximate those of human ratings. Consistent with previous findings of English-language and American biases in LLMs, we find that LLMs tend to perform better on EnglishThe affective connotations of words are central to meaning and important predictors of many social processes. As such, understanding the degree to which commercially-available generative language models (LLMs) replicate human judgements of affective connotations may help better understand human-model interactions. LLMs may also serve as useful tools for researchers seeking affective meaning estimates. We test the ability of three LLMs – GPT-4o, Mistral Large, and Llama 3.1 – to estimate human affective connotation ratings of words representing social identities, behaviours, modifiers, and settings in three language cultures: English (US), French (France), and German (Germany). We find that LLM ratings of terms correlate strongly with human ratings. However, their ratings tend to be overly extreme and patterns of correlations between meaning dimensions only loosely approximate those of human ratings. Consistent with previous findings of English-language and American biases in LLMs, we find that LLMs tend to perform better on English terms, though this pattern varies somewhat by meaning dimension and the type of term in question. We explore how LLMs might contribute to scholarship on affective connotations – by acting as tools for measurement – and how scholarship on affective connotations might contribute to generative language models – by guiding exploration of model biases.zeige mehrzeige weniger

Metadaten exportieren

Weitere Dienste

Suche bei Google Scholar
Metadaten
Verfasserangaben:Aidan Combs, Diego DamettoORCiDGND, Christophe Blaison, Renee Leung, Aarti Malhotra, Tobias SchröderORCiDGND, Jesse Hoey, Lynn Smith-Lovin
DOI:https://doi.org/10.1080/02699931.2025.2568551
ISSN:0269-9931
ISSN:1464-0600
Titel des übergeordneten Werkes (Englisch):Cognition and Emotion
Untertitel (Englisch):Implications for meaning measurement and cultural bias
Verlag:Routledge, Taylor & Francis
Verlagsort:London
Dokumentart:Wissenschaftlicher Artikel
Sprache:Englisch
Jahr der Erstveröffentlichung:2025
Veröffentlichende Institution:Fachhochschule Potsdam
Datum der Freischaltung:22.10.2025
GND-Schlagwort:Affektivität; Bias; Großes Sprachmodell; Konnotation
Seitenzahl:17
Fachbereiche und Zentrale Einrichtungen:FB1 Sozial- und Bildungswissenschaften
Forschungs- und An-Institute / Inst. für angewandte Forschung Urbane Zukunft (IaF)
DDC-Klassifikation:300 Sozialwissenschaften
Open Access:Hybrid Open Access
Lizenz (Deutsch):Creative Commons - CC BY - Namensnennung 4.0 International
Einverstanden ✔
Diese Webseite verwendet technisch erforderliche Session-Cookies. Durch die weitere Nutzung der Webseite stimmen Sie diesem zu. Unsere Datenschutzerklärung finden Sie hier.