Sliding Into Happiness: A New Tool for Measuring Affective Responses to Words

Reliable measurement of affective responses is critical for research into human emotion. Affective evaluation of words is most commonly gauged on multiple dimensions—including valence (positivity) and arousal—using a rating scale. Despite its popularity, this scale is open to criticism: It generates ordinal data that is often misinterpreted as interval, it does not provide the fine resolution that is essential by recent theoretical accounts of emotion, and its extremes may not be properly calibrated. In 5 experiments, the authors introduce a new slider tool for affective evaluation of words on a continuous, well-calibrated and high-resolution scale. In Experiment 1, participants were shown a word and asked to move a manikin representing themselves closer to or farther away from the word. The manikin’s distance from the word strongly correlated with the word’s valence. In Experiment 2, individual differences in shyness and sociability elicited reliable differences in distance from the words. Experiment 3 validated the results of Experiments 1 and 2 using a demographically more diverse population of responders. Finally, Experiment 4 (along with Experiment 2) suggested that task demand is not a potential cause for scale recalibration. In Experiment 5, men and women placed a manikin closer or farther from words that showed sex differences in valence, highlighting the sensitivity of this measure to group differences. These findings shed a new light on interactions among affect, language, and individual differences, and demonstrate the utility of a new tool for measuring word affect. La fiabilité de mesure des réponses affectives est essentielle à la recherche des émotions humaines. L’évaluation affective des mots est généralement mesurée à travers multiples dimensions – y compris la valence (positivité) et l’excitation – au moyen d’une échelle d’évaluation. Malgré sa popularité, cette échelle est critiquée pour les raisons suivantes : Elle génère des données ordinales qui sont souvent interprétées comme des intervalles; elle ne fournit pas la résolution requise par les récents postulats théoriques de l’émotion; et ses extrêmes sont possiblement mal calibrées. À travers cinq expériences, les auteurs présentent un nouveau curseur pour l’évaluation affective des mots sur une échelle continue, bien calibrée et à haute résolution. Dans l’expérience 1, un mot est présenté aux participants puis, l’on demande à ces derniers de déplacer un mannequin les représentant, plus près ou plus loin du mot. La distance du mannequin par rapport au mot était en forte corrélation avec la valence du mot. Dans l’expérience 2, les différences individuelles au niveau de la timidité et de la sociabilité ont entraîné des différences fiables quant à la distance par rapport aux mots. L’expérience 3 a validé les résultats des expériences 1 et 2 en utilisant un échantillon de répondants plus diversifié sur le plan démographique. Finalement, l’expérience 4 (similairement à l’expérience 2) suggérait que la demande liée à la tâche n’était pas un motif potentiel de recalibration de l’échelle. Dans l’expérience 5, les hommes et les femmes plaçaient le mannequin plus près ou plus loin des mots qui affichaient des différences sexuelles en termes de valence, mettant en évidence la sensibilité de cette mesure par rapport aux différences de groupe. Ces constats apportent une nouvelle perspective sur les interactions en termes d’affect, de langue et de différences individuelles et démontrent l’utilité d’un nouvel outil pour mesurer l’affect des mots.

[1]  J. Cheek Revised Cheek and Buss Shyness Scale , 2018 .

[2]  Geoff Hollis,et al.  Extrapolating human judgments from skip-gram vector representations of word meaning , 2017, Quarterly journal of experimental psychology.

[3]  Amy Beth Warriner,et al.  Affective biases in English are bi-dimensional , 2015, Cognition & emotion.

[4]  Jamie L. Metsala,et al.  Individual Differences in Word Recognition , 2015 .

[5]  Benny B. Briesemeister,et al.  Avoid violence, rioting, and outrage; approach celebration, delight, and strength: Using large text corpora to compute valence, arousal, and the basic emotions , 2015, Quarterly journal of experimental psychology.

[6]  Gabriel Recchia,et al.  Reproducing affective norms with lexical co-occurrence statistics: Predicting valence, arousal, and dominance , 2015, Quarterly journal of experimental psychology.

[7]  Reinhold Kliegl,et al.  Smoothing Spline ANOVA Decomposition of Arbitrary Splines: An Application to Eye Movements in Reading , 2015, PloS one.

[8]  Marc Brysbaert,et al.  How useful are corpus-based methods for extrapolating psycholinguistic variables? , 2015, Quarterly journal of experimental psychology.

[9]  C. Westbury You Can’t Drink a Word: Lexical and Individual Emotionality Affect Subjective Familiarity Judgments , 2014, Journal of psycholinguistic research.

[10]  Amy Beth Warriner,et al.  Emotion and language: valence and arousal affect word recognition. , 2014, Journal of experimental psychology. General.

[11]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[12]  Zachary Estes,et al.  Emotion and memory: A recognition advantage for positive and negative words independent of arousal , 2013, Cognition.

[13]  Saif Mohammad,et al.  CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON , 2013, Comput. Intell..

[14]  Zachary Estes,et al.  The unexplained nature of reading. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[15]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[16]  Amy Beth Warriner,et al.  Norms of valence, arousal, and dominance for 13,915 English lemmas , 2013, Behavior Research Methods.

[17]  J. Russell,et al.  The relation between valence and arousal in subjective experience. , 2013, Psychological bulletin.

[18]  Laura Winther Balling,et al.  Probability and surprisal in auditory comprehension of morphologically complex words , 2012, Cognition.

[19]  Benjamin V. Tucker,et al.  Danger and usefulness are detected early in auditory lexical processing: Evidence from electroencephalography , 2012, Brain and Language.

[20]  Bing Liu,et al.  Sentiment Analysis and Opinion Mining , 2012, Synthesis Lectures on Human Language Technologies.

[21]  Yves Bestgen,et al.  Checking and bootstrapping lexical norms by means of word similarity indexes , 2012, Behavior Research Methods.

[22]  Christopher M. Danforth,et al.  Positivity of the English Language , 2011, PloS one.

[23]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[24]  P. Beek,et al.  Walk to me when I smile, step back when I’m angry: emotional faces modulate whole-body approach–avoidance behaviors , 2011, Experimental Brain Research.

[25]  S. Wood Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models , 2011 .

[26]  L. Wurm,et al.  Ratings gathered online vs. in person: Different stimulus sets and different statistical conclusions , 2011 .

[27]  R. Baayen,et al.  Analyzing Reaction Times , 2010 .

[28]  Andreas B. Eder,et al.  Automatic influence of arousal information on evaluative processing: Valence–arousal interactions in an affective Simon task , 2010 .

[29]  Peter D. Turney,et al.  Emotions Evoked by Common Words and Phrases: Using Mechanical Turk to Create an Emotion Lexicon , 2010, HLT-NAACL 2010.

[30]  S. Hofmann,et al.  The Affective Style Questionnaire: Development and Psychometric Properties , 2010, Journal of psychopathology and behavioral assessment.

[31]  A. Reinecke,et al.  Three Indirect Tasks Assessing Implicit Threat Associations and Behavioral Response Tendencies Test-Retest Reliability and Validity , 2010 .

[32]  Cyrus Shaoul,et al.  Exploring lexical co-occurrence space using HiDEx , 2010, Behavior research methods.

[33]  R. H. Baayen,et al.  Frequency effects in compound processing , 2010 .

[34]  G. Norman Likert scales, levels of measurement and the “laws” of statistics , 2010, Advances in health sciences education : theory and practice.

[35]  Lee H. Wurm,et al.  A comparison of stimulus ratings made online and in person: Gender and method effects , 2010, Behavior research methods.

[36]  A. Buss,et al.  Understanding shyness: Four questions and four decades of research. , 2010 .

[37]  Lee H. Wurm,et al.  Stimulus norming: It is too soon to close down brick-and-mortar labs , 2010 .

[38]  David A. Wood,et al.  Perspectives on formulaic language : acquisition and communication , 2010 .

[39]  Marc Brysbaert,et al.  Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English , 2009, Behavior research methods.

[40]  G. Vigliocco,et al.  Emotion words, regardless of polarity, have a processing advantage over neutral words , 2009, Cognition.

[41]  H. Baayen,et al.  Holistic Processing of Regular Four-word Sequences 1 HOLISTIC PROCESSING OF REGULAR FOUR-WORD SEQUENCES Holistic Processing of Regular Four-word Sequences : A Behavioral and ERP study of the effects of structure , frequency , and probability on immediate free recall , 2009 .

[42]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[43]  Nathaniel J. Smith,et al.  Optimal Processing Times in Reading: A Formal Model and Empirical Investigation , 2008 .

[44]  M. Rinck,et al.  Avoidance of emotional facial expressions in social anxiety: The Approach-Avoidance Task. , 2007, Behaviour research and therapy.

[45]  Alan Y. Chiang,et al.  Generalized Additive Models: An Introduction With R , 2007, Technometrics.

[46]  M. Rinck,et al.  Approach and avoidance in fear of spiders. , 2007, Journal of behavior therapy and experimental psychiatry.

[47]  James W. Pennebaker,et al.  Linguistic Inquiry and Word Count (LIWC2007) , 2007 .

[48]  Gerhard Rinkenauer,et al.  Individual differences in approach and avoidance movements: how the avoidance motive influences response force. , 2006, Journal of personality.

[49]  Ayelet Fishbach,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES Self-Control in Action : Implicit Dispositions Toward Goals and Away From Temptations , 2006 .

[50]  G. Alpers,et al.  Automatic behavioural responses to valence: Evidence that facial action is facilitated by evaluative processing , 2005 .

[51]  S. Jamieson Likert scales: how to (ab)use them , 2004, Medical education.

[52]  Michael D. Robinson,et al.  Watch Out! That Could Be Dangerous: Valence-Arousal Interactions in Evaluative Processing , 2004, Personality & social psychology bulletin.

[53]  V. Carey,et al.  Mixed-Effects Models in S and S-Plus , 2001 .

[54]  Stefan Sperlich,et al.  Generalized Additive Models , 2014 .

[55]  D. Balota,et al.  Individual differences in information-processing rate and amount: implications for group differences in response latency. , 1999, Psychological bulletin.

[56]  S. Chaiken,et al.  Personality and Social Psychology Bulle- Tin Chen, Bargh / Consequences of Automatic Evaluation Immediate Behavioral Predispositions to Approach or Avoid the Stimulus , 2022 .

[57]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[58]  C. J. Wang,et al.  Bridging the gap between the pros and cons in treating ordinal scales as interval scales from an analysis point of view. , 1999, Nursing research.

[59]  P. Lang The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[60]  C. Carver,et al.  Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales , 1994 .

[61]  R. Bagby,et al.  The twenty-item Toronto Alexithymia Scale--I. Item selection and cross-validation of the factor structure. , 1994, Journal of psychosomatic research.

[62]  C. Izard,et al.  Four systems for emotion activation: cognitive and noncognitive processes. , 1993, Psychological review.

[63]  T. R. Knapp Treating ordinal scales as interval scales: an attempt to resolve the controversy. , 1990, Nursing research.

[64]  M. Bruch,et al.  Shyness and sociability reexamined: a multicomponent analysis , 1989 .

[65]  M. Paulshock,et al.  Frequency of effects. , 1984, Hospital practice.

[66]  A. Buss,et al.  Shyness and sociability. , 1981 .

[67]  S. Hains,et al.  Individual differences in word recognition latency , 1979 .

[68]  J. M. Kittross The measurement of meaning , 1959 .

[69]  H. Hotelling The Selection of Variates for Use in Prediction with Some Comments on the General Problem of Nuisance Parameters , 1940 .