Lisbon Emoji and Emoticon Database (LEED): Norms for emoji and emoticons in seven evaluative dimensions

The use of emoticons and emoji is increasingly popular across a variety of new platforms of online communication. They have also become popular as stimulus materials in scientific research. However, the assumption that emoji/emoticon users’ interpretations always correspond to the developers’/researchers’ intended meanings might be misleading. This article presents subjective norms of emoji and emoticons provided by everyday users. The Lisbon Emoji and Emoticon Database (LEED) comprises 238 stimuli: 85 emoticons and 153 emoji (collected from iOS, Android, Facebook, and Emojipedia). The sample included 505 Portuguese participants recruited online. Each participant evaluated a random subset of 20 stimuli for seven dimensions: aesthetic appeal, familiarity, visual complexity, concreteness, valence, arousal, and meaningfulness. Participants were additionally asked to attribute a meaning to each stimulus. The norms obtained include quantitative descriptive results (means, standard deviations, and confidence intervals) and a meaning analysis for each stimulus. We also examined the correlations between the dimensions and tested for differences between emoticons and emoji, as well as between the two major operating systems—Android and iOS. The LEED constitutes a readily available normative database (available at www.osf.io/nua4x) with potential applications to different research domains.

[1]  Loren G. Terveen,et al.  "Blissfully Happy" or "Ready toFight": Varying Interpretations of Emoji , 2016, ICWSM.

[2]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[3]  Lisa J. Orchard,et al.  Emoticon convergence in Internet chat rooms , 2013 .

[4]  Hao Wang,et al.  Sentiment expression via emoticons on social media , 2015, 2015 IEEE International Conference on Big Data (Big Data).

[5]  M. Viergever,et al.  Standardized food images: A photographing protocol and image database , 2016, Appetite.

[6]  Marília Prada,et al.  Lisbon Symbol Database (LSD): Subjective norms for 600 symbols , 2016, Behavior research methods.

[7]  Natalie C. Ebner,et al.  FACES—A database of facial expressions in young, middle-aged, and older women and men: Development and validation , 2010, Behavior research methods.

[8]  D. Skiba,et al.  Face with Tears of Joy Is Word of the Year: Are Emoji a Sign of Things to Come in Health Care? , 2016, Nursing education perspectives.

[9]  R. D'Hooge,et al.  Assessing valence indirectly and online , 2009 .

[10]  Darja Fiser,et al.  A Global Analysis of Emoji Usage , 2016, WAC@ACL.

[11]  Ari Rappoport,et al.  Enhanced Sentiment Learning Using Twitter Hashtags and Smileys , 2010, COLING.

[12]  Yi Zhao,et al.  Effects of Emoticons on the Acceptance of Negative Feedback in Computer-Mediated Communication , 2014, J. Assoc. Inf. Syst..

[13]  Lisa Gibbs,et al.  Exploring the use of emoji as a visual research method for eliciting young children’s voices in childhood research , 2018 .

[14]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[15]  Nicole C. Krämer,et al.  Same Same But Different!? The Differential Influence of Smilies and Emoticons on Person Perception , 2012, Cyberpsychology Behav. Soc. Netw..

[16]  J. Mccroskey,et al.  Human Communication Theory and Research: Traditions and Models , 2014 .

[17]  Michael S. Wogalter,et al.  Comprehension of Pictorial Symbols: Effects of Context and Test Method , 1998, Hum. Factors.

[18]  Marília Prada,et al.  The many faces of a face: Comparing stills and videos of facial expressions in eight dimensions (SAVE database) , 2017, Behavior research methods.

[19]  Bernard C. Y. Tan,et al.  The Development of Social Capital in the Collaboration Network of Information Systems Scholars , 2014, J. Assoc. Inf. Syst..

[20]  N. Mukawa,et al.  Brain activity associated with graphic emoticons. The effect of abstract faces in communication over a computer network , 2011 .

[21]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[22]  Heather M. Claypool,et al.  Positivity Can Cue Familiarity , 2004, Personality & social psychology bulletin.

[23]  Manuel Perea,et al.  ERP correlates of masked affective priming with emoticons 1-s2.0-S0747563212003007-fx1 , 2013, Comput. Hum. Behav..

[24]  M. Bradley,et al.  Affective Normsfor English Words (ANEW): Stimuli, instruction manual and affective ratings (Tech Report C-1) , 1999 .

[25]  Panos Panagiotopoulos,et al.  Beyond positive or negative: Qualitative sentiment analysis of social media reactions to unexpected stressful events , 2016, Comput. Hum. Behav..

[26]  Mário J. Silva,et al.  Clues for detecting irony in user-generated contents: oh...!! it's "so easy" ;-) , 2009, TSA@CIKM.

[27]  Philip T. Kortum,et al.  Emoticon Use in Mobile Communications , 2015 .

[28]  Hugh Sweatman,et al.  Persistence and Change in Community Composition of Reef Corals through Present, Past, and Future Climates , 2014, PloS one.

[29]  N. Busch,et al.  Food-pics: an image database for experimental research on eating and appetite , 2014, Front. Psychol..

[30]  Uzay Kaymak,et al.  Exploiting emoticons in sentiment analysis , 2013, SAC '13.

[31]  Patrick Bonin,et al.  A new set of 299 pictures for psycholinguistic studies: French norms for name agreement, image agreement, conceptual familiarity, visual complexity, image variability, age of acquisition, and naming latencies , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[32]  Daantje Derks,et al.  Emoticons and Online Message Interpretation , 2008 .

[33]  Ruth Filik,et al.  Sarcasm in Written Communication: Emoticons are Efficient Markers of Intention , 2016, J. Comput. Mediat. Commun..

[34]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[35]  F. B. Krohn,et al.  A Generational Approach to Using Emoticons as Nonverbal Communication , 2004 .

[36]  Mark Dredze,et al.  You Are What You Tweet: Analyzing Twitter for Public Health , 2011, ICWSM.

[37]  Tüge T. Gülşen.  You Tell Me in Emojis , 2016 .

[38]  J. Walther.  Computer-Mediated Communication , 1996 .

[39]  Karianne Skovholt,et al.  The Communicative Functions of Emoticons in Workplace E-Mails: : -) , 2014, J. Comput. Mediat. Commun..

[40]  R. Siegel,et al.  Emoticon use Increases Plain Milk and Vegetable Purchase in a School Cafeteria without Adversely Affecting Total Milk Purchase. , 2015, Clinical therapeutics.

[41]  Petra Kralj Novak,et al.  Sentiment of Emojis , 2015, PloS one.

[42]  R. Spears,et al.  Paralanguage and social perception in computer‐mediated communication , 1992 .

[43]  Sangharsh Thakur,et al.  Facebook as a Corpus for Emoticons-Based Sentiment Analysis , 2014 .

[44]  P. Lang,et al.  International Affective Picture System (IAPS): Instruction Manual and Affective Ratings (Tech. Rep. No. A-4) , 1999 .

[45]  Jacob Eisenstein,et al.  Emoticons vs. Emojis on Twitter: A Causal Inference Approach , 2015, ArXiv.

[46]  Clayton Shepard,et al.  A longitudinal study of emoticon use in text messaging from smartphones , 2012, Comput. Hum. Behav..

[47]  S. Herring,et al.  Functions of the Nonverbal in CMC: Emoticons and Illocutionary Force , 2010 .

[48]  Mike Thelwall,et al.  Sentiment strength detection for the social web , 2012, J. Assoc. Inf. Sci. Technol..

[49]  Mike Thelwall,et al.  Sentiment in short strength detection informal text , 2010 .

[50]  Gail M Williams,et al.  Internet-based surveillance systems for monitoring emerging infectious diseases , 2013, The Lancet Infectious Diseases.

[51]  Joseph B. Walther,et al.  The Impacts of Emoticons on Message Interpretation in Computer-Mediated Communication , 2001 .

[52]  K. Scherer,et al.  The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance , 2011, Behavior research methods.

[53]  R. Proctor,et al.  Index of norms and ratings published in the Psychonomic Society journals , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[54]  Christina M. Steiner,et al.  Design and Development of an Empirical Smiley-based Affective Instrument , 2013, UMAP Workshops.

[55]  Fang-Wu Tung,et al.  Increasing social presence of social actors in e-learning environments: Effects of dynamic and static emoticons on children , 2007, Displays.

[56]  David C. Yen,et al.  Exploring the potential effects of emoticons , 2008, Inf. Manag..

[57]  Martin B. Curry,et al.  Measuring symbol and icon characteristics: Norms for concreteness, complexity, meaningfulness, familiarity, and semantic distance for 239 symbols , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[58]  S. Jaeger,et al.  Can emoji be used as a direct method to measure emotional associations to food names? Preliminary investigations with consumers in USA and China , 2017 .

[59]  Marcelo Vaiman,et al.  Development and validation of an Argentine set of facial expressions of emotion , 2017, Cognition & emotion.

[60]  Patti M. Valkenburg,et al.  Getting acquainted through social network sites: Testing a model of online uncertainty reduction and social attraction , 2010, Comput. Hum. Behav..

[61]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[62]  M. Thelwall,et al.  Sentiment Strength Detection for the Social Web 1 , 2012 .

[63]  S. Jaeger,et al.  Use of emoticon and emoji in tweets for food-related emotional expression , 2016 .

[64]  Linda K. Kaye,et al.  "Turn that frown upside-down": A contextual account of emoticon usage on different virtual platforms , 2016, Comput. Hum. Behav..

[65]  Gunther Eysenbach,et al.  Infodemiology and infoveillance tracking online health information and cyberbehavior for public health. , 2011, American journal of preventive medicine.

[66]  Minyi Guo,et al.  Emoticon Smoothed Language Models for Twitter Sentiment Analysis , 2012, AAAI.

[67]  M. Cha,et al.  Cross-Cultural Comparison of Nonverbal Cues in Emoticons on Twitter: Evidence from Big Data Analysis , 2014 .

[68]  J. G. Snodgrass,et al.  A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. , 1980, Journal of experimental psychology. Human learning and memory.

[69]  Christine T Chambers,et al.  An intrusive impact of anchors in children's faces pain scales , 1998, Pain.

[70]  Shao-Kang Lo,et al.  The Nonverbal Communication Functions of Emoticons in Computer-Mediated Communication , 2008, Cyberpsychology Behav. Soc. Netw..

[71]  Julie Barnett,et al.  Crisis as seen by the individual: the Norm Deviation Approach / La crisis vista por el individuo: el Enfoque de la Desviación de la Norma , 2015 .

[72]  Rachel Gibson,et al.  140 Characters to Victory?: Using Twitter to Predict the UK 2015 General Election , 2015, ArXiv.

[73]  Patti M. Valkenburg,et al.  Computer-Mediated Communication and Interpersonal Attraction: An Experimental Test of Two Explanatory Hypotheses , 2007, Cyberpsychology Behav. Soc. Netw..