Visual Blending for Concept Representation: A Case Study on Emoji Generation

The emoji connection between visual representation and semantic knowledge, together with its large conceptual coverage have the potential to be exploited in computational approaches to the visual representation of concepts. An example of a system that explores this potential is Emojinating—a system that uses a process of visual blending of existing emoji to represent concepts. In this paper, we use the Emojinating system as a case study to analyse the appropriateness of visual blending for the visual representation of concepts. We conduct three experiments in which we analyse output quality, type of blend used, usefulness to the user and ease of interpretation. Our main contributions are the following: (i) the production of a double-word concept list for testing the system; (ii) an extensive user study using two different concept lists (single-word and double-word); and (iii) a study that compares produced blends with user drawings.

[1]  Edward F. McQuarrie,et al.  Beyond Visual Metaphor: A New Typology of Visual Rhetoric in Advertising , 2004 .

[2]  Pegah Karimi,et al.  A computational model for visual conceptual blends , 2019, IBM J. Res. Dev..

[3]  Murhaf Fares.  A Dataset for Joint Noun-Noun Compound Bracketing and Interpretation , 2016, ACL.

[4]  Markus Egg,et al.  A Large Automatically-Acquired All-Words List of Multiword Expressions Scored for Compositionality , 2018, LREC.

[5]  Nuno Lourenço,et al.  Emojinating: Evolving Emoji Blends , 2019, EvoMUSART.

[6]  Amit P. Sheth,et al.  EmojiNet: An Open Service and API for Emoji Sense Discovery , 2017, ICWSM.

[7]  G. Fauconnier,et al.  The Way We Think , 2002 .

[8]  Penousal Machado,et al.  Assessing Usefulness of a Visual Blending System: "Pictionary Has Used Image-making New Meaning Logic for Decades. We Don't Need a Computational Platform to Explore the Blending Phenomena", Do We? , 2019, ICCC.

[9]  Rynson W. H. Lau,et al.  ICONATE: Automatic Compound Icon Generation and Ideation , 2020, CHI.

[10]  Catherine Havasi,et al.  Representing General Relational Knowledge in ConceptNet 5 , 2012, LREC.

[11]  Lydia B. Chilton,et al.  VisiBlends: A Flexible Workflow for Visual Blends , 2019, CHI.

[12]  Horacio Saggion,et al.  What does this Emoji Mean? A Vector Space Skip-Gram Model for Twitter Emojis , 2016, LREC.

[13]  Patrizia Paggio,et al.  Investigating Redundancy in Emoji Use: Study on a Twitter Based Corpus , 2017, WASSA@EMNLP.

[14]  Amílcar Cardoso,et al.  The Boat-House Visual Blending Experience , 2002 .

[15]  Ping Xiao,et al.  Vismantic: Meaning-making with Images , 2015, ICCC.

[16]  Pedro Martins,et al.  Emojinating: Hooked Beings , 2019, ARTECH.

[17]  Antonios Liapis,et al.  Can Computers Foster Human Users’ Creativity? Theory and Praxis of Mixed-Initiative Co-Creativity , 2016 .

[18]  Marília Prada,et al.  Lisbon Symbol Database (LSD): Subjective norms for 600 symbols , 2016, Behavior research methods.

[19]  Isabelle Augenstein,et al.  emoji2vec: Learning Emoji Representations from their Description , 2016, SocialNLP@EMNLP.

[20]  Petra Kralj Novak,et al.  Sentiment of Emojis , 2015, PloS one.

[21]  Lluís Padró,et al.  FreeLing 3.0: Towards Wider Multilinguality , 2012, LREC.

[22]  Penousal Machado,et al.  How Shell and Horn make a Unicorn: Experimenting with Visual Blending in Emoji , 2018, ICCC.

[23]  Ning Wang,et al.  Untangling Emoji Popularity Through Semantic Embeddings , 2017, ICWSM.

[24]  Viswanath Venkatesh,et al.  Bridging the Qualitative-Quantitative Divide: Guidelines for Conducting Mixed Methods Research in Information Systems , 2013, MIS Q..

[25]  Penousal Machado,et al.  A Pig, an Angel and a Cactus Walk Into a Blender: A Descriptive Approach to Visual Blending , 2017, ICCC.

[26]  Marianna Bolognesi,et al.  Emoji-based semantic representations for abstract and concrete concepts , 2020, Cognitive Processing.

[27]  Charles Browne.  A New General Service List: The Better Mousetrap We've Been Looking for? , 2014 .

[28]  Alison Pease,et al.  Using Argumentation to Evaluate Concept Blends in Combinatorial Creativity , 2015, ICCC.

[29]  Amy Beth Warriner,et al.  Concreteness ratings for 40 thousand generally known English word lemmas , 2014, Behavior research methods.