Beyond Translation: Design and Evaluation of an Emotional and Contextual Knowledge Interface for Foreign Language Social Media Posts

Although many social media sites now provide machine translation (MT) for foreign language posts, translation of a post may not suffice to support understanding of, and engagement with, that post. We present SenseTrans, a tool that provides emotional and contextual annotations generated by natural language analysis in addition to machine translation. We evaluated SenseTrans in a laboratory experiment in which native English speakers browsed five Facebook profiles in foreign languages. One group used the SenseTrans interface while the other group used MT alone. Participants using SenseTrans reported significantly greater understanding of the posts, and greater willingness to engage with the posts. However, no additional cognitive load was associated with using an interface that provided more information. These results provide promising support for the idea of using computational tools to annotate communication to support multilingual sense making and interaction on social media.

[1]  Susan R. Fussell,et al.  Making Sense of Foreign Language Posts in Social Media , 2017, Proc. ACM Hum. Comput. Interact..

[2]  Shaomei Wu,et al.  Automatic Alt-text: Computer-generated Image Descriptions for Blind Users on a Social Network Service , 2017, CSCW.

[3]  Susan R. Fussell,et al.  Understanding How People Attend to and Engage with Foreign Language Posts in Multilingual Newsfeeds , 2017, ICWSM.

[4]  Júlio Cesar dos Reis,et al.  An evaluation of machine translation for multilingual sentence-level sentiment analysis , 2016, SAC.

[5]  C. Wankel.  Developing cross-cultural managerial skills through social media , 2016 .

[6]  D. Y. Wohn,et al.  One Click, Many Meanings: Interpreting Paralinguistic Digital Affordances in Social Media , 2016 .

[7]  C. Berger.  Knowledge Structures and Social Interaction , 2015 .

[8]  Zheng Yao,et al.  Two is Better Than One: Improving Multilingual Collaboration by Giving Two Machine Translation Outputs , 2015, CSCW.

[9]  Steven Skiena,et al.  POLYGLOT-NER: Massive Multilingual Named Entity Recognition , 2014, SDM.

[10]  T. Köhler,et al.  Brain drain: The cognitive neuroscience of foreign language processing in multinational corporations , 2014, Journal of International Business Studies.

[11]  Marlys K. Christianson,et al.  Sensemaking in Organizations: Taking Stock and Moving Forward , 2014 .

[12]  Hao-Chuan Wang,et al.  Supporting second language reading with picture note-taking , 2014, CHI Extended Abstracts.

[13]  Ge Gao,et al.  "Maybe it was a joke": emotion detection in text-only communication by non-native english speakers , 2014, CHI.

[14]  Scott A. Hale.  Global connectivity and multilinguals in the Twitter network , 2014, CHI.

[15]  Alfred Markey,et al.  A study of learners’ perceptions of online intercultural exchange through Web 2.0 technologies , 2014, ReCALL.

[16]  Christine Greenhow,et al.  Adolescents’ visions of their future careers, educational plans, and life pathways , 2014 .

[17]  Daniele Quercia,et al.  Twitter ain't without frontiers: economic, social, and cultural boundaries in international communication , 2014, CSCW.

[18]  Alexandra Balahur,et al.  Comparative experiments using supervised learning and machine translation for multilingual sentiment analysis , 2014, Comput. Speech Lang..

[19]  Kalina Bontcheva,et al.  Making sense of social media streams through semantics: A survey , 2014, Semantic Web.

[20]  Susan R. Fussell,et al.  Same translation but different experience: the effects of highlighting on machine-translated conversations , 2013, CHI.

[21]  Kadir Yoğurtçu.  The Impact of Self-efficacy Perception on Reading Comprehension on Academic Achievement , 2013 .

[22]  Timothy Baldwin,et al.  Lexical normalization for social media text , 2013, TIST.

[23]  Alcides Velasquez,et al.  The gap between producer intentions and consumer behavior in social media , 2012, GROUP '12.

[24]  Cliff Lampe,et al.  Facebook as a toolkit: A uses and gratification approach to unbundling feature use , 2011, Comput. Hum. Behav..

[25]  Ed H. Chi,et al.  Language Matters In Twitter: A Large Scale Study , 2011, ICWSM.

[26]  Robert E. Kraut,et al.  Social capital on facebook: differentiating uses and users , 2011, CHI.

[27]  Marie Hardin,et al.  Less effortful thinking leads to more social networking? The associations between the use of social network sites and personality traits , 2011, Comput. Hum. Behav..

[28]  Cliff Lampe,et al.  Connection strategies: Social capital implications of Facebook-enabled communication practices , 2011, New Media Soc..

[29]  John A. Daly,et al.  The Sage handbook of interpersonal communication , 2011 .

[30]  Cyrus Rashtchian,et al.  Every Picture Tells a Story: Generating Sentences from Images , 2010, ECCV.

[31]  Susan R. Fussell,et al.  Idea expander: supporting group brainstorming with conversationally triggered visual thinking stimuli , 2010, CSCW '10.

[32]  Anna Maria Di Sciullo,et al.  Natural Language Understanding , 2009, SoMeT.

[33]  Mary Beth Rosson,et al.  How and why people Twitter: the role that micro-blogging plays in informal communication at work , 2009, GROUP.

[34]  Steven Skiena,et al.  International Sentiment Analysis for News and Blogs , 2021, ICWSM.

[35]  Adam N. Joinson,et al.  Looking at, looking up or keeping up with people?: motives and use of facebook , 2008, CHI.

[36]  Tomohiro Shigenobu,et al.  Evaluation and Usability of Back Translation for Intercultural Communication , 2007, HCI.

[37]  Cliff Lampe,et al.  A face(book) in the crowd: social Searching vs. social browsing , 2006, CSCW '06.

[38]  Toru Ishida,et al.  Effects of machine translation on collaborative work , 2006, CSCW '06.

[39]  Toru Ishida,et al.  Automatic prediction of misconceptions in multilingual computer-mediated communication , 2006, IUI '06.

[40]  D. Schunk.  SELF-EFFICACY FOR READING AND WRITING: INFLUENCE OF MODELING, GOAL SETTING, AND SELF-EVALUATION , 2003 .

[41]  Allan Bird,et al.  Beyond sophisticated stereotyping: Cultural sensemaking in context , 2000 .

[42]  R. Paige,et al.  Culture Learning in Language Education: A Review of the Literature. , 2000 .

[43]  P. Ekman.  An argument for basic emotions , 1992 .

[44]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[45]  J. Mccroskey,et al.  The measurement of interpersonal attraction , 1974 .