Using affective signals as implicit indicators of information relevance and information processing strategies

Search engines have become increasingly better at providing information to users. However, they still face major challenges, such as determining how searchers process information, how they make relevance judgments, and how their cognitive or emotional states affect their search progress. We address these challenges by exploring searchers' affective dimension. In particular, we investigate how feelings, facial expressions, and electrodermal activity (EDA) could help to understand information relevance, search progress, and information processing strategies (IPS). To meet this goal, we designed an experiment in which 45 participants were exposed to affective stimuli prior to solving a fact‐finding search task. Results indicate that initial affective dimensions are linked to IPSs, search progress, and task completion. However, further analyses suggest that affective‐related features alone have limited utility in the binary classification of relevance using machine learning techniques.

[1]  J. Forgas,et al.  Affective influences on partner choice: role of mood in social decisions. , 1991, Journal of personality and social psychology.

[2]  Rosalind W. Picard Affective computing: (526112012-054) , 1997 .

[3]  Ben Xianye Face detection and tracking in video , 2011 .

[4]  Загоровская Ольга Владимировна,et al.  Исследование влияния пола и психологических характеристик автора на количественные параметры его текста с использованием программы Linguistic Inquiry and Word Count , 2015 .

[5]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[6]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[7]  Chirag Shah,et al.  Investigating positive and negative affects in collaborative information seeking: A pilot study report , 2012, ASIST.

[8]  Gilbert Cockton From doing to being: bringing emotion into interaction (Editorial) , 2001, Interact. Comput..

[9]  Frank E. Pollick,et al.  Understanding Relevance: An fMRI Study , 2013, ECIR.

[10]  Chirag Shah,et al.  Smile! Studying expressivity of happiness as a synergic factor in collaborative information seeking , 2011, ASIST.

[11]  Ian Ruthven,et al.  The emotional impact of search tasks , 2010, IIiX.

[12]  Nicholas J. Belkin,et al.  Personalizing information retrieval for multi-session tasks: the roles of task stage and task type , 2010, SIGIR '10.

[13]  Joemon M. Jose,et al.  An effective implicit relevance feedback technique using affective, physiological and behavioural features , 2013, SIGIR.

[14]  Roberto I. González-Ibáñez Affective Signals as Implicit Indicators of Information Relevancy and Information Processing Strategies , 2015 .

[15]  John J. B. Allen,et al.  The handbook of emotion elicitation and assessment , 2007 .

[16]  Jacek Gwizdka,et al.  Does interactive search results overview help?: an eye tracking study , 2013, CHI Extended Abstracts.

[17]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[18]  B. Fredrickson,et al.  The Undoing Effect of Positive Emotions , 2000, Motivation and emotion.

[19]  Carol Collier Kuhlthau Inside the Search Process: Information Seeking from the User's Perspective. , 1991 .

[20]  Irene Lopatovska,et al.  Does the mood matter? , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[21]  W. Bruce Croft,et al.  Quary Expansion Using Local and Global Document Analysis , 1996, SIGIR Forum.

[22]  Chirag Shah,et al.  Coagmento: A system for supporting collaborative information seeking , 2011, ASIST.

[23]  Barbara Means,et al.  The influence of positive affect on decision-making strategy. , 1983 .

[24]  Joseph P. Forgas,et al.  The Affect Infusion Model (AIM): An integrative theory of mood effects on cognition and judgments. , 2001 .

[25]  Christian Küblbeck,et al.  Face detection and tracking in video sequences using the modifiedcensus transformation , 2006, Image Vis. Comput..

[26]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[27]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[28]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[29]  Irene Lopatovska,et al.  Emotional correlates of information retrieval behaviors , 2011, 2011 IEEE Workshop on Affective Computational Intelligence (WACI).

[30]  Maryanne Martin On the induction of mood , 1990 .

[31]  R. C. Sinclair,et al.  The effects of mood state on judgemental accuracy: Processing strategy as a mechanism , 1995 .

[32]  J. Turner Human Emotions: A Sociological Theory , 2007 .

[33]  T. D. Wilson,et al.  Review of: Nahl, Diane and Bilal, Dania, Eds. Information and emotion: the emergent affective paradigm in information behavior research and theory. Medford, NJ: Information Today, Inc., 2007 , 2008, Inf. Res..

[34]  W. Bruce Croft,et al.  Query expansion using local and global document analysis , 1996, SIGIR '96.

[35]  Joseph P. Forgas,et al.  Affective Influences on Judgments and Behavior in Organizations: An Information Processing Perspective , 2001 .

[36]  M. Saunders,et al.  Maximizing Student Performance in Summary Writing: Managing Cognitive Load , 1991 .

[37]  Elad Yom-Tov,et al.  Social bookmark weighting for search and recommendation , 2010, The VLDB Journal.

[38]  Nicu Sebe,et al.  Authentic Facial Expression Analysis , 2004, FGR.

[39]  F. P. Cantero,et al.  Certezas y controversias en el estudio de la emoción , 2006 .

[40]  Ahmed Hassan Awadallah,et al.  Beyond DCG: user behavior as a predictor of a successful search , 2010, WSDM '10.

[41]  Carson Reynolds,et al.  The HandWave Bluetooth Skin Conductance Sensor , 2005, ACII.

[42]  P. Ekman Emotion in the human face , 1982 .

[43]  Joemon M. Jose,et al.  Affective feedback: an investigation into the role of emotions in the information seeking process , 2008, SIGIR '08.

[44]  ChengXiang Zhai,et al.  Positional relevance model for pseudo-relevance feedback , 2010, SIGIR.

[45]  James Allan,et al.  Predicting searcher frustration , 2010, SIGIR.