Motivational intensity and visual word search: Layout matters

Motivational intensity has been previously linked to information processing. In particular, it has been argued that affects which are high in motivational intensity tend to narrow cognitive scope. A similar effect has been attributed to negative affect, which has been linked to narrowing of cognitive scope. In this paper, we investigated how these phenomena manifest themselves during visual word search. We conducted three studies in which participants were instructed to perform word category identification. We manipulated motivational intensity by controlling reward expectations and affect via reward outcomes. Importantly, we altered visual search paradigms, assessing the effects of affective manipulations as modulated by information arrangement. We recorded multiple physiological signals (EEG, EDA, ECG and eye tracking) to assess whether motivational states can be predicted by physiology. Across the three studies, we found that high motivational intensity narrowed visual attentional scope by altering visual search strategies, especially when information was displayed sparsely. Instead, when information was vertically listed, approach-directed motivational intensity appeared to improve memory encoding. We also observed that physiology, in particular eye tracking, may be used to detect biases induced by motivational intensity, especially when information is sparsely organised.

[1]  B. Fredrickson The role of positive emotions in positive psychology. The broaden-and-build theory of positive emotions. , 2001, The American psychologist.

[2]  Jelena Havelka,et al.  Affective norms for 210 British English and Finnish nouns , 2010, Behavior research methods.

[3]  H. Otani,et al.  Arousal, valence, and memory for detail , 2004, Memory.

[4]  P. Obrist,et al.  The relationship among heart rate, caratid dP/dt, and blood pressure in humans as a function of the type of stress. , 1978, Psychophysiology.

[5]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[6]  Anthony J. Hornof,et al.  A Computational Model of “Active Vision” for Visual Search in Human–Computer Interaction , 2011, Hum. Comput. Interact..

[7]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[8]  A. Elliot,et al.  Approach and Avoidance Motivation , 2001 .

[9]  M. Ohsuga,et al.  Assessment of phasic work stress using autonomic indices. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[10]  Yiyu Yao,et al.  Visualization support for interactive query refinement , 2005, The 2005 IEEE/WIC/ACM International Conference on Web Intelligence (WI'05).

[11]  W. Klimesch EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[12]  A. Shimamura,et al.  Effect of negative emotional pictures on associative memory for peripheral information , 2007, Memory.

[13]  Kimberly S. Chiew,et al.  Temporal Dynamics of Motivation-Cognitive Control Interactions Revealed by High-Resolution Pupillometry , 2013, Front. Psychology.

[14]  Daniela Karin Rosner,et al.  Tag Clouds: Data Analysis Tool or Social Signaller? , 2008, Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008).

[15]  Steve Jones Graphical query specification and dynamic result previews for a digital library , 1998, UIST '98.

[16]  G. Pfurtscheller,et al.  Event-related dynamics of cortical rhythms: frequency-specific features and functional correlates. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[17]  E. Higgins Making a good decision: value from fit. , 2000, The American psychologist.

[18]  Maik C. Stüttgen,et al.  Computation of measures of effect size for neuroscience data sets , 2011, The European journal of neuroscience.

[19]  Bryan D. Poole,et al.  Asymmetrical hemisphere activation enhances global–local processing , 2013, Brain and Cognition.

[20]  Jari Kätsyri,et al.  Aesthetic images modulate emotional responses to reading news messages on a small screen: A psychophysiological investigation , 2012, Int. J. Hum. Comput. Stud..

[21]  H. Gr Compensatory control in the regulation of human performance under stress and high workload; a cognitive-energetical framework. , 1997 .

[22]  M. Botvinick,et al.  Motivation and cognitive control: from behavior to neural mechanism. , 2015, Annual review of psychology.

[23]  Robin S. Edelstein,et al.  Emotion and memory narrowing: A review and goal-relevance approach , 2009 .

[24]  Niklas Ravaja,et al.  Online News and Corporate ReputationA Neurophysiological Investigation , 2015, J. Media Psychol. Theor. Methods Appl..

[25]  Harm Veling,et al.  Boosting or choking – How conscious and unconscious reward processing modulate the active maintenance of goal-relevant information , 2011, Consciousness and Cognition.

[26]  Cristina Conati,et al.  Prediction of individual learning curves across information visualizations , 2016, User Modeling and User-Adapted Interaction.

[27]  Gyanendra K. Verma,et al.  Multimodal fusion framework: A multiresolution approach for emotion classification and recognition from physiological signals , 2014, NeuroImage.

[28]  Giulio Jacucci,et al.  The Psychophysiology Primer: A Guide to Methods and a Broad Review with a Focus on Human-Computer Interaction , 2016, Found. Trends Hum. Comput. Interact..

[29]  Philip A. Gable,et al.  The motivational dimensional model of affect: Implications for breadth of attention, memory, and cognitive categorisation , 2010 .

[30]  D. Casasanto,et al.  Motivation and Motor Control: Hemispheric Specialization for Approach Motivation Reverses with Handedness , 2012, PloS one.

[31]  Jiyoung Park,et al.  Neurobiological Concomitants of Motivational States , 2014 .

[32]  J. Förster,et al.  Enactment of approach and avoidance behavior influences the scope of perceptual and conceptual attention , 2006 .

[33]  Philip A. Gable,et al.  The role of asymmetric frontal cortical activity in emotion-related phenomena: A review and update , 2010, Biological Psychology.

[34]  Dorota Glowacka,et al.  Directing exploratory search with interactive intent modeling , 2013, CIKM.

[35]  Ryota Kanai,et al.  Direct current stimulation (tDCS) reveals parietal asymmetry in local/global and salience-based selection , 2013, Cortex.

[36]  G. R. J. Hockey Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework , 1997, Biological Psychology.

[37]  D. Rubin,et al.  r equivalent: A simple effect size indicator. , 2003, Psychological methods.

[38]  N. Burgess,et al.  Negative affect impairs associative memory but not item memory , 2013, Learning & memory.

[39]  Philip A. Gable,et al.  The effect of low versus high approach-motivated positive affect on memory for peripherally versus centrally presented information. , 2010, Emotion.

[40]  Michael E. Smith,et al.  Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods , 1998, Hum. Factors.

[41]  W. Boucsein Electrodermal activity, 2nd ed. , 2012 .

[42]  John J. B. Allen,et al.  Issues and assumptions on the road from raw signals to metrics of frontal EEG asymmetry in emotion , 2004, Biological Psychology.

[43]  Michael W. Cole,et al.  Reward Motivation Enhances Task Coding in Frontoparietal Cortex. , 2016, Cerebral cortex.

[44]  Philip A. Gable,et al.  The Blues Broaden, but the Nasty Narrows , 2010, Psychological science.

[45]  Cristina Conati,et al.  Predicting Confusion in Information Visualization from Eye Tracking and Interaction Data , 2016, IJCAI.

[46]  G. Stemmler,et al.  Separating emotion and motivational direction in fear and anger: effects on frontal asymmetry. , 2003, Emotion.

[47]  G. Clore,et al.  Affective Arousal as Information: How Affective Arousal Influences Judgments, Learning, and Memory. , 2008, Social and personality psychology compass.

[48]  M. Sheelagh T. Carpendale,et al.  VisGets: Coordinated Visualizations for Web-based Information Exploration and Discovery , 2008, IEEE Transactions on Visualization and Computer Graphics.

[49]  Shlomo Bentin,et al.  Attentional selection of relative SF mediates global versus local processing: evidence from EEG. , 2011, Journal of vision.

[50]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Andrew Howes,et al.  The adaptation of visual search strategy to expected information gain , 2008, CHI.

[52]  Robert Godwin-Jones Tag Clouds in the Blogosphere: Electronic Literacy and Social Networking , 2006 .

[53]  Jeffrey R. Huntsinger Does Emotion Directly Tune the Scope of Attention? , 2013 .

[54]  Mark D'Esposito,et al.  Modulation of working memory function by motivation through loss‐aversion , 2013, Human brain mapping.

[55]  N. Liberman,et al.  Distinguishing Gains from Nonlosses and Losses from Nongains: A Regulatory Focus Perspective on Hedonic Intensity , 2000 .

[56]  Daniel Brandeis,et al.  Frontal Midline Theta Reflects Individual Task Performance in a Working Memory Task , 2014, Brain Topography.

[57]  Cynthia A. Brewer,et al.  ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps , 2003 .

[58]  Jen-Her Wu,et al.  Improving searching and reading performance: the effect of highlighting and text color coding , 2003, Inf. Manag..

[59]  Jonathan R. Zadra,et al.  Emotion and perception: the role of affective information. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[60]  Philip A. Gable,et al.  Anger perceptually and conceptually narrows cognitive scope. , 2015, Journal of personality and social psychology.

[61]  Changchun Liu,et al.  Dynamic Difficulty Adjustment in Computer Games Through Real-Time Anxiety-Based Affective Feedback , 2009, Int. J. Hum. Comput. Interact..

[62]  John J. B. Allen,et al.  Frontal EEG asymmetry as a moderator and mediator of emotion , 2004, Biological Psychology.

[63]  Emily Balcetis,et al.  See what you want to see: motivational influences on visual perception. , 2006, Journal of personality and social psychology.

[64]  P. Welch The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[65]  John J. B. Allen,et al.  Anger and frontal brain activity: EEG asymmetry consistent with approach motivation despite negative affective valence. , 1998, Journal of personality and social psychology.

[66]  G. Clore,et al.  How emotions inform judgment and regulate thought , 2007, Trends in Cognitive Sciences.

[67]  Niklas Ravaja,et al.  Predicting purchase decision: The role of hemispheric asymmetry over the frontal cortex. , 2013 .

[68]  Pietro Perona,et al.  Homo economicus in visual search. , 2009, Journal of vision.

[69]  A. B. Scholey,et al.  Glucose administration, heart rate and cognitive performance: effects of increasing mental effort , 2000, Psychopharmacology.

[70]  D. Fotiadis,et al.  An integrated telemedicine platform for the assessment of affective physiological states , 2006, Diagnostic pathology.

[71]  D. Derryberry,et al.  Anxiety and attentional focusing: trait, state and hemispheric influences , 1998 .

[72]  E. Harmon-Jones,et al.  Attentional Consequences of Pregoal and Postgoal Positive Affects Motivational Intensity in Affects , 2022 .

[73]  Jeffrey Witzel,et al.  Comparisons of Online Reading Paradigms: Eye Tracking, Moving-Window, and Maze , 2012, Journal of psycholinguistic research.

[74]  M. Tomasello,et al.  Young children understand and defend the entitlements of others. , 2013, Journal of experimental child psychology.

[75]  A. Nowicka,et al.  Neural correlates of own and close-other’s name recognition: ERP evidence , 2014, Front. Hum. Neurosci..

[76]  G. Sperling,et al.  Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency. , 2006, Cerebral cortex.

[77]  Philip A. Gable,et al.  Affective motivational direction drives asymmetric frontal hemisphere activation , 2014, Experimental Brain Research.

[78]  M. Benedek,et al.  A continuous measure of phasic electrodermal activity , 2010, Journal of Neuroscience Methods.

[79]  Daniel Senkowski,et al.  Phase-locking and amplitude modulations of EEG alpha: Two measures reflect different cognitive processes in a working memory task. , 2004, Experimental psychology.

[80]  Christian Beste,et al.  Machine learning provides novel neurophysiological features that predict performance to inhibit automated responses , 2018, Scientific Reports.

[81]  N. Cohen,et al.  On opposing effects of emotion on contextual or relational memory , 2013, Front. Psychol..

[82]  Rafał Kuś,et al.  Name recognition in autism: EEG evidence of altered patterns of brain activity and connectivity , 2016, Molecular Autism.

[83]  Masataka Watanabe,et al.  Role of anticipated reward in cognitive behavioral control , 2007, Current Opinion in Neurobiology.

[84]  R. Näsänen,et al.  Eye movements in the visual search of word lists , 2002, Vision Research.

[85]  Glenn F. Wilson,et al.  Real-Time Assessment of Mental Workload Using Psychophysiological Measures and Artificial Neural Networks , 2003, Hum. Factors.

[86]  Thomas Ertl,et al.  Iterative integration of visual insights during patent search and analysis , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[87]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[88]  D. Tranel The effects of monetary incentive and frustrative nonreward on heart rate and electrodermal activity. , 1983, Psychophysiology.

[89]  A. Anderson,et al.  Positive affect increases the breadth of attentional selection , 2007, Proceedings of the National Academy of Sciences.

[90]  Jennifer M. Talarico,et al.  Positive emotions enhance recall of peripheral details , 2009, Cognition & emotion.

[91]  Elizabeth W. Dunn,et al.  Bulletin Personality and Social Psychology Affective Regulation of Stereotype Activation: It's the (accessible) Thought That Counts on Behalf Of: Society for Personality and Social Psychology , 2022 .

[92]  Peter Brusilovsky,et al.  Adaptive Visualization of Search Results: Bringing User Models to Visual Analytics , 2009, Inf. Vis..

[93]  Lynn Hasher,et al.  Delighted and distracted: positive affect increases priming for irrelevant information. , 2011, Emotion.

[94]  L. Pessoa,et al.  Positive emotions broaden the scope of attention and thought‐action repertoires , 2005, Cognition & emotion.

[95]  L. Pessoa On the relationship between emotion and cognition , 2008, Nature Reviews Neuroscience.

[96]  S. Fairclough,et al.  The effect of task demand and incentive on neurophysiological and cardiovascular markers of effort. , 2017, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[97]  H. M. Müller,et al.  EEG beta-power changes reflect motor involvement in abstract action language processing , 2017, Brain and Language.

[98]  Min-Shik Kim,et al.  Visual Search Does Not Remain Efficient When Executive Working Memory Is Working , 2004, Psychological science.