ON THE CONVERGENCE OF AFFECTIVE AND PERSUASIVE TECHNOLOGIES IN COMPUTER-MEDIATED HEALTH-CARE SYSTEMS

This paper offers a portrayal of how affective computing and persuasive technologies can converge into an effective tool for interfacing biomedical engineering with behavioral sciences and medicine. We describe the characteristics, features, applications, present state of the art, perspectives, and trends of both streams of research. In particular, these streams are analyzed in light of the potential contribution of their convergence for improving computer-mediated health-care systems, by facilitating the modification of patients' attitudes and behaviors, such as engagement and compliance. We propose a framework for future research in this emerging area, highlighting how key constructs and intervening variables should be considered. Some specific implications and challenges posed by the convergence of these two technologies in health care, such as paradigm change, multimodality, patients' attitude improvement, and cost reduction, are also briefly addressed and discussed.

[1]  Marcela D. Rodríguez,et al.  Persuasive strategies for motivating elders to exercise , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[2]  Zhaoyu Wang,et al.  Analyses of a Multimodal Spontaneous Facial Expression Database , 2013, IEEE Transactions on Affective Computing.

[3]  N. D. de Vries,et al.  Online Prevention Aimed at Lifestyle Behaviors: A Systematic Review of Reviews , 2013, Journal of medical Internet research.

[4]  Cynthia LeRouge,et al.  Developing multimodal intelligent affective interfaces for tele-home health care , 2003, Int. J. Hum. Comput. Stud..

[5]  Russell Haines,et al.  Computer-mediated group influence on ethical behavior , 2007, Comput. Hum. Behav..

[6]  Hatice Gunes,et al.  From the Lab to the real world: affect recognition using multiple cues and modalities , 2008 .

[7]  Laura R. Olevitch,et al.  Tailoring Health Messages: Customizing Communication With Computer Technology , 1999 .

[8]  Morten Aagaard,et al.  The Application of Persuasive Technology to educational settings: Some theoretical from the HANDS Project , 2010 .

[9]  Jakob E. Bardram,et al.  A framework for overcoming challenges in designing persuasive monitoring and feedback systems for mental illness , 2011, 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[10]  Georgios N. Yannakakis,et al.  Ethical Considerations in Designing Adaptive Persuasive Games , 2012 .

[11]  Erik Cambria,et al.  Towards an intelligent framework for multimodal affective data analysis , 2015, Neural Networks.

[12]  J. Hietanen,et al.  Bodily maps of emotions , 2013, Proceedings of the National Academy of Sciences.

[13]  José Manuel Páez-Borrallo,et al.  Innovative self management system for guided cardiac rehabilitation , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[14]  John E. Steephen,et al.  HED: A Computational Model of Affective Adaptation and Emotion Dynamics , 2013, IEEE Transactions on Affective Computing.

[15]  Tibert Verhagen,et al.  Understanding users' motivations to engage in virtual worlds: A multipurpose model and empirical testing , 2012, Comput. Hum. Behav..

[16]  Maja Pantic,et al.  Web-based database for facial expression analysis , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[17]  Judith Masthoff,et al.  Designing empathic computers: the effect of multimodal empathic feedback using animated agent , 2009, Persuasive '09.

[18]  Tsang-Long Pao,et al.  Segment-based emotion recognition from continuous Mandarin Chinese speech , 2011, Comput. Hum. Behav..

[19]  Luca Milani,et al.  Computer-mediated communication and persuasion: Peripheral vs. central route to opinion shift , 2008, Comput. Hum. Behav..

[20]  Harri Oinas-Kukkonen,et al.  Persuasive system design: state of the art and future directions , 2009, Persuasive '09.

[21]  Gerardo Iovane,et al.  A Computational Model for Managing Emotions and Affections in Emotional Learning Platforms and Learning Experience in Emotional Computing Context , 2012, 2012 Sixth International Conference on Complex, Intelligent, and Software Intensive Systems.

[22]  Tibor Bosse,et al.  Challenges in Computational Modeling of Affective Processes , 2013, IEEE Trans. Affect. Comput..

[23]  Joost Broekens,et al.  AffectButton: A method for reliable and valid affective self-report , 2013, Int. J. Hum. Comput. Stud..

[24]  Tatsuo Nakajima,et al.  Designing motivation using persuasive ambient mirrors , 2011, Personal and Ubiquitous Computing.

[25]  E. Thompson.  Development and Validation of an Internationally Reliable Short-Form of the Positive and Negative Affect Schedule (PANAS) , 2007 .

[26]  Rosalind W. Picard.  Affective Computing for HCI , 1999, HCI.

[27]  Pål Kraft,et al.  Digital Therapy: The Coming Together of Psychology and Technology Can Create a New Generation of Programs for More Sustainable Behavioral Change , 2007, PERSUASIVE.

[28]  T. Mirlacher,et al.  A persuasive interactive mannequin for shop windows , 2009, Persuasive '09.

[29]  E. Vance Wilson,et al.  Communication goals and online persuasion: An empirical examination , 2008, Comput. Hum. Behav..

[30]  Rosa Gil,et al.  Emotions ontology for collaborative modelling and learning of emotional responses , 2015, Comput. Hum. Behav..

[31]  Sean A. Munson,et al.  Pervasive, persuasive health: Some challenges , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[32]  C. Quinn,et al.  Cluster-Randomized Trial of a Mobile Phone Personalized Behavioral Intervention for Blood Glucose Control , 2011, Diabetes Care.

[33]  Mira Kim,et al.  A touch based affective user interface for smartphone , 2013, 2013 IEEE International Conference on Consumer Electronics (ICCE).

[34]  Tom Geller.  How do you feel?: your computer knows , 2014, CACM.

[35]  Jesse Hoey,et al.  Affect Control Processes: Probabilistic and Decision Theoretic Affective Control in Human-Computer Interaction , 2013, ArXiv.

[36]  Martijn H. Vastenburg,et al.  Flowie: A persuasive virtual coach to motivate elderly individuals to walk , 2009, 2009 3rd International Conference on Pervasive Computing Technologies for Healthcare.

[37]  Himalaya Patel,et al.  Receptive to bad reception: Jerky motion can make persuasive messages more effective , 2014, Comput. Hum. Behav..

[38]  Rosalind W. Picard,et al.  Affective medicine: technology with emotional intelligence. , 2002, Studies in health technology and informatics.

[39]  M. Rinck,et al.  Changing automatic behavior through self-monitoring: does overt change also imply implicit change? , 2013, Journal of behavior therapy and experimental psychiatry.

[40]  Dana Ruggiero,et al.  The effect of a persuasive social impact game on affective learning and attitude , 2015, Comput. Hum. Behav..

[41]  Pascale M. Le Blanc,et al.  Do-it-yourself: An online positive psychology intervention to promote positive emotions, self-efficacy, and engagement at work. , 2013 .

[42]  Aleksandra Tesanovic,et al.  HeartCycle: From insights to clinically evaluated ICT solutions for Telehealth , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[43]  M. Cole.  Cross-cultural universals of affective meaning. , 1976 .

[44]  Jonghwa Kim,et al.  In-vehicle Monitoring of Affective Symptoms for Diabetic Drivers - In-vehicle Hypoglycemia Alerting System in EU Project METABO , 2010, HEALTHINF.

[45]  Matthew S. Goodwin,et al.  Technology for just-in-time in-situ learning of facial affect for persons diagnosed with an autism spectrum disorder , 2008, Assets '08.

[46]  Eva Hudlicka,et al.  To feel or not to feel: The role of affect in human-computer interaction , 2003, Int. J. Hum. Comput. Stud..

[47]  Samir Chatterjee,et al.  Designing Effective Persuasive Systems Utilizing the Power of Entanglement: Communication Channel, Strategy and Affect , 2010, PERSUASIVE.

[48]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[49]  Stephen S. Intille,et al.  A new research challenge: persuasive technology to motivate healthy aging , 2004, IEEE Transactions on Information Technology in Biomedicine.

[50]  Harri Oinas-Kukkonen,et al.  Three Approaches to Ethical Considerations in the Design of Behavior Change Support Systems , 2013, PERSUASIVE.

[51]  Shaaron Ainsworth,et al.  Emotion understanding and performance during computer-supported collaboration , 2012, Comput. Hum. Behav..

[52]  P. Ekman,et al.  Emotion in the Human Face: Guidelines for Research and an Integration of Findings , 1972 .

[53]  M. Sriram Iyengar,et al.  GuideView: a system for developing structured, multimodal, multi-platform persuasive applications , 2009, Persuasive '09.

[54]  Paul Richard,et al.  Emotion assessment for affective computing based on physiological responses , 2012, 2012 IEEE International Conference on Fuzzy Systems.

[55]  John T. Stasko,et al.  A taxonomy of ambient information systems: four patterns of design , 2006, AVI '06.

[56]  Gaetano Valenza,et al.  Mood recognition in bipolar patients through the PSYCHE platform: Preliminary evaluations and perspectives , 2013, Artif. Intell. Medicine.

[57]  Panagiotis D. Bamidis,et al.  Affective Computing and Medical Informatics: State Of The Art in Emotion-Aware Medical Applications , 2008, MIE.

[58]  Jessika Schulze,et al.  Handbook Of Affect And Social Cognition , 2016 .

[59]  Mehmet Göktürk,et al.  Involuntary postural responses of users as input to Attentive Computing Systems: An investigation on head movements , 2012, Comput. Hum. Behav..

[60]  Jung-Hsien Chiang,et al.  An Efficient Cloud for Wellness Self-management Devices and Services , 2010, 2010 Fourth International Conference on Genetic and Evolutionary Computing.

[61]  Björn W. Schuller,et al.  AVEC 2013: the continuous audio/visual emotion and depression recognition challenge , 2013, AVEC@ACM Multimedia.

[62]  P D Bamidis,et al.  Affective Medicine , 2010, Methods of Information in Medicine.

[63]  Dongrui Wu,et al.  Fuzzy sets and systems in building closed-loop affective computing systems for human-computer interaction: Advances and new research directions , 2012, 2012 IEEE International Conference on Fuzzy Systems.

[64]  María Teresa Arredondo,et al.  A user centered design approach for patient interfaces to a diabetes IT platform , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[65]  M. Peeples,et al.  Integration of a Mobile-Integrated Therapy with Electronic Health Records: Lessons Learned , 2013, Journal of diabetes science and technology.

[66]  Emiliano Lorini,et al.  Computational Modeling of Emotion: Toward Improving the Inter- and Intradisciplinary Exchange , 2013, IEEE Transactions on Affective Computing.

[67]  Life-Style Risk Factors , 1989 .

[68]  Pj Philip Nickel,et al.  Trust, discourse ethics, and persuasive technology , 2012 .

[69]  Fuji Ren,et al.  Estimation of word emotions based on part of speech and positional information , 2011, Comput. Hum. Behav..

[70]  Antoni Bayes-Genis,et al.  [Noninvasive remote telemonitoring for ambulatory patients with heart failure: effect on number of hospitalizations, days in hospital, and quality of life. CARME (CAtalan Remote Management Evaluation) study]. , 2011, Revista espanola de cardiologia.

[71]  Sarit Kraus,et al.  Adaptive Agent for Player-Specific Fitness and Health Incentives in Mobile Location Based Games , 2013, AAAI Spring Symposium: Data Driven Wellness.

[72]  Nicu Sebe,et al.  Affective multimodal human-computer interaction , 2005, ACM Multimedia.

[73]  Panagiotis D. Bamidis,et al.  Affective computing in the era of contemporary neurophysiology and health informatics , 2004, Interact. Comput..

[74]  Karin Ackermann,et al.  The Nature Of Emotion Fundamental Questions , 2016 .

[75]  Patricia A. Chalmers,et al.  The role of cognitive theory in human-computer interface , 2003, Comput. Hum. Behav..

[76]  María Teresa Arredondo,et al.  Clinical validation of a wearable system for emotional recognition based on biosignals , 2008, Journal of telemedicine and telecare.

[77]  Katie A. Siek,et al.  Persuasive wearable technology design for health and wellness , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[78]  Jing Wang,et al.  Affective Computing Model for the Set Pair Users on Twitter , 2013 .

[79]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[80]  Scott B. MacKenzie,et al.  Construct Measurement and Validation Procedures in MIS and Behavioral Research: Integrating New and Existing Techniques , 2011, MIS Q..

[81]  Toshihisa Tanaka,et al.  Interactive component extraction from fEEG, fNIRS and peripheral biosignals for affective brain-machine interfacing paradigms , 2011, Comput. Hum. Behav..

[82]  Liew Kwei Ping,et al.  A framework of a childhood obesity intervention using persuasive web-mobile technology , 2012, 2012 International Conference on Computer & Information Science (ICCIS).

[83]  C. Osgood,et al.  Language, Meaning, and Culture: The Selected Papers of C.E. Osgood , 1990 .

[84]  Lisa A Marsch,et al.  Web-based behavioral treatment for substance use disorders as a partial replacement of standard methadone maintenance treatment. , 2014, Journal of substance abuse treatment.

[85]  Egon L. van den Broek,et al.  Ubiquitous emotion-aware computing , 2011, Personal and Ubiquitous Computing.

[86]  B. J. Fogg.  Healthy Living with Persuasive Technologies : Framework , Issues , and Challenges , 2009 .

[87]  Djamel Belaïd,et al.  Persuasive Healthcare Self-Management in Intelligent Environments , 2012, 2012 Eighth International Conference on Intelligent Environments.

[88]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[89]  Bilge Mutlu,et al.  Designing motivational agents: The role of praise, social comparison, and embodiment in computer feedback , 2011, Comput. Hum. Behav..

[90]  Chia-Ying Li,et al.  Persuasive messages on information system acceptance: A theoretical extension of elaboration likelihood model and social influence theory , 2013, Comput. Hum. Behav..

[91]  Andrea Kleinsmith,et al.  Affective Body Expression Perception and Recognition: A Survey , 2013, IEEE Transactions on Affective Computing.

[92]  W. Brinkman.  Current and Future Research Directions in Mental Health Computing , 2013 .

[93]  M. Munafo,et al.  Effectiveness and cost-effectiveness of computer and other electronic aids for smoking cessation: a systematic review and network meta-analysis. , 2012, Health technology assessment.

[94]  Johannes Schöning,et al.  Informing intelligent user interfaces by inferring affective states from body postures in ubiquitous computing environments , 2013, IUI '13.

[95]  Rosalind W. Picard.  Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[96]  Weihui Dai,et al.  Emotion recognition and affective computing on vocal social media , 2015, Inf. Manag..

[97]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[98]  Yvonne de Kort,et al.  Persuasive Technology for Human Well-Being: Setting the Scene , 2006, PERSUASIVE.

[99]  Patrick Shen-Pei Wang,et al.  Performance Comparisons of Facial Expression Recognition in Jaffe Database , 2008, Int. J. Pattern Recognit. Artif. Intell..

[100]  C. Lange,et al.  The chronic patient in face of falling ill and the treatment compliance , 2008 .

[101]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[102]  Octavian Postolache,et al.  Treat me well: Affective and physiological feedback for wheelchair users , 2012, 2012 IEEE International Symposium on Medical Measurements and Applications Proceedings.

[103]  B. J. Fogg,et al.  Persuasive technology: using computers to change what we think and do , 2002, UBIQ.

[104]  Boris E. R. de Ruyter,et al.  Personalizing persuasive technologies: Explicit and implicit personalization using persuasion profiles , 2015, Int. J. Hum. Comput. Stud..

[105]  Daniel Berdichevsky,et al.  Toward an ethics of persuasive technology , 1999, CACM.