The EMPATHIC Project: Building an Expressive, Advanced Virtual Coach to Improve Independent Healthy-Life-Years of the Elderly

This paper outlines the EMPATHIC Research & Innovation project, which aims to research, innovate, explore and validate new interaction paradigms and platforms for future generations of Personalized Virtual Coaches to assist elderly people living independently at and around their home. Innovative multimodial face analytics, adaptive spoken dialogue systems, and natural language interfaces are part of what the project investigates and innovates, aiming to help dependent aging persons and their carers. It will use remote, non-intrusive technologies to extract physiological markers of emotional states and adapt respective coach responses. In doing so, it aims to develop causal models for emotionally believable coach-user interactions, which shall engage elders and thus keep off loneliness, sustain health, enhance quality of life, and simplify access to future telecare services. Through measurable end-user validations performed in Spain, Norway and France (and complementary user evaluations in Italy), the proposed methods and solutions will have to demonstrate usefulness, reliability,, flexibility and robustness

[1]  J. Rotter.  Interpersonal trust, trustworthiness, and gullibility. , 1980 .

[2]  Dennis H. Klatt,et al.  Software for a cascade/parallel formant synthesizer , 1980 .

[3]  Mark H. Davis.  Measuring individual differences in empathy: Evidence for a multidimensional approach. , 1983 .

[4]  Francis Charpentier,et al.  Diphone synthesis using an overlap-add technique for speech waveforms concatenation , 1986, ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Godfrey T. Barrett-Lennard,et al.  The Relationship Inventory now: Issues and advances in theory, method, and use. , 1986 .

[6]  Alan W. Black,et al.  Unit selection in a concatenative speech synthesis system using a large speech database , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[7]  Nirbhay N. Singh,et al.  Facial Expressions of Emotion , 1998 .

[8]  Gérard Chollet,et al.  Toward ALISP: A proposal for Automatic Language Independent Speech Processing , 1999 .

[9]  Clifford Nass,et al.  Effects of Humor in Task-Oriented Human-Computer Interaction and Computer-Mediated Communication: A Direct Test of SRCT Theory , 1999, Hum. Comput. Interact..

[10]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[11]  Roberto Pieraccini,et al.  A stochastic model of human-machine interaction for learning dialog strategies , 2000, IEEE Trans. Speech Audio Process..

[12]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[13]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[14]  Charles A. Micchelli,et al.  Kernels for Multi--task Learning , 2004, NIPS.

[15]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[16]  D McDaid,et al.  Promoting mental well-being in the workplace: A European policy perspective , 2005, International review of psychiatry.

[17]  T. Kirkwood,et al.  Understanding the Odd Science of Aging , 2005, Cell.

[18]  Jürgen Schmidhuber,et al.  Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks , 2006, ICML.

[19]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.

[20]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[21]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[22]  Charles A. Micchelli,et al.  Universal Multi-Task Kernels , 2008, J. Mach. Learn. Res..

[23]  Jean-Philippe Vert,et al.  Clustered Multi-Task Learning: A Convex Formulation , 2008, NIPS.

[24]  Milica Gasic,et al.  Spoken language understanding from unaligned data using discriminative classification models , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[25]  E. Keogh,et al.  Technologically-assisted behaviour change: a systematic review of studies of novel technologies for the management of chronic illness , 2009, Journal of telemedicine and telecare.

[26]  D. Siewiorek,et al.  Virtual coach technology for supporting self-care. , 2010, Physical medicine and rehabilitation clinics of North America.

[27]  K. Jöreskog,et al.  Confirmatory Factor Analysis of Ordinal Variables With Misspecified Models , 2010 .

[28]  Adam Niewiadomski,et al.  On Finity, Countability, Cardinalities, and Cylindric Extensions of Type-2 Fuzzy Sets in Linguistic Summarization of Databases , 2010, IEEE Transactions on Fuzzy Systems.

[29]  Björn W. Schuller,et al.  Interacting with Emotional Virtual Agents , 2011, INTETAIN.

[30]  Koby Crammer,et al.  Learning Multiple Tasks using Shared Hypotheses , 2012, NIPS.

[31]  Daniel P. Siewiorek,et al.  Architecture and Applications of Virtual Coaches , 2012, Proceedings of the IEEE.

[32]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[33]  Cynthia Breazeal,et al.  Detecting the Trustworthiness of Novel Partners in Economic Exchange , 2012, Psychological science.

[34]  Vysoké Učení,et al.  Statistical Language Models Based on Neural Networks , 2012 .

[35]  Maja Pantic,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON AFFECTIVE COMPUTING , 2022 .

[36]  Axel Buendia,et al.  From Informative Cooperative Dialogues to Long-Term Social Relation with a Robot , 2012, Natural Interaction with Robots, Knowbots and Smartphones, Putting Spoken Dialog Systems into Practice.

[37]  Hal Daumé,et al.  Learning Task Grouping and Overlap in Multi-task Learning , 2012, ICML.

[38]  Paolo Dario,et al.  Robot Companions For Citizens: Roadmapping The Potential For Future Robots In Empowering Older People , 2012 .

[39]  M. S. Hawley,et al.  A Voice-Input Voice-Output Communication Aid for People With Severe Speech Impairment , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[40]  Kate Cavanagh,et al.  (Inter)personal Computing: The Role of the Therapeutic Relationship in E-mental Health , 2013, Journal of Contemporary Psychotherapy.

[41]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[42]  Stuart G Parker,et al.  Telecare for an ageing population? , 2013, Age and ageing.

[43]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[44]  Giovanni Scapagnini,et al.  Healthy aging diets other than the Mediterranean: A focus on the Okinawan diet , 2014, Mechanisms of Ageing and Development.

[45]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[46]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[47]  Koby Crammer,et al.  Convex Multi-Task Learning by Clustering , 2015, AISTATS.

[48]  Geoffrey Zweig,et al.  Sequence-to-sequence neural net models for grapheme-to-phoneme conversion , 2015, INTERSPEECH.

[49]  Rafael E. Banchs,et al.  The Fourth Dialog State Tracking Challenge , 2016, IWSDS.

[50]  M. Inés Torres,et al.  A Multi-lingual Evaluation of the vAssist Spoken Dialog System. Comparing Disco and RavenClaw , 2016, IWSDS.

[51]  J. Gisbert,et al.  Ustekinumab for the Treatment of Refractory Crohn's Disease: The Spanish Experience in a Large Multicentre Open-label Cohort , 2016, Inflammatory bowel diseases.

[52]  M. Inés Torres,et al.  Entropy-Driven Dialog for Topic Classification: Detecting and Tackling Uncertainty , 2016, IWSDS.

[53]  María Inés Torres,et al.  First time encounters with Roberta : a humanoid assistant for conversational autobiography creation , 2017 .

[54]  Physical Medicine and Rehabilitation Clinics of North America. , 2020, Physical medicine and rehabilitation clinics of North America.