Better Choice? Combining Speech and Touch in Multimodal Interaction for Elderly Persons

This paper presents our work on developing, implementing and evaluating a multimodal interactive guidance system that features spoken language and touch-screen input for elderly persons. The development foundation of the system comprises two systematically designed and empirically improved aspects: a set of development guidelines for elderly-friendly multimodal interaction according to common ageing-related decline of important human abilities, and a hybrid dialogue modelling approach with a formal method triggering and agent-based management for the elderly-centered multimodal interaction. To evaluate the minutely developed and implemented system, an experimental study was conducted with thirty-three elderly persons and empirical data were analyzed by applying an adapted version of a general evaluation framework, which provided overall positive analysis results and validated our effort to develop an effective, efficient and elderly friendly multimodal interaction.

[1]  Anthony Hall,et al.  Correctness by Construction: Developing a Commercial Secure System , 2002, IEEE Softw..

[2]  Cui Jian,et al.  Evaluating a Spoken Language Interface of a Multimodal Interactive Guidance System for Elderly Persons , 2012, HEALTHINF.

[3]  D. Zapf,et al.  Ageing and errors in computer‐based work: An observational field study , 1997 .

[4]  J. Fozard.  Nine – Vision and Hearing in Aging , 1990 .

[5]  W. Sanderson,et al.  The coming acceleration of global population ageing , 2008, Nature.

[6]  Cui Jian,et al.  Towards effective, efficient and elderly-friendly multimodal interaction , 2011, PETRA '11.

[7]  Hui Shi,et al.  Developing Human-Robot Dialogue Management Formally , 2005 .

[8]  Jarek Krajewski,et al.  An Acoustic Framework for Detecting Fatigue in Speech Based Human-Computer-Interaction , 2008, ICCHP.

[9]  Christian Mandel,et al.  Navigating a smart wheelchair with a brain-computer interface interpreting steady-state visual evoked potentials , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  W. Hoyer,et al.  Age and the ability to inhibit distractor information in visual selective attention. , 1995, Experimental aging research.

[11]  Neil Charness,et al.  Twenty Six – Human Factors and Design for Older Adults , 1990 .

[12]  Niko Moritz,et al.  Multimodal Human-Machine Interaction for Service Robots in Home-Care Environments , 2012, SMIAE@ACL.

[13]  David Traum,et al.  The Information State Approach to Dialogue Management , 2003 .

[14]  W. Alston.  Illocutionary acts and sentence meaning , 2000 .

[15]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[16]  D. Hawthorn,et al.  Possible implications of aging for interface designers , 2000, Interact. Comput..

[17]  Marcela D. Rodríguez,et al.  Design dimensions of ambient information systems to facilitate the development of AAL environments , 2011, PETRA '11.

[18]  Donald G. MacKay,et al.  Language, memory, and aging: Distributed deficits and the structure of new-versus-old connections. , 1996 .

[19]  F. Schieber.  Aging and the Senses , 1992 .

[20]  Timothy A. Salthouse,et al.  The aging of working memory. , 1994 .

[21]  Marilyn A. Walker,et al.  PARADISE: A Framework for Evaluating Spoken Dialogue Agents , 1997, ACL.

[22]  S. Folstein,et al.  "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. , 1975, Journal of psychiatric research.

[23]  A. D. Fisk,et al.  Age-related differences in movement control: adjusting submovement structure to optimize performance. , 1997, The journals of gerontology. Series B, Psychological sciences and social sciences.

[24]  F. Craik,et al.  Effects of aging and task difficulty on divided attention performance. , 1988, Journal of experimental psychology. Human perception and performance.

[25]  Maria Klara Wolters,et al.  Corpus Analysis of Spoken Smart-Home Interactions with Older Users , 2008, LREC.

[26]  A. W. Roscoe,et al.  Tutorial on FDR and Its Applications , 2000, SPIN.