Elderly Speech-Gaze Interaction - State of the Art and Challenges for Interaction Design

Elderly people face problems when using current forms of Human-Computer Interaction (HCI). Developing novel and natural methods of interaction would facilitate resolving some of those issues. We propose that HCI can be improved by combining communication modalities, in particular, speech and gaze, in various ways. This study presents elderly speech-gaze interaction as a novel method in HCI, a review of literature for its potential of use, and discusses possible domains of application for further empirical investigations.

[1]  Sharon L. Oviatt,et al.  Ten myths of multimodal interaction , 1999, Commun. ACM.

[2]  Kiyohiro Shikano,et al.  Elderly acoustic model for large vocabulary continuous speech recognition , 2001, INTERSPEECH.

[3]  Kallirroi Georgila,et al.  Speech Input from Older Users in Smart Environments: Challenges and Perspectives , 2009, HCI.

[4]  Martin J. Russell,et al.  Exploiting a ‘gaze-Lombard effect’ to improve ASR performance in acoustically noisy settings , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Tanja Schultz,et al.  Speaker Characteristics , 2007, Speaker Classification.

[6]  D. Munoz,et al.  Age-related performance of human subjects on saccadic eye movement tasks , 1998, Experimental Brain Research.

[7]  Ralf Engbert,et al.  Tracking the mind during reading: the influence of past, present, and future words on fixation durations. , 2006, Journal of experimental psychology. General.

[8]  Keith Vertanen,et al.  Speech dasher: fast writing using speech and gaze , 2010, CHI.

[9]  Andreas Stolcke,et al.  Gaze-enhanced speech recognition , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  David Schlangen,et al.  Interpreting Situated Dialogue Utterances: an Update Model that Uses Speech, Gaze, and Gesture Information , 2013, SIGDIAL Conference.

[11]  Jeff K. Caird,et al.  Visual Search for Traffic Signs: The Effects of Clutter, Luminance, and Aging , 2001, Hum. Factors.

[12]  W. Haynes,et al.  Topic manipulation and cohesive adequacy in conversations of normal adults between the ages of 30 and 90. , 1989, Clinical linguistics & phonetics.

[13]  Isabel Trancoso,et al.  A corpus-based study of elderly and young speakers of European Portuguese: acoustic correlates and their impact on speech recognition performance , 2013, INTERSPEECH.

[14]  Rashid Ansari,et al.  Multimodal human discourse: gesture and speech , 2002, TCHI.

[15]  Keith Vertanen,et al.  Speech dasher: a demonstration of text input using speech and approximate pointing , 2014, ASSETS.

[16]  José Miguel Salles Dias,et al.  Multimodal User Interfaces to Improve Social Integration of Elderly and Mobility Impaired , 2012, pHealth.

[17]  C. A. Weaver,et al.  Psychology of Reading , 2012 .

[18]  Géza Németh,et al.  The EASR Corpora of European Portuguese, French, Hungarian and Polish Elderly Speech , 2014, LREC.

[19]  Otmar Bock,et al.  Age-related decline of peripheral visual processing: the role of eye movements , 2011, Experimental Brain Research.

[20]  Steve An Xue,et al.  Changes in the human vocal tract due to aging and the acoustic correlates of speech production: a pilot study. , 2003, Journal of speech, language, and hearing research : JSLHR.

[21]  Isabel Trancoso,et al.  Impact of Age in ASR for the Elderly: Preliminary Experiments in European Portuguese , 2012, IberSPEECH.

[22]  Constantine Stephanidis,et al.  Universal accessibility in HCI: Process-oriented design guidelines and tool requirements , 1999 .

[23]  Natalie Liberman,et al.  Recognition of elderly speech and voice-driven document retrieval , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[24]  K. Rayner,et al.  Eye movements and the perceptual span in older and younger readers. , 2009, Psychology and aging.

[25]  Yan Li,et al.  Senior Citizens' Acceptance of Information Systems: A Study in the Context of e-Government Services , 2006, IEEE Transactions on Engineering Management.

[26]  Frank Schieber,et al.  Age Differences in the Useful Field of View during Real-World Driving , 2005 .

[27]  Martin J. Russell,et al.  Gaze-contingent automatic speech recognition , 2008 .

[28]  Michael Jessen,et al.  Speaker Classification in Forensic Phonetics and Acoustics , 2007, Speaker Classification.

[29]  W. Einhäuser,et al.  Effects of aging on eye movements in the real world , 2015, Front. Hum. Neurosci..

[30]  Stefanie E. Kuchinsky,et al.  Pupil size varies with word listening and response selection difficulty in older adults with hearing loss. , 2013, Psychophysiology.

[31]  C. Wojcik.  Springer international publishing switzerland , 2016 .

[32]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[33]  E. Szabadi,et al.  Changes in autonomic function with age: a study of pupillary kinetics in healthy young and old people. , 1996, Age and ageing.

[34]  Rakesh Gupta,et al.  Situated multi-modal dialog system in vehicles , 2013, GazeIn '13.

[35]  C. Scialfa,et al.  Age differences in the useful field of view: an eye movement analysis. , 1994, Optometry and vision science : official publication of the American Academy of Optometry.

[36]  K. Rayner,et al.  The psychology of reading , 1989 .

[37]  Gökhan Tür,et al.  Multi-Modal Conversational Search and Browse , 2013, SLAM@INTERSPEECH.

[38]  M. Mather,et al.  Aging and motivated cognition: the positivity effect in attention and memory , 2005, Trends in Cognitive Sciences.

[39]  Jeffrey G Malins,et al.  The roles of tonal and segmental information in Mandarin spoken word recognition: An eyetracking study , 2010 .

[40]  H. Mitterer,et al.  No delays in application of perceptual learning in speech recognition: Evidence from eye tracking , 2013 .

[41]  Edwin S. Dalmaijer,et al.  Is the low-cost EyeTribe eye tracker any good for research? , 2014 .

[42]  Alison C Bowling,et al.  Using saccadic eye movements to assess cognitive decline with ageing , 2014 .

[43]  T. D. Hanley,et al.  Vocal aging. , 1959, Geriatrics.

[44]  Alexander Pollatsek,et al.  Psychology of reading, 2nd ed. , 2012 .

[45]  D. Munoz,et al.  Look away: the anti-saccade task and the voluntary control of eye movement , 2004, Nature Reviews Neuroscience.

[46]  Joyce Yue Chai,et al.  What's in a gaze?: the role of eye-gaze in reference resolution in multimodal conversational interfaces , 2008, IUI '08.

[47]  K. Rayner,et al.  The effect of foveal and parafoveal masks on the eye movements of older and younger readers. , 2014, Psychology and aging.

[48]  David Shinar,et al.  Eye Movements of Younger and Older Drivers , 1997, Hum. Factors.

[49]  K. Rayner.  Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[50]  R Freedman,et al.  Age Diminishes Performance on an Antisaccade Eye Movement Task , 1997, Neurobiology of Aging.

[51]  Normand Teasdale,et al.  Aging Yields a Smaller Number of Fixations and a Reduced Gaze Amplitude When Driving in a Simulator , 2006 .

[52]  Alan Kennedy,et al.  Book Review: Eye Tracking: A Comprehensive Guide to Methods and Measures , 2016, Quarterly journal of experimental psychology.

[53]  António J. S. Teixeira,et al.  Impact of age in the production of European Portuguese vowels , 2014, INTERSPEECH.

[54]  J. Enns,et al.  LIFESPAN CHANGES IN ATTENTION: THE VISUAL SEARCH TASK , 1998 .

[55]  Dilek Z. Hakkani-Tür,et al.  Eye Gaze for Spoken Language Understanding in Multi-modal Conversational Interactions , 2014, ICMI.

[56]  Dilek Z. Hakkani-Tür,et al.  The Relation of Eye Gaze and Face Pose: Potential Impact on Speech Recognition , 2014, ICMI.

[57]  Jay G. Wilpon,et al.  A study of speech recognition for children and the elderly , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[58]  Miguel Sales Dias,et al.  Towards Elderly Social Integration using a Multimodal Human-computer Interface , 2016 .