Speech interface dialog with smart glasses

This paper describes design of elderly-user-friendly multi-mode user interface with different modules. Use of eye glasses is common among senior citizens, and it inspired us to implement interface modules on it. Indicator-based Glasses contains the Eye Blinking Detection module integrated with visual cues indicators as system feedback. The multi-mode interface provides five interaction channels by proposing audio input/output modules and Android application on smartphone device. The VoiceXML Dialog Manager implementation (VoiceON) is described and proposed for speech enabled computer initiated dialogues. Senior citizens suffering from mild and moderate dementia are the primary target group of the proposed system. The human factors of the multi-mode interface will be tested in experiment with senior citizens, and different scenarios will be evaluated.

[1]  Dirk Schnelle,et al.  Context aware voice user interfaces for workflow support: voice based support for mobile workers , 2008 .

[2]  Matthias Peissner,et al.  Voice User Interface Design , 2004, UP.

[3]  Niina Keränen,et al.  Twinkle Megane: Near-Eye LED Indicators on Glasses for Simple and Smart Navigation in Daily Life , 2016, eHealth 360°.

[4]  Gregor Rozinaj,et al.  Development of slovak GALAXY/voiceXML based spoken language dialogue system to retrieve information from the internet , 2006, INTERSPEECH.

[5]  Jozef Juhar,et al.  A Study of Acoustic Features for Emotional Speaker Recognition in I-Vector Representation , 2015 .

[6]  Jozef Juhár,et al.  Multimodal dialogue system with NAO and VoiceXML dialogue manager , 2017, 2017 8th IEEE International Conference on Cognitive Infocommunications (CogInfoCom).

[7]  Paul Lukowicz,et al.  In the blink of an eye: combining head motion and eye blink frequency for activity recognition with Google Glass , 2014, AH.

[8]  Kentaro Go,et al.  A gaze and speech multimodal interface , 2004, 24th International Conference on Distributed Computing Systems Workshops, 2004. Proceedings..

[9]  Christian Holz,et al.  DualBlink: A Wearable Device to Continuously Detect, Track, and Actuate Blinking For Alleviating Dry Eyes and Computer Vision Syndrome , 2017, IMWUT.

[10]  Andreas Wendemuth,et al.  Zanzibar OpenIVR: An Open-Source Framework for Development of Spoken Dialog Systems , 2011, TSD.

[11]  Jozef Juhar,et al.  HIDDEN MARKOV MODEL BASED SPEECH SYNTHESIS SYSTEM IN SLOVAK LANGUAGE WITH SPEAKER INTERPOLATION , 2015 .

[12]  Mari Ervasti,et al.  European Conference on Information Systems ( ECIS ) Summer 10-6-2011 WAYFINDING AID FOR THE ELDERLY WITH MEMORY DISTURBANCES , 2017 .

[13]  Marek Bundzel,et al.  IMPLEMENTATION OF INTELLIGENT SOFTWARE USING IBM WATSON AND BLUEMIX , 2017 .

[14]  Mary Zajicek,et al.  Successful and available: interface design exemplars for older users , 2004, Interact. Comput..

[15]  Liberios Vokorokos,et al.  DATA ACQUISITION IN NON-INVASIVE BRAIN-COMPUTER INTERFACE USING EMOTIV EPOC NEUROHEADSET , 2012 .

[16]  Amit Konar,et al.  Electrooculography based blink detection to prevent Computer Vision Syndrome , 2014, 2014 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT).

[17]  Margrit Betke,et al.  Real Time Eye Tracking and Blink Detection with USB Cameras , 2005 .

[18]  Fabio Lo Castro,et al.  Class I infrared eye blinking detector , 2008 .

[19]  Andreas Bulling,et al.  Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction , 2014, UbiComp Adjunct.