The state-of-the-art in autonomous wheelchairs controlled through natural language: A survey

Abstract Natural language is a flexible and powerful control modality which can transform a wheelchair from a vehicle into a genuine helper. While autonomous wheelchairs are increasingly designed to use natural language for control, most of them only handle a small number of rigid commands. To establish the state-of-the-art in language-enabled wheelchairs and determine how to improve natural language capabilities, we introduce a framework for analyzing and classifying properties of language-enabled wheelchairs. We then apply the framework to the twenty-four most recent natural language-enabled wheelchair projects, in order to compare their achievements and identify areas for improvement.

[1]  Nguyen Thanh Hai,et al.  MFCC-DTW Algorithm for Speech Recognition in an Intelligent Wheelchair , 2015 .

[2]  Muhammad Tahir Qadri,et al.  Voice Controlled Wheelchair Using DSK TMS320C6711 , 2009, 2009 International Conference on Signal Acquisition and Processing.

[3]  Matthias Scheutz The Inherent Dangers of Unidirectional Emotional Bonds between Humans and Social Robots , 2009 .

[4]  Rubén Posada-Gómez,et al.  Development of a Dual Control System Applied to a Smart Wheelchair, using Magnetic and Speech Control☆ , 2013 .

[5]  S. Fitzgerald,et al.  Outcomes of Wheelchair Systems Intervention With Residents of Long-Term Care Facilities , 2004, Assistive technology : the official journal of RESNA.

[6]  Guanghui Wang,et al.  A Novel Economical Embedded Multi-mode Intelligent Control System for Powered Wheelchair , 2010 .

[7]  Kiyoshi Irie,et al.  Outdoor Localization Using Stereo Vision Under Various Illumination Conditions , 2012, Adv. Robotics.

[8]  Christian Mandel,et al.  15 Years of Rolland , 2009 .

[9]  Rytis Maskeliunas,et al.  Multimodal Wheelchair Control for the Paralyzed People , 2011 .

[10]  Sachithra Hemachandra Narrated guided tour following and interpretation by an autonomous wheelchair , 2010 .

[11]  L Fehr,et al.  Adequacy of power wheelchair control interfaces for persons with severe disabilities: a clinical survey. , 2000, Journal of rehabilitation research and development.

[12]  Seth J. Teller,et al.  Following and interpreting narrated guided tours , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Matthias Scheutz,et al.  POWER: A domain-independent algorithm for Probabilistic, Open-World Entity Resolution , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[14]  Joelle Pineau,et al.  On the Design and Validation of an Intelligent Powered Wheelchair: Lessons from the SmartWheeler Project , 2010 .

[15]  Matthias Scheutz,et al.  Grounding Natural Language References to Unvisited and Hypothetical Locations , 2013, AAAI.

[16]  Shafaque Anjum Mohd Shakir Sheikh,et al.  An evolutionary approach for smart wheelchair system , 2015, 2015 International Conference on Communications and Signal Processing (ICCSP).

[17]  Jeanette K. Gundel,et al.  Cognitive Status and the form of Referring Expressions in Discourse , 1993, The Oxford Handbook of Reference.

[18]  R B Roemer,et al.  Voice controlled wheelchair. , 1977, Archives of physical medicine and rehabilitation.

[19]  Fillia Makedon,et al.  A development and evaluation platform for non-tactile power wheelchair controls , 2013, PETRA '13.

[20]  R. K. Megalingam,et al.  Automated voice based home navigation system for the elderly and the physically challenged , 2011, 13th International Conference on Advanced Communication Technology (ICACT2011).

[21]  Benjamin Kuipers,et al.  An Intellectual History of the Spatial Semantic Hierarchy , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[22]  Rory A. Cooper,et al.  Rehabilitation Engineering Applied to Mobility and Manipulation , 1995 .

[23]  Matthias Scheutz,et al.  A Framework for Resolving Open-World Referential Expressions in Distributed Heterogeneous Knowledge Bases , 2016, AAAI.

[24]  Uvais Qidwai,et al.  Arabic speech-controlled wheelchair: A fuzzy scenario , 2010, 10th International Conference on Information Science, Signal Processing and their Applications (ISSPA 2010).

[25]  Yi Zhang,et al.  A novel intelligent wheelchair control system based on hand gesture recognition , 2011, The 2011 IEEE/ICME International Conference on Complex Medical Engineering.

[26]  Takeshi Saitoh,et al.  Elevator available voice activated wheelchair , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[27]  David P. Miller,et al.  A demonstration of a conversationally guided smart wheelchair , 2007, Assets '07.

[28]  Evan A. Krause,et al.  Novel Mechanisms for Natural Human-Robot Interactions in the DIARC Architecture , 2013 .

[29]  Jean Oh,et al.  Inferring Maps and Behaviors from Natural Language Instructions , 2015, ISER.

[30]  Muhammad Asif,et al.  Dynamic Finger Movement Tracking and Voice Commands Based Smart Wheelchair , 2011 .

[31]  Gabriel Villarrubia,et al.  Alternative Human-Machine Interface system for powered wheelchairs , 2011, 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH).

[32]  Hongtao Wang,et al.  A Wheelchair Platform Controlled by a Multimodal Interface , 2015, 2015 2nd International Conference on Information Science and Control Engineering.

[33]  Matthias Scheutz,et al.  A Hybrid Architectural Approach to Understanding and Appropriately Generating Indirect Speech Acts , 2013, AAAI.

[34]  Roger B. Davis,et al.  Mobility difficulties are not only a problem of old age , 2001, Journal of General Internal Medicine.

[35]  Holly A. Yanco,et al.  Development and Testing of a Robotic Wheelchair System for Outdoor Navigation , 2001 .

[36]  Luís Paulo Reis,et al.  A Methodology for Creating an Adapted Command Language for Driving an Intelligent Wheelchair , 2015, J. Intell. Robotic Syst..

[37]  John A. Bateman,et al.  Towards Dialogue Based Shared Control of Navigating Robots , 2004, Spatial Cognition.

[38]  Radovan Stojanovic,et al.  Prototype of speech controlled cloud based wheelchair platform for disabled persons , 2014, 2014 3rd Mediterranean Conference on Embedded Computing (MECO).

[39]  Hyun-Yeol Chung,et al.  Voice/Non-Voice Classification Using Reliable Fundamental Frequency Estimator for Voice Activated Powered Wheelchair Control , 2007, ICESS.

[40]  Matthias Scheutz,et al.  Resolution of Referential Ambiguity in Human-Robot Dialogue Using Dempster-Shafer Theoretic Pragmatics , 2017, Robotics: Science and Systems.

[41]  Luís Paulo Reis,et al.  IntellWheels: Intelligent wheelchair with user-centered design , 2013, 2013 IEEE 15th International Conference on e-Health Networking, Applications and Services (Healthcom 2013).

[42]  Matthew R. Walter,et al.  Learning Semantic Maps from Natural Language Descriptions , 2013, Robotics: Science and Systems.

[43]  Matthias Scheutz,et al.  A Tale of Two Architectures: A Dual-Citizenship Integration of Natural Language and the Cognitive Map , 2017, AAMAS.

[44]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[45]  Gary Geunbae Lee,et al.  Recent Approaches to Dialog Management for Spoken Dialog Systems , 2010, J. Comput. Sci. Eng..

[46]  António Paulo Moreira,et al.  IntellWheels: modular development platform for intelligent wheelchairs. , 2011, Journal of rehabilitation research and development.

[47]  Claude Vincent,et al.  Development of an obstacle course assessment of wheelchair user performance (OCAWUP): A content validity study , 2004 .

[48]  Masayoshi Tabuse,et al.  Outdoor autonomous navigation using SURF features , 2011, Artificial Life and Robotics.

[49]  Clark Ja,et al.  Voice controlled wheelchair. , 1977 .

[50]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[51]  Mohd Murtadha Mohamad,et al.  Ubiquitous Positioning: Integrated GPS/Wireless LAN Positioning for Wheelchair Navigation System , 2013, ACIIDS.

[52]  Nina Dethlefs,et al.  Route instructions in map-based human-human and human-computer dialogue: A comparative analysis , 2010, J. Vis. Lang. Comput..

[53]  Joelle Pineau,et al.  Bayesian reinforcement learning for POMDP-based dialogue systems , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  Yong Tao,et al.  A behavior control method based on hierarchical POMDP for intelligent wheelchair , 2009, 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[55]  Matthias Scheutz,et al.  Going Beyond Literal Command-Based Instructions: Extending Robotic Natural Language Interaction Capabilities , 2015, AAAI.

[56]  Kazue Nishihara,et al.  Operation assistance of a voice-controlled electric wheelchair , 2007, ICMIT: Mechatronics and Information Technology.

[57]  Matthias Scheutz,et al.  Situated open world reference resolution for human-robot dialogue , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).