A Dual Mode Human-Robot Teleoperation Interface Based on Airflow in the Aural Cavity

Robot teleoperation systems have been limited in their utility due to the need for operator motion, lack of portability and limitation to singular input modalities. In this article, the design and construction of a dual-mode human—machine interface system for robot teleoperation addressing all these issues is presented. The interface is capable of directing robotic devices in response to tongue movement and/or speech without insertion of any device in the vicinity of the oral cavity. The interface is centered on the unique properties of the human ear as an acoustic output device. Specifically, we present: (1) an analysis of the sensitivity of human ear canals as acoustic output device; (2) the design of a new sensor for monitoring airflow in the aural canal; (3) pattern recognition procedures for recognition of both speech and tongue movement by monitoring aural flow across several human test subjects; and (4) a conceptual design and simulation of the machine interface system. We believe this work will lay the foundation for a new generation of human machine interface systems for all manner of robotic applications.

[1]  Lawrence R. Rabiner,et al.  An algorithm for determining the endpoints of isolated utterances , 1975, Bell Syst. Tech. J..

[2]  Stan Davis,et al.  Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Se , 1980 .

[3]  John H. L. Hansen,et al.  Discrete-Time Processing of Speech Signals , 1993 .

[4]  Leah H. Jamieson,et al.  Endpoint detection of isolated utterances based on a modified Teager energy measurement , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Hidekazu Yoshikawa,et al.  A study on space interface for teleoperation system , 1994, Proceedings of 1994 3rd IEEE International Workshop on Robot and Human Communication.

[6]  A. Syrdal,et al.  Applied speech technology , 1995 .

[7]  He Qiang,et al.  On prefiltering and endpoint detection of speech signal , 1998, ICSP '98. 1998 Fourth International Conference on Signal Processing (Cat. No.98TH8344).

[8]  Jong-Oh Park,et al.  KIST teleoperation system for humanoid robot , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[9]  Claudio Becchetti,et al.  Speech Recognition: Theory and C++ Implementation , 1999 .

[10]  Roland Siegwart,et al.  Robots on the web [Guest Editorial] , 2000, IEEE Robotics Autom. Mag..

[11]  R. Siegwart,et al.  Special issues on Robots on the Web , 2000 .

[12]  Tomomasa Sato,et al.  Human motion tracking system based on skeleton and surface integration model using pressure sensors distribution bed , 2000, Proceedings Workshop on Human Motion.

[13]  Mattias Dahl,et al.  In-Ear Microphone Equalization Exploiting an Active Noise Control. , 2001 .

[14]  Al Richardson,et al.  Vision-based semi-autonomous outdoor robot system to reduce soldier workload , 2001, SPIE Defense + Commercial Sensing.

[15]  Mattias Dahl,et al.  Speech Recognition in Severely Disturbed Environments Combining Ear-Mic and Active Noise Control , 2002 .

[16]  Raúl Marín,et al.  Automatic speech recognition to teleoperate a robot via Web , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Daniel A. Kingsley,et al.  Improved mobility through abstracted biological principles , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Simon Ferguson,et al.  Grasp Recognition From Myoelectric Signals , 2002 .

[19]  Kuu-young Young,et al.  Challenges in VR-based robot teleoperation , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[20]  Sabri Tosunoglu,et al.  A REVIEW OF TELEOPERATION SYSTEM CONTROL , 2003 .

[21]  Kalevi Huhtala,et al.  Teleoperation interfaces for a remote controlled hydraulic mobile machine , 2003, Proceedings 2003 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2003).

[22]  Toshio Tsuji,et al.  A human-assisting manipulator teleoperated by EMG signals and arm motions , 2003, IEEE Trans. Robotics Autom..

[23]  Dong-Soo Kwon,et al.  Human friendly interfaces of robotic manipulator control for handicapped persons , 2003, Proceedings 2003 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2003).

[24]  Peter Xiaoping Liu,et al.  Visual gesture recognition for human-machine interface of robot teleoperation , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[25]  Roger D. Quinn,et al.  Human-machine interface for tele-robotic operation: mapping of tongue movements based on aural flow monitoring , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[26]  José del R. Millán,et al.  Noninvasive brain-actuated control of a mobile robot by human EEG , 2004, IEEE Transactions on Biomedical Engineering.

[27]  Meng Wang,et al.  A novel teleoperation paradigm for human-robot interaction , 2004, IEEE Conference on Robotics, Automation and Mechatronics, 2004..

[28]  Wyatt S. Newman,et al.  A human-robot interface based on electrooculography , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[29]  M. Urban,et al.  Fusion of voice, gesture, and human-computer interface controls for remotely operated robot , 2005, 2005 7th International Conference on Information Fusion.

[30]  Jonathan Kofman,et al.  Teleoperation of a robot manipulator using a vision-based human-robot interface , 2005, IEEE Transactions on Industrial Electronics.

[31]  Raul Wirz,et al.  A multimodal interface to control a robot arm via the web: a case study on remote programming , 2005, IEEE Transactions on Industrial Electronics.

[32]  Roger D. Quinn,et al.  Insect-like Antennal Sensing for Climbing and Tunneling Behavior in a Biologically-inspired Mobile Robot , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[33]  P.X. Liu,et al.  Voice based robot control , 2005, 2005 IEEE International Conference on Information Acquisition.

[34]  Kazuo Tanaka,et al.  Electroencephalogram-based control of an electric wheelchair , 2005, IEEE Transactions on Robotics.

[35]  Xiaoping Yun,et al.  Design, Implementation, and Experimental Results of a Quaternion-Based Kalman Filter for Human Body Motion Tracking , 2006, IEEE Trans. Robotics.

[36]  Gary E. Birch,et al.  An experimental study to investigate the effects of a motion tracking electromagnetic sensor during EEG data acquisition , 2006, IEEE Transactions on Biomedical Engineering.

[37]  C. Galindo,et al.  Control Architecture for Human–Robot Integration: Application to a Robotic Wheelchair , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  G. Bulbuller,et al.  In-Ear Microphone Speech Data Segmentation and Recognition using Neural Networks , 2006, 2006 IEEE 12th Digital Signal Processing Workshop & 4th IEEE Signal Processing Education Workshop.

[39]  Remzi Serdar Kurcan.  Isolated word recognition from in-ear microphone data using Hidden Markov Models (HMM) , 2006 .

[40]  R. Vaidyanathan,et al.  A Dual-Mode Human-Machine Interface for Robotic Control based on Acoustic Sensitivity of the Aural Cavity , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[41]  Dimitrios S. Koliousis.  Real-time speech recognition system for robotic control applications using an ear-microphone , 2007 .

[42]  Ravi Vaidyanathan,et al.  Tongue-Movement Communication and Control Concept for Hands-Free Human–Machine Interfaces , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.