Shared user-computer control of a robotic wheelchair system

An assistive robotic wheelchair system should allow its user to travel more efficiently and with greater ease. The robotic wheelchair system described in this thesis, Wheelesley, performs semi-autonomous navigation for its user, taking high-level directional commands and performing the low-level navigation required to avoid obstacles and stay on a safe path. The system consists of a standard electric wheelchair with an on-board computer for motor control, a vision system running on an off-the-shelf notebook computer, sensors, and a graphical user interface running on a tray mounted notebook computer. Most other research on robotic wheelchairs only addresses indoor navigation. The Wheelesley system can travel both indoors and outdoors using specialized navigation modes; there is a control algorithm for indoor navigation and a control algorithm for outdoor navigation. User tests have been conducted for both control modes to compare robotic assisted control against manual control. Robotic assisted control requires 71% less effort for indoor navigation and 88% less effort for outdoor navigation. In addition, the total time needed to travel between two points is reduced since less time is spent waiting to scan to the desired commands. The system switches automatically between navigation modes through the use of a novel indoor/outdoor detector. The detector is comprised of an ultrasonic transducer, three light-to-voltage optical sensors and a thermistor. A decision tree learned by C4.5 using data collected in a variety of indoor and outdoor conditions classified a test set correctly 98.3% of the time. The system can be easily customized for the access method(s) required by each user. This thesis describes customization of the user interface for two different access methods: eye tracking, an uncommon access method for a wheelchair, and single switch scanning, which is considered the driving method of last resort on standard powered wheelchairs. The wheelchair system and its interface was evaluated by a group of physical therapists. Thesis Supervisor: Rodney A. Brooks Title: Professor Shared User-Computer Control of a Robotic Wheelchair

[1]  G. Honderd,et al.  Rehabilitation robotics: the MANUS concept , 1991, Fifth International Conference on Advanced Robotics 'Robots in Unstructured Environments.

[2]  E. Guglielmelli,et al.  Robotics in medicine , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[3]  Minoru Abe,et al.  Electrocutaneous Communication in a Guide Dog Robot (MELDOG) , 1985, IEEE Transactions on Biomedical Engineering.

[4]  Matthew Turk,et al.  VITS-A Vision System for Autonomous Land Vehicle Navigation , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Peter J. Burt,et al.  Techniques for Autonomous, Off-Road Navigation , 1998, IEEE Intell. Syst..

[6]  George A. Bekey,et al.  On autonomous robots , 1998, The Knowledge Engineering Review.

[7]  Alan Wrench,et al.  Assessment of need for special seating and/or electronic control systems for wheelchairs among people with severe physical disabilities , 1993 .

[8]  Debbie Field,et al.  Powered Mobility: A Literature Review Illustrating the Importance of a Multifaceted Approach , 1999 .

[9]  Hideo Mori,et al.  Visual vehicle detection and tracking based on the 'sign pattern' , 1994, Adv. Robotics.

[10]  Yoshiaki Shirai,et al.  Intelligent wheelchair using visual information on human faces , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[11]  M. Mazo,et al.  Guidance of a wheelchair for handicapped people by face tracking , 1999, 1999 7th IEEE International Conference on Emerging Technologies and Factory Automation. Proceedings ETFA '99 (Cat. No.99TH8467).

[12]  William K. Durfee,et al.  IEEE/RSJ/GI International Conference on Intelligent Robots and Systems , 1994 .

[13]  J. Borenstein,et al.  The NavBelt-a computerized travel aid for the blind based on mobile robotics technology , 1998, IEEE Transactions on Biomedical Engineering.

[14]  Rosalind W. Picard,et al.  Texture orientation for sorting photos "at a glance" , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[15]  Tariq Rahman,et al.  A review of design issues in rehabilitation robotics with reference to North American research , 1995 .

[16]  Marek Perkowski,et al.  PSUBOT-a voice-controlled wheelchair for the handicapped , 1990, Proceedings of the 33rd Midwest Symposium on Circuits and Systems.

[17]  R. Lathe.  Phd by thesis , 1988, Nature.

[18]  H Wakaumi,et al.  Development of an automated wheelchair guided by a magnetic ferrite marker lane. , 1992, Journal of rehabilitation research and development.

[19]  G. Wyszecki,et al.  Color Science Concepts and Methods , 1982 .

[20]  W JACOB,et al.  [Clinical rehabilitation]. , 1961, Zeitschrift fur arztliche Fortbildung.

[21]  Markus Maurer,et al.  A compact vision system for road vehicle guidance , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[22]  Håkan Eftring,et al.  Robotics in rehabilitation , 1995 .

[23]  S B Skaar,et al.  Initial results in the development of a guidance system for a powered wheelchair. , 1996, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[24]  Hideo Mori,et al.  A new approach for real time moving vehicle detection , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[25]  Illah R. Nourbakhsh,et al.  Topological robot localization by training a vision-based transition detector , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[26]  James Gips,et al.  On Building Intelligence into EagleEyes , 1998, Assistive Technology and Artificial Intelligence.

[27]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[28]  Takakazu Ishimatsu,et al.  Self-localization for an electric wheelchair , 1997 .

[29]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[30]  Y. Agostini,et al.  Man-machine Cooperation for the Control of an Intelligent Powered Wheelchair , 1998, J. Intell. Robotic Syst..

[31]  F. Warren McFarlan,et al.  Trinity College (A) , 1996 .

[32]  Spyros G. Tzafestas,et al.  The autonomous mobile robot SENARIO: a sensor aided intelligent navigation system for powered wheelchairs , 1997, IEEE Robotics Autom. Mag..

[33]  Roger Orpwood,et al.  A WHEELCHAIR MOUNTED ASSISTIVE ROBOT , 1999 .

[34]  Jill Jepson,et al.  The design of a wheelchair mounted robot , 1997 .

[35]  Y. Agostini,et al.  The Vahm Robotized Wheelchair: System Architecture and Human-Machine Interaction , 1998, J. Intell. Robotic Syst..

[36]  C G Warren.  Powered mobility and its implications. , 1990, Journal of rehabilitation research and development. Clinical supplement.

[37]  Paul A. Viola,et al.  Cooperative control of a semi-autonomous mobile robot , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[38]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[39]  S P Levine,et al.  Automatic adaptation in the NavChair Assistive Wheelchair Navigation System. , 1999, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[40]  S J Sheredos,et al.  Ultrasonic head controller for powered wheelchairs. , 1995, Journal of rehabilitation research and development.

[41]  Ian Horswill,et al.  Specialization of perceptual processes , 1993 .

[42]  Carol H. Gwin,et al.  Assistive Technology Sourcebook , 1991 .

[43]  E. D. Dickmanns,et al.  A camera platform for intelligent vehicles , 1994, Proceedings of the Intelligent Vehicles '94 Symposium.

[44]  Hideo Mori,et al.  Danger estimation of vehicles at intersection , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[45]  P. D. BEATTIE,et al.  Self-Localisation in the ‘Senario’ Autonomous Wheelchair , 1998, J. Intell. Robotic Syst..

[46]  J. Davenport.  Editor , 1960 .

[47]  James Gips,et al.  Using EagleEyes—an electrodes based device for controlling the computer with your eyes—to help people with special needs , 1996 .

[48]  Robin R. Murphy,et al.  Artificial intelligence and mobile robots: case studies of successful robot systems , 1998 .

[49]  J. C. Rosier,et al.  The MANUS wheelchair-borne manipulator system , 1988 .

[50]  S P Levine,et al.  The NavChair Assistive Wheelchair Navigation System. , 1999, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[51]  Michael E. Cleary,et al.  Progress on the Deictically Controlled Wheelchair , 1998, Assistive Technology and Artificial Intelligence.

[52]  Benjamin Kuipers,et al.  Integrating Vision and Spatial Reasoning for Assistive Navigation , 1998, Assistive Technology and Artificial Intelligence.

[53]  Yoshiaki Shirai,et al.  Intelligent wheelchair based on the integration of human and environment observations , 1999, Proceedings 1999 International Conference on Information Intelligence and Systems (Cat. No.PR00446).

[54]  Melvyn Kettle,et al.  A national survey of wheelchair users , 1992 .

[55]  G Bourhis,et al.  Mobile robotics and mobility assistance for people with motor impairments: rational justification for the VAHM Project. , 1996, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[56]  H.F.M. Van der Loos,et al.  VA/Stanford rehabilitation robotics research and development program: lessons learned in the application of robotics technology to the field of rehabilitation , 1995 .

[57]  Hobart R. Everett,et al.  Sensors for Mobile Robots: Theory and Application , 1995 .

[58]  Martial Hebert,et al.  Vision and navigation for the Carnegie-Mellon Navlab , 1988 .

[59]  J A Sanford,et al.  People with mobility impairments in the United States today and in 2010. , 1996, Assistive technology : the official journal of RESNA.

[60]  Yoram Koren,et al.  Potential field methods and their inherent limitations for mobile robot navigation , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[61]  Warren Cg,et al.  Powered mobility and its implications. , 1990 .

[62]  Robert F. Cromp,et al.  The design of an autonomous vehicle for the disabled , 1986, IEEE J. Robotics Autom..

[63]  Noriaki Kiyohiro,et al.  HITOMI: Design and Development of a Robotic Travel Aid , 1998, Assistive Technology and Artificial Intelligence.

[64]  Sheredos Sj,et al.  Ultrasonic head controller for powered wheelchairs. , 1995 .

[65]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[66]  Ulrich Borgolte,et al.  Intelligent Control of a Semi-Autonomous Omnidirectional Wheelchair , 1995 .

[67]  Richard C. Simpson,et al.  EVALUATION OF THE HEPHAESTUS SMART WHEELCHAIR SYSTEM , 1999 .

[68]  Kenji Inoue,et al.  Development of an Intelligent Wheelchair Using Computer Graphics Animation and Simulation , 1991, Comput. Graph. Forum.

[69]  Yoshiaki Shirai,et al.  Robotic wheelchair with three control modes , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[70]  R. D. Jackson,et al.  Rehabilitation robotics in Europe , 1995 .

[71]  P. Rabischong.  Robotics for the Handicapped , 1982 .

[72]  Ulrich Borgolte,et al.  Architectural Concepts of a Semi-autonomous Wheelchair , 1998, J. Intell. Robotic Syst..

[73]  Ulrich Borgolte,et al.  An Omnidirectional Wheelchair with Enhanced Comfort Features , 1997 .

[74]  H. Harry Asada,et al.  A semi-autonomous control architecture applied to robotic wheelchairs , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[75]  Martial Hebert,et al.  A complete navigation system for goal acquisition in unknown environments , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[76]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[77]  Rodney A. Brooks,et al.  Visually-guided obstacle avoidance in unstructured environments , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[78]  David A. Bell,et al.  NavChair: An Assistive Wheelchair Navigation System with Automatic Adaptation , 1998, Assistive Technology and Artificial Intelligence.

[79]  Zunaid Kazi,et al.  Speech and Gesture Mediated Intelligent Teleoperation , 1998, Assistive Technology and Artificial Intelligence.

[80]  José Luis Lázaro,et al.  Wheelchair for physically disabled people with voice, ultrasonic and infrared sensor control , 1995, Auton. Robots.

[81]  Martial Hebert,et al.  A complete navigation system for goal acquisition in unknown environments , 1995, Auton. Robots.

[82]  David P. Miller,et al.  Design and testing of a low-cost robotic wheelchair prototype , 1995, Auton. Robots.

[83]  梶田 尚志,et al.  IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'97) , 1998 .

[84]  John Craig,et al.  Smart wheelchairs for mobility training , 1996 .

[85]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .

[86]  Jane Smith,et al.  The development of handy 1. A robotic system to assist the severely disabled , 1999 .

[87]  P. Rabischong.  ROBOTICS FOR THE HANDICAPPED , 1983 .

[88]  Shane MacNamara,et al.  Personal Adaptive Mobility Aid for the Infirm and Elderly Blind , 1995, Assistive Technology and Artificial Intelligence.

[89]  Holly A. Yanco,et al.  PRELIMINARY INVESTIGATION OF A SEMI-AUTONOMOUS ROBOTIC WHEELCHAIR DIRECTED THROUGH ELECTRODES , 1997 .

[90]  Paul Nisbet,et al.  The 'Smart' Wheelchair , 2000 .

[91]  Takashi Gomi,et al.  Developing Intelligent Wheelchairs for the Handicapped , 1998, Assistive Technology and Artificial Intelligence.

[92]  Anil K. Jain,et al.  Lane boundary detection using a multiresolution Hough transform , 1997, Proceedings of International Conference on Image Processing.

[93]  R A Cooper,et al.  Engineering manual and electric powered wheelchairs. , 1999, Critical reviews in biomedical engineering.