An egocentric computer vision based co-robot wheelchair

Motivated by the emerging needs to improve the quality of life for the elderly and disabled individuals who rely on wheelchairs for mobility, and who might have limited or no hand functionality at all, we propose an egocentric computer vision based co-robot wheelchair to enhance their mobility without hand usage. The co-robot wheelchair is built upon a typical commercial power wheelchair. The user can access 360 degrees of motion direction as well as a continuous range of speed without the use of hands via the egocentric computer vision based control we developed. The user wears an egocentric camera and collaborates with the robotic wheelchair by conveying the motion commands with head motions. Compared with previous sip-n-puff, chin-control and tongue-operated solutions to hands-free mobility, this egocentric computer vision based control system provides a more natural human robot interface. Our experiments show that this design is of higher usability and users can quickly learn to control and operate the wheelchair. Besides its convenience in manual navigation, the egocentric camera also supports novel user-robot interaction modes by enabling autonomous navigation towards a detected person or object of interest. User studies demonstrate the usability and efficiency of the proposed egocentric computer vision co-robot wheelchair.

[1]  Vicente Mut,et al.  Vision-Based Interfaces Applied to Assistive Robots , 2013 .

[2]  Martin Jägersand,et al.  VIBI: Assistive vision-based interface for robot manipulation , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[3]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[4]  Urbano Nunes,et al.  RobChair: Experiments evaluating Brain-Computer Interface to steer a semi-autonomous wheelchair , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Kohei Arai,et al.  Electric wheelchair control with gaze direction and eye blinking , 2009, Artificial Life and Robotics.

[6]  Gang Hua,et al.  A convolutional neural network cascade for face detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Éric Marchand,et al.  ViSP for visual servoing: a generic software platform with a wide class of robot control skills , 2005, IEEE Robotics & Automation Magazine.

[8]  Ravi Vaidyanathan,et al.  Real-time implementation of a non-invasive tongue-based human-robot interface , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  R. Simpson.  Smart wheelchairs: A literature review. , 2005, Journal of rehabilitation research and development.

[10]  Huosheng Hu,et al.  Head gesture recognition for hands-free control of an intelligent wheelchair , 2007, Ind. Robot.

[11]  Mohamed Fezari,et al.  Speech and sensor in guiding an electric wheelchair , 2005, 2005 ICSC Congress on Computational Intelligence Methods and Applications.

[12]  Alexandre Krupa,et al.  Vision-based assistance for wheelchair navigation along corridors , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Eun Yi Kim,et al.  Intelligent wheelchair (IW) interface using face and mouth recognition , 2009, IUI.

[14]  Huosheng Hu,et al.  EMG-based hands-free wheelchair control with EOG attention shift detection , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[15]  Te-Son Kuo,et al.  Head-controlled device with M3S-based for people with disabilities , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[16]  Lotte N. S. Andreasen Struijk,et al.  Inductive tongue control of powered wheelchairs , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[17]  R. Simpson,et al.  How many people would benefit from a smart wheelchair? , 2008, Journal of rehabilitation research and development.

[18]  T. Saitoh,et al.  Voice controlled intelligent wheelchair , 2007, SICE Annual Conference 2007.

[19]  Bernd Freisleben,et al.  HaWCoS: the "hands-free" wheelchair control system , 2002, ASSETS.

[20]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[21]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[22]  Roman P. Pflugfelder,et al.  Consensus-based matching and tracking of keypoints for object tracking , 2014, IEEE Winter Conference on Applications of Computer Vision.

[23]  R Lipskin.  An evaluation program for powered wheelchair control systems. , 1970, Bulletin of prosthetics research.

[24]  Ricardo Chavarriaga,et al.  The birth of the brain-controlled wheelchair , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Haibo Li,et al.  Active vision for controlling an electric wheelchair , 2011, Intelligent Service Robotics.

[26]  José del R. Millán,et al.  Brain-Controlled Wheelchairs: A Robotic Architecture , 2013, IEEE Robotics & Automation Magazine.

[27]  W. Marsden.  I and J , 2012 .

[28]  H.T. Nguyen,et al.  Real-time head movement system and embedded Linux implementation for the control of power wheelchairs , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[29]  Gang Hua,et al.  Probabilistic Elastic Matching for Pose Variant Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[31]  Hideyuki Tanaka,et al.  Robotic wheelchair using a high accuracy visual marker LentiBar and its application to door crossing navigation , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).