A Stereo Vision Approach for Cooperative Robotic Movement Therapy

Movement therapy is an integrating part of stroke rehabilitation. The positive influence of intensive, repetitive motion training and the importance of active patient participation trigger the development of cooperative robotic assistants. We suggest a device for the re-education of upper limb movements in hemiparetic patients where a light-weight robotic arm that supports the deficient arm is equipped with a stereoscopic camera system. It follows the movements of the healthy arm that wears a sleeve equipped with flat round reflective markers detected by the cameras. We introduce an advanced robust and real-time algorithm to provide the tracking information. It performs a sparse marker based point cloud registration based on subpixel precision contour fits to enable high accuracy pose estimates while being capable of online model adjustments. The update rate of the tracking is 9 ms and the precision of the system is measured to be 0.5 mm. Tests with healthy subjects show that the system is able to accurately reproduce the movement of the healthy arm on an impaired arm.

[1]  Milan Sonka,et al.  Image pre-processing , 1993 .

[2]  Eric Foxlin,et al.  Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[3]  Tony Lindeberg Edge Detection and Ridge Detection with Automatic Scale Selection , 2004, International Journal of Computer Vision.

[4]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Paul J. Zsombor-Murray,et al.  Direct and specific least-square fitting of hyperbolæ and ellipses , 2004, J. Electronic Imaging.

[6]  Jrgen Richter-Gebert,et al.  Perspectives on Projective Geometry: A Guided Tour Through Real and Complex Geometry , 2011 .

[7]  Andrew W. Fitzgibbon,et al.  Direct Least Square Fitting of Ellipses , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[9]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[10]  A. U. Pehlivan,et al.  Current Trends in Robot-Assisted Upper-Limb Stroke Rehabilitation: Promoting Patient Engagement in Therapy , 2014, Current Physical Medicine and Rehabilitation Reports.

[11]  V. Feigin,et al.  Global and regional burden of stroke during 1990–2010: findings from the Global Burden of Disease Study 2010 , 2014, The Lancet.

[12]  Richard A. Volz,et al.  Estimating 3-D location parameters using dual number quaternions , 1991, CVGIP Image Underst..

[13]  J P Frisby,et al.  PMF: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit , 1985, Perception.

[14]  Farshid Amirabdollahian,et al.  Training modalities in robot-mediated upper limb rehabilitation in stroke: a framework for classification based on a systematic review , 2014, Journal of NeuroEngineering and Rehabilitation.

[15]  Mark Fiala,et al.  Designing Highly Reliable Fiducial Markers , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Éric Marchand,et al.  ViSP for visual servoing: a generic software platform with a wide class of robot control skills , 2005, IEEE Robotics & Automation Magazine.

[17]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[18]  J. Pulman,et al.  Assessing the Efficacy of Different Upper Limb Hemiparesis Interventions on Improving Health-Related Quality of Life in Stroke Patients: A Systematic Review , 2013, Topics in stroke rehabilitation.

[19]  Eric Mjolsness,et al.  New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence , 1998, NIPS.

[20]  Thomas S. Huang,et al.  Image processing , 1971 .

[21]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[23]  Will Traves,et al.  Perspectives in Projective Geometry. A Guided Tour Through Real and Complex Geometry , 2015, Am. Math. Mon..

[24]  Jürgen Richter-Gebert,et al.  Perspectives on Projective Geometry , 2011 .

[25]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[26]  Steven Gold,et al.  A Graduated Assignment Algorithm for Graph Matching , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Tony Lindeberg,et al.  Edge Detection and Ridge Detection with Automatic Scale Selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Gabor Fichtinger,et al.  OpenIGTLink: an open network protocol for image‐guided therapy environment , 2009, The international journal of medical robotics + computer assisted surgery : MRCAS.

[29]  Richard Sinkhorn A Relationship Between Arbitrary Positive Matrices and Doubly Stochastic Matrices , 1964 .

[30]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[31]  Matheen Siddiqui,et al.  Robust real-time upper body limb detection and tracking , 2006, VSSN '06.

[32]  Roger Y. Tsai,et al.  A new technique for fully autonomous and efficient 3D robotics hand/eye calibration , 1988, IEEE Trans. Robotics Autom..

[33]  Nassir Navab,et al.  Visual marker detection and decoding in AR systems: a comparative study , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[34]  John S. Bridle,et al.  Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters , 1989, NIPS.

[35]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[36]  Andrew Zisserman,et al.  Upper Body Detection and Tracking in Extended Signing Sequences , 2011, International Journal of Computer Vision.