Hand tracking in bimanual movements

A general hand-tracking algorithm is presented for tracking hands in bimanual movements. The problem is approached from a neuroscience point of view. Using a dynamic model and some motor control phenomena, the movement of the hands during a bimanual movement is recognised. By capturing the hands velocities and recognising the bimanual synchronisation, the hands are tracked and reacquired in different types of movements. This includes tracking hands when they are separated, and reacquiring them at the end of hand-hand occlusion. Different applications are demonstrated including active vision where the camera view direction and position change.

[1]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[2]  Tetsushi Wakabayashi,et al.  Extraction of hand region and specification of finger tips from color image , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[3]  Nebojsa Jojic,et al.  Tracking self-occluding articulated objects in dense disparity maps , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Markus Kohler.  System architecture and techniques for gesture recognition in unconstrained environments , 1997, Proceedings. International Conference on Virtual Systems and MultiMedia VSMM '97 (Cat. No.97TB100182).

[5]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[6]  W. Prinz,et al.  Perceptual basis of bimanual coordination , 2001, Nature.

[7]  Joseph A. Paradiso,et al.  Tracking hands above large interactive surfaces with a low-cost scanning laser rangefinder , 1998, CHI Conference Summary.

[8]  Alex Pentland,et al.  Looking at People: Sensing for Ubiquitous and Wearable Computing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  I. Jolliffe.  Principal Component Analysis , 2002 .

[10]  Sergios Theodoridis,et al.  Pattern Recognition, Third Edition , 2006 .

[11]  Mubarak Shah,et al.  Toward 3-D Gesture Recognition , 1999, Int. J. Pattern Recognit. Artif. Intell..

[12]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[13]  A. Murat Tekalp,et al.  Tracking multiple objects in the presence of articulated and occluded motion , 2000, Proceedings Workshop on Human Motion.

[14]  Subhasis Chaudhuri,et al.  Simultaneous Tracking of Both Hands by Estimation of Erroneous Observations , 2001, BMVC.

[15]  Laurent Bonnaud,et al.  Multiple occluding objects tracking using a non-redundant boundary-based representation for image sequence interpolation after decoding , 1997, Proceedings of International Conference on Image Processing.

[16]  R. Ivry,et al.  Callosotomy patients exhibit temporal uncoupling during continuous bimanual movements , 2002, Nature Neuroscience.

[17]  Guanrong Chen,et al.  Kalman filtering: with real-time applications (2nd ed.) , 1991 .

[18]  Ioannis Pitas,et al.  Digital Image Processing Algorithms , 1993 .

[19]  Yoichi Sato,et al.  Fast tracking of hands and fingertips in infrared images for augmented desk interface , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[20]  M Husain,et al.  The coordination of bimanual prehension movements in a centrally deafferented patient. , 2000, Brain : a journal of neurology.

[21]  Giancarlo Iannizzotto,et al.  Hand tracking for human-computer interaction with Graylevel VisualGlove: turning back to the simple way , 2001, PUI '01.

[22]  Suat Akyol,et al.  Hands Tracking from Frontal View for Vision-Based Gesture Recognition , 2002, DAGM-Symposium.

[23]  Shaogang Gong,et al.  Modelling facial colour and identity with Gaussian mixtures , 1998, Pattern Recognit..

[24]  Shaogang Gong,et al.  On the semantics of visual behaviour, structured events and trajectories of human action , 2002, Image Vis. Comput..

[25]  Daniel M. Wolpert,et al.  Making smooth moves , 2022 .

[26]  Shaogang Gong,et al.  Segmentation and Tracking Using Color Mixture Models , 1998, ACCV.

[27]  Guanrong Chen,et al.  Kalman Filtering with Real-time Applications , 1987 .

[28]  Stephen J. McKenna,et al.  Hand tracking for behaviour understanding , 2002, Image Vis. Comput..

[29]  Richard A. Brown,et al.  Introduction to random signals and applied kalman filtering (3rd ed , 2012 .

[30]  Atid Shamaie.  Hand tracking and bimanual movement understanding , 2003 .

[31]  E. Vaadia,et al.  Primary motor cortex is involved in bimanual coordination , 1998, Nature.

[32]  R. Ivry,et al.  Moving to Directly Cued Locations Abolishes Spatial Interference During Bimanual Actions , 2001, Psychological science.

[33]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[34]  Shan Lu,et al.  Using multiple cues for hand tracking and model refinement , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[35]  Zoubin Ghahramani,et al.  Perspectives and problems in motor learning , 2001, Trends in Cognitive Sciences.

[36]  Mohammed Yeasin,et al.  Visual modelling and evaluation of surgical skill , 2003, Pattern Analysis & Applications.

[37]  J. Ohya,et al.  Automatic skin-color distribution extraction for face detection and tracking , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.

[38]  Shaogang Gong,et al.  Resolving Visual Uncertainty and Occlusion through Probabilistic Reasoning , 2000, BMVC.

[39]  Paulo R. S. Mendonça,et al.  Model-Based Hand Tracking Using an Unscented Kalman Filter , 2001, BMVC.