Dynamics based 3D skeletal hand tracking

Natural human computer interaction motivates hand tracking research, preferably without requiring the user to wear special hardware or markers. Ideally, a hand tracking solution would provide not only points of interest, but the full state of an entire hand. [Oikonomidis et al. 2011] demonstrated a particle swarm optimization that tracked a 3D skeletal hand model from a single depth camera, albeit using significant computing resources. In contrast, we track the hand from a single depth camera using an efficient physical simulation, which incrementally updates a model's fit and explores alternative candidate poses based on a variety of heuristics. Our approach enables real-time, robust 3D skeletal tracking of a user's hand, while utilizing a single x86 CPU core for processing.

[1]  Anthony A. Maciejewski,et al.  Computational modeling for the computer animation of legged figures , 1985, SIGGRAPH.

[2]  John C. Platt,et al.  Elastically deformable models , 1987, SIGGRAPH.

[3]  Jaron Lanier,et al.  A hand gesture interface device , 1987, CHI 1987.

[4]  Jane Wilhelms,et al.  Collision Detection and Response for Computer Animation , 1988, SIGGRAPH.

[5]  Thomas S. Huang,et al.  Vision based hand modeling and tracking for virtual teleconferencing and telecollaboration , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  John Canny,et al.  Impulse-Based Dynamic Simulation , 1995 .

[7]  Andrew Witkin,et al.  Physically Based Modeling: Principles and Practice , 1997 .

[8]  Ying Wu,et al.  Modeling the constraints of human hand motion , 2000, Proceedings Workshop on Human Motion.

[9]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[10]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[12]  S. Malik.  Real-time Hand Tracking and Finger Tracking for Interaction CSC 2503 F Project Report , 2003 .

[13]  Hans-Peter Seidel,et al.  Construction and animation of anatomically based human hand models , 2003, SCA '03.

[14]  Taku Komura,et al.  Motion editing with data glove , 2004, ACE '04.

[15]  Luc Van Gool,et al.  Smart particle filtering for 3D hand tracking , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[16]  Erin S. Catto.  Iterative Dynamics with Temporal Coherence , 2005 .

[17]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[18]  Grant Palmer,et al.  Physics for Game Programmers , 2005 .

[19]  Kenny Erleben,et al.  Physics Based Animation (Graphics Series) , 2005 .

[20]  Ben Weiss,et al.  Fast median and bilateral filtering , 2006, ACM Trans. Graph..

[21]  John P. Lewis,et al.  Human hand modeling from surface anatomy , 2006, I3D '06.

[22]  S. Mitra,et al.  Gesture Recognition: A Survey , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[24]  Jovan Popović,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH 2009.

[25]  C. Karen Liu.  Dextrous manipulation from a grasping pose , 2009, SIGGRAPH 2009.

[26]  A. Buryanov,et al.  Proportions of Hand Segments , 2010 .

[27]  Fotis Sotiropoulos,et al.  Slice WIM: a multi-surface, multi-touch interface for overview+detail exploration of volume datasets in virtual reality , 2011, SI3D.

[28]  Miguel A. Otaduy,et al.  Interactive simulation of a deformable hand for haptic rendering , 2011, 2011 IEEE World Haptics Conference.

[29]  Joseph J. LaViola,et al.  3D spatial interaction: applications for art, design, and science , 2011, SIGGRAPH '11.

[30]  Rod McCall,et al.  Lightweight palm and finger tracking for real-time 3D gesture control , 2011, 2011 IEEE Virtual Reality Conference.

[31]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[32]  Sylvain Paris,et al.  6D hands: markerless hand-tracking for computer aided design , 2011, UIST.

[33]  Andrew Wilson,et al.  MirageTable: freehand interaction on a projected augmented reality tabletop , 2012, CHI.

[34]  Patrick Olivier,et al.  Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor , 2012, UIST.

[35]  Sebastian Thrun,et al.  Real-Time Human Pose Tracking from Range Data , 2012, ECCV.

[36]  David Kim,et al.  HoloDesk: direct 3d interactions with a situated see-through display , 2012, CHI.

[37]  Ming C. Lin,et al.  Tabletop Ensemble: touch-enabled virtual percussion instruments , 2012, I3D '12.

[38]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.