Active voodoo dolls: a vision based input device for nonrigid control

A vision based technique for nonrigid control is presented that can be used for animation and video game applications. The user grasps a soft, squishable object in front of a camera that can be moved and deformed in order to specify motion. Active Blobs, a nonrigid tracking technique is used to recover the position, rotation and nonrigid deformations of the object. The resulting transformations can be applied to a texture mapped mesh, thus allowing the user to control it interactively. Our use of texture mapping hardware in tracking makes the system responsive enough for interactive animation and video game character control.

[1]  Anil K. Jain,et al.  Object Matching Using Deformable Templates , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Alex Pentland,et al.  Shape analysis of brain structures using physical and experimental modes , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Jim Ruppert,et al.  A Delaunay Refinement Algorithm for Quality 2-Dimensional Mesh Generation , 1995, J. Algorithms.

[4]  Jonathan Richard Shewchuk,et al.  Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator , 1996, WACG.

[5]  A F Bobick,et al.  Movement, activity and action: the role of knowledge in the perception of motion. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[6]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[7]  S. Sclaroff.  Modal matching: a method for describing, comparing, and manipulating digital signals , 1995 .

[8]  John Law,et al.  Robust Statistics—The Approach Based on Influence Functions , 1986 .

[9]  P PentlandAlex,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997 .

[10]  Michael Gleicher,et al.  Projective registration with difference decomposition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Timothy F. Cootes,et al.  Trainable method of parametric shape description , 1992, Image Vis. Comput..

[12]  James S. Duncan,et al.  Measurement of non-rigid motion using contour shape descriptors , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  William T. Freeman,et al.  Television control by hand gestures , 1994 .

[15]  David C. Hogg,et al.  Learning Flexible Models from Image Sequences , 1994, ECCV.

[16]  Lance Williams,et al.  Performance-driven facial animation , 1990, SIGGRAPH.

[17]  Lance Williams,et al.  Pyramidal parametrics , 1983, SIGGRAPH.

[18]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[19]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[20]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Alex Pentland,et al.  Recovery of Nonrigid Motion and Structure , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Marco La Cascia,et al.  Head tracking via robust registration in texture map images , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[23]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[24]  Alex Pentland,et al.  Coding, Analysis, Interpretation, and Recognition of Facial Expressions , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Irfan Essa,et al.  Tracking facial motion , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[26]  Azriel Rosenfeld,et al.  Digital Picture Processing , 1976 .

[27]  Richard Szeliski,et al.  Hierarchical spline-based image registration , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Alex Pentland,et al.  Visually Controlled Graphics , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Alex Pentland,et al.  Closed-Form Solutions for Physically Based Shape Modeling and Recognition , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Alex Pentland,et al.  Recovery of non-rigid motion and structure , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Timothy F. Cootes,et al.  Combining point distribution models with shape models based on finite element analysis , 1994, Image Vis. Comput..

[32]  Rangachar Kasturi,et al.  Machine vision , 1995 .

[33]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[34]  Gregory D. Hager,et al.  Real-time tracking of image regions with changes in geometry and illumination , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  A. Pentland,et al.  Physically-based combinations of views: representing rigid and nonrigid motion , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[36]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Michael Isard,et al.  3D position, attitude and shape input using video tracking of hands and lips , 1994, SIGGRAPH.

[38]  Demetri Terzopoulos,et al.  Matching deformable models to images: Direct and iterative solutions , 1980 .

[39]  Alex Pentland,et al.  The ALIVE system: wireless, full-body interaction with autonomous agents , 1997, Multimedia Systems.

[40]  Alex Pentland,et al.  Good vibrations: modal dynamics for graphics and animation , 1989, SIGGRAPH.

[41]  H. Saunders,et al.  Finite element procedures in engineering analysis , 1982 .

[42]  Stan Sclaroff,et al.  Active blobs , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[43]  Dimitris N. Metaxas,et al.  Deformable model-based face shape and motion estimation , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[44]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..