Linearized Motion Estimation for Articulated Planes

In this paper, we describe the explicit application of articulation constraints for estimating the motion of a system of articulated planes. We relate articulations to the relative homography between planes and show that these articulations translate into linearized equality constraints on a linear least-squares system, which can be solved efficiently using a Karush-Kuhn-Tucker system. The articulation constraints can be applied for both gradient-based and feature-based motion estimation algorithms and to illustrate this, we describe a gradient-based motion estimation algorithm for an affine camera and a feature-based motion estimation algorithm for a projective camera that explicitly enforces articulation constraints. We show that explicit application of articulation constraints leads to numerically stable estimates of motion. The simultaneous computation of motion estimates for all of the articulated planes in a scene allows us to handle scene areas where there is limited texture information and areas that leave the field of view. Our results demonstrate the wide applicability of the algorithm in a variety of challenging real-world cases such as human body tracking, motion estimation of rigid, piecewise planar scenes, and motion estimation of triangulated meshes.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[3]  Katsushi Ikeuchi,et al.  Deformable surfaces: a free-form shape representation , 1991, Optics & Photonics.

[4]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[5]  Larry S. Davis,et al.  Learned Models for Estimation of Rigid and Articulated Human Motion from Stationary or Moving Camera , 2004, International Journal of Computer Vision.

[6]  Hsi-Jian Lee,et al.  Determination of 3D human body postures from a single view , 1985, Comput. Vis. Graph. Image Process..

[7]  Ioannis A. Kakadiaris,et al.  Model-Based Estimation of 3D Human Motion , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Luc Van Gool,et al.  Grouping and invariants using planar homologies , 1995 .

[9]  Laurent D. Cohen,et al.  Finite-Element Methods for Active Contour Models and Balloons for 2-D and 3-D Images , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[11]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[12]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[13]  Takeo Kanade,et al.  On the sustained tracking of human motion , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[14]  Michael J. Black,et al.  Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , 1995, Proceedings of IEEE International Conference on Computer Vision.

[15]  Michael J. Black,et al.  Cardboard people: a parameterized model of articulated image motion , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[16]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[17]  Hans-Hellmut Nagel,et al.  Displacement vectors derived from second-order intensity variations in image sequences , 1983, Comput. Vis. Graph. Image Process..

[18]  Jitendra Malik,et al.  Twist Based Acquisition and Tracking of Animal and Human Kinematics , 2004, International Journal of Computer Vision.

[19]  Alex Pentland,et al.  Automatic extraction of deformable part models , 1990, International Journal of Computer Vision.

[20]  A. Verri,et al.  A computational approach to motion perception , 1988, Biological Cybernetics.

[21]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[22]  David J. Fleet,et al.  Optical Flow Estimation , 2006, Handbook of Mathematical Models in Computer Vision.

[23]  Vincent Gay-Bellile,et al.  Feature-Driven Direct Non-Rigid Image Registration , 2007, BMVC.

[24]  Michael J. Black,et al.  Skin and bones: multi-layer, locally affine, optical flow and regularization with transparency , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[26]  Ankur Agarwal,et al.  Tracking Articulated Motion Using a Mixture of Autoregressive Models , 2004, ECCV.

[27]  Philip E. Gill,et al.  Practical optimization , 1981 .

[28]  Stan Sclaroff,et al.  Active blobs , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[29]  Reinhard Koch,et al.  3D Structure from Multiple Images of Large-Scale Environments , 1998, Lecture Notes in Computer Science.

[30]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[31]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  H. Opower Multiple view geometry in computer vision , 2002 .

[33]  Masanobu Yamamoto,et al.  Scene constraints-aided tracking of human body , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[34]  James J. Little,et al.  Parallel Optical Flow Using Local Voting , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[35]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Alessio Del Bue,et al.  Non-rigid 3D Factorization for Projective Reconstruction , 2005, BMVC.

[37]  Demetri Terzopoulos,et al.  A finite element model for 3D shape reconstruction and nonrigid motion tracking , 1993, 1993 (4th) International Conference on Computer Vision.

[38]  Timothy F. Cootes,et al.  Groupwise Diffeomorphic Non-rigid Registration for Automatic Model Building , 2004, ECCV.

[39]  Larry S. Davis,et al.  W4S: A real-time system detecting and tracking people in 2 1/2D , 1998, ECCV.

[40]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Ming-Hsuan Yang,et al.  A direct method for modeling non-rigid motion with thin plate spline , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[43]  Lihi Zelnik-Manor,et al.  Multiview Constraints on Homographies , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  James M. Rehg,et al.  A multiple hypothesis approach to figure tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[45]  Sidharth Bhatia,et al.  Tracking loose-limbed people , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[46]  Pascal Fua,et al.  Local deformation models for monocular 3D shape recovery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  William T. Freeman,et al.  Bayesian Reconstruction of 3D Human Motion from Single-Camera Video , 1999, NIPS.

[48]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[49]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[50]  Harpreet S. Sawhney,et al.  Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding , 1995, Proceedings of IEEE International Conference on Computer Vision.

[51]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[52]  Takeo Kanade,et al.  A subspace approach to layer extraction , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[53]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[54]  Richard Szeliski,et al.  An integrated Bayesian approach to layer extraction from image sequences , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[55]  Yair Weiss,et al.  Smoothness in layers: Motion segmentation using nonparametric mixture estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[56]  Aaron Hertzmann,et al.  Learning Non-Rigid 3D Shape from 2D Motion , 2003, NIPS.

[57]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[58]  Florent Brunet,et al.  Feature-Driven Direct Non-Rigid Image Registration , 2010, International Journal of Computer Vision.

[59]  Mathieu Perriollat,et al.  Image Registration by Combining Thin-Plate Splines with a 3D Morphable Model , 2006, 2006 International Conference on Image Processing.

[60]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[61]  Björn Johansson View synthesis and 3D reconstruction of piecewise planar scenes using intersection lines between the planes , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[62]  Thomas S. Huang,et al.  Model-based human body tracking , 2002, Object recognition supported by user interaction for service robots.

[63]  Michael J. Black,et al.  Measure Locally, Reason Globally: Occlusion-sensitive Articulated Pose Estimation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[64]  Adrien Bartoli,et al.  Direct Estimation of Non-Rigid Registration , 2004, BMVC.

[65]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[66]  Radu Horaud,et al.  Rigid and articulated motion seen with an uncalibrated stereo rig , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[67]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[68]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[69]  Trevor Darrell,et al.  Constraining human body tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.