Motion capture from body-mounted cameras

Motion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large areas. In this paper, we present the theory and practice of using body-mounted cameras to reconstruct the motion of a subject. Outward-looking cameras are attached to the limbs of the subject, and the joint angles and root pose are estimated through non-linear optimization. The optimization objective function incorporates terms for image matching error and temporal continuity of motion. Structure-from-motion is used to estimate the skeleton structure and to provide initialization for the non-linear optimization procedure. Global motion is estimated and drift is controlled by matching the captured set of videos to reference imagery. We show results in settings where capture would be difficult or impossible with traditional motion capture systems, including walking outside and swinging on monkey bars. The quality of the motion reconstruction is evaluated by comparing our results against motion capture data produced by a commercially available optical system.

[1]  Wojciech Matusik,et al.  Practical motion capture in everyday surroundings , 2007, SIGGRAPH 2007.

[2]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[3]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[4]  Greg Welch,et al.  The HiBall Tracker: high-performance wide-area tracking for virtual and augmented environments , 1999, VRST '99.

[5]  Woltring Hj,et al.  New possibilities for human motion studies by real-time light spot position measurement. , 1974 .

[6]  Hans-Peter Seidel,et al.  Markerless Motion Capture with unsynchronized moving cameras , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Jessica K. Hodgins,et al.  Action capture with accelerometers , 2008, SCA '08.

[8]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[9]  C. Cobelli,et al.  A Markerless Motion Capture System to Study Musculoskeletal Biomechanics: Visual Hull and Simulated Annealing Approach , 2006, Annals of Biomedical Engineering.

[10]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[11]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[12]  Nassir Navab,et al.  Multiple-Activity Human Body Tracking in Unconstrained Environments , 2010, AMDO.

[13]  Yaser Sheikh,et al.  Motion capture from body-mounted cameras , 2011, SIGGRAPH 2011.

[14]  Francis K. H. Quek,et al.  Data-driven motion estimation with low-cost sensors , 2008 .

[15]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[16]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[17]  Greg Welch,et al.  Motion Tracking: No Silver Bullet, but a Respectable Arsenal , 2002, IEEE Computer Graphics and Applications.

[18]  Ramesh Raskar,et al.  Prakash: lighting aware motion capture using photosensing markers and multiplexed illuminators , 2007, ACM Trans. Graph..

[19]  Supun Samarasekera,et al.  Real-time global localization with a pre-built visual landmark database , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Jessica K. Hodgins,et al.  Automatic Joint Parameter Estimation from Magnetic Motion Capture Data , 2023, Graphics Interface.

[21]  Ian D. Reid,et al.  Articulated Body Motion Capture by Stochastic Search , 2005, International Journal of Computer Vision.

[22]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[23]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[24]  Supun Samarasekera,et al.  Ten-fold Improvement in Visual Odometry Using Landmark Matching , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[25]  Supun Samarasekera,et al.  Visual Odometry System Using Multiple Stereo Cameras and Inertial Measurement Unit , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Takeo Kanade,et al.  Shape-from-silhouette of articulated objects and its use for human body kinematics estimation and motion capture , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[28]  Zhiqiang Zhang,et al.  Ubiquitous human body motion capture using micro-sensors , 2009, 2009 IEEE International Conference on Pervasive Computing and Communications.

[29]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[30]  Olivier D. Faugeras,et al.  Automatic calibration and removal of distortion from scenes of structured environments , 1995, Optics & Photonics.

[31]  Hans-Peter Seidel,et al.  Motion reconstruction using sparse accelerometer data , 2011, TOGS.

[32]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[33]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[34]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Melissa R. Plaufcan.  Avatar , 2010 .

[36]  Marc Pollefeys,et al.  Unstructured video-based rendering: interactive exploration of casually captured videos , 2010, SIGGRAPH 2010.

[37]  James R. Bergen,et al.  Visual odometry for ground vehicle applications , 2006, J. Field Robotics.

[38]  Emiliano Gambaretto,et al.  Automatic Generation of a Subject-Specific Model for Accurate Markerless Motion Capture and Biomechanical Applications , 2010, IEEE Transactions on Biomedical Engineering.

[39]  Jessica K. Hodgins,et al.  Human Motion Reconstruction using Wearable Accelerometers , 2010 .