Dynamic 2D/3D registration for the Kinect

Image and geometry registration algorithms are essential components of many computer graphics and computer vision systems. With recent technological advances in RGB-D sensors, robust algorithms that combine 2D image and 3D geometry registration have become an active area of research. This course introduces the basics of 2D/3D registration algorithms and provides theoretical explanations and practical tools for designing computer vision and computer graphics systems based on RGB-D devices such as the Microsoft Kinect or Asus Xtion Live. To illustrate the theory and demonstrate practical relevance, the course briefly discusses three applications: rigid scanning, non-rigid modeling, and real-time face tracking.

[1]  M. Pauly,et al.  Embedded deformation for shape manipulation , 2007, SIGGRAPH 2007.

[2]  Gérard G. Medioni,et al.  Object modeling by registration of multiple range images , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[3]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[4]  Luc Van Gool,et al.  Accurate and robust registration for in-hand modeling , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Robert B. Fisher,et al.  Estimating 3-D rigid body transformations: a comparison of four major algorithms , 1997, Machine Vision and Applications.

[6]  Mark Pauly,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[7]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[8]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[9]  Antonis A. Argyros,et al.  Tracking the articulated motion of two strongly interacting hands , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Luc Van Gool,et al.  In-hand scanning with online loop closure , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[11]  Markus H. Gross,et al.  PriMo: coupled prisms for intuitive surface modeling , 2006, SGP '06.

[12]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Mark Pauly,et al.  Shape‐Up: Shaping Discrete Geometry with Projections , 2012, Comput. Graph. Forum.

[14]  Jinxiang Chai,et al.  Accurate realtime full-body motion capture using a single depth camera , 2012, ACM Trans. Graph..

[15]  Timothy F. Cootes,et al.  Statistical models of appearance for computer vision , 1999 .

[16]  Ligang Liu,et al.  Scanning 3D Full Human Bodies Using Kinects , 2012, IEEE Transactions on Visualization and Computer Graphics.

[17]  Kim L. Boyer,et al.  Performance evaluation of a class of M-estimators for surface parameter estimation in noisy range data , 1993, IEEE Trans. Robotics Autom..

[18]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[19]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[20]  Berthold K. P. Horn,et al.  "Determining optical flow": A Retrospective , 1993, Artif. Intell..

[21]  Leonidas J. Guibas,et al.  Robust single-view geometry and motion reconstruction , 2009, ACM Trans. Graph..

[22]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[23]  Yangang Wang,et al.  Online modeling for realtime facial animation , 2013, ACM Trans. Graph..