Spatial augmented reality based high accuracy human face projection

This paper discusses the imaging principles and the technical difficulties of spatial augmented reality based human face projection. A novel geometry correction method is proposed to realize fast, high-accuracy face model projection. Using a depth camera to reconstruct the projected object, the relative position from the rendered model to the projector can be accessed and the initial projection image is generated. Then the projected image is distorted by using Bezier interpolation to guarantee that the projected texture matches with the object surface. The proposed method is under a simple process flow and can achieve high perception registration of virtual and real object. In addition, this method has a good performance in the condition that the reconstructed model is not exactly same with the rendered virtual model which extends its application area in the spatial augmented reality based human face projection.

[1]  Gordon Wetzstein,et al.  Consistent illumination within optical see-through augmented environments , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[2]  Markus H. Gross,et al.  Adaptive Instant Displays: Continuously Calibrated Projections Using Per‐Pixel Light Control , 2005, Comput. Graph. Forum.

[3]  Aditi Majumder,et al.  Markerless View-Independent Registration of Multiple Distorted Projectors on Extruded Surfaces Using an Uncalibrated Camera , 2009, IEEE Transactions on Visualization and Computer Graphics.

[4]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[5]  Camillo J. Taylor.  Surface reconstruction from feature based stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Li Renju.  Data registration in 3-D scanning systems , 2004 .

[7]  Ruigang Yang.  Automatic Projector Display Surface Estimation Using Every-Day Imagery , 2010 .

[8]  Henry Fuchs,et al.  Real-Time Projector Tracking on Complex Geometry Using Ordinary Imagery , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Aditi Majumder,et al.  Autocalibrating Tiled Projectors on Piecewise Smooth Vertically Extruded Surfaces , 2011, IEEE Transactions on Visualization and Computer Graphics.

[10]  W. Brent Seales,et al.  Multi-projector displays using camera-based registration , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[11]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[12]  Michael Harville,et al.  Practical Methods for Geometric and Photometric Correction of Tiled Projector , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[13]  Gabriel Taubin,et al.  Simple, Accurate, and Robust Projector-Camera Calibration , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[14]  Greg Welch,et al.  Shader Lamps: Animating Real Objects With Image-Based Illumination , 2001, Rendering Techniques.

[15]  Zhou Lai.  Research of Multi-View Registration and Integration on Measured Point Cloud Data , 2003 .

[16]  Oliver Bimber,et al.  Passive-Active Geometric Calibration for View-Dependent Projections onto Arbitrary Surfaces , 2007, J. Virtual Real. Broadcast..

[17]  Greg Welch,et al.  A Distributed Cooperative Framework for Continuous Multi-Projector Pose Estimation , 2009, 2009 IEEE Virtual Reality Conference.

[18]  Ivan Poupyrev,et al.  SideBySide: ad-hoc multi-user interaction with handheld projectors , 2011, UIST.

[19]  Ruigang Yang,et al.  Multi-projector display with continuous self-calibration , 2008, PROCAMS '08.

[20]  Markus H. Gross,et al.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[21]  Ruigang Yang,et al.  Camera-based calibration techniques for seamless multiprojector displays , 2005, IEEE Transactions on Visualization and Computer Graphics.

[22]  Aditi Majumder,et al.  Auto-calibration of cylindrical multi-projector systems , 2010, 2010 IEEE Virtual Reality Conference (VR).

[23]  Ramesh Raskar,et al.  Dynamic shader lamps : painting on movable objects , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[24]  W. Brent Seales,et al.  A practical and flexible tiled display system , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[25]  Dongdong Weng,et al.  iSarProjection: A KinectFusion Based Handheld Dynamic Spatial Augmented Reality System , 2013, 2013 International Conference on Computer-Aided Design and Computer Graphics.

[26]  Daniel G. Aliaga.  Digital inspection: an interactive stage for viewing surface details , 2008, I3D '08.

[27]  Aditi Majumder,et al.  Registration Techniques for Using Imperfect and Par tially Calibrated Devices in Planar Multi-Projector Displays , 2007, IEEE Transactions on Visualization and Computer Graphics.

[28]  Joaquim Salvi,et al.  Overview of coded light projection techniques for automatic 3D profiling , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[29]  Ramesh Raskar,et al.  Quadric Transfer for Immersive Curved Screen Displays , 2004, Comput. Graph. Forum.

[30]  K. Niski,et al.  Registration Techniques for Using Imperfect and Partially Calibrated Devices in Planar Multi-Projector Displays , 2007, IEEE Transactions on Visualization and Computer Graphics.

[31]  Daniel G. Aliaga,et al.  Photogeometric structured light: A self-calibrating and multi-viewpoint framework for accurate 3D modeling , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Wei Sun,et al.  Calibrating multi-projector cylindrically curved displays for "wallpaper" projection , 2008, PROCAMS '08.