Dynamic Projection Mapping on Multiple Non-rigid Moving Objects for Stage Performance Applications

In this paper we have proposed and developed a projection mapping system that can project 3D images on multiple moving objects that change their positions and shapes. Although projection mapping has become a new type of image projection method, so far most of the projection mappings have focused on the projection of 3D images on objects such as buildings that stay static without changing their shapes. If projection mapping on non-rigid moving objects such as human performers is realized, application area of projection mapping will become far wider.

[1]  Ramesh Raskar,et al.  Modern approaches to augmented reality: introduction to current approaches , 2006, SIGGRAPH Courses.

[2]  Ryohei Nakatsu,et al.  Projection Mapping Celebrating RIMPA 400th Anniversary , 2015, 2015 International Conference on Culture and Computing (Culture Computing).

[3]  A. L. Sadler,et al.  Japanese Plays: Classic Noh, Kyogen and Kabuki Works , 2010 .

[4]  Anthony Rowe Designing for engagement in mixed reality experiences that combine projection mapping and camera-based interaction , 2014, Digit. Creativity.

[5]  Igor Guskov,et al.  Efficient tracking of regular patterns on non-rigid geometry , 2002, Object recognition supported by user interaction for service robots.

[6]  Gordon Wetzstein,et al.  The Visual Computing of Projector‐Camera Systems , 2008, SIGGRAPH '08.

[7]  Hideaki Uchiyama,et al.  Object Detection and Pose Tracking for Augmented Reality: Recent Approaches , 2012 .

[8]  Masatoshi Ishikawa,et al.  Dynamic Projection Mapping onto Deforming Non-Rigid Surface Using Deformable Dot Cluster Marker , 2017, IEEE Transactions on Visualization and Computer Graphics.

[9]  Ryohei Nakatsu,et al.  Creation of Media Art Utilizing Fluid Dynamics , 2017, 2017 International Conference on Culture and Computing (Culture and Computing).

[10]  Vincent Gay-Bellile,et al.  Deformable Surface Augmentation in Spite of Self-Occlusions , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[11]  Eyal Ofek,et al.  IllumiRoom: peripheral projected illusions for interactive experiences , 2013, SIGGRAPH '13.

[12]  Pascal Fua,et al.  Object-centered surface reconstruction: Combining multi-image stereo and shading , 1995, International Journal of Computer Vision.

[13]  Greg Welch,et al.  Shader Lamps: Animating Real Objects With Image-Based Illumination , 2001, Rendering Techniques.

[14]  Naohiro Ishii,et al.  Development of Projection Mapping with Utility of Digital Signage , 2014, 2014 IIAI 3rd International Conference on Advanced Applied Informatics.

[15]  Justus Thies,et al.  Real-time pixel luminance optimization for dynamic multi-projection mapping , 2015, ACM Trans. Graph..

[16]  Anselm Grundhöfer,et al.  Projection-Based Augmented Reality in Disney Theme Parks , 2012, Computer.

[17]  Jong-Il Park,et al.  Invisible marker based augmented reality system , 2005, Visual Communications and Image Processing.