Visualization/AR/VR/MR Systems

As is shown in previous chapters, active lighting is a powerful tool that satisfies various demands. Succeeding chapters provide several application examples of active lighting. Projecting active-lighting images onto real objects augments their appearance. Such projection mapping is now widely used in various fields, including the entertainment industry. Actively lighting multispectral lights onto oil paintings enables novel art modifications. Multispectral light can deceive human eyes because of our RGB limitations, because the light is represented using continuous wavelengths. Therefore, multispectral light can augment the visualization of real objects. Active lighting also enables us to capture the depth of the human body. Thus, a human pose can be estimated from the analysis of its depth. To represent a digital character, capturing actual human motion enables realistic duplication. Estimating human body positions is necessary for representing the motion of digital character and user positions in augmented-, virtual-, and mixed-reality systems.

[1]  Tomás Pajdla,et al.  Multi-view reconstruction preserving weakly-supported surfaces , 2011, CVPR 2011.

[2]  Masashi Nishiyama,et al.  Virtual Photometric Environment using Projector , 2004 .

[3]  Kosuke Sato,et al.  Shape optimization of fabricated transparent layer for pixel density uniformalization in non-planar rear projection , 2017, SCF.

[4]  Kosuke Sato,et al.  Non-Contact Thermo-Visual Augmentation by IR-RGB Projection , 2019, IEEE Transactions on Visualization and Computer Graphics.

[5]  Zicheng Liu,et al.  Tensor-Based Human Body Modeling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[7]  Peter V. Gehler,et al.  Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image , 2016, ECCV.

[8]  Bernt Schiele,et al.  Building statistical shape spaces for 3D human modeling , 2015, Pattern Recognit..

[9]  Kosuke Sato,et al.  SoftAR: Visually Manipulating Haptic Softness Perception in Spatial Augmented Reality , 2015, IEEE Transactions on Visualization and Computer Graphics.

[10]  Kosuke Sato,et al.  Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture , 2015, Virtual Reality.

[11]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[12]  Ramesh Raskar,et al.  Merging Fossil Specimens with Computer-Generated Information , 2002, Computer.

[13]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Jinlong Yang,et al.  Analyzing Clothing Layer Deformation Statistics of 3D Human Motions , 2018, ECCV.

[15]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Seiji Inokuchi,et al.  3-D tele-direction interface using video projector , 2003, SIGGRAPH '03.

[17]  Wojciech Matusik,et al.  Articulated mesh animation from multi-view silhouettes , 2008, ACM Trans. Graph..

[18]  Kosuke Sato,et al.  Stereoscopic Capture in Projection Mapping , 2018, IEEE Access.

[19]  Toshiyuki Amano.  Shading illusion: A novel way for 3-D representation on paper media , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[21]  Michael J. Black,et al.  Detailed Full-Body Reconstructions of Moving People from Monocular RGB-D Sequences , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Katsushi Ikeuchi,et al.  Eigen-Texture Method: Appearance Compression and Synthesis Based on a 3D Model , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Leonidas J. Guibas,et al.  Robust single-view geometry and motion reconstruction , 2009, SIGGRAPH 2009.

[24]  Kosuke Sato,et al.  Pseudo-Shape Sensation by Stereoscopic Projection Mapping , 2018, IEEE Access.

[25]  Shree K. Nayar,et al.  Multispectral Imaging Using Multiplexed Illumination , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[26]  Qionghai Dai,et al.  DoubleFusion: Real-Time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Hao Li,et al.  Global Correspondence Optimization for Non‐Rigid Registration of Depth Scans , 2008, Comput. Graph. Forum.

[28]  Darren Leigh,et al.  RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors , 2004, SIGGRAPH 2004.

[29]  Katsushi Ikeuchi,et al.  Representing a Partially Observed Non-Rigid 3D Human Using Eigen-Texture and Eigen-Deformation , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[30]  Anselm Grundhöfer,et al.  Projection-Based Augmented Reality in Disney Theme Parks , 2012, Computer.

[31]  Marc Alexa,et al.  Laplacian mesh optimization , 2006, GRAPHITE '06.

[32]  Yuta Nakashima,et al.  Human Shape Reconstruction with Loose Clothes from Partially Observed Data by Pose Specific Deformation , 2019, PSIVT.

[33]  Katsushi Ikeuchi,et al.  Realtime Novel View Synthesis with Eigen-Texture Regression , 2017, BMVC.

[34]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[35]  Shin'ya Nishida,et al.  Deformation Lamps: A Projection Technique to Make Static Objects Perceptually Dynamic , 2015, TAP.

[36]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH 2005.

[37]  Magnus Norgren,et al.  Liquid Exfoliation of Layered Materials in Water for Inkjet Printing , 2016 .

[38]  Kosuke Sato,et al.  FleXeen: Visually Manipulating Perceived Fabric Bending Stiffness in Spatial Augmented Reality , 2020, IEEE Transactions on Visualization and Computer Graphics.

[39]  Andrew W. Fitzgibbon,et al.  Real-time non-rigid reconstruction using an RGB-D camera , 2014, ACM Trans. Graph..

[40]  Sami Romdhani,et al.  Optimal Step Nonrigid ICP Algorithms for Surface Registration , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Kosuke Sato,et al.  Fabricating Diminishable Visual Markers for Geometric Registration in Projection Mapping , 2018, IEEE Transactions on Visualization and Computer Graphics.

[42]  Masashi Baba,et al.  Optimization of LED Illumination for Generating Metamerism , 2016 .

[43]  John J. Leonard,et al.  Kintinuous: Spatially Extended KinectFusion , 2012, AAAI 2012.