Illusory light: Perceptual appearance control using a projection-induced illusion

Abstract With projection mapping, we can control the appearance of real-world objects by adding illumination. A projector can be used to control light reflected from an object, where the reflected light depends not only on the projection but also on the reflectance and environmental light. Because the resulting colors are affected by the reflectance and environmental light, the presentable color range of a projector is limited. The purpose of this work is to broaden this limited range by focusing on the perceived colors. Although our eyes capture reflected light to perceive the colors of an object, the colors perceived by humans are not always the same as the actual colors, and there are often significant differences between them because of the human visual system. To overcome the limitations of a projector based on human perception, we intentionally generate this difference by inducing a visual illusion, namely, color constancy. In this work, we designed an algorithm to determine the projected colors for presenting the desired colors perceptually by employing a color constancy effect. In addition, we conducted a user study and confirmed that our algorithm can (1) create a misperception regarding the color of illumination, (2) broaden the presentable color range of a projector, and (3) shift the perceptual colors in the desirable direction.

[1]  Greg Welch,et al.  Shader Lamps: Animating Real Objects With Image-Based Illumination , 2001, Rendering Techniques.

[2]  Mark Ashdown,et al.  Robust Content-Dependent Photometric Projector Compensation , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[3]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[4]  Hirokazu Kato,et al.  Perceptual Appearance Control by Projection-Induced Illusion , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[5]  K. Chihara,et al.  PiTaSu: wearable interface for assisting senior citizens with memory problems , 2011 .

[6]  Hirokazu Kato,et al.  Geometrically-Correct Projection-Based Texture Mapping onto a Deformable Object , 2014, IEEE Transactions on Visualization and Computer Graphics.

[7]  Anselm Grundhöfer,et al.  Projection-Based Augmented Reality in Disney Theme Parks , 2012, Computer.

[8]  M. Fioravante,et al.  Visualizing Motional Correlations in Molecular Dynamics using Geometric Deformations , 2013, Comput. Graph. Forum.

[9]  Oliver Bimber,et al.  Real-Time Adaptive Radiometric Compensation , 2006, IEEE Transactions on Visualization and Computer Graphics.

[10]  Shree K. Nayar,et al.  A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments , 2005, CVPR.

[11]  Shree K. Nayar,et al.  A Projection System with Radiometric Compensation for Screen Imperfections , 2003 .

[12]  YangRuigang,et al.  Camera-Based Calibration Techniques for Seamless Multiprojector Displays , 2005 .

[13]  Mark D. Fairchild,et al.  Refinement of the RLAB Color Space , 1996 .

[14]  Sheng-Jyh Wang,et al.  Information Preserving Color Transformation for Protanopia and Deuteranopia , 2007, IEEE Signal Processing Letters.

[15]  Hirokazu Kato,et al.  Perceptual Appearance Control by Projection-Induced Illusion , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[16]  Hiroshi Date,et al.  Clinical application of projection mapping technology for surgical resection of lung metastasis. , 2017, Interactive cardiovascular and thoracic surgery.

[17]  Blair MacIntyre,et al.  RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units , 2014, UIST.

[18]  Kim Halskov,et al.  Projections on museum exhibits: engaging visitors in the museum setting , 2010, OZCHI '10.

[19]  Hirokazu Kato,et al.  Appearance control by projector camera feedback for visually impaired , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[20]  Adam Reeves,et al.  The dependence of color constancy and brightness constancy on saturation , 2018 .

[21]  Shin'ya Nishida,et al.  Perceptually Based Adaptive Motion Retargeting to Animate Real Objects by Light Projection , 2019, IEEE Transactions on Visualization and Computer Graphics.

[22]  Hirokazu Kato,et al.  Light Projection-Induced Illusion for Controlling Object Color , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[23]  Ramesh Raskar,et al.  Modern approaches to augmented reality: introduction to current approaches , 2006, SIGGRAPH Courses.

[24]  Christoph Resch,et al.  Sticky Projections-A Model-Based Approach to Interactive Shader Lamps Tracking , 2016, IEEE Transactions on Visualization and Computer Graphics.

[25]  D. Paulus,et al.  Mobile services supporting color vision deficiency , 2012, 2012 13th International Conference on Optimization of Electrical and Electronic Equipment (OPTIM).

[26]  Christian Sandor,et al.  Robust Reflectance Estimation for Projection-Based Appearance Control in a Dynamic Light Environment , 2019, IEEE Transactions on Visualization and Computer Graphics.

[27]  Haider Butt,et al.  Contact Lenses for Color Blindness , 2018, Advanced healthcare materials.

[28]  Toshiyuki Amano,et al.  Appearance control for human material perception manipulation , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[29]  Hirokazu Kato,et al.  Imperceptible On-Screen Markers for Mobile Interaction on Public Large Displays , 2017, IEICE Trans. Inf. Syst..

[30]  J. Frisby.  Seeing: Illusion, Brain and Mind , 1979 .

[31]  Anselm Grundhöfer,et al.  Makeup Lamps: Live Augmentation of Human Faces via Projection , 2017, Comput. Graph. Forum.

[32]  Hirokazu Kato,et al.  Appearance Control Using Projection with Model Predictive Control , 2010, 2010 20th International Conference on Pattern Recognition.

[33]  Holger Regenbrecht,et al.  ChromaGlasses: Computational Glasses for Compensating Colour Blindness , 2018, CHI.

[34]  Shin'ya Nishida,et al.  Deformation Lamps: A Projection Technique to Make Static Objects Perceptually Dynamic , 2015, TAP.

[35]  Koichi Hashimoto,et al.  Sticky projection mapping: 450-fps tracking projection onto a moving planar surface , 2015, SIGGRAPH Asia Emerging Technologies.

[36]  D. L. Macadam.  Visual Sensitivities to Color Differences in Daylight , 1942 .

[37]  Djemel Ziou,et al.  Color constancy for visual compensation of projector displayed image , 2014, Displays.

[38]  Anselm Grundhöfer,et al.  Seamless Multi-Projection Revisited , 2018, IEEE Transactions on Visualization and Computer Graphics.

[39]  William R. Mathew,et al.  Color as a Science , 2005 .

[40]  D Purves,et al.  An Empirical Explanation of the Cornsweet Effect , 1999, The Journal of Neuroscience.

[41]  Ichiro Kuriki,et al.  A Novel Method of Color Appearance Simulation Using Achromatic Point Locus With Lightness Dependence , 2018, i-Perception.

[42]  Masatoshi Ishikawa,et al.  Dynamic projection mapping onto a deformable object with occlusion based on high-speed tracking of dot marker array , 2015, VRST.

[43]  Shree K. Nayar,et al.  Making one object look like another: controlling appearance using a projector-camera system , 2004, CVPR 2004.

[44]  B. Pinna,et al.  Surface color from boundaries: a new ‘watercolor’ illusion , 2001, Vision Research.

[45]  Takuji Narumi,et al.  Swinging 3D lamps: a projection technique to convert a static 2D picture to 3D using wiggle stereoscopy , 2017, SIGGRAPH Posters.

[46]  Takahiro Kawabe,et al.  Spatially augmented depth and transparency in paper materials , 2018, SIGGRAPH ASIA Emerging Technologies.