Color Harmonization, Deharmonization and Balancing in Augmented Reality

Color schemes play a crucial role in blending virtual objects with the real environment. Good color schemes improve user’s perception, which is of crucial importance for augmented reality. In this paper, we propose a set of novel methods based on the color harmonization methodology to recolor augmented reality content according to the real background. Three different strategies are proposed—harmonic, disharmonic, and balance—that allow for satisfying different needs in different settings depending on the application field. The first approach aims to harmonize the colors of virtual objects to make them consistent with the colors of the real background and reach a more pleasing effect to a human eye. The second approach, instead, can be adopted to generate a set of disharmonious colors with respect to real ones to be associated with the augmented virtual content to improve its distinctiveness from the real background. The third approach balances these goals by achieving a compromise between harmony and good visibility among virtual and real objects. Furthermore, the proposed re-coloring method is applied to three different case studies by adopting the three strategies to meet three different objectives, which are specific for each case study. Several parameters are calculated for each test, such as the covered area, the color distribution, and the set of generated colors. Results confirm the great potential of the proposed approaches to improve the AR visualization in different scenarios.

[1]  Niloy J. Mitra,et al.  Color Harmonization for Videos , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[2]  Stephen DiVerdi,et al.  Palette-based photo recoloring , 2015, ACM Trans. Graph..

[3]  Giuseppe Monno,et al.  Text Readability in Head-Worn Displays: Color and Style Optimization in Video versus Optical See-Through Devices , 2014, IEEE Transactions on Visualization and Computer Graphics.

[4]  Maria Beatriz Carmo,et al.  Improving Symbol Salience in Augmented Reality , 2013, GRAPP/IVAPP.

[5]  Lei Deng,et al.  Colorization Using Quaternion Algebra with Automatic Scribble Generation , 2012, MMM.

[6]  Giuseppe Monno,et al.  Text legibility for projected Augmented Reality on industrial workbenches , 2015, Comput. Ind..

[7]  Tien-Tsin Wong,et al.  Manga colorization , 2006, SIGGRAPH 2006.

[8]  Shi-Min Hu,et al.  Instant Propagation of Sparse Edits on Images and Videos , 2010, Comput. Graph. Forum.

[9]  Sanni Siltanen,et al.  Diminished reality for augmented reality interior design , 2017, The Visual Computer.

[10]  Julie Dorsey,et al.  Understanding and improving the realism of image composites , 2012, ACM Trans. Graph..

[11]  Junxiong Cai,et al.  Temporally Coherent Video Harmonization Using Adversarial Networks , 2018, IEEE Transactions on Image Processing.

[12]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[13]  Pan Hui,et al.  Mobile Augmented Reality Survey: From Where We Are to Where We Go , 2017, IEEE Access.

[14]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[15]  Szymon Rusinkiewicz,et al.  Multiscale shape and detail enhancement from multi-light image collections , 2007, ACM Trans. Graph..

[16]  Yotam I. Gingold,et al.  Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry , 2018, ACM Trans. Graph..

[17]  Frédo Durand,et al.  Two-scale tone management for photographic look , 2006, SIGGRAPH 2006.

[18]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[19]  Fabio Pellacini,et al.  AppProp: all-pairs appearance-space edit propagation , 2008, ACM Trans. Graph..

[20]  Wojciech Matusik,et al.  Multi-scale image harmonization , 2010, SIGGRAPH 2010.

[21]  Yoo-Joo Choi,et al.  Augmented-Reality Survey: from Concept to Application , 2017, KSII Trans. Internet Inf. Syst..

[22]  Katherine A. Hughes Color in Design , 2019, Graphic Design.

[23]  Yotam I. Gingold,et al.  Palette-based image decomposition, harmonization, and color transfer , 2018, ArXiv.

[24]  Kwan H. Lee,et al.  Local color transfer between images using dominant colors , 2013, J. Electronic Imaging.

[25]  J. Edward Swan,et al.  Usability Engineering for Augmented Reality: Employing User-Based Studies to Inform Design , 2008, IEEE Transactions on Visualization and Computer Graphics.

[26]  Edward H. Adelson,et al.  Eurographics Symposium on Rendering 2008 Scribbleboost: Adding Classification to Edge-aware Interpolation of Local Image and Video Adjustments , 2022 .

[27]  Jian Sun,et al.  Drag-and-drop pasting , 2006, SIGGRAPH 2006.

[28]  Forrest Fabian Jesse,et al.  Colour harmonisation for images and videos via two-level graph cut , 2011 .

[29]  Sos S. Agaian,et al.  ARFurniture: Augmented Reality Interior Decoration Style Colorization , 2019 .

[30]  Lianhong Cai,et al.  Affective image adjustment with a single word , 2013, The Visual Computer.

[31]  Pierre Hellier,et al.  Saliency-Guided Consistent Color Harmonization , 2013, CCIW.

[32]  Zhengjun Zha,et al.  Deep Palette-Based Color Decomposition for Image Recoloring with Aesthetic Suggestion , 2020, MMM.

[33]  Sylvain Paris,et al.  Error-Tolerant Image Compositing , 2010, International Journal of Computer Vision.

[34]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[35]  A MOBILE AUGMENTED REALITY APPLICATION FOR INTERACTIVE PAINTING RECOLORING , 2017 .

[36]  Seung Yeon Choo,et al.  Interior Design in Augmented Reality Environment , 2010 .

[37]  Mark A. Livingston,et al.  Issues in Human Factors Evaluations of Augmented Reality Systems , 2013 .

[38]  Chunxia Xiao,et al.  Palette-Based Image Recoloring Using Color Decomposition Optimization , 2017, IEEE Transactions on Image Processing.

[39]  Antoni B. Chan,et al.  Color Orchestra: Ordering Color Palettes for Interpolation and Prediction , 2017, IEEE Transactions on Visualization and Computer Graphics.

[40]  Daniel Cohen-Or,et al.  Color harmonization , 2006, ACM Trans. Graph..

[41]  Federico Manuri,et al.  A Survey on Applications of Augmented Reality , 2016 .

[42]  Tsuhan Chen,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < , 2022 .