Interactive Design of Gallery Walls via Mixed Reality

We present a novel interactive design tool that allows users to create and visualize gallery walls via a mixed reality device. To use our tool, a user selects a wall to decorate and chooses a focal art item. Our tool then helps the user complete their design by optionally recommending additional art items or automatically completing both the selection and placement of additional art items. Our tool holistically considers common design criteria such as alignment, color, and style compatibility in the synthesis of a gallery wall. Through a mixed reality device, such as a Magic Leap One headset, the user can instantly visualize the gallery wall design in situ and can interactively modify the design in collaboration with our tool’s suggestion engine. We describe the suggestion engine and its adaptability to users with different design goals. We also evaluate our mixedreality-based tool for creating gallery wall designs and compare it with a 2D interface, providing insights for devising mixed reality interior design applications.

[1]  Demetri Terzopoulos,et al.  The Clutterpalette: An Interactive Tool for Detailing Indoor Scenes , 2016, IEEE Transactions on Visualization and Computer Graphics.

[2]  Ian Mackenzie,et al.  Fitts' law as a performance model in human-computer interaction , 1992 .

[3]  Jan P. Allebach,et al.  Recommendation system for automatic design of magazine covers , 2013, IUI '13.

[4]  Shi-Min Hu,et al.  Magic decorator , 2015, ACM Trans. Graph..

[5]  Zhi-Hua Zhou,et al.  Learning to Generate Posters of Scientific Papers , 2016, AAAI.

[6]  Maneesh Agrawala,et al.  Interactive furniture layout using interior design guidelines , 2011, SIGGRAPH 2011.

[7]  Jun Xiao,et al.  Mixed-initiative photo collage authoring , 2008, ACM Multimedia.

[8]  Chenfanfu Jiang,et al.  Fast and Scalable Position-Based Layout Synthesis , 2018, IEEE Transactions on Visualization and Computer Graphics.

[9]  David Lindlbauer,et al.  Remixed Reality: Manipulating Space and Time in Augmented Reality , 2018, CHI.

[10]  Angel X. Chang,et al.  Deep convolutional priors for indoor scene synthesis , 2018, ACM Trans. Graph..

[11]  Aaron Hertzmann,et al.  Learning Layouts for Single-PageGraphic Designs , 2014, IEEE Transactions on Visualization and Computer Graphics.

[12]  Yi Ji,et al.  H Space: Interactive Augmented Reality Art , 2020, Tangible and Embedded Interaction.

[13]  Eyal Ofek,et al.  SnapToReality: Aligning Augmented Reality to the Real World , 2016, CHI.

[14]  Shi-Min Hu,et al.  Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models , 2013, ACM Trans. Graph..

[15]  John Locke,et al.  Project discover: an application of generative design for architectural space planning , 2017 .

[16]  Aaron Hertzmann,et al.  DesignScape: Design with Interactive Layout Suggestions , 2015, CHI.

[17]  Pat Hanrahan,et al.  Characterizing structural relationships in scenes using graph kernels , 2011, ACM Trans. Graph..

[18]  Paul Coulton,et al.  User Engagement Continuum: Art Engagement and Exploration with Augmented Reality , 2018 .

[19]  Tovi Grossman,et al.  DreamRooms: Prototyping Rooms in Collaboration with a Generative Process , 2019, Graphics Interface.

[20]  Takeo Igarashi,et al.  GACA: Group-Aware Command-based Arrangement of Graphic Elements , 2015, CHI.

[21]  Takeo Igarashi,et al.  Global beautification of layouts with interactive ambiguity resolution , 2014, UIST.

[22]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Pat Hanrahan,et al.  Context-based search for 3D models , 2010, ACM Trans. Graph..

[24]  Timothy Jung,et al.  Google Glass Augmented Reality: Generic Learning Outcomes for Art Galleries , 2015, ENTER.

[25]  Gang Ren,et al.  SceneCtrl: Mixed Reality Enhancement via Efficient Scene Editing , 2017, UIST.

[26]  Rynson W. H. Lau,et al.  Directing user attention via visual flow on web designs , 2016, ACM Trans. Graph..

[27]  Pat Hanrahan,et al.  Example-based synthesis of 3D object arrangements , 2012, ACM Trans. Graph..

[28]  Michael F. Cohen,et al.  Emptying, refurnishing, and relighting indoor spaces , 2016, ACM Trans. Graph..

[29]  Paul Coulton,et al.  User curated augmented reality art exhibitions , 2014, NordiCHI.

[30]  Chi-Keung Tang,et al.  Make it home: automatic optimization of furniture arrangement , 2011, ACM Trans. Graph..

[31]  Dario tom Dieck,et al.  Enhancing art gallery visitors’ learning experience using wearable augmented reality: generic learning outcomes perspective , 2018 .

[32]  Pengfei Xu,et al.  SweepCanvas: Sketch-based 3D Prototyping on an RGB-D Image , 2017, UIST.

[33]  Paul Coulton,et al.  Designing mobile augmented reality art applications: addressing the views of the galleries and the artists , 2014, MindTrek.

[34]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[35]  Konstantinos N. Plataniotis,et al.  Distance measures for color image retrieval , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[36]  Jinwook Seo,et al.  Wall-based Space Manipulation Technique for Efficient Placement of Distant Objects in Augmented Reality , 2018, UIST.