ClayVision: the (elastic) image of the city

In this paper we describe ClayVision, a new quasi-immersive urban navigation system that rethinks the design conventions of existing Augmented Reality (AR) applications, by aggressively incorporating knowledge from non-Computer Science fields - namely Information Design and Urban Planning. Instead of the prevailing approach of pasting "information bubbles" onto the existing urban scenery, ClayVision communicates through real-time 3D transformations of city elements. In other words, the system dynamically probes and reassembles the city into a better-designed copy of the original, that is both easier to navigate and tailored to suit the user's needs and preferences. We provide extensive discussions that cover the technical details of the system, the types of city-morphing operations that can be effectively applied, and what people's experiences will be in the newly "elastic" city.

[1]  Steven K. Feiner,et al.  Windows on the world: 2D windows for 3D augmented reality , 1993, UIST '93.

[2]  David Ingram,et al.  Augmented reality in a wide area sentient environment , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[3]  Dieter Schmalstieg,et al.  Multiple target detection and tracking with guaranteed framerates on mobile phones , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[4]  Jan-Michael Frahm,et al.  Towards Urban 3D Reconstruction from Video , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[5]  P. Fayers,et al.  The Visual Display of Quantitative Information , 1990 .

[6]  Bernd Girod,et al.  Outdoors augmented reality on mobile phone using loxel-based visual feature organization , 2008, MIR '08.

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  Richard Szeliski,et al.  Multi-image matching using multi-scale oriented patches , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Tony Schuman.  A history of housing in New York City : dwelling type and social change in the American metropolis , 1993 .

[10]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[11]  Jacques Bertin,et al.  Semiology of Graphics - Diagrams, Networks, Maps , 2010 .

[12]  Katashi Nagao,et al.  The world through the computer: computer augmented interaction with real world environments , 1995, UIST '95.

[13]  Jaron Lanier,et al.  You Are Not a Gadget: A Manifesto , 2010 .

[14]  Alexandru Dancu,et al.  The Ultimate Display , 2014 .

[15]  Steve Mann,et al.  EyeTap Devices for Augmented, Deliberately Diminished, or Otherwise Altered Visual Perception of Rigid Planar Patches of Real-World Scenes , 2002, Presence: Teleoperators & Virtual Environments.

[16]  Kevin Lynch,et al.  The Image of the City , 1960 .

[17]  Lucas Paletta,et al.  A Mobile Vision System for Urban Detection with Informative Local Descriptors , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[18]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[19]  Edward Rolf Tufte,et al.  The visual display of quantitative information , 1985 .

[20]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[21]  J. Jacobs.  The Death and Life of Great American Cities , 1962 .

[22]  Virpi Roto,et al.  Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI , 2005, CHI.

[23]  Steven K. Feiner,et al.  View management for virtual and augmented reality , 2001, UIST '01.

[24]  Donald Appleyard,et al.  The View From the Road , 1965 .

[25]  Bernd Kleinjohann,et al.  Mobile AR4ALL , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[26]  Mihran Tuceryan,et al.  Automatic determination of text readability over textured backgrounds for augmented reality systems , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[27]  Hiroshi Ishii,et al.  Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information , 1998, CoBuild.

[28]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[29]  Jun Rekimoto,et al.  CyberCode: designing augmented reality environments with visual tags , 2000, DARE '00.

[30]  Ingrid Mulder,et al.  An information overload study: using design methods for understanding , 2006, OZCHI.

[31]  George W. Fitzmaurice,et al.  Situated information spaces and spatially aware palmtop computers , 1993, CACM.

[32]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[33]  Wolfgang Broll,et al.  Advanced self-contained object removal for realizing real-time Diminished Reality in unconstrained environments , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[34]  Richard Plunz,et al.  A History of Housing in New York City: Dwelling Type and Social Change in the American Metropolis. , 1991 .

[35]  Steinar Kristoffersen,et al.  “Making place” to make IT work: empirical explorations of HCI for mobile CSCW , 1999, GROUP.