ARQuery: Hallucinating Analytics over Real-World Data using Augmented Reality

In addition to the virtual, there is a vast amount of data present in the real world. Given recent advances in computer vision, augmented reality, and cloud services, we are faced with a tremendous opportunity to augment the structured data around end-users with insights. Coinciding with these trends, the number of data-rich end-user activities is also rapidly increasing. Thus, it is useful to investigate the process of data exploration and analysis in augmented and mixed reality settings. In this paper, we describe ARQuery, a query platform that utilizes augmented reality to enable querying over real-world data. We provide an interaction and visualization grammar that is designed to augment realworld data, and a performant framework that enables query exploration in real-time. Our studies show that ARQuery provides a fluid, low-latency query experience for the enduser that is significantly faster than traditional approaches.

[1]  Christopher Ré,et al.  Fonduer: Knowledge Base Construction from Richly Formatted Data , 2017, SIGMOD Conference.

[2]  Arnab Nandi,et al.  Querying Without Keyboards , 2013, CIDR.

[3]  Arnab Nandi,et al.  Gestural Query Specification , 2013, Proc. VLDB Endow..

[4]  Stratos Idreos,et al.  dbTouch: Analytics at your Fingertips , 2013, CIDR.

[5]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[6]  Pat Hanrahan,et al.  VizQL: a language for query, analysis and visualization , 2006, SIGMOD Conference.

[7]  Anthony K. H. Tung,et al.  ARShop: A Cloud-based Augmented Reality System for Shopping , 2017, Proc. VLDB Endow..

[8]  Xie Yuan-dan,et al.  Survey on Image Segmentation , 2002 .

[9]  Arnab Nandi,et al.  Evaluating Interactive Data Systems: Workloads, Metrics, and Guidelines , 2018, SIGMOD Conference.

[10]  Hadley Wickham,et al.  A Layered Grammar of Graphics , 2010 .

[11]  Pat Hanrahan,et al.  Polaris: a system for query, analysis and visualization of multi-dimensional relational databases , 2000, IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings.

[12]  Allen Newell,et al.  The keystroke-level model for user performance time with interactive systems , 1980, CACM.

[13]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[14]  Filipe Costa Luz,et al.  Augmented reality for games , 2008, DIMEA.

[15]  Carsten Binnig,et al.  Vizdom: Interactive Analytics through Pen and Touch , 2015, Proc. VLDB Endow..

[16]  Mark Ollila,et al.  UMAR: Ubiquitous Mobile Augmented Reality , 2004, MUM '04.

[17]  Ce Zhang,et al.  DeepDive: A Data Management System for Automatic Knowledge Base Construction , 2015 .

[18]  Joseph M. Hellerstein,et al.  Data Tweening: Incremental Visualization of Data Transforms , 2017, Proc. VLDB Endow..

[19]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[20]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[21]  Joseph M. Hellerstein,et al.  Shreddr: pipelined paper digitization for low-resource organizations , 2012, ACM DEV '12.

[22]  Donald R. Jones,et al.  Using Visual Representations of Data to Enhance Sensemaking in Data Exploration Tasks , 2009, J. Assoc. Inf. Syst..

[23]  Egui Zhu,et al.  Augmented reality in healthcare education: an integrative review , 2014, PeerJ.

[24]  Aniket Kittur,et al.  Kinetica: naturalistic multi-touch data visualization , 2014, CHI.