SketchInsight: Natural data exploration on interactive whiteboards leveraging pen and touch interaction

In this work, we advance research efforts in combining the casual sketching approach of whiteboards with the machine's computing power. We present SketchInsight, a system that applies the familiar and collaborative features of a whiteboard interface to the accurate data exploration capabilities of interactive visualizations. SketchInsight enables data analysis with more fluid interaction, allowing people to visually explore their data by drawing simple charts and directly manipulating them. In addition, we report results from a qualitative study conducted to evaluate user experience in exploring data with SketchInsight, expanding our understanding on how people use a pen- and touch-enabled digital whiteboard for data exploration. We also discuss the challenges in building a working system that supports data analytic capabilities with pen and touch interaction and freeform annotation.

[1]  Ivan E. Sutherland,et al.  Sketchpad: a man-machine graphical communication system , 1899, AFIPS '63 (Spring).

[2]  James A. Landay,et al.  Interactive sketching for the early stages of user interface design , 1995, CHI '95.

[3]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[4]  Martin Wattenberg,et al.  Sketching a graph to query a time-series database , 2001, CHI Extended Abstracts.

[5]  Mark W. Newman,et al.  DENIM: an informal tool for early stage web site design , 2001, CHI Extended Abstracts.

[6]  Joseph J. LaViola,et al.  MathPad2: a system for the creation and exploration of mathematical sketches , 2004, SIGGRAPH 2004.

[7]  Joseph J. LaViola,et al.  MathPad2: a system for the creation and exploration of mathematical sketches , 2004, ACM Trans. Graph..

[8]  James A. Landay,et al.  K-sketch: a 'kinetic' sketch pad for novice animators , 2008, CHI.

[9]  Ravin Balakrishnan,et al.  ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models , 2008, UIST '08.

[10]  Daniel J. Wigdor,et al.  Combining and Measuring the Benefits of Bimanual Pen and Direct-Touch Interaction of Interfaces , 2008 .

[11]  Chris North,et al.  Understanding Multi-touch Manipulation for Surface Computing , 2009, INTERACT.

[12]  M. Sheelagh T. Carpendale,et al.  Getting practical with interactive tabletop displays: designing for dense data, "fat fingers," diverse interactions, and face-to-face collaboration , 2009, ITS '09.

[13]  William O. Chao NapkinVis: Rapid Pen-Centric Authoring of Improvisational Visualizations , 2009 .

[14]  Raimund Dachselt,et al.  Investigating multi-touch and pen gestures for diagram editing on interactive surfaces , 2009, ITS '09.

[15]  Steven K. Feiner,et al.  Relaxed selection techniques for querying time-series graphs , 2009, UIST '09.

[16]  William Buxton,et al.  Pen + touch = new tools , 2010, UIST.

[17]  Joseph J. LaViola,et al.  A Pen-Based Tool for Visualizing Vector Mathematics , 2010, SBIM.

[18]  Mary Czerwinski,et al.  Gender differences and programming environments: across programming populations , 2010, ESEM '10.

[19]  Heidrun Schumann,et al.  Tangible views for information visualization , 2010, ITS '10.

[20]  Tobias Isenberg,et al.  Integrating 2D mouse emulation with 3D manipulation for visualizations on a multi-touch table , 2010, ITS '10.

[21]  M. Sheelagh T. Carpendale,et al.  Digital Tables for Collaborative Information Exploration , 2010, Tabletops.

[22]  M. Sheelagh T. Carpendale,et al.  A set of multi-touch graph interaction techniques , 2010, ITS '10.

[23]  Michael Haller,et al.  NiCE formula editor , 2010, SIGGRAPH '10.

[24]  M. Sheelagh T. Carpendale,et al.  Data analysis on interactive whiteboards through sketch-based interaction , 2011, ITS '11.

[25]  Niklas Elmqvist,et al.  Fluid interaction for information visualization , 2011, Inf. Vis..

[26]  M. Sheelagh T. Carpendale,et al.  Gestures in the wild: studying multi-touch gesture sequences on interactive tabletop exhibits , 2011, CHI.

[27]  M. Sheelagh T. Carpendale,et al.  Visual Thinking In Action: Visualizations As Used On Whiteboards , 2011, IEEE Transactions on Visualization and Computer Graphics.

[28]  M. Sheelagh T. Carpendale,et al.  Beyond Mouse and Keyboard: Expanding Design Considerations for Information Visualization Interactions , 2012, IEEE Transactions on Visualization and Computer Graphics.

[29]  Sumit Gulwani,et al.  QuickDraw: improving drawing experience for geometric diagrams , 2012, CHI.

[30]  Takeo Igarashi,et al.  Vignette: interactive texture design and manipulation with freeform gestures for pen-and-ink illustration , 2012, CHI.

[31]  Paul Johns,et al.  Understanding Pen and Touch Interaction for Data Exploration on Interactive Whiteboards , 2012, IEEE Transactions on Visualization and Computer Graphics.

[32]  M. Sheelagh T. Carpendale,et al.  TouchWave: kinetic multi-touch manipulation for hierarchical stacked graphs , 2012, ITS.

[33]  Monica M. C. Schraefel,et al.  TouchViz: a case study comparing two interfaces for data analytics on tablets , 2013, CHI.

[34]  Tobias Isenberg,et al.  Data Visualization on Interactive Surfaces: A Research Agenda , 2013, IEEE Computer Graphics and Applications.

[35]  M. Sheelagh T. Carpendale,et al.  Transmogrification: causal manipulation of visualizations , 2013, UIST.

[36]  Bongshin Lee,et al.  SketchStory: Telling More Engaging Stories with Data through Freeform Sketching , 2013, IEEE Transactions on Visualization and Computer Graphics.

[37]  John T. Stasko,et al.  Designing and implementing an interactive scatterplot visualization for a tablet computer , 2014, AVI.

[38]  Rubaiat Habib Kazi,et al.  Draco: bringing life to illustrations with kinetic textures , 2014, CHI.

[39]  Aniket Kittur,et al.  Kinetica: naturalistic multi-touch data visualization , 2014, CHI.

[40]  Emanuel Zgraggen,et al.  PanoramicData: Data Analysis through Pen & Touch , 2014, IEEE Transactions on Visualization and Computer Graphics.