Declarative interaction design for data visualization

Declarative visualization grammars can accelerate development, facilitate retargeting across platforms, and allow language-level optimizations. However, existing declarative visualization languages are primarily concerned with visual encoding, and rely on imperative event handlers for interactive behaviors. In response, we introduce a model of declarative interaction design for data visualizations. Adopting methods from reactive programming, we model low-level events as composable data streams from which we form higher-level semantic signals. Signals feed predicates and scale inversions, which allow us to generalize interactive selections at the level of item geometry (pixels) into interactive queries over the data domain. Production rules then use these queries to manipulate the visualization's appearance. To facilitate reuse and sharing, these constructs can be encapsulated as named interactors: standalone, purely declarative specifications of interaction techniques. We assess our model's feasibility and expressivity by instantiating it with extensions to the Vega visualization grammar. Through a diverse range of examples, we demonstrate coverage over an established taxonomy of visualization interaction techniques.

[1]  Brian P. Bailey,et al.  Getting inspired!: understanding how and why examples are used in creative design practice , 2009, CHI.

[2]  Arvind Satyanarayan,et al.  Lyra: An Interactive Visualization Design Environment , 2014, Comput. Graph. Forum.

[3]  Gregory H. Cooper,et al.  Integrating Dataflow Evaluation into a Practical Higher-Order Call-by-Value Language , 2008 .

[4]  Brad A. Myers Separating application code from toolkits: eliminating the spaghetti of call-backs , 1991, UIST '91.

[5]  Scott E. Hudson,et al.  Ultra-lightweight constraints , 1996, UIST '96.

[6]  Steven F. Roth,et al.  An interactive visual query environment for exploring data , 1997, UIST '97.

[7]  Peter J. Stuckey,et al.  The Cassowary linear arithmetic constraint solving algorithm , 2001, TCHI.

[8]  Ken Perlin,et al.  Pad: an alternative approach to the computer interface , 1993, SIGGRAPH.

[9]  E.A. Lee,et al.  Synchronous data flow , 1987, Proceedings of the IEEE.

[10]  Jeffrey Heer,et al.  prefuse: a toolkit for interactive information visualization , 2005, CHI.

[11]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .

[12]  Mark Bailey,et al.  The Grammar of Graphics , 2007, Technometrics.

[13]  Jeffrey Heer,et al.  Declarative Language Design for Interactive Visualization , 2010, IEEE Transactions on Visualization and Computer Graphics.

[14]  Paul Hudak,et al.  Event-Driven FRP , 2002, PADL.

[15]  Brian Campbell,et al.  Amortised Memory Analysis Using the Depth of Data Structures , 2009, ESOP.

[16]  Jeffrey Heer,et al.  Software Design Patterns for Information Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[17]  Shriram Krishnamurthi,et al.  Embedding Dynamic Dataflow in a Call-by-Value Language , 2006, ESOP.

[18]  Jeffrey Heer,et al.  SpanningAspectRatioBank Easing FunctionS ArrayIn ColorIn Date Interpolator MatrixInterpola NumObjecPointI Rectang ISchedu Parallel Pause Scheduler Sequen Transition Transitioner Transiti Tween Co DelimGraphMLCon IData JSONCon DataField DataSc Dat DataSource Data DataUtil DirtySprite LineS RectSprite , 2011 .

[19]  Leland Wilkinson,et al.  The Grammar of Graphics (Statistics and Computing) , 2005 .

[20]  Jonathan Edwards,et al.  Coherent reaction , 2009, OOPSLA Companion.

[21]  Wolfgang De Meuter,et al.  A survey on reactive programming , 2013, CSUR.

[22]  Brad A. Myers A new model for handling input , 1990, TOIS.

[23]  John T. Stasko,et al.  Mental Models, Visual Reasoning and Interaction in Information Visualization: A Top-down Perspective , 2010, IEEE Transactions on Visualization and Computer Graphics.

[24]  Martin Wattenberg,et al.  Voyagers and voyeurs: supporting asynchronous collaborative information visualization , 2007, CHI.

[25]  Chris Weaver Building Highly-Coordinated Visualizations in Improvise , 2004, IEEE Symposium on Information Visualization.

[26]  John T. Stasko,et al.  The Science of Interaction , 2009, Inf. Vis..

[27]  Tony DeRose,et al.  Proton++: a customizable declarative multitouch framework , 2012, UIST.

[28]  HeerJeffrey,et al.  D3 Data-Driven Documents , 2011 .

[29]  Jeffrey Heer,et al.  Generalized selection via interactive query relaxation , 2008, CHI.

[30]  Maria Kutar,et al.  Cognitive Dimensions of Notations: Design Tools for Cognitive Technology , 2001, Cognitive Technology.

[31]  Jennifer Widom,et al.  STREAM: The Stanford Data Stream Management System , 2016, Data Stream Management.

[32]  Brad A. Myers,et al.  ConstraintJS: programming interactive behaviors for the web by integrating constraints and states , 2012, UIST '12.

[33]  John T. Stasko,et al.  Toward a Deeper Understanding of the Role of Interaction in Information Visualization , 2007, IEEE Transactions on Visualization and Computer Graphics.

[34]  Donald A. Sch The reflective practitioner: how professionals think in action , 1983 .

[35]  Joseph A. Cottam,et al.  Stencil: A Conceptual Model for Representation and Interaction , 2008, 2008 12th International Conference Information Visualisation.

[36]  Shriram Krishnamurthi,et al.  Flapjax: a programming language for Ajax applications , 2009, OOPSLA '09.

[37]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[38]  Paul Hudak,et al.  Functional reactive animation , 1997, ICFP '97.

[39]  Stephen Chong,et al.  Asynchronous functional reactive programming for GUIs , 2013, PLDI.

[40]  Jeffrey Heer,et al.  Protovis: A Graphical Toolkit for Visualization , 2009, IEEE Transactions on Visualization and Computer Graphics.