The semantics of sketch: Flexibility in visual query systems for time series data

Sketching allows analysts to specify complex and free-form patterns of interest. Visual query systems can make use of sketches to locate these patterns of interest in large datasets. However, sketching is ambiguous: the same drawing could represent a multitude of potential queries. In this work, we investigate these ambiguities as they apply to visual query systems for time series data. We define a class of “invariants” — the properties of a time series that the analyst wishes to ignore when performing a sketch-based query. We present the results of a crowd-sourced study, showing that these invariants are key components of how people rate the strength of match between sketch and target. We adapt a number of algorithms for time series matching to support invariants in sketches. Lastly, we present a web-deployed prototype sketch-based visual query system that relies on these invariants. We apply the prototype to data from finance, the digital humanities, and political science.

[1]  Eamonn J. Keogh,et al.  A Complexity-Invariant Distance Measure for Time Series , 2011, SDM.

[2]  Bongshin Lee,et al.  SketchStory: Telling More Engaging Stories with Data through Freeform Sketching , 2013, IEEE Transactions on Visualization and Computer Graphics.

[3]  Ross T. Whitaker,et al.  Curve Boxplot: Generalization of Boxplot for Ensembles of Curves , 2014, IEEE Transactions on Visualization and Computer Graphics.

[4]  Stan Salvador,et al.  FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space , 2004 .

[5]  James A. Landay,et al.  Sketching Interfaces: Toward More Human Interface Design , 2001, Computer.

[6]  Steven K. Feiner,et al.  Relaxed selection techniques for querying time-series graphs , 2009, UIST '09.

[7]  Anthony K. H. Tung,et al.  SpADe: On Shape-based Pattern Detection in Streaming Time Series , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[8]  B. Shneiderman,et al.  Temporal Search and Replace : An Interactive Tool for the Analysis of Temporal Event Sequences , 2013 .

[9]  Jeffrey Heer,et al.  The Effects of Interactive Latency on Exploratory Visual Analysis , 2014, IEEE Transactions on Visualization and Computer Graphics.

[10]  Emanuel Zgraggen,et al.  Evaluating Subjective Accuracy in Time Series Pattern-Matching Using Human-Annotated Rankings , 2015, IUI.

[11]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[12]  Ada Wai-Chee Fu,et al.  Efficient time series matching by wavelets , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[13]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[14]  Ben Shneiderman,et al.  Interactive pattern search in time series , 2005, IS&T/SPIE Electronic Imaging.

[15]  Ben Shneiderman,et al.  An Augmented Visual Query Mechanism for Finding Patterns in Time Series Data , 2002, FQAS.

[16]  Martin Wattenberg,et al.  Sketching a graph to query a time-series database , 2001, CHI Extended Abstracts.

[17]  Jonathan Stephen Fish,et al.  Amplifying the Mind’s Eye: Sketching and Visual Cognition , 1990 .

[18]  T. Saito,et al.  Two-tone pseudo coloring: compact visualization for one-dimensional data , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[19]  Jarke J. van Wijk,et al.  Small Multiples, Large Singles: A New Approach for Visual Data Exploration , 2013, Comput. Graph. Forum.

[20]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[21]  Michael Gleicher,et al.  Task-driven evaluation of aggregation in time series visualization , 2014, CHI.

[22]  Ben Shneiderman,et al.  Shape Identification in Temporal Data Sets , 2012, Expanding the Frontiers of Visual Analytics and Visualization.

[23]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[24]  Hagit Shatkay,et al.  Approximate queries and representations for large data sequences , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[25]  Ben Shneiderman,et al.  Dynamic Query Tools for Time Series Data Sets: Timebox Widgets for Interactive Exploration , 2004, Inf. Vis..

[26]  John Lee,et al.  zenvisage: Effortless Visual Data Exploration , 2016, ArXiv.

[27]  Shigeru Makino,et al.  QueryLines: approximate query for visual browsing , 2005, CHI 2005.

[28]  Josef Kittler,et al.  A survey of the hough transform , 1988, Comput. Vis. Graph. Image Process..

[29]  Ben Shneiderman,et al.  Exploring Point and Interval Event Patterns: Display Methods and Interactive Visual Query , 2012 .

[30]  M. Sheelagh T. Carpendale,et al.  Beyond Mouse and Keyboard: Expanding Design Considerations for Information Visualization Interactions , 2012, IEEE Transactions on Visualization and Computer Graphics.

[31]  Niklas Elmqvist,et al.  Graphical Perception of Multiple Time Series , 2010, IEEE Transactions on Visualization and Computer Graphics.

[32]  Joachim Meyer,et al.  The effect of user characteristics on the efficiency of visual querying , 2011, Behav. Inf. Technol..

[33]  Toshikazu Kato,et al.  A sketch retrieval method for full color image database-query by visual example , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[34]  Monica M. C. Schraefel,et al.  Trust me, i'm partially right: incremental visualization lets analysts explore large datasets faster , 2012, CHI.