From Sketching to Natural Language

Data visualization is the primary means by which data analysts explore patterns, trends, and insights in their data. Unfortunately, existing visual analytics tools offer limited expressiveness and scalability when it comes to searching for visualizations over large datasets, making visual data exploration labor-intensive and timeconsuming. We first discuss our prior work on Zenvisage that helps accelerate exploratory data analysis via an interactive interface and an expressive visualization query language, but offers limited flexibility when the pattern of interest is under-specified and approximate. Motivated from our findings from Zenvisage, we develop ShapeSearch, an efficient and flexible pattern-searching tool that enables the search for desired patterns via multiple mechanisms: sketch, natural-language, and visual regular expressions. ShapeSearch leverages a novel shape querying algebra that can express a large class of shape queries and supports query-aware and perceptually-aware optimizations to execute shape queries within interactive response times. To further improve the usability and performance of both Zenvisage and ShapeSearch, we discuss a number of open research problems.

[1]  Christos Faloutsos,et al.  Fast subsequence matching in time-series databases , 1994, SIGMOD '94.

[2]  Karrie Karahalios,et al.  ShapeSearch: A Flexible and Efficient System for Shape-based Exploration of Trendlines , 2018, SIGMOD Conference.

[3]  Abraham Silberschatz,et al.  Playful Query Specification with DataPlay , 2012, Proc. VLDB Endow..

[4]  Karrie Karahalios,et al.  ShapeSearch: Flexible Pattern-based Querying of Trend Line Visualizations , 2018, Proc. VLDB Endow..

[5]  Judith A. Blake,et al.  The Mouse Genome Database (MGD): mouse biology and model systems , 2007, Nucleic Acids Res..

[6]  Matthew D. Cooper,et al.  Shape grammar extraction for efficient query-by-sketch pattern matching in long time series , 2016, 2016 IEEE Conference on Visual Analytics Science and Technology (VAST).

[7]  Giuseppe Psaila,et al.  Querying Shapes of Histories , 1995, VLDB.

[8]  Ben Shneiderman,et al.  Interactive pattern search in time series , 2005, IS&T/SPIE Electronic Imaging.

[9]  Martin Wattenberg,et al.  Sketching a graph to query a time-series database , 2001, CHI Extended Abstracts.

[10]  Shigeru Makino,et al.  QueryLines: approximate query for visual browsing , 2005, CHI 2005.

[11]  Reimund P. Rötter,et al.  Integration of Systems Network (SysNet) tools for regional land use scenario analysis in Asia , 2005, Environ. Model. Softw..

[12]  C. Zaniolo,et al.  Expressing and optimizing sequence queries in database systems , 2004, TODS.

[13]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[14]  Hagit Shatkay,et al.  Approximate queries and representations for large data sequences , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[15]  John Lee,et al.  Effortless Data Exploration with zenvisage: An Expressive and Interactive Visual Analytics System , 2016, Proc. VLDB Endow..

[16]  Sanjiv Kumar,et al.  Google Correlate Whitepaper , 2011 .

[17]  S. Levinson,et al.  Considerations in dynamic time warping algorithms for discrete word recognition , 1978 .

[18]  Azza Abouzeid,et al.  Expressive Time Series Querying with Hand-Drawn Scale-Free Sketches , 2018, CHI.

[19]  Kyuseok Shim,et al.  Fast Similarity Search in the Presence of Noise, Scaling, and Translation in Time-Series Databases , 1995, VLDB.

[20]  John Lee,et al.  Fast-Forwarding to Desired Visualizations with Zenvisage , 2017, CIDR.

[21]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[22]  Karrie Karahalios,et al.  You can't always sketch what you want: Understanding Sensemaking in Visual Query Systems , 2017, IEEE Transactions on Visualization and Computer Graphics.

[23]  Kyuseok Shim,et al.  SPIRIT: Sequential Pattern Mining with Regular Expression Constraints , 1999, VLDB.

[24]  Arnab Nandi,et al.  Gestural Query Specification , 2013, Proc. VLDB Endow..