Expressive Time Series Querying with Hand-Drawn Scale-Free Sketches

We present Qetch, a tool where users freely sketch patterns on a scale-less canvas to query time series data without specifying query length or amplitude. We study how humans sketch time series patterns --- humans preserve visually salient perceptual features but often non-uniformly scale and locally distort a pattern --- and we develop a novel matching algorithm that accounts for human sketching errors. Qetch enables the easy construction of complex and expressive queries with two key features: regular expressions over sketches and relative positioning of sketches to query multiple time-aligned series. Through user studies, we demonstrate the effectiveness of Qetch's different interaction features. We also demonstrate the effectiveness of Qetch's matching algorithm compared to popular algorithms on targeted, and exploratory query-by-sketch search tasks on a variety of data sets.

[1]  Eamonn J. Keogh,et al.  Searching and Mining Trillions of Time Series Subsequences under Dynamic Time Warping , 2012, KDD.

[2]  Ben Shneiderman,et al.  Dynamic Query Tools for Time Series Data Sets: Timebox Widgets for Interactive Exploration , 2004, Inf. Vis..

[3]  K. Hipel,et al.  Time series modelling of water resources and environmental systems , 1994 .

[4]  Eamonn J. Keogh,et al.  On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration , 2002, Data Mining and Knowledge Discovery.

[5]  Themis Palpanas,et al.  RINSE: Interactive Data Series Exploration with ADS+ , 2015, Proc. VLDB Endow..

[6]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[7]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[8]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[9]  Wesley W. Chu,et al.  Segment-based approach for subsequence searches in sequence databases , 2001, Comput. Syst. Sci. Eng..

[10]  Stephen DiVerdi,et al.  Exploring collections of 3D models using fuzzy correspondences , 2012, ACM Trans. Graph..

[11]  Eamonn J. Keogh.  Efficiently Finding Arbitrarily Scaled Patterns in Massive Time Series Databases , 2003, PKDD.

[12]  Donald D. Hoffman,et al.  Salience of visual parts , 1997, Cognition.

[13]  W. Bruce Croft,et al.  Search Engines - Information Retrieval in Practice , 2009 .

[14]  Steven K. Feiner,et al.  Relaxed selection techniques for querying time-series graphs , 2009, UIST '09.

[15]  Dennis Shasha,et al.  Warping indexes with envelope transforms for query by humming , 2003, SIGMOD '03.

[16]  Eamonn Keogh.  Indexing and Mining Time Series Data , 2008 .

[17]  Martin Wattenberg,et al.  Sketching a graph to query a time-series database , 2001, CHI Extended Abstracts.

[18]  Nuno Constantino Castro,et al.  Time Series Data Mining , 2009, Encyclopedia of Database Systems.

[19]  M. Wertheimer.  Laws of organization in perceptual forms. , 1938 .

[20]  Eamonn J. Keogh,et al.  Extracting Optimal Performance from Dynamic Time Warping , 2016, KDD.

[21]  Marc Alexa,et al.  PhotoSketch: a sketch based image query and compositing system , 2009, SIGGRAPH '09.

[22]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[23]  Leonidas J. Guibas,et al.  Exploration of continuous variability in collections of 3D shapes , 2011, ACM Trans. Graph..

[24]  Miguel Rio,et al.  Internet Traffic Forecasting using Neural Networks , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[25]  Pavlos Protopapas,et al.  Supporting exact indexing of arbitrarily rotated shapes and periodic time series under Euclidean and warping distance measures , 2008, The VLDB Journal.

[26]  Giuseppe Psaila,et al.  Querying Shapes of Histories , 1995, VLDB.

[27]  Eamonn J. Keogh,et al.  A Complexity-Invariant Distance Measure for Time Series , 2011, SDM.

[28]  Emanuel Zgraggen,et al.  Evaluating Subjective Accuracy in Time Series Pattern-Matching Using Human-Annotated Rankings , 2015, IUI.

[29]  Eamonn J. Keogh,et al.  Scaling and time warping in time series querying , 2005, The VLDB Journal.

[30]  Nick Craswell.  Mean Reciprocal Rank , 2009, Encyclopedia of Database Systems.

[31]  Harry Hochheiser.  Interactive querying of time series data , 2002, CHI Extended Abstracts.

[32]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[33]  Shigeru Makino,et al.  QueryLines: approximate query for visual browsing , 2005, CHI 2005.

[34]  Eamonn J. Keogh,et al.  Segmenting Time Series: A Survey and Novel Approach , 2002 .

[35]  Panagiotis Papapetrou,et al.  Benchmarking dynamic time warping for music retrieval , 2010, PETRA '10.

[36]  Eamonn J. Keogh.  Indexing and Mining Time Series Data , 2008, Encyclopedia of GIS.

[37]  Maneesh Agrawala,et al.  Perceptual interpretation of ink annotations on line charts , 2009, UIST '09.

[38]  Eamonn J. Keogh,et al.  A Probabilistic Approach to Fast Pattern Matching in Time Series Databases , 1997, KDD.

[39]  Jeffrey P. Morrill.  Distributed recognition of patterns in time series data , 1998, CACM.

[40]  Ben Shneiderman,et al.  Interactive Exploration of Time Series Data , 2001, Discovery Science.

[41]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[42]  Anthony K. H. Tung,et al.  SpADe: On Shape-based Pattern Detection in Streaming Time Series , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[43]  Miguel Rio,et al.  Topology Aware Internet Traffic Forecasting Using Neural Networks , 2007, ICANN.