VESPa: A Pattern-based Visual Query Language for Event Sequences

Movement data can often be enriched with additional information that enables analysts to ask new questions, for instance about POIs visited and meetings that imply interactions between persons. Information on spatiotemporal events such as visits or meetings can be especially valuable for digital forensics, marketing analysis, and urban planning. Most existing query languages for movement data, however, do not take that additional information into account. We address this gap by proposing VESPa, a pattern-based graphical query language to express, check, and refine hypotheses about spatio-temporal event sequences. Using VESPa, the analyst can sketch abstract assumptions and use the pattern to query the data for matches. The applicability of our approach is demonstrated in two case studies with different datasets. We also report on a small user study in which several construction and comprehension tasks were successfully solved in an interactive implementation

[1]  Stefano Spaccapietra,et al.  Semantic trajectories modeling and analysis , 2013, CSUR.

[2]  Evangelos E. Milios,et al.  Storage and retrieval of system log events using a structured schema based on message type transformation , 2011, SAC '11.

[3]  Daniel A. Keim,et al.  CloudLines: Compact Display of Event Episodes in Multiple Time-Series , 2011, IEEE Transactions on Visualization and Computer Graphics.

[4]  Ben Shneiderman,et al.  LifeLines: visualizing personal histories , 1996, CHI.

[5]  Donna Peuquet,et al.  An Event-Based Spatiotemporal Data Model (ESTDM) for Temporal Analysis of Geographical Data , 1995, Int. J. Geogr. Inf. Sci..

[6]  Yang Liu,et al.  Embedding Temporal Display into Maps for Occlusion-Free Visualization of Spatio-temporal Data , 2014, 2014 IEEE Pacific Visualization Symposium.

[7]  Thomas Ertl,et al.  Visual Analysis of Movement Behavior Using Web Data for Context Enrichment , 2014, 2014 IEEE Pacific Visualization Symposium.

[8]  Ben Shneiderman,et al.  Dynamic queries for visual information seeking , 1994, IEEE Software.

[9]  Vijay Kumar,et al.  Metadata visualization for digital libraries: interactive timeline editing and review , 1998, DL '98.

[10]  Heidrun Schumann,et al.  Stacking-Based Visualization of Trajectory Attribute Data , 2012, IEEE Transactions on Visualization and Computer Graphics.

[11]  Daniel A. Keim,et al.  Real-time visual analytics for event data streams , 2012, SAC '12.

[12]  Xinyan Zhu,et al.  PAN-INFORMATION LOCATION MAP , 2013 .

[13]  Thomas Ertl,et al.  TrajectoryLenses – A Set‐based Filtering and Exploration Technique for Long‐term Trajectory Data , 2013, Comput. Graph. Forum.

[14]  Michael S. Bernstein,et al.  Twitinfo: aggregating and visualizing microblogs for event exploration , 2011, CHI.

[15]  Fausto Giunchiglia,et al.  Life logging practice for human behavior modeling , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[16]  William Wright,et al.  GeoTime Information Visualization , 2004, IEEE Symposium on Information Visualization.

[17]  David Gotz,et al.  DecisionFlow: Visual Analytics for High-Dimensional Temporal Event Sequence Data , 2014, IEEE Transactions on Visualization and Computer Graphics.

[18]  Ian Horrocks,et al.  OptiqueVQS: towards an ontology-based visual query system for big data , 2013, MEDES.

[19]  Tuan A. Nguyen,et al.  The Community Stack: Concept and Prototype , 2007, 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW'07).

[20]  Ben Shneiderman,et al.  A Visual Interface for Multivariate Temporal Data: Finding Patterns of Events across Multiple Histories , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.

[21]  Alia I. Abdelmoty,et al.  A Filter Flow Visual Querying Language and Interface for Spatial Databases , 2004, GeoInformatica.

[22]  Juan-Zi Li,et al.  NEI: A Framework for Dynamic News Event Exploration and Visualization , 2014, VINCI '14.

[23]  Yan Huang,et al.  A Framework for Mining Sequential Patterns from Spatio-Temporal Event Data Sets , 2008, IEEE Transactions on Knowledge and Data Engineering.

[24]  Ben Shneiderman,et al.  Towards event sequence representation, reasoning and visualization for EHR data , 2012, IHI '12.

[25]  Inessa Seifert A Pool of Queries: Interactive Multidimensional Query Visualization for Information Seeking in Digital Libraries , 2011, Inf. Vis..

[26]  Emanuel Zgraggen,et al.  (s|qu)eries: Visual Regular Expressions for Querying and Exploring Event Sequences , 2015, CHI.

[27]  Leonidas Fegaras,et al.  VOODOO: A Visual Object-Oriented Database Language for ODMG OQL , 1999, ECOOP Workshops.

[28]  Walid Gaaloul,et al.  Discovering Workflow Transactional Behavior from Event-Based Log , 2004, CoopIS/DOA/ODBASE.

[29]  Patrizia Grifoni,et al.  Moving GeoPQL: a pictorial language towards spatio-temporal queries , 2012, GeoInformatica.

[30]  Nigel Shadbolt,et al.  NITELIGHT: A Graphical Tool for Semantic Query Construction , 2008 .

[31]  Ramesh Jain,et al.  Toward a Common Event Model for Multimedia Applications , 2007, IEEE MultiMedia.

[32]  Aggelos K. Katsaggelos,et al.  Anomalous video event detection using spatiotemporal context , 2011 .

[33]  Mohan S. Kankanhalli,et al.  Audio Based Event Detection for Multimedia Surveillance , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[34]  Thomas Ertl,et al.  Inspector Gadget: Integrating Data Preprocessing and Orchestration in the Visual Analysis Loop , 2015, EuroVA@EuroVis.

[35]  Dan Roth,et al.  Joint Inference for Event Timeline Construction , 2012, EMNLP.

[36]  John David N. Dionisio,et al.  MQuery: A Visual Query Language for Multimedia, Timeline and Simulation Data , 1996, J. Vis. Lang. Comput..

[37]  Jing Jin,et al.  QueryMarvel: A visual query language for temporal patterns using comic strips , 2009, 2009 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[38]  Denis Lalanne,et al.  Flowstrates: An Approach for Visual Exploration of Temporal Origin‐Destination Data , 2011, Comput. Graph. Forum.

[39]  Xiaoru Yuan,et al.  TripVista: Triple Perspective Visual Trajectory Analytics and its application on microscopic traffic data at a road intersection , 2011, 2011 IEEE Pacific Visualization Symposium.

[40]  Susanne Boll,et al.  Geographical queries beyond conventional boundaries: regional search and exploration , 2013, GIR '13.

[41]  Andreas Nürnberger,et al.  On the Interactive Visualization of a Logistics Scenario : Requirements and Possible Solutions , 2022 .

[42]  Lorenzo Bracciale,et al.  CRAWDAD dataset roma/taxi (v.2014-07-17) , 2014 .

[43]  Lucy T. Nowell,et al.  ThemeRiver: visualizing theme changes over time , 2000, IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings.

[44]  Ben Shneiderman,et al.  Querying event sequences by exact match or similarity search: Design and empirical evaluation , 2012, Interact. Comput..

[45]  José Luis Cabral de Moura Borges,et al.  Time Automaton: A visual mechanism for temporal querying , 2013, J. Vis. Lang. Comput..

[46]  Ben Shneiderman,et al.  The challenges of specifying intervals and absences in temporal queries: a graphical language approach , 2013, CHI.

[47]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2007 .

[48]  Samir Otmane,et al.  Design of a Visual Query Language for Geographic Information System on a Touch Screen , 2013, HCI.

[49]  Robert Laurini,et al.  A visual language for querying spatio-temporal databases , 1999, GIS '99.

[50]  P. Atrey,et al.  Timeline-based information assimilation in multimedia surveillance and monitoring systems , 2005, VSSN@MM.

[51]  Georg Fuchs,et al.  Towards Privacy-Preserving Semantic Mobility Analysis , 2013, EuroVA@EuroVis.