Sketch-Based Spatial Queries for Retrieving Human Locomotion Patterns From Continuously Archived GPS Data

We propose a system for retrieving human locomotion patterns from tracking data captured within a large geographical area, over a long period of time. A GPS receiver continuously captures data regarding the location of the person carrying it. A constrained agglomerative hierarchical clustering algorithm segments these data according to the person's navigational behavior. Sketches made on a map displayed on a computer screen are used for specifying queries regarding locomotion patterns. Two basic sketch primitives, selected based on a user study, are combined to form five different types of queries. We implement algorithms to analyze a sketch made by a user, identify the query, and retrieve results from the collection of data. A graphical user interface combines the user interaction strategy and algorithms, and allows hierarchical querying and visualization of intermediate results. We evaluate the system using a collection of data captured during nine months. The constrained hierarchical clustering algorithm is able to segment GPS data at an overall accuracy of 94% despite the presence of location-dependent noise. A user study was conducted to evaluate the proposed user interaction strategy and the usability of the overall system. The results of this study demonstrate that the proposed user interaction strategy facilitates fast querying, and efficient and accurate retrieval, in an intuitive manner.

[1]  Kiyoharu Aizawa,et al.  10-12 Video Summarization for a Large Number of Cameras Using Floor Sensors in a Ubiquitous Environment , 2005 .

[2]  Kiyoharu Aizawa,et al.  Interactive experience retrieval for a ubiquitous home , 2006, CARPE '06.

[3]  Kiyoharu Aizawa,et al.  A Sensor Network for Event Retrieval in a Home Like Ubiquitous Environment , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[4]  Kiyoharu Aizawa,et al.  A-16-14 Audio Segmentation using a Large Number of Microphones for Multimedia Retrieval in a Ubiquitous Environment , 2006 .

[5]  Kiyoharu Aizawa,et al.  A-15-15 Retrieval of Locomotion Patterns Using Sketch Based Spatial Queries , 2008 .

[6]  Kiyoharu Aizawa,et al.  Practical experience recording and indexing of Life Log video , 2005, CARPE '05.

[7]  Kiyoharu Aizawa,et al.  Spatial querying for retrieval of locomotion patterns in smart environments , 2007, ACM Multimedia.

[8]  Kiyoharu Aizawa,et al.  Sketch-Based Spatial Queries for the Retrieval of Human Locomotion Patterns in Smart Environments , 2009, Adv. Multim..

[9]  Ig-Jae Kim,et al.  PERSONE: personalized experience recoding and searching on networked environment , 2006, CARPE '06.

[10]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[11]  Svetha Venkatesh,et al.  Extraction of social context and application to personal multimedia exploration , 2006, MM '06.

[12]  Christopher R. Wren,et al.  Toward Spatial Queries for Spatial Surveillance Tasks , 2006 .

[13]  Kiyoharu Aizawa,et al.  Person Tracking and Multicamera Video Retrieval Using Floor Sensors in a Ubiquitous Environment , 2005, CIVR.

[14]  Jim Gemmell,et al.  Telling Stories with Mylifebits , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[15]  Henry A. Kautz,et al.  Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields , 2007, Int. J. Robotics Res..

[16]  Shinjiro Kawato,et al.  Human Factors Evaluation of a Vision-Based Facial Gesture Interface , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[17]  Kiyoharu Aizawa,et al.  A-20-12 Interactive Multimedia Retrieval Based on Illumination Changes in a Multi Camera Ubiquitous Environment , 2007 .

[18]  Kiyoharu Aizawa,et al.  Experience retrieval in a ubiquitous home , 2005, CARPE '05.

[19]  Kobus Barnard,et al.  Probabilistic travel modeling using GPS , 2005 .

[20]  Kiyoharu Aizawa,et al.  M-025 Selection from a Large Number of Audio and Video Sources for Personalized Video Retrieval in a Ubiquitous Environment , 2005 .

[21]  Björn Gottfried,et al.  Spatial Health Systems , 2006, 2006 Pervasive Health Conference and Workshops.

[22]  Frank M. Shipman,et al.  Trailblazing: Video Playback Control by Direct Object Manipulation , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[23]  Kiyoharu Aizawa,et al.  Embedded Tags and Visual Querying for Face Photo Retrieval , 2008, PCM.

[24]  Michael J. Lyons,et al.  A Novel Face-tracking Mouth Controller and its Application to Interacting with Bioacoustic Models , 2004, NIME.

[25]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[26]  Kiyoharu Aizawa,et al.  Evaluation of video summarization for a large number of cameras in ubiquitous home , 2005, MULTIMEDIA '05.

[27]  Kiyoharu Aizawa,et al.  Creation of an Electronic Chronicle for a Ubiquitous Home: Sensing, Analysis and Evaluation , 2006, 22nd International Conference on Data Engineering Workshops (ICDEW'06).

[28]  Kiyoharu Aizawa,et al.  TOWARDS AUTOMATED RETRIEVAL OF CONTINUOUSLY CAPTURED MULTIMEDIA AT HOME , 2007 .

[29]  Kiyoharu Aizawa,et al.  Audio Analysis for Multimedia Retrieval from a Ubiquitous Home , 2008, MMM.

[30]  Kiyoharu Aizawa,et al.  21-2 Human Locomotion Segmentation by Constrained Hierarchical Clustering of GPS Data , 2008 .

[31]  相澤 清晴,et al.  LK_011 Sound Source Localization for Multimedia Retrieval in a Ubiquitous Environment , 2006 .

[32]  Max J. Egenhofer,et al.  Query Processing in Spatial-Query-by-Sketch , 1997, J. Vis. Lang. Comput..

[33]  Kiyoharu Aizawa,et al.  Sketch-on-Map: Spatial Queries for Retrieving Human Locomotion Patterns from Continuously Archived GPS Data , 2009, MMM.

[34]  Kiyoharu Aizawa,et al.  Video handover for retrieval in a ubiquitous environment using floor sensor data , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[35]  B.W. Parkinson,et al.  NAVSTAR: Global positioning system—Ten years later , 1983, Proceedings of the IEEE.

[36]  M. J. Enenhofer.  Spatial-Query-by-Sketch , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.