Location-Based Activity Recognition

Learning patterns of human behavior from sensor data is extremely important for high-level activity inference. We show how to extract and label a person's activities and significant places from traces of GPS data. In contrast to existing techniques, our approach simultaneously detects and classifies the significant locations of a person and takes the high-level context into account. Our system uses relational Markov networks to represent the hierarchical activity model that encodes the complex relations among GPS readings, activities and significant places. We apply FFT-based message passing to perform efficient summation over large numbers of nodes in the networks. We present experiments that show significant improvements over existing techniques.

[1]  Avi Pfeffer,et al.  Asynchronous Dynamic Bayesian Networks , 2005, UAI.

[2]  Henry A. Kautz.  A formal theory of plan recognition , 1987 .

[3]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[4]  Peter D. Welch,et al.  The Fast Fourier Transform and Its Applications , 1969 .

[5]  Paul A. Viola,et al.  Interactive Information Extraction with Constrained Conditional Random Fields , 2004, AAAI.

[6]  Hung Hai Bui,et al.  A General Model for Online Probabilistic Plan Recognition , 2003, IJCAI.

[7]  Alex Pentland,et al.  Learning communities: connectivity and dynamics of interacting agents , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[8]  David Poole,et al.  Probabilistic Horn Abduction and Bayesian Networks , 1993, Artif. Intell..

[9]  Thomas L. Dean,et al.  Probabilistic Temporal Reasoning , 1988, AAAI.

[10]  Ben Taskar,et al.  Link Prediction in Relational Data , 2003, NIPS.

[11]  Dieter Fox,et al.  Relational Object Maps for Mobile Robots , 2005, IJCAI.

[12]  Bill N. Schilit,et al.  Place Lab: Device Positioning Using Radio Beacons in the Wild , 2005, Pervasive.

[13]  Kevin P. Murphy,et al.  Bayesian Map Learning in Dynamic Environments , 1999, NIPS.

[14]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[15]  Henry A. Kautz,et al.  Extracting Places and Activities from GPS Traces , 2005 .

[16]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[17]  Eric Horvitz,et al.  Layered representations for learning and inferring office activity from multiple sensory channels , 2004, Comput. Vis. Image Underst..

[18]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[19]  Andy Hopper,et al.  The active badge location system , 1992, TOIS.

[20]  Henry A. Kautz,et al.  Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields , 2007, Int. J. Robotics Res..

[21]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[22]  Henry A. Kautz,et al.  Location-Based Activity Recognition using Relational Markov Networks , 2005, IJCAI.

[23]  Michael P. Wellman,et al.  Probabilistic grammars for plan recognition , 1999 .

[24]  Krzysztof Z. Gajos,et al.  Opportunity Knocks: A System to Provide Cognitive Assistance with Transportation Services , 2004, UbiComp.

[25]  Henry A. Kautz,et al.  Training Conditional Random Fields Using Virtual Evidence Boosting , 2007, IJCAI.

[26]  George Casella,et al.  Implementations of the Monte Carlo EM Algorithm , 2001 .

[27]  Nando de Freitas,et al.  Sequential Monte Carlo in Practice , 2001 .

[28]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[29]  David Heckerman,et al.  Probabilistic Models for Relational Data , 2004 .

[30]  David C. Minnen,et al.  Propagation networks for recognition of partially ordered sequential action , 2004, CVPR 2004.

[31]  Avi Pfeffer,et al.  Probabilistic Frame-Based Systems , 1998, AAAI/IAAI.

[32]  Dieter Fox,et al.  Knowledge Compilation Properties of Trees-of-BDDs, Revisited , 2009, IJCAI.

[33]  Friedrich M. Wahl,et al.  Camera-based monitoring system for mobile robot guidance , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[34]  Stuart J. Russell,et al.  Identity Uncertainty and Citation Matching , 2002, NIPS.

[35]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[36]  Ben Taskar,et al.  Discriminative Probabilistic Models for Relational Data , 2002, UAI.

[37]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[38]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[39]  Ashish Kapoor,et al.  Learning discriminative models with incomplete data , 2006 .

[40]  Paramvir Bahl,et al.  RADAR: an in-building RF-based user location and tracking system , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[41]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[42]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[43]  Thomas G. Dietterich,et al.  Training conditional random fields via gradient tree boosting , 2004, ICML.

[44]  Henry A. Kautz,et al.  Inferring High-Level Behavior from Low-Level Sensors , 2003, UbiComp.

[45]  Andrew McCallum,et al.  Efficiently Inducing Features of Conditional Random Fields , 2002, UAI.

[46]  Svetha Venkatesh,et al.  On the Recognition of Abstract Markov Policies , 2000, AAAI/IAAI.

[47]  Nando de Freitas,et al.  Real-Time Monitoring of Complex Industrial Processes with Particle Filters , 2002, NIPS.

[48]  Brendan J. Frey,et al.  Convolutional Factor Graphs as Probabilistic Models , 2004, UAI.

[49]  Ben Taskar,et al.  Max-Margin Markov Networks , 2003, NIPS.

[50]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[51]  Dieter Fox,et al.  Map-Based Multiple Model Tracking of a Moving Object , 2004, RoboCup.

[52]  Thad Starner,et al.  Using GPS to learn significant locations and predict movement across multiple users , 2003, Personal and Ubiquitous Computing.

[53]  Henry A. Kautz,et al.  Extending Continuous Time Bayesian Networks , 2005, AAAI.

[54]  Luc De Raedt,et al.  Towards Combining Inductive Logic Programming with Bayesian Networks , 2001, ILP.

[55]  Ajo Fod,et al.  Laser-Based People Tracking , 2002 .

[56]  Edmund H. Durfee,et al.  The Automated Mapping of Plans for Plan Recognition , 1994, AAAI.

[57]  Howie Choset,et al.  Sensor based motion planning: the hierarchical generalized Voronoi graph , 1996 .

[58]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[59]  Dirk Haehnel,et al.  Are GSM Phones THE Solution for Localization? , 2006, WMCSA.

[60]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[61]  Wolfram Burgard,et al.  Using EM to learn motion behaviors of persons with mobile robots , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[63]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[64]  Lin Liao.  Locati[o]n-based activity recognition , 2006 .

[65]  Svetha Venkatesh,et al.  Policy Recognition in the Abstract Hidden Markov Model , 2002, J. Artif. Intell. Res..

[66]  Nando de Freitas,et al.  Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks , 2000, UAI.

[67]  Robert P. Goldman,et al.  A Bayesian Model of Plan Recognition , 1993, Artif. Intell..

[68]  Michael P. Wellman,et al.  Probabilistic State-Dependent Grammars for Plan Recognition , 2000, UAI.

[69]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[70]  Robert P. Goldman,et al.  A New Model of Plan Recognition , 1999, UAI.

[71]  Antonio Torralba,et al.  Contextual Models for Object Detection Using Boosted Random Fields , 2004, NIPS.

[72]  Martin Szummer.  Learning diagram parts with hidden random fields , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[73]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[74]  C. Geyer,et al.  Constrained Monte Carlo Maximum Likelihood for Dependent Data , 1992 .

[75]  Martial Hebert,et al.  Discriminative random fields: a discriminative framework for contextual interaction in classification , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[76]  Ben Taskar,et al.  Learning Probabilistic Models of Relational Structure , 2001, ICML.

[77]  Henry A. Kautz,et al.  Hierarchical Conditional Random Fields for GPS-Based Activity Recognition , 2005, ISRR.

[78]  Andrew McCallum,et al.  Accurate Information Extraction from Research Papers using Conditional Random Fields , 2004, NAACL.

[79]  Gaetano Borriello,et al.  The location stack , 2004 .

[80]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[81]  Trevor Darrell,et al.  Conditional Random Fields for Object Recognition , 2004, NIPS.

[82]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[83]  John Langford,et al.  Monte Carlo Hidden Markov Models: Learning Non-Parametric Models of Partially Observable Stochastic Processes , 1999, ICML.

[84]  Aaron F. Bobick,et al.  Action recognition using probabilistic parsing , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[85]  Blake Hannaford,et al.  A Hybrid Discriminative/Generative Approach for Modeling Human Activities , 2005, IJCAI.

[86]  Avi Pfeffer,et al.  Learning Probabilities for Noisy First-Order Rules , 1997, IJCAI.

[87]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[88]  Brendan J. Frey,et al.  A Revolution: Belief Propagation in Graphs with Cycles , 1997, NIPS.

[89]  Wolfram Burgard,et al.  People Tracking with Mobile Robots Using Sample-Based Joint Probabilistic Data Association Filters , 2003, Int. J. Robotics Res..

[90]  J. Besag.  Statistical Analysis of Non-Lattice Data , 1975 .

[91]  Kentaro Toyama,et al.  Project Lachesis: Parsing and Modeling Location Histories , 2004, GIScience.

[92]  Stuart J. Russell,et al.  Approximate inference for first-order probabilistic languages , 2001, IJCAI.

[93]  Martha E. Pollack,et al.  Execution monitoring with quantitative temporal Bayesian networks , 2002 .

[94]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[95]  William T. Freeman,et al.  Nonparametric belief propagation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[96]  Dieter Fox,et al.  Large-Scale Localization from Wireless Signal Strength , 2005, AAAI.

[97]  Vibhav Gogate,et al.  Modeling Transportation Routines using Hybrid Dynamic Mixed Networks , 2005, UAI.

[98]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[99]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[100]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[101]  N. D. Freitas.  Rao-Blackwellised particle filtering for fault diagnosis , 2002 .

[102]  Michael C. Horsch,et al.  Dynamic Bayesian networks , 1990 .

[103]  Michael I. Jordan,et al.  On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes , 2001, NIPS.

[104]  Wolfram Burgard,et al.  Learning Motion Patterns of People for Compliant Robot Motion , 2005, Int. J. Robotics Res..

[105]  Pedro M. Domingos,et al.  Dynamic Probabilistic Relational Models , 2003, IJCAI.

[106]  Thiagalingam Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation , 2001 .

[107]  Sebastian Thrun,et al.  FastSLAM: a factored solution to the simultaneous localization and mapping problem , 2002, AAAI/IAAI.

[108]  Jeff A. Bilmes,et al.  Recognizing Activities and Spatial Context Using Wearable Sensors , 2006, UAI.

[109]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[110]  Dieter Fox,et al.  Gaussian Processes for Signal Strength-Based Location Estimation , 2006, Robotics: Science and Systems.

[111]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[112]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[113]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.