Feature selection in conditional random fields for activity recognition

Temporal classification, such as activity recognition, is a key component for creating intelligent robot systems. In the case of robots, classification algorithms must robustly incorporate complex, non-independent features extracted from streams of sensor data. Conditional random fields are discriminatively trained temporal models that can easily incorporate such features. However, robots have few computational resources to spare for computing a large number of features from high bandwidth sensor data, which creates opportunities for feature selection. Creating models that contain only the most relevant features reduces the computational burden of temporal classification. In this paper, we show that lscr1 regularization is an effective technique for feature selection in conditional random fields. We present results from a multi-robot tag domain with data from both real and simulated robots that compare the classification accuracy of models trained with lscr1 regularization, which simultaneously smoothes the model and selects features; lscr2 regularization, which smoothes to avoid over-fitting, but performs no feature selection; and models trained with no smoothing.

[1]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[2]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[3]  Martial Hebert,et al.  Discriminative random fields: a discriminative framework for contextual interaction in classification , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[5]  Mee Young Park,et al.  L 1-regularization path algorithm for generalized linear models , 2006 .

[6]  Cristian Sminchisescu,et al.  Conditional Random Fields for Contextual Human Motion Recognition , 2005, ICCV.

[7]  Manuela M. Veloso,et al.  Fast and accurate vision-based pattern detection and identification , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[8]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[9]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[10]  Honglak Lee,et al.  Efficient L1 Regularized Logistic Regression , 2006, AAAI.

[11]  Henry A. Kautz,et al.  Hierarchical Conditional Random Fields for GPS-Based Activity Recognition , 2005, ISRR.

[12]  北野 宏明,et al.  RoboCup-97 : robot soccer World Cup I , 1998 .

[13]  Mee Young Park,et al.  L1‐regularization path algorithm for generalized linear models , 2007 .

[14]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[15]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[16]  Manuela M. Veloso,et al.  Conditional random fields for activity recognition , 2007, AAMAS '07.

[17]  Brett Browning,et al.  Multi-robot team response to a multi-robot opponent team , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[18]  Richard S. Zemel,et al.  Combining discriminative features to infer complex trajectories , 2006, ICML.

[19]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, CDC 1981.

[20]  Hanna M. Wallach,et al.  Efficient Training of Conditional Random Fields , 2002 .