Probabilistic models for concurrent chatting activity recognition

Recognition of chatting activities in social interactions is useful for constructing human social networks. However, the existence of multiple people involved in multiple dialogues presents special challenges. To model the conversational dynamics of concurrent chatting behaviors, this article advocates Factorial Conditional Random Fields (FCRFs) as a model to accommodate co-temporal relationships among multiple activity states. In addition, to avoid the use of inefficient Loopy Belief Propagation (LBP) algorithm, we propose using Iterative Classification Algorithm (ICA) as the inference method for FCRFs. We designed experiments to compare our FCRFs model with two dynamic probabilistic models, Parallel Condition Random Fields (PCRFs) and Hidden Markov Models (HMMs), in learning and decoding based on auditory data. The experimental results show that FCRFs outperform PCRFs and HMMs-like models. We also discover that FCRFs using the ICA inference approach not only improves the recognition accuracy but also takes significantly less time than the LBP inference method.

[1]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[2]  F. Lin,et al.  Modeling of discrete event systems using finite state machines with parameters , 2000, Proceedings of the 2000. IEEE International Conference on Control Applications. Conference Proceedings (Cat. No.00CH37162).

[3]  Jennifer Neville,et al.  Iterative Classification in Relational Data , 2000 .

[4]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[5]  Alex Pentland,et al.  Mapping human networks , 2006, Fourth Annual IEEE International Conference on Pervasive Computing and Communications (PERCOM'06).

[6]  Dimitris N. Metaxas,et al.  A Framework for Recognizing the Simultaneous Aspects of American Sign Language , 2001, Comput. Vis. Image Underst..

[7]  Andrew McCallum,et al.  Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data , 2004, J. Mach. Learn. Res..

[8]  Gregory ProvanRockwell Exploiting System Structure in Model-Based Diagnosis of Discrete-Event Systems , 1996 .

[9]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[10]  William W. Cohen,et al.  Proceedings of the 23rd international conference on Machine learning , 2006, ICML 2008.

[11]  Oliver Brdiczka,et al.  Automatic detection of interaction groups , 2005, ICMI '05.

[12]  Mark W. Schmidt,et al.  Accelerated training of conditional random fields with stochastic gradient methods , 2006, ICML.

[13]  Masamichi Shimosaka,et al.  Robust Action Recognition and Segmentation with Multi-Task Conditional Random Fields , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[14]  Jeff A. Bilmes,et al.  A Privacy-Sensitive Approach to Modeling Multi-Person Conversations , 2007, IJCAI.

[15]  Sumit Basu,et al.  Conversational scene analysis , 2002 .

[16]  Henry A. Kautz,et al.  Hierarchical Conditional Random Fields for GPS-Based Activity Recognition , 2005, ISRR.

[17]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  J. Hsu,et al.  Joint Recognition of Multiple Concurrent Activities using Factorial Conditional Random Fields , 2007 .

[19]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[20]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[21]  Adnan Darwiche,et al.  Model-Based Diagnosis using Structured System Descriptions , 1998, J. Artif. Intell. Res..

[22]  Lise Getoor,et al.  Link-Based Classification , 2003, Encyclopedia of Machine Learning and Data Mining.

[23]  Sumit Basu,et al.  Modeling Conversational Dynamics as a Mixed-Memory Markov Process , 2004, NIPS.