A hybrid probabilistic neural model for person tracking based on a ceiling-mounted camera

Person tracking is an important topic in ambient living systems as well as in computer vision. In particular, detecting a person from a ceiling-mounted camera is a challenge since the person's appearance is very different from the top or from the side view, and the shape of the person changes significantly when moving around the room. This article presents a novel approach for a real-time person tracking system based on particle filters with input from different visual streams. A new architecture is developed that integrates different vision streams by means of a Sigma-Pi-like network. Moreover, a short-term memory mechanism is modeled to enhance the robustness of the tracking system. Based on this architecture, the system can start localizing a person with several cues and learn the features of other cues online. The experimental results show that robust real-time person tracking can be achieved.

[1]  Jean-Marc Odobez,et al.  Using particles to track varying numbers of interacting people , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[3]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[5]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[6]  Paul Lukowicz,et al.  Developing a Sub Room Level Indoor Location System for Wide Scale Deployment in Assisted Living Systems , 2008, ICCHP.

[7]  Karsten Berns,et al.  Indoor Localisation of Humans, Objects, and mobile Robots with RFID Infrastructure , 2007, 7th International Conference on Hybrid Intelligent Systems (HIS 2007).

[8]  Shamik Sural,et al.  Segmentation and histogram generation using the HSV color space for image retrieval , 2002, Proceedings. International Conference on Image Processing.

[9]  Michael R. W. Dawson,et al.  The Multilayer Perceptron , 2008 .

[10]  Rafael Muñoz-Salinas,et al.  People detection and tracking using stereo vision and color , 2007, Image Vis. Comput..

[11]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[12]  Jochen Triesch,et al.  Democratic Integration: Self-Organized Integration of Adaptive Cues , 2001, Neural Computation.

[13]  Gérard G. Medioni,et al.  Multiple Target Tracking Using Spatio-Temporal Markov Chain Monte Carlo Data Association , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Gregory D. Abowd,et al.  The Aware Home: A Living Laboratory for Ubiquitous Computing Research , 1999, CoBuild.

[15]  Simone Frintrop,et al.  A Component-Based Approach to Visual Person Tracking from a Mobile Platform , 2010, Int. J. Soc. Robotics.

[16]  Stephen J. McKenna,et al.  Activity summarisation and fall detection in a supportive home environment , 2004, ICPR 2004.

[17]  B. Kröse,et al.  An EM-like algorithm for color-histogram-based object tracking , 2004, CVPR 2004.

[18]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[19]  Jordi Luque,et al.  Multimodal identification and localization of users in a smart environment , 2008, Journal on Multimodal User Interfaces.

[20]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[21]  Gwenn Englebienne,et al.  UvA-DARE ( Digital Academic Repository ) Activity recognition using semi-Markov models on real world smart home datasets , 2010 .

[22]  Byoung-Tak Zhang,et al.  Synthesis of sigma-pi neural networks by the breeder genetic programming , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[23]  Stefan Wermter,et al.  A self-organizing map of sigma-pi units , 2007, Neurocomputing.

[24]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[25]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[26]  Wolfram Burgard,et al.  Particle Filters for Mobile Robot Localization , 2001, Sequential Monte Carlo Methods in Practice.

[27]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[28]  David A. Forsyth,et al.  Tracking People by Learning Their Appearance , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Daniele Nardi,et al.  Real-time people localization and tracking through fixed stereo vision , 2005, Applied Intelligence.

[30]  Rüdiger Dillmann,et al.  RFID-based topological and metrical self-localization in a structured environment , 2009, 2009 International Conference on Advanced Robotics.

[31]  C. Helmick,et al.  Projections of US prevalence of arthritis and associated activity limitations. , 2006, Arthritis and rheumatism.

[32]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[33]  Patrick Pérez,et al.  Color-Based Probabilistic Tracking , 2002, ECCV.

[34]  Roberto Brunelli,et al.  An Appearance-Based Particle Filter for Visual Tracking in Smart Rooms , 2007, CLEAR.

[35]  Mubarak Shah,et al.  Tracking Multiple Occluding People by Localizing on Multiple Scene Planes , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Larry Rudolph,et al.  Project Oxygen: Pervasive, Human-Centric Computing - An Initial Experience , 2001, CAiSE.

[37]  Keiichi Kemmotsu,et al.  Human behavior recognition using unconscious cameras and a visible robot in a network robot system , 2008, Robotics Auton. Syst..

[38]  Naresh Marturi,et al.  Integration of the humanoid robot Nao inside a smart home : a case study , 2010 .

[39]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[40]  Pierre Blazevic,et al.  Mechatronic design of NAO humanoid , 2009, 2009 IEEE International Conference on Robotics and Automation.

[41]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[42]  Bernt Schiele,et al.  Visual People Detection - Different Models, Comparison and Discussion , 2009, ICRA 2009.

[43]  Günther Palm,et al.  Biomimetic Neural Learning for Intelligent Robots - Intelligent Systems, Cognitive Robotics, and Neuroscience , 2005, Biomimetic Neural Learning for Intelligent Robots.

[44]  Hélène Pigot.  From smart homes to smart care : ICOST'2005, 3rd International Conference on Smart Homes and Health Telematics , 2005 .

[45]  Emile H. L. Aarts,et al.  Ambient intelligence in HomeLab , 2002 .

[46]  Rainer Stiefelhagen,et al.  Multi-level Particle Filter Fusion of Features and Cues for Audio-Visual Person Tracking , 2007, CLEAR.

[47]  Allen Y. Yang,et al.  Distributed recognition of human actions using wearable motion sensor networks , 2009, J. Ambient Intell. Smart Environ..

[48]  Rüdiger Dillmann,et al.  Markerless human motion tracking with a flexible model and appearance learning , 2009, 2009 IEEE International Conference on Robotics and Automation.

[49]  Sankar K. Pal,et al.  Multilayer perceptron, fuzzy sets, and classification , 1992, IEEE Trans. Neural Networks.

[50]  Donald E. Brown,et al.  Health-status monitoring through analysis of behavioral patterns , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[51]  John W. McDonough,et al.  A joint particle filter for audio-visual speaker tracking , 2005, ICMI '05.

[52]  Gaurav S. Sukhatme,et al.  People tracking and following with mobile robot using an omnidirectional camera and a laser , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[53]  Alessandro Saffiotti,et al.  PEIS Ecology: integrating robots into smart environments , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[54]  M. Mercimek,et al.  Real object recognition using moment invariants , 2005 .

[55]  Alireza Khotanzad,et al.  Classification of invariant image representations using a neural network , 1990, IEEE Trans. Acoust. Speech Signal Process..

[56]  WermterStefan,et al.  A hybrid probabilistic neural model for person tracking based on a ceiling-mounted camera , 2011 .

[57]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[58]  Arthur I. Karshmer,et al.  Living assistance systems: an ambient intelligence approach , 2006, ICSE.

[59]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[60]  Peter J. Denning,et al.  The invisible future: the seamless integration of technology into everyday life , 2001 .

[61]  Branko Ristic,et al.  A particle filter for joint detection and tracking of color objects , 2007, Image Vis. Comput..

[62]  Andrew Howard,et al.  Multi-robot Simultaneous Localization and Mapping using Particle Filters , 2005, Int. J. Robotics Res..