Integrated vision system for the semantic interpretation of activities where a person handles objects

Interpretation of human activity is primarily known from surveillance and video analysis tasks and concerned with the persons alone. In this paper we present an integrated system that gives a natural language interpretation of activities where a person handles objects. The system integrates low-level image components such as hand and object tracking, detection and recognition, with high-level processes such as spatio-temporal object relationship generation, posture and gesture recognition, and activity reasoning. A task-oriented approach focuses processing to achieve near real-time and to react depending on the situation context.

[1]  Hans-Hellmut Nagel.  Reflections on Cognitive Vision Systems , 2003, ICVS.

[2]  Wolfgang Ponweiser,et al.  A software framework to integrate vision and reasoning components for Cognitive Vision Systems , 2005, Robotics Auton. Syst..

[3]  Anthony G. Cohn,et al.  Cognitive Vision: Integrating Symbolic Qualitative Representations with Computer Vision , 2006, Cognitive Vision Systems.

[4]  Hans-Hellmut Nagel.  Cognitive Vision Systems: From Ideas to Specifications , 2006, Cognitive Vision Systems.

[5]  Kjell Brunnström,et al.  Active Detection and Classsification of Junctions by Foveation with a Head-Eye System Guided by the Scale-Space Primal Sketch , 1992, ECCV.

[6]  Stepán Obdrzálek,et al.  Object Recognition using Local Affine Frames on Distinguished Regions , 2002, BMVC.

[7]  Motoki Takagi,et al.  Control of Redundant Attentive and Investigative Behaviors in an Active Cognitive Vision System , 1998 .

[8]  Katsushi Ikeuchi,et al.  Task analysis based on observing hands and objects by vision , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Giulio Sandini,et al.  Dynamic vergence , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[11]  Stepán Obdrzálek,et al.  Image Retrieval Using Local Compact DCT-Based Representation , 2003, DAGM-Symposium.

[12]  Markus Vincze,et al.  Vision for Robotics: a tool for model-based object tracking , 2005, IEEE Robotics & Automation Magazine.

[13]  E. Rivlin,et al.  Zoom tracking , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[14]  H. Christensen.  Cognitive Vision , 2004, The AI Magazine.

[15]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Hilary Buxton,et al.  Learning and understanding dynamic scene activity: a review , 2003, Image Vis. Comput..

[17]  Hilary Buxton,et al.  Developing Task-Specific RBF Hand Gesture Recognition , 2003, Gesture Workshop.

[18]  Manolis I. A. Lourakis,et al.  Real-Time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera , 2004, ECCV.

[19]  Hans-Hellmut Nagel,et al.  Cognitive Vision Systems, Sampling the Spectrum of Approaches [based on a Dagstuhl seminar] , 2006, Cognitive Vision Systems.

[20]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[21]  Helge J. Ritter,et al.  Visual recognition of continuous hand postures , 2002, IEEE Trans. Neural Networks.

[22]  Chiraz Ben Abdelkader.  Detection of People Carrying Objects: A Motion-Based Recognition Approach , 2002 .

[23]  Liu Zhuang.  Cognition of Actions , 2005 .

[24]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[25]  Sebastian Lang,et al.  Multi-modal anchoring for human-robot interaction , 2003, Robotics Auton. Syst..

[26]  James L. Crowley,et al.  Autonomic Computer Vision Systems , 2007 .

[27]  R. Dillmann,et al.  Using gesture and speech control for commanding a robot assistant , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.

[28]  Yuntao Cui,et al.  Appearance-Based Hand Sign Recognition from Intensity Image Sequences , 2000, Comput. Vis. Image Underst..

[29]  Wolfgang Ponweiser,et al.  Edge-Projected Integration of Image and Model Cues for Robust Model-Based Object Tracking , 2001, Int. J. Robotics Res..

[30]  Alessandro Saffiotti,et al.  An introduction to the anchoring problem , 2003, Robotics Auton. Syst..

[31]  Jochen Triesch,et al.  A System for Person-Independent Hand Posture Recognition against Complex Backgrounds , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Larry S. Davis,et al.  Detection of people carrying objects : a motion-based recognition approach , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[33]  Recognition of Action, Activity and Behaviour in the ActIPret Project , 2005, Künstliche Intell..