Pipeline-Architecture Based Real-Time Active-Vision for Human-Action Recognition

This paper presents a generic framework for on-line reconfiguration of a multi-camera active-vision system for time-varying-geometry object/subject action recognition. The proposed methodology utilizes customizable pipeline architecture to select optimal camera poses in real time. Subject visibility is optimized via a depth-limited search algorithm. All stages are developed with real-time operation as the central focus. A human action-sensing implementation example demonstrates viability. Controlled experiments, first with a human analogue and, subsequently, with a real human, illustrate the workings of the proposed framework. A tangible increase in action-recognition success rate over other strategies, particularly those with static cameras, is noteworthy. The proposed framework is also shown to operate in real-time. Further experiments examine the effect of scaling the number of obstacles and cameras, sensing-system mobility, and library actions on real-time performance.

[1]  Richard Pito,et al.  A Solution to the Next Best View Problem for Automated Surface Acquisition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Mohamed S. Kamel,et al.  An agent-based approach to multisensor coordination , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[3]  Tolga K. Çapin,et al.  A dead-reckoning algorithm for virtual human figures , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[4]  Beno Benhabib,et al.  Active-Vision System Reconfiguration for Form Recognition in the Presence of Dynamic Obstacles , 2008, AMDO.

[5]  Peter K. Allen,et al.  Constraint-Based Sensor Planning for Scene Modeling , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Masayoshi Kakikura,et al.  Occlusion avoidance of visual sensors based on a hand-eye action simulator system: HEAVEN , 1987, Adv. Robotics.

[7]  Shirley Williams.  Programming models for parallel systems , 1990, Wiley series in parallel computing.

[8]  Michael M. Marefat,et al.  A robust approach for sensor placement in automated vision dimensional inspection , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[9]  Konstantinos A. Tarabanis,et al.  A survey of sensor planning in computer vision , 1995, IEEE Trans. Robotics Autom..

[10]  Mark S. Nixon,et al.  What image information is important in silhouette-based gait recognition? , 2004, CVPR 2004.

[11]  Sudeep Sarkar,et al.  Effect of silhouette quality on hard problems in gait recognition , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Dimitrios Tzovaras,et al.  Gait Identification using the 3D Protrusion Transform , 2007, 2007 IEEE International Conference on Image Processing.

[13]  Raul Poler,et al.  Non-Linear Programming , 2014 .

[14]  Beno Benhabib,et al.  A Multi-Camera Active-Vision System for Dynamic Form Recognition , 2008 .

[15]  Beno Benhabib,et al.  An Active Vision System for Multitarget Surveillance in Dynamic Environments , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Raymond S. T. Lee,et al.  A New Representation for Human Gait Recognition: Motion Silhouettes Image (MSI) , 2006, ICB.

[17]  S. Sural,et al.  A fuzzy inferencing system for gait recognition , 2009, NAFIPS 2009 - 2009 Annual Meeting of the North American Fuzzy Information Processing Society.

[18]  Hans-Peter Seidel,et al.  Motion capture using joint skeleton tracking and surface estimation , 2009, CVPR.

[19]  Ahmed M. Elgammal,et al.  Dynamic shape outlier detection for human locomotion , 2009, Comput. Vis. Image Underst..

[20]  Jiwen Lu,et al.  Gait recognition for human identification based on ICA and fuzzy SVM through multiple views fusion , 2007, Pattern Recognit. Lett..

[21]  Beno Benhabib,et al.  Coordinated dispatching of proximity sensors for the surveillance of manoeuvring targets , 2003 .

[22]  Katsushi Ikeuchi,et al.  Task-Oriented Generation of Visual Sensing Strategies in Assembly Tasks , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Beno Benhabib,et al.  TIME-VARYING-GEOMETRY OBJECT SURVEILLANCE USING A MULTI-CAMERA ACTIVE-VISION SYSTEM , 2008 .

[24]  Beno Benhabib,et al.  Object-of-Interest Selection for Model-Based 3D Pose Tracking with Background Clutter , 2008 .

[25]  Peter Kovesi,et al.  Automatic Sensor Placement from Vision Task Requirements , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Xuelong Li,et al.  Human Gait Recognition With Matrix Representation , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Anurag Mittal,et al.  Generalized Multi-sensor Planning , 2006, ECCV.

[28]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[29]  Rafael Murrieta-Cid,et al.  A Sampling-Based Motion Planning Approach to Maintain Visibility of Unpredictable Targets , 2005, Auton. Robots.

[30]  Mark S. Nixon,et al.  Gait Feature Subset Selection by Mutual Information , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[31]  Xue Wang,et al.  Distributed Visual-Target-Surveillance System in Wireless Sensor Networks , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[32]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[33]  Beno Benhabib,et al.  Active-Vision for the Autonomous Surveillance of Dynamic, Multi-Object Environments , 2009, J. Intell. Robotic Syst..

[34]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.