Real-Time Multiview Recognition of Human Gestures by Distributed Image Processing

Since a gesture involves a dynamic and complex motion, multiview observation and recognition are desirable. For the better representation of gestures, one needs to know, in the first place, from which views a gesture should be observed. Furthermore, it becomes increasingly important how the recognition results are integrated when larger numbers of camera views are considered. To investigate these problems, we propose a framework under which multiview recognition is carried out, and an integration scheme by which the recognition results are integrated online and in realtime. For performance evaluation, we use the ViHASi (Virtual Human Action Silhouette) public image database as a benchmark and our Japanese sign language (JSL) image database that contains 18 kinds of hand signs. By examining the recognition rates of each gesture for each view, we found gestures that exhibit view dependency and the gestures that do not. Also, we found that the view dependency itself could vary depending on the target gesture sets. By integrating the recognition results of different views, our swarm-based integration provides more robust and better recognition performance than individual fixed-view recognition agents.

[1]  Masahiko Yachida,et al.  Multiple-human tracking using multiple cameras , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[2]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[3]  Baozong Yuan,et al.  Model-Based Markerless Human Body Motion Capture using Multiple Cameras , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[4]  A. Corradini,et al.  Dynamic time warping for off-line recognition of a small gesture vocabulary , 2001, Proceedings IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems.

[5]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Roberto Cipolla,et al.  Continuous Gesture Recognition using a Sparse Bayesian Classifier , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[7]  Wei Pan,et al.  An evolutionary system development approach in a pervasive computing environment , 2004, 2004 International Conference on Cyberworlds.

[8]  Pradeep K. Khosla,et al.  Optimal sensor placement for cooperative distributed vision , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[9]  Sadegh Abbasi,et al.  Automatic view selection in multi-view object recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[10]  Mohiuddin Ahmad,et al.  HMM-based Human Action Recognition Using Multiview Image Sequences , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Matthias Scheutz,et al.  Real-Time Hierarchical Swarms for Rapid Adaptive Multi-Level Pattern Detection and Tracking , 2007, 2007 IEEE Swarm Intelligence Symposium.

[12]  Norimichi Ukita,et al.  Real-time multitarget tracking by a cooperative distributed vision system , 2002, Proc. IEEE.

[13]  M. Weiser,et al.  Hot topics-ubiquitous computing , 1993 .

[14]  L.M. Fernandez-Carrasco,et al.  On the Path Towards Autonomic Computing: Combining Swarm Intelligence and Excitable Media Models , 2008, 2008 Seventh Mexican International Conference on Artificial Intelligence.

[15]  Payam Saisan,et al.  Multi-View Classifier Swarms for Pedestrian Detection and Tracking , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[16]  Hermann Ney,et al.  Tracking using dynamic programming for appearance-based sign language recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[17]  Tim J. Ellis,et al.  ViHASi: Virtual human action silhouette data for the performance evaluation of silhouette-based action recognition methods , 2008, ICDSC.

[18]  Roy Sterritt,et al.  Swarms and Swarm Intelligence , 2007, Computer.

[19]  Jae Wook Jeon,et al.  Multiple Objects Tracking Circuit using Particle Filters with Multiple Features , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[20]  Ruan Qiuqi,et al.  Visual gesture recognition with color segmentation and support vector machines , 2004, Proceedings 7th International Conference on Signal Processing, 2004. Proceedings. ICSP '04. 2004..

[21]  K. Sato,et al.  Multi-rate recognition of human gestures by concurrent frame rate control , 2008, 2008 23rd International Conference Image and Vision Computing New Zealand.

[22]  Kosuke Sato,et al.  Real-time gesture recognition by learning and selective control of visual interest points , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.