Robust face detection and hand posture recognition in color images for human-machine interaction

A system for the detection of human faces and for the classification of hand postures in color images is presented. We first propose to apply a combination of a skin chrominance-based image segmentation with a color vector gradient-based edge detection to efficiently detect faces and hands. A statistical model for face detection based on invariant moments is then used to discriminate between faces and hands in the segmented images. A novel approach to hand posture recognition based on phase-only correlation is finally applied to classify a subset of static hand postures of the Japanese sign language, each posture representing a given phoneme, and also to discriminate between hand postures and the image scene background. Experiments show that the additional use of the color gradient significantly improves the correct rate of face detection, and that the phase-only correlation filter yields a high rate of discrimination between different static hand postures as well as between hand postures and the scene background.

[1]  Hsien-Che Lee,et al.  Detecting boundaries in a vector field , 1991, IEEE Trans. Signal Process..

[2]  Jochen Triesch,et al.  Robust classification of hand postures against complex backgrounds , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[3]  Shigeru Akamatsu,et al.  Invariant neural-network based face detection with orthogonal Fourier-Mellin moments , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[4]  Alexander H. Waibel,et al.  Segmenting hands of arbitrary color , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[5]  J. Horner,et al.  Phase-only matched filtering. , 1984, Applied optics.

[6]  Shigeru Akamatsu,et al.  Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[7]  A. Pentland.  Smart rooms, smart clothes , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[8]  A.V. Oppenheim,et al.  The importance of phase in signals , 1980, Proceedings of the IEEE.

[9]  Y. Sheng,et al.  Orthogonal Fourier–Mellin moments for invariant pattern recognition , 1994 .

[10]  Shengrui Wang,et al.  Sign Language Recognition using Moment-Based Size Functions , 1999 .

[11]  Silvano Di Zenzo,et al.  A note on the gradient of a multi-image , 1986, Comput. Vis. Graph. Image Process..

[12]  Kazuhiko Yamamoto,et al.  Face Direction Estimation Using Multiple Cameras for Human Computer Interaction , 2000, ICMI.

[13]  Yajun Li,et al.  Reforming the theory of invariant moments for pattern recognition , 1992, Pattern Recognit..

[14]  Shigeru Akamatsu,et al.  Automatic detection of human faces in natural scene images by use of a skin color model and of invariant moments , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.