Nonstationary color tracking for vision-based human-computer interaction

Skin color offers a strong cue for efficient localization and tracking of human body parts in video sequences for vision-based human-computer interaction. Color-based target localization could be achieved by analyzing segmented skin color regions. However, one of the challenges of color-based target tracking is that color distributions would change in different lighting conditions such that fixed color models would be inadequate to capture nonstationary color distributions over time. Meanwhile, using a fixed skin color model trained by the data of a specific person would probably not work well for other people. Although some work has been done on adaptive color models, this problem still needs further studies. We present our investigation of color-based image segmentation and nonstationary color-based target tracking, by studying two different representations for color distributions. We propose the structure adaptive self-organizing map (SASOM) neural network that serves as a new color model. Our experiments show that such a representation is powerful for efficient image segmentation. Then, we formulate the nonstationary color tracking problem as a model transduction problem, the solution of which offers a way to adapt and transduce color classifiers in nonstationary color distributions. To fulfill model transduction, we propose two algorithms, the SASOM transduction and the discriminant expectation-maximization (EM), based on the SASOM color model and the Gaussian mixture color model, respectively. Our extensive experiments on the task of real-time face/hand localization show that these two algorithms can successfully handle some difficulties in nonstationary color tracking. We also implemented a real-time face/hand localization system based on such algorithms for vision-based human-computer interaction.

[1]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[2]  Zhen-Ping Lo,et al.  Analysis of the convergence properties of topology preserving neural networks , 1993, IEEE Trans. Neural Networks.

[3]  Michael Isard,et al.  Active Contours , 2000, Springer London.

[4]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[6]  Vladimir Cherkassky,et al.  Statistical analysis of self-organization , 1995, Neural Networks.

[7]  John R. Kender,et al.  Finding skin in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[8]  Helge J. Ritter,et al.  Self-organizing maps: Local competition and evolutionary optimization , 1994, Neural Networks.

[9]  Ying Wu,et al.  Capturing articulated human hand motion: a divide-and-conquer approach , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[11]  Paul Mueller.  Computation of temporal pattern primitives in a neural net for speech recognition , 1988, Neural Networks.

[12]  Shan Lu,et al.  Color-based hands tracking system for sign language recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[13]  K. Schulten,et al.  On the stationary state of Kohonen's self-organizing sensory mapping , 2004, Biological Cybernetics.

[14]  J. Fort,et al.  A stochastic model of retinotopy: A self organizing process , 1986, Biological Cybernetics.

[15]  Alex Pentland,et al.  Active gesture recognition using partially observable Markov decision processes , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[16]  Ying Wu,et al.  An Adaptive Self-Organizing Color Segmentation Algorithm with Application to Robust Real-time Human Hand Localization , 2000 .

[17]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Ying Wu,et al.  Robust Visual Tracking by Integrating Multiple Cues Based on Co-Inference Learning , 2004, International Journal of Computer Vision.

[19]  Dorin Comaniciu,et al.  Robust analysis of feature spaces: color image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[21]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[22]  Klaus Schulten,et al.  Self-organizing maps: stationary states, metastability and convergence rate , 1992, Biological Cybernetics.

[23]  Ying Wu,et al.  Bootstrap Initialization of Nonparametric Texture Models for Tracking , 2000, ECCV.

[24]  Zhen-Ping Lo,et al.  On the rate of convergence in topology preserving neural networks , 1991, Biological Cybernetics.

[25]  Allen Gersho,et al.  On the structure of vector quantizers , 1982, IEEE Trans. Inf. Theory.

[26]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 2004, International Journal of Computer Vision.

[27]  Yi Zheng,et al.  The effect of concave and convex weight adjustments on self-organizing maps , 1996, IEEE Trans. Neural Networks.

[28]  Ying Wu,et al.  Human hand modeling, analysis and animation in the context of HCI , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[29]  Klaus Schulten,et al.  Self-organizing maps: ordering, convergence properties and energy functions , 1992, Biological Cybernetics.

[30]  Michael Isard,et al.  ICONDENSATION: Unifying Low-Level and High-Level Tracking in a Stochastic Framework , 1998, ECCV.

[31]  Sang-Hui Park,et al.  Self-creating and organizing neural networks , 1994, IEEE Trans. Neural Networks.

[32]  Stanley C. Ahalt,et al.  Competitive learning algorithms for vector quantization , 1990, Neural Networks.

[33]  Tsu-Chang Lee,et al.  Structure level adaptation for artificial neural networks , 1991 .

[34]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[35]  Alexander H. Waibel,et al.  Skin-Color Modeling and Adaptation , 1998, ACCV.