Gesture Recognition Based on Fusion Features from Multiple Spiking Neural Networks

Gesture recognition is one of challenging image processing. In this paper, a method of gesture segmentation is proposed, which is based on fusion of multi-information from multiple neural networks inspired by the human visual system. In this method, the gesture region is segmented from the video image sequence, innovatively using integration of the outputs from two kinds of spiking neural networks. The structures and the properties of the two networks are detailed in this paper. Based on the integrated outputs, the features of distance distribution histograms and outline moments are extracted and fused to form the mixed features. Finally, gestures are classified by the multi-class Support Vector Machine. Experimental results show that the proposed algorithm works efficiently and can perform gesture segmentation and gesture recognition with the satisfying accuracy for dynamic visual image sequence under complex background. It is promising to apply this approach to video processing domain and robotic visual systems.

[1]  L A JEFFRESS,et al.  A place theory of sound localization. , 1948, Journal of comparative and physiological psychology.

[2]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[3]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Qingxiang Wu,et al.  Motion Detection Using Spiking Neural Network Model , 2008, ICIC.

[5]  Guo Li.  Distance Distribution Histogram and its Application in Trademark Image Retrieval , 2002 .

[6]  James M. Bower,et al.  The Role of Axonal Delay in the Synchronization of Networks of Coupled Cortical Oscillators , 1997, Journal of Computational Neuroscience.

[7]  Luis Enrique Sucar,et al.  On Selecting an Appropriate Colour Space for Skin Detection , 2002, MICAI.

[8]  Qingxiang Wu,et al.  Information Processing Functionality of Spiking Neurons for Image Feature Extraction , 2007 .

[9]  Agnès Just,et al.  Hand Posture Classification and Recognition using the Modified Census Transform , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[10]  P. V. V. Kishore,et al.  Segment, Track, Extract, Recognize and Convert Sign Language Videos to Voice/Text , 2012 .

[11]  Yoichi Sato,et al.  Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[12]  William T. Freeman,et al.  Television control by hand gestures , 1994 .

[13]  Jen-Wei Lin,et al.  Modulation of synaptic delay during synaptic plasticity , 2002, Trends in Neurosciences.

[14]  Chen Shuo.  Hand Gesture Recognition Based on Fourier Descriptor-BP Neural Network , 2009 .

[15]  Martin Schneider,et al.  Activity-Dependent Development of Axonal and Dendritic Delays, or, Why Synaptic Transmission Should Be Unreliable , 2002, Neural Computation.

[16]  Masakazu Konishi,et al.  Cochlear and Neural Delays for Coincidence Detection in Owls , 2001, The Journal of Neuroscience.

[17]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[18]  L A JEFFRESS.  Interaural phase difference and pitch variation; day-to-day changes. , 1949, The American journal of psychology.

[19]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[20]  M. Konishi,et al.  Axonal delay lines for time measurement in the owl's brainstem. , 1988, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Qingxiang Wu,et al.  Edge Detection Based on Spiking Neural Network Model , 2009, ICIC.

[22]  Qingxiang Wu,et al.  Segmentation of Leukocytes in Blood Smeare Images Using Color Processing Mechanism Inspired by the Visual System , 2009, 2009 2nd International Conference on Biomedical Engineering and Informatics.

[23]  Lloyd A. Jeffress.  Binaural Phase Difference and Pitch Variation , 1948 .

[24]  Tracy L. Faber.  Next generation artificial vision systems: reverse engineering the human visual system , 2009 .

[25]  Konstantinos G. P. Derpanis,et al.  A Review of Vision-Based Hand Gestures , 2004 .

[26]  Zhu Min-chen.  Real-time human face detection and tracking based on HSV model space of skin color , 2006 .