Temporally sequenced intelligent block-matching and motion-segmentation using locally coupled networks

Motion-based segmentation is a very important capability for computer vision and video analysis. It depends fundamentally on the system's ability to estimate optic flow using temporally proximate image frames. This is often done using block-matching. However, block-matching is sensitive to the presence of observational noise, which is inevitable in real images. Also, images often include regions of homogeneous intensity, where block-matching is problematic. A better method in this case is to estimate motion at the region level. In the approach described in this paper, we have attempted to address the noise-sensitivity and texture-insufficiency problems using a two-pathway system. The pixel-level pathway is a multilayer pulse-coupled neural network (PCNN)-like locally coupled network used to correct outliers in the block-matching motion estimates and produce improved estimates in regions with sufficient texture. In contrast, the region-level pathway is used to estimate the motion for regions with little intensity variation. In this pathway, a PCNN network first partitions intensity images into homogeneous regions, and a motion vector is then determined for the whole region. The optic flows from both pathways are fused together based on the estimated intensity variation. The fused optic flow is then segmented by a one-layer PCNN network. Results on synthetic and real images are presented to demonstrate that the accuracy of segmentation is improved significantly by taking advantage of the complementary strengths and weaknesses of the two pathways.

[1]  J. B. Ranck,et al.  Hippocampal theta rhythm and the firing of neurons in walking and urethane anesthetized rats , 2004, Experimental Brain Research.

[2]  DeLiang Wang,et al.  An oscillatory correlation model of visual motion analysis , 2002, Perception & psychophysics.

[3]  Heggere S. Ranganath,et al.  Perfect image segmentation using pulse coupled neural networks , 1999, IEEE Trans. Neural Networks.

[4]  M. Hasselmo,et al.  Dynamics of learning and recall at excitatory recurrent synapses and cholinergic modulation in rat hippocampal region CA3 , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  H. J. Reitboeck,et al.  A Multi-Electrode Matrix for Studies of Temporal Signal Correlations Within Neural Assemblies , 1983 .

[6]  Marshall C. Yovits,et al.  Ohio State University , 1974, SGAR.

[7]  Byung Cheol Song,et al.  A fast multi-resolution block matching algorithm and its LSI architecture for low bit-rate video coding , 2001, IEEE Trans. Circuits Syst. Video Technol..

[8]  Xie Yuan-dan,et al.  Survey on Image Segmentation , 2002 .

[9]  Teddy Surya Gunawan,et al.  Motion estimation using adaptive matching and multiscale methods , 2000, Visual Communications and Image Processing.

[10]  DeLiang Wang,et al.  Locally excitatory globally inhibitory oscillator networks , 1995, IEEE Transactions on Neural Networks.

[11]  Reinhard Eckhorn,et al.  A neural network for scene segmentation by temporal coding , 1996, Neurocomputing.

[12]  John L. Johnson,et al.  PCNN models and applications , 1999, IEEE Trans. Neural Networks.

[13]  P. Milner.  A model for visual shape recognition. , 1974, Psychological review.

[14]  Andrew Lippman,et al.  Spatio-temporal segmentation based on motion and static segmentation , 1995, Proceedings., International Conference on Image Processing.

[15]  J. Bourassa,et al.  The behavior of mossy cells of the rat dentate gyrus during theta oscillationsin vivo , 1993, Neuroscience.

[16]  W. Singer,et al.  Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Guy J. Brown,et al.  Separation of speech from interfering sounds based on oscillatory correlation , 1999, IEEE Trans. Neural Networks.

[18]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[19]  Francesc J. Ferri,et al.  Variable-size block matching algorithm for motion estimation using a perceptual-based splitting criterion , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[20]  Jason M. Kinser.  Foveation by a pulse-coupled neural network , 1999, IEEE Trans. Neural Networks.

[21]  R. Eckhorn,et al.  Coherent oscillations: A mechanism of feature linking in the visual cortex? , 1988, Biological Cybernetics.

[22]  Heggere S. Ranganath,et al.  Object detection using pulse coupled neural networks , 1999, IEEE Trans. Neural Networks.

[23]  H. John Caulfield,et al.  O(No) pulse-coupled neural network performing humanlike logic , 1996, Defense + Commercial Sensing.

[24]  Ch. von der Malsburg,et al.  A neural cocktail-party processor , 1986, Biological Cybernetics.

[25]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[26]  DeLiang Wang,et al.  Motion segmentation based on motion/brightness integration and oscillatory correlation , 2000, IEEE Trans. Neural Networks Learn. Syst..

[27]  Michael E. Hasselmo,et al.  Acetylcholine and Learning in a Cortical Associative Memory , 1993, Neural Computation.

[28]  Luciano Alparone,et al.  Adaptively weighted vector-median filters for motion-fields smoothing , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[29]  G. Buzsáki,et al.  Phase relations of hippocampal projection cells and interneurons to theta activity in the anesthetized rat , 1983, Brain Research.

[30]  DeLiang Wang,et al.  Image Segmentation Based on Oscillatory Correlation , 1997, Neural Computation.

[31]  Reinhard Eckhorn,et al.  Feature Linking via Synchronization among Distributed Assemblies: Simulations of Results from Cat Visual Cortex , 1990, Neural Computation.

[32]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[33]  O. Reiser,et al.  Principles Of Gestalt Psychology , 1936 .

[34]  M. Hasselmo,et al.  Encoding and retrieval of episodic memories: Role of cholinergic and GABAergic modulation in the hippocampus , 1998, Hippocampus.