Tangent Bundle Elastica and Computer Vision

Visual curve completion, an early visual process that completes the occluded parts between observed boundary fragments (a.k.a. inducers), is a major problem in perceptual organization and a critical step toward higher level visual tasks in both biological and machine vision. Most computational contributions to solving this problem suggest desired perceptual properties that the completed contour should satisfy in the image plane, and then seek the mathematical curves that provide them. Alternatively, few studies (including by the authors) have suggested to frame the problem not in the image plane but rather in the unit tangent bundleR 2 × S1, the space that abstracts the primary visual cortex, where curve completion allegedly occurs. Combining both schools, here we propose and develop a biologically plausible theory of elastica in the tangent bundle that provides not only perceptually superior completion results but also a rigorous computational prediction that inducer curvatures greatly affects the shape of the completed curve, as indeed indicated by human perception.

[1]  Jacqueline M. Fulvio,et al.  Visual extrapolation of contour geometry. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Manish Singh Modal and Amodal Completion Generate Different Shapes , 2004, Psychological science.

[3]  D. Fitzpatrick,et al.  Orientation Selectivity and the Arrangement of Horizontal Connections in Tree Shrew Striate Cortex , 1997, The Journal of Neuroscience.

[4]  T. Wiesel,et al.  Functional architecture of macaque monkey visual cortex , 1977 .

[5]  R. von der Heydt,et al.  Illusory contours and cortical neuron responses. , 1984, Science.

[6]  Ohad Ben-Shahar,et al.  A Tangent Bundle Theory for Visual Curve Completion , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Rüdiger von der Heydt,et al.  Simulation of neural contour mechanisms: representing anomalous contours , 1998, Image Vis. Comput..

[8]  M. Wertheimer Laws of organization in perceptual forms. , 1938 .

[9]  T. S. Lee,et al.  Dynamics of subjective contour formation in the early visual cortex. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[10]  D D Hoffman,et al.  Completing visual contours: The relationship between relatability and minimizing inflections , 1999, Perception & psychophysics.

[11]  Steven W. Zucker,et al.  Sketches with Curvature: The Curve Indicator Random Field and Markov Processes , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  David J. Field,et al.  Contour integration by the human visual system: Evidence for a local “association field” , 1993, Vision Research.

[13]  T. Wiesel,et al.  Clustered intrinsic connections in cat visual cortex , 1983, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  W. H. Reid,et al.  The Theory of Elasticity , 1960 .

[15]  C. Gilbert,et al.  Spatial distribution of contextual interactions in primary visual cortex and in visual perception. , 2000, Journal of neurophysiology.

[16]  Steven W. Zucker,et al.  Radial Projection: An Efficient Update Rule for Relaxation Labeling , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  S. Zucker,et al.  Endstopped neurons in the visual cortex as a substrate for calculating curvature , 1987, Nature.

[18]  Walter Gerbino,et al.  Visual interpolation is not scale invariant , 2006, Vision Research.

[19]  S. Ullman,et al.  Filling-in the gaps: The shape of subjective contours and a model for their generation , 1976, Biological Cybernetics.

[20]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[21]  Giovanna Citti,et al.  Functional geometry of the horizontal connectivity in the primary visual cortex , 2009, Journal of Physiology-Paris.

[22]  S. Pollmann,et al.  Retinotopic Activation in Response to Subjective Contours in Primary Visual Cortex , 2008, Frontiers in human neuroscience.

[23]  Alfred M. Bruckstein,et al.  On Minimal Energy Trajectories , 1990, Comput. Vis. Graph. Image Process..

[24]  Benjamin B. Kimia,et al.  Euler Spiral for Shape Completion , 2003, International Journal of Computer Vision.

[25]  D. Hubel,et al.  Ferrier lecture - Functional architecture of macaque monkey visual cortex , 1977, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[26]  T. Chan,et al.  Variational image inpainting , 2005 .

[27]  V. Caselles,et al.  A geometric model for active contours in image processing , 1993 .

[28]  P. Kellman,et al.  A theory of visual interpolation in object perception , 1991, Cognitive Psychology.

[29]  Ohad Ben-Shahar,et al.  Geometrical Computations Explain Projection Patterns of Long-Range Horizontal Connections in Visual Cortex , 2004, Neural Computation.

[30]  B. O'neill Elementary Differential Geometry , 1966 .

[31]  C. K. Ogden A Source Book Of Gestalt Psychology , 2013 .

[32]  Farzin Mokhtarian,et al.  A Theory of Multiscale, Curvature-Based Shape Representation for Planar Curves , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Sharon E. Guttman,et al.  Contour interpolation revealed by a dot localization paradigm , 2004, Vision Research.

[34]  MokhtarianFarzin,et al.  A Theory of Multiscale, Curvature-Based Shape Representation for Planar Curves , 1992 .

[35]  Jeffrey S. Perry,et al.  Edge co-occurrence in natural images predicts contour grouping performance , 2001, Vision Research.

[36]  Giovanna Citti,et al.  A Cortical Based Model of Perceptual Completion in the Roto-Translation Space , 2006, Journal of Mathematical Imaging and Vision.

[37]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[38]  Kunihiko Fukushima,et al.  Neural network model for completing occluded contours , 2010, Neural Networks.

[39]  Philip J. Kellman,et al.  Interpolation processes in the visual perception of objects , 2003, Neural Networks.

[40]  Eric Saund,et al.  Perceptual organization of occluding contours generated by opaque surfaces , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[41]  G. Orban,et al.  Responses of visual cortical neurons to curved stimuli and chevrons , 1990, Vision Research.

[42]  Isaac Weiss 3-D Shape Representation by Contours , 1985, IJCAI.

[43]  Ronen Basri,et al.  Completion energies and scale , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  John K. Tsotsos,et al.  Shape Representation and Recognition from Multiscale Curvature , 1997, Comput. Vis. Image Underst..

[45]  S. Maier,et al.  Widespread Periodic Intrinsic Connections in the Tree Shrew Visual Cortex , 2005 .

[46]  D. Mumford Elastica and Computer Vision , 1994 .

[47]  G. M.,et al.  A Treatise on the Mathematical Theory of Elasticity , 1906, Nature.

[48]  C. Gilbert Horizontal integration and cortical dynamics , 1992, Neuron.

[49]  S Shimojo,et al.  The Theory of the Curvature-Constraint Line for Amodal Completion , 1995, Perception.

[50]  R Malladi,et al.  Subjective surfaces: a method for completing missing boundaries. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[51]  Ohad Ben-Shahar,et al.  The Perceptual Organization of Texture Flow: A Contextual Inference Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  S. Timoshenko,et al.  Theory of elasticity , 1975 .

[53]  W. Hoffman The visual cortex is a contact bundle , 1989 .

[54]  H Takeichi,et al.  The Effect of Curvature on Visual Interpolation , 1995, Perception.

[55]  A. Grinvald,et al.  Relationship between intrinsic connections and functional architecture revealed by optical imaging and in vivo targeted biocytin injections in primate striate cortex. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[56]  Jacqueline M. Fulvio,et al.  Precision and consistency of contour interpolation , 2008, Vision Research.

[57]  Mubarak Shah,et al.  A Fast algorithm for active contours and curvature estimation , 1992, CVGIP Image Underst..

[58]  David G. Lowe,et al.  Organization of smooth image curves at multiple scales , 1988, International Journal of Computer Vision.

[59]  Jacqueline M. Fulvio,et al.  Bayesian contour extrapolation: Geometric determinants of good continuation , 2007, Vision Research.

[60]  Du-Ming Tsai,et al.  Curve fitting approach for tangent angle and curvature measurements , 1994, Pattern Recognit..

[61]  Berthold K. P. Horn The Curve of Least Energy , 1983, TOMS.

[62]  G. Kanizsa,et al.  Organization in Vision: Essays on Gestalt Perception , 1979 .

[63]  Steven W. Zucker,et al.  Trace Inference, Curvature Consistency, and Curve Detection , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  W. Gerbino,et al.  Contour interpolation by vector-field combination. , 2003, Journal of vision.

[65]  Michael J. Hawken,et al.  Macaque VI neurons can signal ‘illusory’ contours , 1993, Nature.

[66]  W. Marsden I and J , 2012 .

[67]  Lance R. Williams,et al.  Stochastic Completion Fields: A Neural Model of Illusory Contour Shape and Salience , 1995, Neural Computation.

[68]  W. Eric L. Grimson,et al.  Shape Encoding and Subjective Contours , 1980, AAAI.

[69]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[70]  Jean Petitot,et al.  Neurogeometry of V1 and Kanizsa Contours , 2002 .

[71]  G. Mitchison,et al.  Long axons within the striate cortex: their distribution, orientation, and patterns of connection. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[72]  Ohad Ben-Shahar,et al.  Tangent Bundle Curve Completion with Locally Connected Parallel Networks , 2012, Neural Computation.