The construction of labeled line drawings from intensity images

The Marr paradigm for object recognition has been widely used in computational vision (?). This paradigm emphasizes the data driven reconstruction of 3D shape from intensity images. There are a large number of paradigms that can perform this reconstruction in a limited sense, but no paradigm or group of paradigms has yet been shown to perform this reconstruction in a general setting. A shortcoming of the Marr paradigm, as demonstrated by Lowe (?), is the failure to include grouping processes that do not directly reconstruct shape. These grouping processes are collectively called perceptual organization. Lowe proposed the addition of four paths to the Marr paradigm to rectify this shortcoming. Two of Lowe's paths are used in this work. The first is the collection of tokens in the image plane into perceptual groupings. The second is 3D inference from these groupings. The collection of tokens in the image plane into perceptual groupings is done using an integration framework implemented as a blackboard system. Four modules were used as knowledge sources: weak membrane edge detection, curvilinear grouping, proximity grouping, and curvilinear line labeling. An initial representation of the image data is built using the first three knowledge sources. This representation is analyzed using a modified curvilinear line labeling algorithm developed in this thesis that uses figure-ground separation to constrain legal line labelings more closely than the Malik algorithm (?). This modified line labeling algorithm can diagnose problems with the initial representation. Errors in the representation can be fixed using a set of heuristics that were created to repair common mistakes. If no irreparable errors are found in the representation, then the modified line labeling algorithm produces a 3D interpretation of the data in the input image. The 3D interpretation is created without explicitly recovering 3D shape, and is therefore similar to the 3D inference processes proposed by Lowe.

[1]  D. Corkill.  Blackboard Systems , 1991 .

[2]  Mihran Tuceryan.  Extraction of perceptual structure in dot patterns (vision, voronoi, clustering) , 1986 .

[3]  Roland T. Chin,et al.  Shape from texture using the Wigner distribution , 1990, Comput. Vis. Graph. Image Process..

[4]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Indranil Chakravarty,et al.  A Generalized Line and Junction Labeling Scheme with Application to scene Analysis , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Geoff A. W. West,et al.  Detection and verification of surfaces of revolution by perceptual grouping , 1992, Pattern Recognit. Lett..

[8]  Lisa M. Brown,et al.  Surface orientation from projective foreshortening of isotropic texture autocorrelation , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Makoto Nagao,et al.  A Structural Analysis of Complex Aerial Photographs , 1980, Advanced Applications in Pattern Recognition.

[10]  A. Macworth.  Interpreting pictures of polyhedral scenes , 1973 .

[11]  David G. Lowe,et al.  Perceptual Organization and Visual Recognition , 2012 .

[12]  Heinrich H. Bülthoff,et al.  Integration of Visual Modules , 1992 .

[13]  K. Sugihara.  Machine interpretation of line drawings , 1986, MIT Press series in artificial intelligence.

[14]  John Y. Aloimonos,et al.  Unification and integration of visual modules: an extension of the Marr Paradigm , 1989 .

[15]  V. Jagannathan,et al.  Blackboard Architectures and Applications , 1989 .

[16]  Sateesha Gopalakrishna Nadabar.  Markov random field contextual models in computer vision , 1992 .

[17]  Andrew Blake,et al.  Comparison of the Efficiency of Deterministic and Stochastic Algorithms for Visual Reconstruction , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[19]  Victor R. Lesser,et al.  The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty , 1980, CSUR.

[20]  Demetri Terzopoulos,et al.  Regularization of Inverse Visual Problems Involving Discontinuities , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  I. Chakravarty.  The use of characteristic views as a basis for recognition of three-dimensional objects , 1982 .

[23]  M. Wertheimer.  Laws of organization in perceptual forms. , 1938 .

[24]  Jaakko Hintikka,et al.  On the Logic of Perception , 1969 .

[25]  Takeo Kanade,et al.  A Theory of Origami World , 1979, Artif. Intell..

[26]  Irving Biederman,et al.  Human image understanding: Recent research and a theory , 1985, Comput. Vis. Graph. Image Process..

[27]  H. P Nii,et al.  Blackboard Systems , 1986 .

[28]  T. Poggio.  Vision by man and machine. , 1984, Scientific American.

[29]  M. Wertheimer.  A source book of Gestalt psychology. , 1939 .

[30]  Narendra Ahuja,et al.  Shape From Texture: Integrating Texture-Element Extraction and Surface Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  J. Malik,et al.  Recovering Three Dimensional Shape from a Single Image of Curved Objects , 1987, IJCAI.

[32]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[33]  Sharath Pankanti,et al.  On integration of vision modules , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Andrew Blake,et al.  Surface descriptions from stereo and shading , 1986, Image Vis. Comput..

[35]  D. Blostein.  Recovering the orientation of textured surfaces in natural scenes (image processing) , 1987 .

[36]  Daniel D. Corkill,et al.  GBB Reference Manual , 1988 .

[37]  Hans P. Moravec.  Sensor Fusion in Certainty Grids for Mobile Robots , 1988, AI Mag..

[38]  Azriel Rosenfeld,et al.  Human and Machine Vision , 1983 .

[39]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[41]  Gilbert Falk,et al.  Interpretation of Imperfect Line Data as a Three-Dimensional Scene , 1970, Artif. Intell..

[42]  Oscar Firschein,et al.  Readings in computer vision: issues, problems, principles, and paradigms , 1987 .

[43]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[44]  R. Nevatia,et al.  Perceptual Organization for Scene Segmentation and Description , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  David L. Waltz,et al.  Understanding Line drawings of Scenes with Shadows , 1975 .

[46]  E. Dubois,et al.  Digital picture processing , 1985, Proceedings of the IEEE.

[47]  Patrick Joseph Flynn,et al.  Cad-based computer vision: modeling and recognition strategies , 1990 .

[48]  Mihran Tüceryan,et al.  Segmentation and grouping of object boundaries using energy minimization , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  Berthold K. P. Horn.  Obtaining shape from shading information , 1989 .

[50]  Daphna Weinshall,et al.  Integration of vision modules and labeling of surface discontinuities , 1989, IEEE Trans. Syst. Man Cybern..

[51]  A. Lynn Abbott,et al.  Surface Reconstruction By Dynamic Integration Of Focus, Camera Vergence, And Stereo , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[52]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[53]  S. Sutherland.  Seeing things , 1989, Nature.

[54]  Narendra Ahuja,et al.  Dot Pattern Processing Using Voronoi Neighborhoods , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  A. Witkin,et al.  On the Role of Structure in Vision , 1983 .

[56]  M.D. Levine,et al.  Where and Why Local Shading Analysis Works , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Allen R. Hanson,et al.  Extracting Straight Lines , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  P. Burger,et al.  Calculation of surface position and orientation using the photometric stereo method , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[59]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .