Integrating Graph Partition and Matching with Editing

Many computer vision tasks can be posed as either a graph part itioning (segmentation) problem or a graph matching (correspondence) problem. In this paper, we present an integrated solution for simultaneous graph partition and matching with graph editing. Our object ive is to find and match an unknown number of common graph structures (objects) in two images – a problem a rising in recent object recognition research, such as object categorization and unsupervised learning. U like previous segmentation and correspondence problems motivated by large motion or wide baseline stereo, the scenes and objects in the two images may have quite different appearance but share similar graphic s tru tures. Given two images, we first extract their primal sketches [11] as attributed graphs which repre sent both textures (by key points) and structures (by line segments and primitives). The points, line segment s, and primitives are vertices of the graphs. Both graphs are partitioned into K + 1 layers of subgraphs so that K pairs of subgraphs are matched and the remaining layer contains unmatched background. Eac h p ir of matched subgraphs represent a common object under geometric deformations and structural editing with graph operators. We design a data-driven Markov Chain Monte Carlo (DDMCMC) algorithm to explore the joint partition and matching space effectively. It has two iterative components: (i) A Sw endsen-Wang Cut component in the partition space that can flip, in each step, a chunk (connected componen t) of the graph where the vertices are often strongly coupled due to proximity and similar appeara nce; (ii) A Gibbs sampling step for matching some connected components to the other graph. To prune the ma tching space, we grow a number of vertices to form some connected components so that each conn e ted component has a small number of candidate matches. We apply the algorithm to a number of appl ications with comparison to the state of the art methods: (i) multi-object wide-baseline matching (und erlying both rigid and non-rigid motions) with occlusions , (ii) automatic detection and recognition of co mmon objects from templates, and (iii) human articulate motion analysis between images from a video sequ ence. Index Terms Graph Matching, Graph Partitioning, DDMCMC, Motion Analys is, Swendsen-Wang Cut. This manuscript is submitted to IEEE Trans. on PAMI. A short version was published in CVPR2007 [14]. email contacts: llin@lotushill.org, xbliu@lotushill.orgxbliu@lotushill.org, sczhu@stat.ucla.edu

[1]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[2]  Narendra Ahuja,et al.  Unsupervised Category Modeling, Recognition, and Segmentation in Images , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Zhuowen Tu,et al.  Shape matching and registration by data-driven EM , 2008, Comput. Vis. Image Underst..

[4]  Yongtian Wang,et al.  An Empirical Study of Object Category Recognition: Sequential Testing with Generalized Samples , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Benjamin Z. Yao,et al.  Introduction to a Large-Scale General Purpose Ground Truth Database: Methodology, Annotation Tool and Benchmarks , 2007, EMMCVPR.

[6]  Yongtian Wang,et al.  Layered Graph Match with Graph Editing , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Song-Chun Zhu,et al.  Primal sketch: Integrating structure and texture , 2007, Comput. Vis. Image Underst..

[8]  Stefano Soatto,et al.  Local Features, All Grown Up , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[9]  Serge J. Belongie,et al.  A Feature-based Approach for Dense Segmentation and Estimation of Large Disparity Motion , 2006, International Journal of Computer Vision.

[10]  Song-Chun Zhu,et al.  Perceptual Scale Space and its Applications , 2005, ICCV.

[11]  Ramesh Raskar,et al.  Discontinuity preserving stereo with small baseline multi-flash illumination , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[12]  Adrian Barbu,et al.  Generalizing Swendsen-Wang to sampling arbitrary posterior probabilities , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Luc Van Gool,et al.  Wide-baseline stereo matching with line segments , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Song-Chun Zhu,et al.  Multigrid and multi-level Swendsen-Wang cuts for hierarchic graph partition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[15]  Adrian Barbu,et al.  On the Relationship Between Image and Motion Segmentation , 2004, SCVMA.

[16]  Paul Smith,et al.  Layered motion segmentation and depth ordering by tracking edges , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  David J. Fleet,et al.  Probabilistic Detection and Tracking of Motion Boundaries , 2000, International Journal of Computer Vision.

[18]  Shree K. Nayar,et al.  What does motion reveal about transparency? , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[19]  Song-Chun Zhu,et al.  Towards a mathematical theory of primal sketch and sketchability , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[21]  Sang Wook Lee,et al.  ICP Registration Using Invariant Features , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Harry Shum,et al.  Image segmentation by data driven Markov chain Monte Carlo , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[23]  PaperNo.  Recognition of shapes by editing shock graphs , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[24]  Rong Zhang,et al.  Integrating bottom-up/top-down for object recognition by data driven Markov chain Monte Carlo , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[25]  Daniel Snow,et al.  Efficient Deformable Template Detection and Localization without User Initialization , 2000, Comput. Vis. Image Underst..

[26]  Dong-Gyu Sim,et al.  Object matching algorithms using robust Hausdorff distance measures , 1999, IEEE Trans. Image Process..

[27]  E. Adelson,et al.  Slow and Smooth: A Bayesian theory for the combination of local motion signals in human vision , 1998 .

[28]  Sven J. Dickinson,et al.  Shock Graphs and Shape Matching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[29]  Yair Weiss,et al.  Smoothness in layers: Motion segmentation using nonparametric mixture estimation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Alan L. Yuille,et al.  FORMS: A flexible object recognition and modelling system , 1995, Proceedings of IEEE International Conference on Computer Vision.

[31]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[32]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[34]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.