A Framework for Performance Characterization of Intermediate-Level Grouping Modules

We present five performance measures to evaluate grouping modules in the context of constrained search and indexing based object recognition. Using these measures, we demonstrate a sound experimental framework, based on statistical ANOVA tests, to compare and contrast three edge based organization modules, namely, those of Etemadi et al. (1991), Jacobs (1996), and Sarkar-Boyer (1993) in the domain of aerial objects using 50 images. With adapted parameters, the Jacobs module performs overall the best for constraint based recognition. For fixed parameters, the Sarkar-Boyer module is the best in terms of recognition accuracy and indexing speedup. Etemadi et al.'s module performs equally well with fixed and adapted parameters while the Jacobs module is most sensitive to fixed and adapted parameter choices. The overall performance ranking of the modules is Jacobs, Sarkar-Boyer, and Etemadi et al.

[1]  Thomas O. Binford,et al.  On Detecting Edges , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  W. Eric L. Grimson.  The Combinatorics of Heuristic Search Termination for Object Recognition in Cluttered Environments , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Kim L. Boyer,et al.  Optimal infinite impulse response zero crossing based edge detectors , 1991, CVGIP Image Underst..

[5]  Jiri Matas,et al.  Low-level Grouping of Straight Line Segments , 1991, BMVC.

[6]  David W. Jacobs,et al.  Space and Time Bounds on Indexing 3D Models from 2D Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  R. Nevatia,et al.  Perceptual Organization for Scene Segmentation and Description , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Robert M. Haralick.  Performance Characterization in Computer Vision , 1992, BMVC.

[9]  Kim L. Boyer,et al.  Integration, Inference, and Management of Spatial Information Using Bayesian Networks: Perceptual Organization , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Mubarak Shah,et al.  Analysis of shape from shading techniques , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Jake K. Aggarwal,et al.  On comparing the performance of object recognition systems , 1995, Proceedings., International Conference on Image Processing.

[12]  Robert M. Haralick,et al.  A methodology for quantitative performance evaluation of detection algorithms , 1995, IEEE Trans. Image Process..

[13]  Josef Kittler,et al.  A Performance Measure for Boundary Detection Algorithms , 1996, Comput. Vis. Image Underst..

[14]  David W. Jacobs,et al.  Robust and Efficient Detection of Salient Convex Groups , 1996, IEEE Trans. Pattern Anal. Mach. Intell..