Comparative study of metrics for evaluation of object localisation by bounding boxes.

This paper deals with a comparative study of metrics allowing the evaluation of results provided by object localization algorithms. We particularly focus on localization by the bounding box representation. 26 metrics are studied in this paper. A protocol is presented for the creation of ground truths and synthetic results of localization algorithms. These synthetic results permit to simulate several inaccuracies (translation, scale errors... ) and to study the metrics behaviors face to the considered alterations. Finally, some conclusions and perspectives are given.

[1]  William A. Yasnoff,et al.  Error measures for scene segmentation , 1977, Pattern Recognit..

[2]  Olivier D. Faugeras,et al.  Visual Discrimination of Stochastic Texture Fields , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[3]  M. Basseville.  Distance measures for signal processing and pattern recognition , 1989 .

[4]  L. Vinet.  Segmentation et mise en correspondance de régions de paires d'images stéréoscopiques , 1991 .

[5]  Daniel P. Huttenlocher,et al.  A multi-resolution technique for comparing images using the Hausdorff distance , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  P. Bolon,et al.  Analyse d'images: filtrage et segmentation , 1995 .

[7]  Qian Huang,et al.  Quantitative methods of evaluating image segmentation , 1995, Proceedings., International Conference on Image Processing.

[8]  E. Baltsavias,et al.  Automatic Extraction of Man-Made Objects from Aerial and Space Images (II) , 1995 .

[9]  Ph. Bolon,et al.  Evaluation quantitative d'images filtrées , 1997 .

[10]  E. Baltsavias,et al.  3-D Building Reconstruction with ARUBA: A Qualitative and Quantitative Evaluation , 1997 .

[11]  M. Beauchemin,et al.  On the Hausdorff Distance Used for the Evaluation of Segmentation Results , 1998 .

[12]  David S. Doermann,et al.  Detection of slow-motion replay sequences for identifying sports videos , 1999, 1999 IEEE Third Workshop on Multimedia Signal Processing (Cat. No.99TH8451).

[13]  Peter J. L. van Beek,et al.  Detection of slow-motion replay segments in sports video for highlights generation , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[14]  B. S. Manjunath,et al.  Unsupervised Segmentation of Color-Texture Regions in Images and Video , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Hugues Benoit-Cattin,et al.  Scalable discrepancy measures for segmentation evaluation , 2002, Proceedings. International Conference on Image Processing.

[17]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Baoxin Li,et al.  Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  A. Murat Tekalp,et al.  Automatic soccer video analysis and summarization , 2003, IEEE Trans. Image Process..

[20]  Hanqing Lu,et al.  Replay detection in broadcasting sports video , 2004, Third International Conference on Image and Graphics (ICIG'04).

[21]  Robyn A. Owens,et al.  A New Metric for Grey-Scale Image Comparison , 1997, International Journal of Computer Vision.

[22]  Adel Hafiane,et al.  Caracterisation de Textures et Segmentation pour la recherche d'images par le contenu. (Texture characterization and Segmentation for Content Based Image Retrieval) , 2005 .

[23]  Yifan Zhang,et al.  Highlight ranking for sports video browsing , 2005, MULTIMEDIA '05.

[24]  Cordelia Schmid,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[25]  E. D'Angelo,et al.  ROBIN Challenge Evaluation principles and metrics , 2006 .

[26]  Hélène Laurent,et al.  Unsupervised and Supervised Image Segmentation Evaluation , 2006 .

[27]  章 毓晋,et al.  Advances in image and video segmentation , 2006 .