Perceptually-weighted evaluation criteria for segmentation masks in video sequences

In order to complement subjective evaluation of the quality of segmentation masks, this paper introduces a procedure for automatically assessing this quality. Algorithmically computed figures of merit are proposed. Assuming the existence of a perfect reference mask (ground truth), generated manually or with a reliable procedure over a test set, these figures of merit take into account visually desirable properties of a segmentation mask in order to provide the user with metrics that best quantify the spatial and temporal accuracy of the segmentation masks. For the sake of easy interpretation, results are presented on a peaked signal-to-noise ratio-like logarithmic scale.

[1]  Bülent Sankur,et al.  Performance evaluation metrics for object-based video segmentation , 2000, 2000 10th European Signal Processing Conference.

[2]  Christoph Zetzsche,et al.  Multiple Channel Model For The Prediction Of Subjective Image Quality , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[3]  Pillip Greenway,et al.  Metrics for image segmentation , 1998, Defense, Security, and Sensing.

[4]  David J. Sakrison,et al.  The effects of a visual fidelity criterion of the encoding of images , 1974, IEEE Trans. Inf. Theory.

[5]  Y. J. Zhang,et al.  A survey on evaluation methods for image segmentation , 1996, Pattern Recognit..

[6]  Touradj Ebrahimi,et al.  Evaluation of video segmentation methods for surveillance applications , 2000, 2000 10th European Signal Processing Conference.

[7]  A. Murat Tekalp,et al.  Shape similarity matching for query-by-example , 1998, Pattern Recognit..

[8]  Brendan McCane,et al.  On the Evaluation of Image Segmentation Algorithms , 1997 .

[9]  Moncef Gabbouj,et al.  Shape Similarity Estimation Using Ordinal Measures , 2001 .

[10]  George A. Goehrig,et al.  Analysis Of Image Segmentation Approaches With Emphasis On Performance Evaluation Criteria , 1980, Optics & Photonics.

[11]  Michael S. Lew,et al.  IRUS: image retrieval using shape , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[12]  A. Murat Tekalp,et al.  Metrics for performance evaluation of video object segmentation and tracking without ground-truth , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[13]  Fernando Pereira,et al.  Objective evaluation of relative segmentation quality , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[14]  Jan J. Gerbrands,et al.  Three-dimensional image segmentation using a split, merge and group approach , 1991, Pattern Recognit. Lett..

[15]  Paola Campadelli,et al.  Quantitative evaluation of color image segmentation results , 1998, Pattern Recognit. Lett..

[16]  Paulo Villegas,et al.  Objective evaluation of segmentation masks in video sequences , 2000, 2000 10th European Signal Processing Conference.

[17]  King Ngi Ngan,et al.  Special Issue on segmentation, description, and retrieval of video content , 1998 .

[18]  Touradj Ebrahimi,et al.  Objective evaluation of segmentation quality using spatio-temporal context , 2002, Proceedings. International Conference on Image Processing.

[19]  William A. Yasnoff,et al.  Error measures for scene segmentation , 1977, Pattern Recognit..

[20]  R. Koenen,et al.  MPEG-4 multimedia for our time , 1999 .

[21]  Martin D. Levine,et al.  Dynamic Measurement of Computer Generated Image Segmentations , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  RolandMech.  Objective Evaluation Criteria for 2 D-Shape Estimation Results of Moving Objects , 2002 .

[23]  Z. L. Budrikis,et al.  Picture Quality Prediction Based on a Visual Model , 1982, IEEE Trans. Commun..

[24]  Rosa Lancini,et al.  A perceptual PSNR based on the utilization of a linear model of HVS, motion vectors and DFT-3D , 2000, 2000 10th European Signal Processing Conference.

[25]  Benoit M. Macq,et al.  Image quality criterion based on the cancellation of the masked noise , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[26]  Fernando Pereira,et al.  Estimation of video object's relevance , 2000, 2000 10th European Signal Processing Conference.

[27]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[28]  Yu Jin Zhang,et al.  Evaluation and comparison of different segmentation algorithms , 1997, Pattern Recognit. Lett..

[29]  Levent Onural,et al.  Image sequence analysis for emerging interactive multimedia services-the European COST 211 framework , 1998, IEEE Trans. Circuits Syst. Video Technol..

[30]  Benoit M. Macq,et al.  Postprocessing of images by filtering the unmasked coding noise , 1999, IEEE Trans. Image Process..