Fair evaluation methods for image watermarking systems

Since the early 1990s a number of papers on ''robust'' digital watermarking systems have been presented but none of them uses the same robustness criteria. This is not practical at all for comparison and slows down progress in this area. To address this issue, we present an evaluation procedure of image watermark- ing systems. First we identify all necessary parameters for proper benchmarking and investigate how to quantitatively describe the im- age degradation introduced by the watermarking process. For this, we show the weaknesses of usual image quality measures in the context watermarking and propose a novel measure adapted to the human visual system. Then we show how to efficiently evaluate the watermark performance in such a way that fair comparisons be- tween different methods are possible. The usefulness of three graphs: ''attack versus visual-quality,'' ''bit error versus visual qual- ity,'' and ''bit error versus attack'' are investigated. In addition the receiver operating characteristic graphs are reviewed and proposed to describe statistical detection behavior of watermarking methods. Finally we review a number of attacks that any system should sur- vive to be really useful and propose a benchmark and a set of dif- ferent suitable images. © 2000 SPIE and IS&T. (S1017-9909(00)00604-8)

[1]  M. Zweig,et al.  Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. , 1993, Clinical chemistry.

[2]  Gabriela Csurka,et al.  Template based recovery of Fourier-based watermarks using log-polar and log-log maps , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[3]  Thierry Pun,et al.  Rotation, scale and translation invariant spread spectrum digital image watermarking , 1998, Signal Process..

[4]  Gordon W. Braudaway,et al.  Results of attacks on a claimed robust digital image watermark , 1998, Electronic Imaging.

[5]  Frank Bossen,et al.  Digital watermarking of color images using amplitude modulation , 1998, J. Electronic Imaging.

[6]  Ross J. Anderson.  Why cryptosystems fail , 1993, CCS '93.

[7]  Adrian Perrig,et al.  A Copyright Protection Environment for Digital Images , 1997, Verläßliche IT-Systeme.

[8]  Stefan Winkler,et al.  Perceptual distortion metric for digital color video , 1999, Electronic Imaging.

[9]  Khalid Sayood,et al.  Introduction to Data Compression , 1996 .

[10]  Jian Zhao,et al.  Embedding Robust Labels into Images for Copyright Protection , 1995, KnowRight.

[11]  Thierry Pun,et al.  Secure Copyright Protection Techniques for Digital Images , 1998, Information Hiding.

[12]  Ingemar J. Cox,et al.  Review of watermarking and the importance of perceptual modeling , 1997, Electronic Imaging.

[13]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..

[14]  Bernd Girod,et al.  Watermarking of uncompressed and compressed video , 1998, Signal Process..

[15]  Stefan Winkler,et al.  A perceptual distortion metric for digital color images , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[16]  Ingemar J. Cox,et al.  A Secure, Robust Watermark for Multimedia , 1996, Information Hiding.

[17]  Ioannis Pitas,et al.  Digital watermarking: An overview , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[18]  Markus G. Kuhn,et al.  Attacks on Copyright Marking Systems , 1998, Information Hiding.

[19]  Eckhard Koch,et al.  Towards Robust and Hidden Image Copyright Labeling , 1995 .

[20]  Reginald L. Lagendijk,et al.  Robust labeling methods for copy protection of images , 1997, Electronic Imaging.

[21]  Bernice E. Rogowitz,et al.  Human Vision and Electronic Imaging II , 1997 .

[22]  Christian J. Van Den Branden Lambrecht.  Color moving pictures quality metric , 1996, ICIP.

[23]  Ahmed H. Tewfik,et al.  Multimedia data-embedding and watermarking technologies , 1998, Proc. IEEE.

[24]  Joyce E. Farrell,et al.  Perceptual quality metric for digitally coded color images , 1996, 1996 8th European Signal Processing Conference (EUSIPCO 1996).

[25]  Kiyoshi Tanaka,et al.  Embedding the attribute information into a dithered image , 1990, Systems and Computers in Japan.

[26]  Edward J. Delp,et al.  A WATERMARKING TECHNIQUE FOR DIGITAL IMAGERY: FURTHER STUDIES , 2001 .

[27]  Deepa Kundur,et al.  Digital watermarking using multiresolution wavelet decomposition , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[28]  Ralf Steinmetz,et al.  Multimedia and Security Workshop at ACM Multimedia , 1998 .

[29]  Deepa Kundur,et al.  A robust digital image watermarking method using wavelet-based fusion , 1997, Proceedings of International Conference on Image Processing.

[30]  Ioannis Pitas,et al.  Robust image watermarking in the spatial domain , 1998, Signal Process..

[31]  D. E. Pearson,et al.  Frequency mode LR attack operator for digitally watermarked images , 1998 .

[32]  Michael G. Strintzis,et al.  Robust image watermarking in the subband or discrete cosine transform domain , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[33]  Ennio Ottaviani,et al.  Non-invertible statistical wavelet watermarking , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[34]  Germano Caronni.  Assuring Ownership Rights for Digital Images , 1995, VIS.

[35]  Christophe De Vleeschouwer,et al.  Watermarking algorithm based on a human visual model , 1998, Signal Process..

[36]  Andrew Z. Tirkel,et al.  Image and watermark registration , 1998, Signal Process..

[37]  L Rudolf,et al.  Optical Security and Counterfeit Deterrence Techniques III , 1996 .

[38]  Bin Zhu,et al.  Robust data hiding for images , 1996, 1996 IEEE Digital Signal Processing Workshop Proceedings.

[39]  M. Kutter.  Watermaking resisting to translation, rotation, and scaling , 1998 .

[40]  Jean-Paul M. G. Linnartz,et al.  Analysis of the Sensitivity Attack against Electronic Watermarks in Images , 1998, Information Hiding.

[41]  K. Tanaka,et al.  Embedding secret information into a dithered multi-level image , 1990, IEEE Conference on Military Communications.

[42]  Fabien A. P. Petitcolas,et al.  Weaknesses of copyright marking systems , 1998 .

[43]  Wenjun Zeng,et al.  Digital image watermarking using visual models , 1997, Electronic Imaging.

[44]  Mauro Barni,et al.  A DCT-domain system for robust image watermarking , 1998, Signal Process..

[45]  Fabien A. P. Petitcolas,et al.  Weakness of existing watermarking schemes , 1997 .

[46]  Edward J. Delp,et al.  A watermark for digital images , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[47]  Reginald L. Lagendijk,et al.  Perceptual image quality based on a multiple channel HVS model , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[48]  C. F. Osborne,et al.  A digital watermark , 1994, Proceedings of 1st International Conference on Image Processing.