Twenty years of digital audio watermarking - a comprehensive review

Digital audio watermarking is an important technique to secure and authenticate audio media. This paper provides a comprehensive review of the twenty years' research and development works for digital audio watermarking, based on an exhaustive literature survey and careful selections of representative solutions. We generally classify the existing designs into time domain and transform domain methods, and relate all the reviewed works using two generic watermark embedding equations in the two domains. The most important designing criteria, i.e., imperceptibility and robustness, are thoroughly reviewed. For imperceptibility, the existing measurement and control approaches are classified into heuristic and analytical types, followed by intensive analysis and discussions. Then, we investigate the robustness of the existing solutions against a wide range of critical attacks categorized into basic, desynchronization, and replacement attacks, respectively. This reveals current challenges in developing a global solution robust against all the attacks considered in this paper. Some remaining problems as well as research potentials for better system designs are also discussed. In addition, audio watermarking applications in terms of US patents and commercialized solutions are reviewed. This paper serves as a comprehensive tutorial for interested readers to gain a historical, technical, and also commercial view of digital audio watermarking. HighlightsThe paper systematically categorizes all important existing audio watermark embedding schemes in a concise and effective way based on two generic embedding functions, followed by extensive discussions and analysis.The measurement and control approaches to ensure the imperceptibility property of existing audio watermarking systems are exhausted and categorized into heuristic and analytical groups, with detailed analysis and comparison.Existing attacks to audio watermarking systems are comprehensively studied and rigorously evaluated against a series of representative audio watermarking systems.Current open challenges and future research potentials are sufficiently addressed in this paper.Audio watermarking applications in terms of US patents and commercial products are extensively reviewed.

[1]  Oscal T.-C. Chen,et al.  Highly Robust, Secure, and Perceptual-Quality Echo Hiding Scheme , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[2]  Yongfeng Huang,et al.  Steganography in Inactive Frames of VoIP Streams Encoded by Source Codec , 2011, IEEE Transactions on Information Forensics and Security.

[3]  Song Guo,et al.  Robust Patchwork-Based Embedding and Decoding Scheme for Digital Audio Watermarking , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[4]  Jiwu Huang,et al.  Histogram-Based Audio Watermarking Against Time-Scale Modification and Cropping Attacks , 2007, IEEE Transactions on Multimedia.

[5]  Chi-Man Pun,et al.  Robust Segments Detector for De-Synchronization Resilient Audio Watermarking , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[6]  Wanlei Zhou,et al.  A Dual-Channel Time-Spread Echo Method for Audio Watermarking , 2012, IEEE Transactions on Information Forensics and Security.

[7]  David Megías,et al.  Audio Watermarking Based on Fibonacci Numbers , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[8]  Teddy Furon,et al.  A New Measure of Watermarking Security: The Effective Key Length , 2013, IEEE Transactions on Information Forensics and Security.

[9]  Dezhong Peng,et al.  Robust time-spread echo watermarking using characteristics of host signals , 2016 .

[10]  Ryouichi Nishimura,et al.  Audio Watermarking Using Spatial Masking and Ambisonics , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[11]  Ahmed H. Tewfik,et al.  Digital watermarks for audio signals , 1996, 1996 8th European Signal Processing Conference (EUSIPCO 1996).

[12]  Darko Kirovski,et al.  Blind pattern matching attack on watermarking systems , 2003, IEEE Trans. Signal Process..

[13]  Hyoung Joong Kim,et al.  A novel echo-hiding scheme with backward and forward kernels , 2003, IEEE Trans. Circuits Syst. Video Technol..

[14]  Walter Bender,et al.  Techniques for Data Hiding , 1996, IBM Syst. J..

[15]  Ton Kalker,et al.  Pairwise Boosted Audio Fingerprint , 2009, IEEE Transactions on Information Forensics and Security.

[16]  Mohammad Ali Akhaee,et al.  Robust Multiplicative Patchwork Method for Audio Watermarking , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[17]  Hyoung Joong Kim,et al.  Modified Patchwork Algorithm: a novel audio watermarking scheme , 2001, Proceedings International Conference on Information Technology: Coding and Computing.

[18]  Akio Ogihara,et al.  Objective evaluation of sound quality for attacks on robust audio watermarking , 2013 .

[19]  Ioannis Pitas,et al.  Robust audio watermarking in the time domain , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[20]  Werner Oomen,et al.  A temporal domain audio watermarking technique , 2003, IEEE Trans. Signal Process..

[21]  Wen-Nung Lie,et al.  Robust and high-quality time-domain audio watermarking based on low-frequency amplitude modification , 2006, IEEE Transactions on Multimedia.

[22]  ChenB.,et al.  Quantization index modulation , 2006 .

[23]  Brian M. Kurkoski,et al.  Full-Index-Embedding Patchwork Algorithm for Audio Watermarking , 2008, IEICE Trans. Inf. Syst..

[24]  Xiang-Yang Wang,et al.  A Novel Synchronization Invariant Audio Watermarking Scheme Based on DWT and DCT , 2006, IEEE Transactions on Signal Processing.

[25]  Xiangyang Wang,et al.  A Robust, Digital-Audio Watermarking Method , 2009, IEEE MultiMedia.

[26]  E. Owens,et al.  An Introduction to the Psychology of Hearing , 1997 .

[27]  Justin Picard,et al.  On the use of masking models for image and audio watermarking , 2005, IEEE Transactions on Multimedia.

[28]  Xiao-Ming Chen,et al.  A Phase-Based Audio Watermarking System Robust to Acoustic Path Propagation , 2014, IEEE Transactions on Information Forensics and Security.

[29]  Frank Hartung,et al.  Multimedia watermarking techniques , 1999, Proc. IEEE.

[30]  Michael Arnold Audio watermarking: features, applications and algorithms , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[31]  Stanley A. Gelfand,et al.  Hearing: An Introduction to Psychological and Physiological Acoustics, Fourth Edition , 1998 .

[32]  Klara Nahrstedt,et al.  Noninvertible watermarking methods for MPEG-encoded audio , 1998, Electronic Imaging.

[33]  Masashi Unoki,et al.  Robust, Blindly-Detectable, and Semi-Reversible Technique of Audio Watermarking Based on Cochlear Delay Characteristics , 2015, IEICE Trans. Inf. Syst..

[34]  Yun Q. Shi,et al.  Reversible Data Hiding Using Controlled Contrast Enhancement and Integer Wavelet Transform , 2015, IEEE Signal Processing Letters.

[35]  Z. Jane Wang,et al.  An Improved Multiplicative Spread Spectrum Embedding Scheme for Data Hiding , 2012, IEEE Transactions on Information Forensics and Security.

[36]  Masashi Unoki,et al.  Embedding Limitations with Digital-audio Watermarking Method Based on Cochlear Delay Characteristics , 2011, J. Inf. Hiding Multim. Signal Process..

[37]  Darko Kirovski,et al.  On the Need for Signal-Coherent Watermarks , 2006, IEEE Transactions on Multimedia.

[38]  Przemyslaw Dymarski,et al.  Controlling the inaudibility and maximizing the robustness in an audio annotation watermarking system , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[39]  Mauro Barni,et al.  Optimum decoding and detection of multiplicative watermarks , 2003, IEEE Trans. Signal Process..

[40]  GuoSong,et al.  Spread spectrum-based high embedding capacity watermarking method for audio signals , 2015 .

[41]  Yi Hu,et al.  Evaluation of Objective Quality Measures for Speech Enhancement , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[42]  R.L. Lagendijk,et al.  Amplitude Scale Estimation for Quantization-Based Watermarking , 2006, IEEE Transactions on Signal Processing.

[43]  Nazim Fatès,et al.  StirMark benchmark: audio watermarking attacks , 2001, Proceedings International Conference on Information Technology: Coding and Computing.

[44]  Walter Bender,et al.  Echo Hiding , 1996, Information Hiding.

[45]  Jinwoo Hong,et al.  New echo embedding technique for robust and imperceptible audio watermarking , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[46]  D.P. Skinner,et al.  The cepstrum: A guide to processing , 1977, Proceedings of the IEEE.

[47]  R.W. Schafer,et al.  From frequency to quefrency: a history of the cepstrum , 2004, IEEE Signal Processing Magazine.

[48]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[49]  Yôiti Suzuki,et al.  Time-spread echo method for digital audio watermarking , 2005, IEEE Transactions on Multimedia.

[50]  Zheng Liu,et al.  Audio watermarking techniques using sinusoidal patterns based on pseudorandom sequences , 2003, IEEE Trans. Circuits Syst. Video Technol..

[51]  Vrizlynn L. L. Thing,et al.  Time-Spread Echo-Based Audio Watermarking With Optimized Imperceptibility and Robustness , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[52]  David Megías,et al.  Efficient self-synchronised blind audio watermarking system based on time domain and FFT amplitude modification , 2010, Signal Process..

[53]  Henrique S. Malvar,et al.  Improved spread spectrum: a new modulation technique for robust watermarking , 2003, IEEE Trans. Signal Process..

[54]  Gregory W. Wornell,et al.  Quantization index modulation: A class of provably good methods for digital watermarking and information embedding , 2001, IEEE Trans. Inf. Theory.

[55]  Masashi Unoki,et al.  Speech Watermarking Method Based on Formant Tuning , 2015, IEICE Trans. Inf. Syst..

[56]  Xiangyang Wang,et al.  A New Adaptive Digital Audio Watermarking Based on Support Vector Regression , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[57]  Minghua Chen,et al.  A fragile watermark error detection scheme for wireless video communications , 2005, IEEE Transactions on Multimedia.

[58]  B. Girod,et al.  A blind watermarking scheme based on structured codebooks , 2000 .

[59]  Ee-Leng Tan,et al.  Robust SVD-Based Audio Watermarking Scheme With Differential Evolution Optimization , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[60]  Masashi Unoki,et al.  Method of digital-audio watermarking based on cochlear delay characteristics , 2013 .

[61]  Masashi Unoki,et al.  Tampering Detection Scheme for Speech Signals using Formant Enhancement based Watermarking , 2015, J. Inf. Hiding Multim. Signal Process..

[62]  Jiwu Huang,et al.  Efficiently self-synchronized audio watermarking for assured audio data transmission , 2005, IEEE Transactions on Broadcasting.

[63]  渡辺馨 Objective measurement method of audio quality in accordance with ITU-R Recommendation BS. 1387 , 2001 .

[64]  Thomas S. Huang,et al.  Robust optimum detection of transform domain multiplicative watermarks , 2003, IEEE Trans. Signal Process..

[65]  Ted Painter,et al.  Audio Signal Processing and Coding , 2007 .

[66]  Kazuhiro Kondo Multimedia Information Hiding Technologies and Methodologies for Controlling Data , 2012 .

[67]  Darko Kirovski,et al.  The Replacement Attack , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[68]  Meriem Jaïdane,et al.  Audio watermarking: a way to stationnarize audio signals , 2005, IEEE Transactions on Signal Processing.

[69]  Ingemar J. Cox,et al.  Watermarking Is Not Cryptography , 2006, IWDW.

[70]  Rui Yang,et al.  Geometric Invariant Audio Watermarking Based on an LCM Feature , 2011, IEEE Transactions on Multimedia.

[71]  Pierre Duhamel,et al.  Audio watermarking under desynchronization and additive noise attacks , 2006, IEEE Transactions on Signal Processing.

[72]  Markus G. Kuhn,et al.  Information hiding-a survey , 1999, Proc. IEEE.

[73]  Masashi Unoki,et al.  Method of Audio Watermarking Based on Adaptive Phase Modulation , 2016, IEICE Trans. Inf. Syst..

[74]  Frank Boland,et al.  Watermarking digital images for copyright protection , 1995 .

[75]  Song Guo,et al.  Spread Spectrum-Based High Embedding Capacity Watermarking Method for Audio Signals , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[76]  Xiangyang Xue,et al.  Localized audio watermarking technique robust against time-scale modification , 2006, IEEE Trans. Multim..

[77]  RECOMMENDATION ITU-R BS.1387-1 - Method for objective measurements of perceived audio quality , 2002 .

[78]  Akira Nishimura Reversible and Robust Audio Watermarking Based on Spread Spectrum and Amplitude Expansion , 2014, IWDW.

[79]  Akira Nishimura Audio watermarking based on subband amplitude modulation , 2010 .

[80]  Abdel-Ouahab Boudraa,et al.  Audio Watermarking Via EMD , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[81]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[82]  Wanlei Zhou,et al.  Effective Pseudonoise Sequence and Decoding Function for Imperceptibility and Robustness Enhancement in Time-Spread Echo-Based Audio Watermarking , 2011, IEEE Transactions on Multimedia.

[83]  Ahmed H. Tewfik,et al.  Multimedia data-embedding and watermarking technologies , 1998, Proc. IEEE.

[84]  Max H. M. Costa,et al.  Writing on dirty paper , 1983, IEEE Trans. Inf. Theory.

[85]  Pierre Moulin,et al.  The zero-rate spread-spectrum watermarking game , 2003, IEEE Trans. Signal Process..

[86]  Song Guo,et al.  Patchwork-Based Audio Watermarking Method Robust to De-synchronization Attacks , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[87]  Ingemar J. Cox,et al.  Secure spread spectrum watermarking for multimedia , 1997, IEEE Trans. Image Process..

[88]  Darko Kirovski,et al.  Spread-spectrum watermarking of audio signals , 2003, IEEE Trans. Signal Process..

[89]  Joseph A. O'Sullivan,et al.  Information-theoretic analysis of information hiding , 2003, IEEE Trans. Inf. Theory.