Fast intelligent watermarking of heterogeneous image streams through mixture modeling of PSO populations

In intelligent watermarking (IW), evolutionary computing (EC) is employed in order to automatically set the embedding parameters of digital watermarking systems for each image. However, the computational complexity of EC techniques makes IW unfeasible for large scale applications involving heterogeneous images. In this paper, we propose a dynamic particle swarm optimization (DPSO) technique which relies on a memory of Gaussian mixture models (GMMs) of solutions in the optimization space. This technique is employed in the optimization of embedding parameters of a multi-level (robust/fragile) bi-tonal watermarking system in high data rate applications. A compact density representation of previously-found DPSO solutions is created through GMM in the optimization space, and stored in memory. Solutions are re-sampled from this memory, re-evaluated for new images and have their distribution of fitness values compared with that stored in the memory. When the distributions are similar, memory solutions are employed in a straightforward manner, avoiding costly re-optimization operations. A specialized memory management mechanism allows to maintain and adapt GMM distributions over time, as the image stream changes. This memory of GMMs allows an accurate representation of the topology of a stream of optimization problems. Consequently, new cases of optimization can be matched against previous cases more precisely (when compared with a memory of static solutions), leading to considerable decrease in computational burden. Simulation results on heterogeneous streams of images indicate that compared to full re-optimization for each document image, the proposed approach allows to decrease the computational requirement linked to EC by up to 97.7% with little impact on the accuracy for detecting watermarks. Comparable results were obtained for homogeneous streams of document images.

[1]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[2]  Kalyanmoy Deb,et al.  Dynamic multiobjective optimization problems: test cases, approximations, and applications , 2004, IEEE Transactions on Evolutionary Computation.

[3]  Min Wu,et al.  Data hiding in binary image for authentication and annotation , 2004, IEEE Transactions on Multimedia.

[4]  Tim Blackwell,et al.  Particle Swarm Optimization in Dynamic Environments , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[5]  Borko Furht,et al.  Adaptive two-level watermarking for binary document images , 2004 .

[6]  Yong Liang Guan,et al.  Perception based binary image watermarking , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[7]  Haiping Lu,et al.  Watermark embedding in DC components of DCT for binary images , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[8]  Markus G. Kuhn,et al.  Attacks on Copyright Marking Systems , 1998, Information Hiding.

[9]  Asifullah Khan,et al.  Genetic perceptual shaping: Utilizing cover image and conceivable attack information during watermark embedding , 2007, Inf. Fusion.

[10]  David G. Stork,et al.  Pattern Classification , 1973 .

[11]  C. H. Chen,et al.  Handbook of Pattern Recognition and Computer Vision , 1993 .

[12]  Jürgen Branke,et al.  Memory enhanced evolutionary algorithms for changing optimization problems , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[13]  Muhammad Arsalan,et al.  Intelligent reversible watermarking in integer wavelet domain for medical images , 2012, J. Syst. Softw..

[14]  Jufeng Dai,et al.  Image Watermarking based on Genetic Algorithm , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[15]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[16]  Yu-Chee Tseng,et al.  A secure data hiding scheme for two-color images , 2000, Proceedings ISCC 2000. Fifth IEEE Symposium on Computers and Communications.

[17]  Robert Sabourin,et al.  Intelligent watermarking systems : a survey , 2010 .

[18]  Asifullah Khan,et al.  BCH coding and intelligent watermark embedding: Employing both frequency and strength selection , 2010, Appl. Soft Comput..

[19]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[20]  Tae-Sun Choi,et al.  Machine learning based adaptive watermark decoding in view of anticipated attack , 2008, Pattern Recognit..

[21]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[22]  Christian Hennig,et al.  Methods for merging Gaussian mixture components , 2010, Adv. Data Anal. Classif..

[23]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[24]  P. Kumsawat,et al.  A new approach for optimization in image watermarking by using genetic algorithms , 2005, IEEE Transactions on Signal Processing.

[25]  S. A. Shah,et al.  Utilization of Maximum Data Hiding Capacity in Object-Based Text Document Authentication , 2006, 2006 International Conference on Intelligent Information Hiding and Multimedia.

[26]  Nathalie Japkowicz,et al.  A Novelty Detection Approach to Classification , 1995, IJCAI.

[27]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[28]  Graham J. Williams,et al.  On-Line Unsupervised Outlier Detection Using Finite Mixtures with Discounting Learning Algorithms , 2000, KDD '00.

[29]  Haiping Lu,et al.  Distance-reciprocal distortion measure for binary document images , 2004, IEEE Signal Processing Letters.

[30]  Frank Y. Shih,et al.  Genetic algorithm based methodology for breaking the steganalytic systems , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Sanjay Jain,et al.  Towards a Better Understanding of Incremental Learning , 2006, ALT.

[32]  Changhe Li,et al.  Particle swarm optimization in dynamic environments. , 2013 .

[33]  Ingemar J. Cox,et al.  Digital Watermarking , 2003, Lecture Notes in Computer Science.

[34]  Nikolas P. Galatsanos,et al.  An Analytic Distance Metric for Gaussian Mixture Models with Application in Image Retrieval , 2005, ICANN.

[35]  James J. Filliben,et al.  NIST/SEMATECH e-Handbook of Statistical Methods; Chapter 1: Exploratory Data Analysis , 2003 .

[36]  Muhammad Arsalan,et al.  Intelligent threshold selection for reversible watermarking of medical images , 2010, GECCO '10.

[37]  Sameer Singh,et al.  Novelty detection: a review - part 2: : neural network based approaches , 2003, Signal Process..

[38]  Kalyanmoy Deb,et al.  Dynamic Multiobjective Optimization Problems: Test Cases, Approximation, and Applications , 2003, EMO.

[39]  Lakhmi C. Jain,et al.  Genetic Watermarking on Spatial Domain , 2004 .

[40]  Robert Sabourin,et al.  A high throughput system for intelligent watermarking of bi-tonal images , 2011, Appl. Soft Comput..

[41]  A. Carlisle,et al.  Tracking changing extrema with adaptive particle swarm optimizer , 2002, Proceedings of the 5th Biannual World Automation Congress.

[42]  Isaac E. Lagaris,et al.  Split-Merge Incremental LEarning (SMILE) of Mixture Models , 2007, ICANN.

[43]  Shengxiang Yang,et al.  Evolutionary Computation in Dynamic and Uncertain Environments , 2007, Studies in Computational Intelligence.

[44]  Bo Zhang,et al.  Tracking concept drifting with Gaussian mixture model , 2005, Visual Communications and Image Processing.

[45]  Patrick Siarry,et al.  On the sensitivity of aggregative multiobjective optimization methods , 2008, J. Comput. Inf. Technol..

[46]  Junshui Ma,et al.  Online novelty detection on temporal sequences , 2003, KDD '03.

[47]  Frank Y. Shih,et al.  Enhancement of image watermark retrieval based on genetic algorithms , 2005, J. Vis. Commun. Image Represent..

[48]  Xin Yao,et al.  Population-Based Incremental Learning With Associative Memory for Dynamic Environments , 2008, IEEE Transactions on Evolutionary Computation.

[49]  Ingemar J. Cox,et al.  A Secure, Robust Watermark for Multimedia , 1996, Information Hiding.

[50]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[51]  Robert Sabourin,et al.  Review and Study of Genotypic Diversity Measures for Real-Coded Representations , 2012, IEEE Transactions on Evolutionary Computation.

[52]  Yazid M. Sharaiha,et al.  Binary digital image processing - a discrete approach , 1999 .

[53]  Shengxiang Yang,et al.  Triggered Memory-Based Swarm Optimization in Dynamic Environments , 2007, EvoWorkshops.

[54]  Chien-Chang Chen,et al.  A GA-BASED NEARLY OPTIMAL IMAGE AUTHENTICATION APPROACH , 2007 .

[55]  Geoffrey E. Hinton,et al.  SMEM Algorithm for Mixture Models , 1998, Neural Computation.

[56]  Ajai Jain,et al.  The Handbook of Pattern Recognition and Computer Vision , 1993 .

[57]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[58]  Robert Sabourin,et al.  Intelligent Watermarking of Document Images as a Dynamic Optimization Problem , 2010, 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[59]  T.E. Areef,et al.  Optimal transform domain watermark embedding via genetic algorithms , 2005, 2005 International Conference on Information and Communication Technology.

[60]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[61]  Pat Langley,et al.  Models of Incremental Concept Formation , 1990, Artif. Intell..

[62]  Jeng-Shyang Pan,et al.  Genetic watermarking based on transform-domain techniques , 2004, Pattern Recognit..

[63]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  Fernando Pérez-Cruz,et al.  Kullback-Leibler divergence estimation of continuous distributions , 2008, 2008 IEEE International Symposium on Information Theory.

[65]  Rongrong Ji,et al.  Genetic Algorithm Based Optimal Block Mapping Method for LSB Substitution , 2006, 2006 International Conference on Intelligent Information Hiding and Multimedia.

[66]  S. Pereira,et al.  Attacks on digital watermarks: classification, estimation based attacks, and benchmarks , 2001, IEEE Communications Magazine.

[67]  Alex ChiChung Kot,et al.  Binary Image Authentication With Tampering Localization by Embedding Cryptographic Signature and Block Identifier , 2006, IEEE Signal Processing Letters.