Robust Fingerprinting Method for Webtoon Identification in Large-Scale Databases

Webtoon, a portmanteau of web and cartoon, denotes a cartoon that has been published on a website. Recently, webtoons have become popular in the global Internet market. Unfortunately, the copyright infringement has emerged as a new challenge resulting in illegal profit gains. Moreover, it is difficult to apply watermarking to published webtoons, because they need to be watermarked prior to publication. In order to deal with a large number of published webtoons, it is necessary to identify each webtoon using fingerprints extracted from its webtoon image. In this paper, we propose an identification framework to detect copyright infringement due to the illegal copying and sharing of webtoons. The proposed identification framework consists of the following main stages: fingerprint generation, indexing, and fingerprint matching. In the fingerprint generation stage, the translation invariant and temporally localized fingerprints are created for distortion-robust identification. An inverted indexing of the database is implemented, using the visual word clustering method and the MapReduce framework, to store the fingerprints efficiently and to minimize the searching time. In addition, we propose a two-step matching process for faster implementation. Moreover, we measured the identification accuracy and the matching time of a large-scale database in the presence of various distortions. Through rigorous simulations, we achieved an identification accuracy of 97.5% within 10 s for each webtoon.

[1]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Hyewon Song,et al.  Natural scene statistics based publication classification algorithm using convolutional neural network , 2017, 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[3]  Grigorios Tsoumakas,et al.  A Comprehensive Study Over VLAD and Product Quantization in Large-Scale Image Retrieval , 2014, IEEE Transactions on Multimedia.

[4]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[5]  Ajit Kumar Mahapatra,et al.  Inverted indexes: Types and techniques , 2011 .

[6]  Jongyoo Kim,et al.  An identification framework for print-scan books in a large database , 2017, Inf. Sci..

[7]  Thierry Pun,et al.  Robust template matching for affine resistant image watermarks , 2000, IEEE Trans. Image Process..

[8]  Ioannis Pitas,et al.  Color-based descriptors for image fingerprinting , 2006, IEEE Transactions on Multimedia.

[9]  Avery Wang,et al.  An Industrial Strength Audio Search Algorithm , 2003, ISMIR.

[10]  Ke Jiang,et al.  Revisiting kernelized locality-sensitive hashing for improved large-scale image retrieval , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Shekhar Verma,et al.  Watermark based digital rights management , 2005, International Conference on Information Technology: Coding and Computing (ITCC'05) - Volume II.

[12]  Cheeyong Kim,et al.  A Study on Kitschy Characteristics and its Consumer s of Webtoon , 2015 .

[13]  Sukmoon Chang,et al.  An efficient audio fingerprint search algorithm for music retrieval , 2013, IEEE Transactions on Consumer Electronics.

[14]  Sang-Hoon Lee,et al.  A restoration method for distorted comics to improve comic contents identification , 2017, International Journal on Document Analysis and Recognition (IJDAR).

[15]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[16]  Heinrich A. van Nieuwenhuizen,et al.  The Study and Implementation of Shazam ’ s Audio Fingerprinting Algorithm for Advertisement Identification , 2011 .

[17]  Kyoungro Yoon,et al.  Sub-fingerprint masking for a robust audio fingerprinting system in a real-noise environment for portable consumer devices , 2010, 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE).

[18]  Shree K. Nayar,et al.  Ordinal Measures for Image Correspondence , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[21]  Jian Lu,et al.  Video fingerprinting for copy identification: from research to industry applications , 2009, Electronic Imaging.

[22]  Roberto Brunelli,et al.  Template Matching Techniques in Computer Vision: Theory and Practice , 2009 .

[23]  Hanjiang Lai,et al.  Supervised Hashing for Image Retrieval via Image Representation Learning , 2014, AAAI.

[24]  Ton Kalker,et al.  Feature Extraction and a Database Strategy for Video Fingerprinting , 2002, VISUAL.

[25]  Uwe D. Hanebeck,et al.  Template matching using fast normalized cross correlation , 2001, SPIE Defense + Commercial Sensing.

[26]  Doyoung Kim,et al.  Low-complexity and robust comic fingerprint method for comic identification , 2015, Signal Process. Image Commun..

[27]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[28]  Sanghoon Lee,et al.  Content-based webtoon fingerprint method , 2016, 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[29]  Shiguang Shan,et al.  Deep Supervised Hashing for Fast Image Retrieval , 2016, International Journal of Computer Vision.

[30]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..