Adaptive Contour Classification of Comics Speech Balloons

Comic books digitization combined with subsequent comic book understanding give rise to a variety of new applications, including content reflowing, mobile reading and multi-modal search. Document understanding in this domain is challenging as comics are semi-structured documents, with semantic information shared between the graphical and textual parts. Speech balloon contour analysis reveals the speech tone which is an essential step towards a fully automatic comics understanding. In this paper we present the first approach for classifying speech balloon in scanned comic books where we separate and analyze their contour variations to classify them as “smooth” (normal speech), “wavy” (thought) or “zigzag” (exclamation). The experiments show a global accuracy classification of 85.2 % on a wide variety of balloons from the eBDtheque dataset.

[1]  Martin Lopatka,et al.  Automated shape annotation for illicit tablet preparations: a contour angle based classification from digital images. , 2013, Science & justice : journal of the Forensic Science Society.

[2]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[3]  Ralph Roskies,et al.  Fourier Descriptors for Plane Closed Curves , 1972, IEEE Transactions on Computers.

[4]  Jürgen Beyerer,et al.  Fast Invariant Contour-Based Classification of Hand Symbols for HCI , 2009, CAIP.

[5]  Joost van de Weijer,et al.  An Active Contour Model for Speech Balloon Detection in Comics , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[6]  Miroslaw Bober,et al.  MPEG-7 visual shape descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[7]  Guojun Lu,et al.  Review of shape representation and description techniques , 2004, Pattern Recognit..

[8]  Alain Bouju,et al.  eBDtheque: A Representative Database of Comics , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[9]  Tsuhan Chen,et al.  Trademark retrieval using contour-skeleton stroke classification , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[10]  Remco C. Veltkamp,et al.  Features in Content-based Image Retrieval Systems: a Survey , 1999, State-of-the-Art in Content-Based Image and Video Retrieval.

[11]  Mandyam D. Srinath,et al.  Partial Shape Classification Using Contour Matching in Distance Transformation , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  R. W. King,et al.  An automatic intonation tone contour labelling and classification algorithm , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[13]  Christian Cenker.  Wavelet contour classification , 1996 .

[14]  Remco C. Veltkamp,et al.  Content-based image retrieval systems: A survey , 2000 .

[15]  Sadegh Abbasi,et al.  Shape similarity retrieval under affine transforms , 2002, Pattern Recognit..

[16]  Eamonn J. Keogh,et al.  LB_Keogh supports exact indexing of shapes under rotation invariance with arbitrary representations and distance measures , 2006, VLDB.

[17]  Josef Kittler,et al.  Curvature scale space image in shape similarity retrieval , 1999, Multimedia Systems.

[18]  R. Mukundan,et al.  Moment Functions in Image Analysis: Theory and Applications , 1998 .

[19]  Zhiyong Wang,et al.  Shape based leaf image retrieval , 2003 .

[20]  Boaz J. Super,et al.  Classification of contour shapes using class segment sets , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Jean-Christophe Burie,et al.  Panel and Speech Balloon Extraction from Comic Books , 2012, 2012 10th IAPR International Workshop on Document Analysis Systems.

[22]  Gerald Kühne,et al.  Contour-based classification of video objects , 2001, IS&T/SPIE Electronic Imaging.

[23]  Kohei Arai,et al.  Method for Real Time Text Extraction of Digital Manga Comic , 2011 .

[24]  Gerald Kühne,et al.  Motion-based segmentation and contour-based classification of video objects , 2001, MULTIMEDIA '01.