Recognizing Cartoon Image Gestures for Retrieval and Interactive Cartoon Clip Synthesis

In this paper, we propose a new method to recognize gestures of cartoon images with two practical applications, i.e., content-based cartoon image retrieval and interactive cartoon clip synthesis. Upon analyzing the unique properties of four types of features including global color histogram, local color histogram (LCH), edge feature (EF), and motion direction feature (MDF), we propose to employ different features for different purposes and in various phases. We use EF to define a graph and then refine its local structure by LCH. Based on this graph, we adopt a transductive learning algorithm to construct local patches for each cartoon image. A spectral method is then proposed to optimize the local structure of each patch and then align these patches globally. MDF is fused with EF and LCH and a cartoon gesture space is constructed for cartoon image gesture recognition. We apply the proposed method to content-based cartoon image retrieval and interactive cartoon clip synthesis. The experiments demonstrate the effectiveness of our method.

[1]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[2]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Jean-Daniel Fekete,et al.  TicTacToon: a paperless system for professional 2D animation , 1995, SIGGRAPH.

[4]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[5]  Rama Chellappa,et al.  Accuracy vs Efficiency Trade-offs in Optical Flow Algorithms , 1996, Comput. Vis. Image Underst..

[6]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[8]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[9]  Alistair Sutherland,et al.  Dynamic gesture recognition using PCA with multiscale theory and HMM , 2001, International Symposium on Multispectral Image Processing and Pattern Recognition.

[10]  Harry Shum,et al.  Speech-driven cartoon animation with emotions , 2001, MULTIMEDIA '01.

[11]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[12]  Narendra Ahuja,et al.  Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Harry Shum,et al.  Motion texture: a two-level statistical model for character motion synthesis , 2002, ACM Trans. Graph..

[14]  Alexander Kort,et al.  Computer aided inbetweening , 2002, NPAR '02.

[15]  Harry Shum,et al.  PicToon: a personalized image-based cartoon system , 2002, MULTIMEDIA '02.

[16]  Riad I. Hammoud,et al.  Estimating the photorealism of images: distinguishing paintings from photographs , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Bernhard Schölkopf,et al.  Ranking on Data Manifolds , 2003, NIPS.

[18]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[19]  Miki Haseyama,et al.  A cartoon character retrieval system including trainable scheme , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[20]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[21]  Miki Haseyama,et al.  A trainable retrieval system for cartoon character images , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[22]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[23]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[24]  Bobby Bodenheimer,et al.  Cartoon textures , 2004, SCA '04.

[25]  Jingrui He,et al.  Manifold-ranking based image retrieval , 2004, MULTIMEDIA '04.

[26]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[27]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Ashish Kapoor,et al.  Mixture of Gaussian Processes for Combining Multiple Modalities , 2005, Multiple Classifier Systems.

[29]  Alexei A. Efros,et al.  Putting Objects in Perspective , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[30]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[31]  Stefano Soatto,et al.  Fast Human Pose Estimation using Appearance and Motion via Multi-Dimensional Boosting Regression , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[33]  Ankita Kumar,et al.  Support Kernel Machines for Object Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[34]  Feiping Nie,et al.  Neighborhood MinMax Projections , 2007, IJCAI.

[35]  Zhongfei Zhang,et al.  Effective Image Retrieval Based on Hidden Concept Discovery in Image Database , 2007, IEEE Transactions on Image Processing.

[36]  Yueting Zhuang,et al.  Perspective‐aware cartoon clips synthesis , 2008, Comput. Animat. Virtual Worlds.

[37]  Dacheng Tao,et al.  Discriminative Locality Alignment , 2008, ECCV.

[38]  Yi Yang,et al.  Mining Semantic Correlation of Heterogeneous Multimedia Data for Cross-Media Retrieval , 2008, IEEE Transactions on Multimedia.

[39]  Jong-Min Kim,et al.  Three Dimensional Gesture Recognition Using PCA of Stereo Images and Modified Matching Algorithm , 2008, 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery.

[40]  Mohiuddin Ahmad,et al.  Human action recognition using shape and CLG-motion flow from multi-view image sequences , 2008, Pattern Recognit..

[41]  Xuelong Li,et al.  Image categorization: Graph edit distance+edge direction histogram , 2008, Pattern Recognit..

[42]  Chao-Hung Lin,et al.  Animation Key-Frame Extraction and Simplification Using Deformation Analysis , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Yi Yang,et al.  Harmonizing Hierarchical Manifolds for Multimedia Document Semantics Understanding and Cross-Media Retrieval , 2008, IEEE Transactions on Multimedia.

[44]  Xuelong Li,et al.  Modality Mixture Projections for Semantic Video Event Detection , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Xuelong Li,et al.  Bayesian Tensor Approach for 3-D Face Modeling , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[46]  Yi Yang,et al.  Ranking with local regression and global alignment for cross media retrieval , 2009, ACM Multimedia.

[47]  Xuelong Li,et al.  Geometric Mean for Subspace Selection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Feiping Nie,et al.  Nonlinear Dimensionality Reduction with Local Spline Embedding , 2009, IEEE Transactions on Knowledge and Data Engineering.

[49]  Sebastian Nowozin,et al.  On feature combination for multiclass object classification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[50]  Feiping Nie,et al.  Interactive Natural Image Segmentation via Spline Regression , 2009, IEEE Transactions on Image Processing.

[51]  Yi Yang,et al.  Retrieval based interactive cartoon synthesis via unsupervised bi-distance metric learning , 2009, ACM Multimedia.

[52]  Dacheng Tao,et al.  Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval , 2010, IEEE Transactions on Image Processing.

[53]  Wenhua Wang,et al.  Local and Global Regressive Mapping for Manifold Learning with Out-of-Sample Extrapolation , 2010, AAAI.

[54]  Yi Yang,et al.  Image Clustering Using Local Discriminant Models and Global Integration , 2010, IEEE Transactions on Image Processing.