Semi‐Supervised Learning in Reconstructed Manifold Space for 3D Caricature Generation

Recently, automatic 3D caricature generation has attracted much attention from both the research community and the game industry. Machine learning has been proven effective in the automatic generation of caricatures. However, the lack of 3D caricature samples makes it challenging to train a good model. This paper addresses this problem by two steps. First, the training set is enlarged by reconstructing 3D caricatures. We reconstruct 3D caricatures based on some 2D caricature samples with a Principal Component Analysis (PCA)‐based method. Secondly, between the 2D real faces and the enlarged 3D caricatures, a regressive model is learnt by the semi‐supervised manifold regularization (MR) method. We then predict 3D caricatures for 2D real faces with the learnt model. The experiments show that our novel approach synthesizes the 3D caricature more effectively than traditional methods. Moreover, our system has been applied successfully in a massive multi‐user educational game to provide human‐like avatars.

[1]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[2]  Ergun Akleman,et al.  Modeling expressive 3D caricatures , 2004, SIGGRAPH '04.

[3]  Wen Gao,et al.  Efficient 3D reconstruction for face recognition , 2005, Pattern Recognit..

[4]  Wen Gao,et al.  Mapping learning in eigenspace for harmonious caricature generation , 2006, MM '06.

[5]  Hans-Peter Seidel,et al.  Creating Face Models from Vague Mental Images , 2005, SIGGRAPH '05.

[6]  Nanning Zheng,et al.  Example-based facial sketch generation with non-parametric sampling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[7]  George Wolberg,et al.  Digital image warping , 1990 .

[8]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[9]  Hiroyasu Koshimizu,et al.  3D modeling system of human face and full 3D facial caricaturing , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[10]  Alexander Fedorov,et al.  3D Caricature Generation System on the Mobile Handset Using a Single Photograph , 2007, 2007 International Conference on Parallel Processing Workshops (ICPPW 2007).

[11]  Harry Shum,et al.  Example-based caricature generation with exaggeration , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[12]  Kunihito Kato,et al.  On the detection of feature points of 3D facial image and its application to 3D facial caricature , 1999, Second International Conference on 3-D Digital Imaging and Modeling (Cat. No.PR00062).

[13]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[14]  Paul W. H. Chung,et al.  Use of Neural Networks in Automatic Caricature Generation: An Approach Based on Drawing Style Capture , 2005, IbPRIA.

[15]  Hiroyasu Koshimizu,et al.  A method for 3D face modeling and caricatured figure generation , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[16]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[17]  Hiroyasu Koshimizu,et al.  From coarse to fine correspondence of 3-D facial images and its application to 3-D facial caricaturing , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[18]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..