Artistic Content Representation and Modelling based on Visual Style Features

This thesis aims to understand visual style in the context of computer science, using traditionally intangible artistic properties to enhance existing content manipulation algorithms and develop new content creation methods. The developed algorithms can be used to apply extracted properties to other drawings automatically; transfer a selected style; categorise images based upon perceived style; build 3D models using style features from concept artwork; and other style-based actions that change our perception of an object without changing our ability to recognise it. The research in this thesis aims to provide the style manipulation abilities that are missing from modern digital art creation pipelines.

[1]  Dov Dori,et al.  Sparse Pixel Vectorization: An Algorithm and Its Performance Evaluation , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  John S. Gero,et al.  Visual style: Qualitative and context-dependent categorization , 2006, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[3]  Zhiyong Huang,et al.  Interactive 3D modeling using only one image , 2000, VRST '00.

[4]  Ayellet Tal,et al.  Hierarchical mesh decomposition using fuzzy clustering and cuts , 2003, ACM Trans. Graph..

[5]  Gueesang Lee,et al.  Morphological gradient applied to new active contour model for color image segmentation , 2012, ICUIMC.

[6]  Georgios Tziritas,et al.  Maximum likelihood texture classification and Bayesian texture segmentation using discrete wavelet frames , 1997, Proceedings of 13th International Conference on Digital Signal Processing.

[7]  Luiz Velho,et al.  Sketching Variational Hermite-RBF Implicits , 2010, SBIM.

[8]  Tong-Yee Lee,et al.  Skeleton extraction by mesh contraction , 2008, SIGGRAPH 2008.

[9]  Golam Ashraf,et al.  Learning Primitive Shapes in Cartoon Designs , 2011, Trans. Mach. Learn. Data Min..

[10]  Yaonan Wang.  Texture Classification Using Discrete Multiwavelet Transform , 2022 .

[11]  Ming-Ming Cheng.  Curve Structure Extraction for Cartoon Images , 1996 .

[12]  Michael Gleicher,et al.  Retargetting motion to new characters , 1998, SIGGRAPH.

[13]  Vladimir Kolmogorov,et al.  Feature Correspondence Via Graph Matching: Models and Global Optimization , 2008, ECCV.

[14]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[15]  F. Gregory Ashby,et al.  Multidimensional Models of Perception and Cognition , 2014 .

[16]  Masayuki Nakajima,et al.  Single-View Sketch Based Surface Modeling , 2009, IEICE Trans. Inf. Syst..

[17]  Robert L. Goldstone,et al.  The development of features in object concepts , 1998, Behavioral and Brain Sciences.

[18]  Gilles F. Houle,et al.  Handwriting stroke extraction using a new XYTC transform , 2001, Proceedings of Sixth International Conference on Document Analysis and Recognition.

[19]  J. Suh.  Stroke Extraction from GrayScale Character Image , 2007 .

[20]  Markus H. Gross,et al.  Topology-driven vectorization of clean line drawings , 2013, ACM Trans. Graph..

[21]  Nuria Oliver,et al.  Curve Analogies , 2002, Rendering Techniques.

[22]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[23]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[24]  Adam Finkelstein,et al.  Line drawings from 3D models , 2005, SIGGRAPH '08.

[25]  Arvin Agah,et al.  A survey of sketch-based 3-D modeling techniques , 2009, Interact. Comput..

[26]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, CAIP.

[27]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[28]  Martin D. Levine,et al.  Recovering parametric geons from multiview range data , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Faramarz F. Samavati,et al.  Stroke Extraction and Classification for Mesh Inflation , 2010, SBIM.

[30]  Steven J. Gortler,et al.  A perception-based color space for illumination-invariant image processing , 2008, ACM Trans. Graph..

[31]  Stephen DiVerdi,et al.  A Sketch-based Interface for Photo Pop-up , 2009, SBIM.

[32]  Yaser Sheikh,et al.  Three-dimensional proxies for hand-drawn characters , 2012, TOGS.

[33]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Takeo Igarashi,et al.  Smooth meshes for sketch-based freeform modeling , 2003, I3D '03.

[35]  Philip H. S. Torr,et al.  VideoTrace: rapid interactive scene modelling from video , 2007, SIGGRAPH 2007.

[36]  Robert L. Goldstone.  Unitization during category learning. , 2000, Journal of experimental psychology. Human perception and performance.

[37]  Lior Shamir,et al.  Impressionism, expressionism, surrealism: Automated recognition of painters and schools of art , 2010, TAP.

[38]  Natapon Pantuwong,et al.  Skeleton-growing: a vector-field-based 3D curve-skeleton extraction algorithm , 2010, SA '10.

[39]  J. Hughes,et al.  SmoothSketch: 3D free-form shapes from complex sketches , 2006, ACM Trans. Graph..

[40]  Aaron Hertzmann,et al.  Style-based inverse kinematics , 2004, ACM Trans. Graph..

[41]  Philip J. Willis,et al.  Automatic rigging for animation characters with 3D silhouette , 2009, Comput. Animat. Virtual Worlds.

[42]  Ariel Shamir,et al.  Seam Carving for Content-Aware Image Resizing , 2007, ACM Trans. Graph..

[43]  Wan-Chun Ma,et al.  Automatic animation skeleton using repulsive force field , 2003, 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings..

[44]  Edward R. Dougherty,et al.  An introduction to morphological image processing , 1992 .

[45]  Young J. Kim,et al.  Interactive Hausdorff distance computation for general polygonal models , 2009, SIGGRAPH '09.

[46]  Serge J. Belongie,et al.  Context based object categorization: A critical survey , 2010, Comput. Vis. Image Underst..

[47]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[48]  Yi Hu,et al.  Matching of characters in scene images by using local shape feature vectors , 2003, 12th International Conference on Image Analysis and Processing, 2003.Proceedings..

[49]  Ilya Baran,et al.  Automatic rigging and animation of 3D characters , 2007, SIGGRAPH 2007.

[50]  Gabriella Sanniti di Baja,et al.  Euclidean skeleton via centre-of-maximal-disc extraction , 1993, Image Vis. Comput..

[51]  Wolfgang Effelsberg,et al.  Automatic recognition of film genres , 1995, MULTIMEDIA '95.

[52]  Faramarz F. Samavati,et al.  Sketch-based subdivision models , 2009, SBIM '09.

[53]  Michael Gleicher,et al.  Building efficient, accurate character skins from examples , 2003, ACM Trans. Graph..

[54]  Natapon Pantuwong,et al.  A novel template‐based automatic rigging algorithm for articulated‐character animation , 2012, Comput. Animat. Virtual Worlds.

[55]  Manuel Contero,et al.  Smart Sketch System for 3D Reconstruction Based Modeling , 2003, Smart Graphics.

[56]  Bjarne Sode Funch.  The psychology of art appreciation , 1997 .

[57]  Thomas A. Funkhouser,et al.  Shape-based retrieval and analysis of 3d models , 2005, CACM.

[58]  R. J. Bhiwani,et al.  Texture Based Pattern Classification , 2010 .

[59]  M. Punithavalli,et al.  A Survey on Skeletons in Digital Image Processing , 2009, 2009 International Conference on Digital Image Processing.

[60]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[61]  Nicu Sebe,et al.  Wavelet based texture classification , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[62]  M. Emre Celebi,et al.  Effective initialization of k-means for color quantization , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[63]  Chris Hecker,et al.  Real-time motion retargeting to highly varied user-created morphologies , 2008, SIGGRAPH 2008.

[64]  Christof Lutteroth,et al.  Poisson Blended Exemplar-based Texture Completion , 2014, ACSC.

[65]  Adam Krzyzak,et al.  Piecewise Linear Skeletonization Using Principal Curves , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[66]  Ravin Balakrishnan,et al.  ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models , 2008, UIST '08.

[67]  Touradj Ebrahimi,et al.  MESH: measuring errors between surfaces using the Hausdorff distance , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[68]  P. Selinger.  Potrace : a polygon-based tracing algorithm , 2003 .

[69]  Zhao-Ying Zhou,et al.  A complex nonlinear exponential autoregressive model approach to shape recognition using neural networks , 2000, IEEE Trans. Instrum. Meas..

[70]  Chen Mao,et al.  Sketching-out virtual humans: from 2D storyboarding to immediate 3D character animation , 2006, Advances in Computer Entertainment Technology.

[71]  B. Cyganek.  An Introduction to 3D Computer Vision Techniques and Algorithms , 2009 .

[72]  Cindy Grimm,et al.  Just DrawIt: a 3D sketching system , 2012, SBIM '12.

[73]  Petros Maragos,et al.  Morphological skeleton representation and coding of binary images , 1984, IEEE Trans. Acoust. Speech Signal Process..

[74]  Hiromasa Suzuki,et al.  3D Sketch: Sketch-based Model Reconstruction and Rendering , 2000, Workshop on Geometric Modeling.

[75]  Dave Elliman.  A Really Useful Vectorization Algorithm , 1999, GREC.

[76]  Byeungwoo Jeon,et al.  An efficient method of image identification by combining image features , 2009, International Conference on Ubiquitous Information Management and Communication.

[77]  Christian Rössl,et al.  Animation collage , 2007, SCA '07.

[78]  Mario Costa Sousa,et al.  Sketch-based modeling: A survey , 2009, Comput. Graph..

[79]  M. Sivalingamaiah,et al.  Texture Segmentation Using Multichannel Gabor Filtering , 2012 .

[80]  Tainchi Lu,et al.  A Rapid Mesh Fusion Method to Create 3D Virtual Characters in Games , 2009, 2009 Fourth International Conference on Computer Sciences and Convergence Information Technology.

[81]  Daniel Cohen-Or,et al.  3-Sweep , 2013, ACM Trans. Graph..

[82]  Li-Yi Wei,et al.  Deterministic texture analysis and synthesis using tree structure vector quantization , 1999, XII Brazilian Symposium on Computer Graphics and Image Processing (Cat. No.PR00481).

[83]  Xiongcai Cai,et al.  Perceptual Evaluation of Automatic 2.5D Cartoon Modelling , 2012, PKAW.

[84]  Oliver Deussen,et al.  Wang Tiles for image and texture generation , 2003, ACM Trans. Graph..

[85]  Pascal Barla,et al.  Diffusion curves: a vector representation for smooth-shaded images , 2008, SIGGRAPH 2008.

[86]  Margrit Betke,et al.  Empathic painting: interactive stylization through observed emotional state , 2006, NPAR.

[87]  Charlie C. L. Wang,et al.  Virtual human modeling from photographs for garment industry , 2003, Comput. Aided Des..

[88]  Lakshman Prasad,et al.  Morphological Analysis of Shapes , 1997 .

[89]  Ken-ichi Anjyo,et al.  Fourier principles for emotion-based human figure animation , 1995, SIGGRAPH.

[90]  Jochen Lang,et al.  Skeleton pruning by contour approximation and the integer medial axis transform , 2012, Comput. Graph..

[91]  Hua Huang,et al.  EXCOL: An EXtract-and-COmplete Layering Approach to Cartoon Animation Reusing , 2012, IEEE Transactions on Visualization and Computer Graphics.

[92]  Philip Buchanan,et al.  Transferring Characteristic Proportions to Modify the Artistic Style of Cartoons , 2012, CGI 2012.

[93]  Siddhartha Chaudhuri,et al.  A probabilistic model for component-based shape synthesis , 2012, ACM Trans. Graph..

[94]  Anil K. Jain,et al.  Unsupervised texture segmentation using Gabor filters , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[95]  Frederick W. B. Li,et al.  Feature-varying skeletonization , 2012, The Visual Computer.

[96]  Xiaofeng Mi,et al.  Abstraction of 2D shapes in terms of parts , 2009, NPAR '09.

[97]  D. Cohen-Or,et al.  Curve skeleton extraction from incomplete point cloud , 2009, SIGGRAPH 2009.

[98]  V. T. Rajan,et al.  Voronoi diagrams of polygons: A framework for shape representation , 2004, Journal of Mathematical Imaging and Vision.

[99]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[100]  Ralph R. Martin,et al.  Vectorizing Cartoon Animations , 2009, IEEE Transactions on Visualization and Computer Graphics.

[101]  Marc Alexa,et al.  FiberMesh: designing freeform surfaces with 3D curves , 2007, SIGGRAPH 2007.

[102]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[103]  R. Mukundan,et al.  Structural vectorization of raster images , 2012, IVCNZ '12.

[104]  Eike Falk Anderson.  Real-Time Character Animation for Computer Games , 2001 .

[105]  C. Taylor,et al.  Active shape models - 'Smart Snakes'. , 1992 .

[106]  Ramakrishnan Mukundan,et al.  Automatic single-view character model reconstruction , 2013, SBIM '13.

[107]  Johan Wagemans,et al.  The effect of category learning on the representation of shape: dimensions can be biased but not differentiated. , 2003, Journal of experimental psychology. General.

[108]  Henrik Hagtvedt,et al.  The Perception and Evaluation of Visual Art , 2008 .

[109]  Koji Abe,et al.  Vector Image Segmentation for Content-Based Vector Image Retrieval , 2007, 7th IEEE International Conference on Computer and Information Technology (CIT 2007).

[110]  Hermann Ney,et al.  Pixel-to-Pixel Matching for Image Recognition Using Hungarian Graph Matching , 2004, DAGM-Symposium.

[111]  Joshua B. Tenenbaum,et al.  Learning style translation for the lines of a drawing , 2003, TOGS.

[112]  Yong S. Suh.  Reconstructing Polyhedral Swept Volumes from a Single-View Sketch , 2006, 2006 IEEE International Conference on Information Reuse & Integration.

[113]  Beryl Plimmer,et al.  Automated labeling of ink stroke data , 2012, SBIM '12.

[114]  Deborah Silver,et al.  Curve-Skeleton Properties, Applications, and Algorithms , 2007, IEEE Trans. Vis. Comput. Graph..

[115]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[116]  P. Wayner,et al.  Identification of artistic styles using a local statistical metric , 1991, [1991] Proceedings. The Seventh IEEE Conference on Artificial Intelligence Application.

[117]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[118]  Wei Chen,et al.  Curve reconstruction based on an interval B-spline curve , 2005, The Visual Computer.

[119]  Joshua B. Tenenbaum,et al.  Separating Style and Content with Bilinear Models , 2000, Neural Computation.

[120]  Gregory Joy,et al.  Center-cut for color-image quantization , 2005, The Visual Computer.

[121]  Sebastiano Battiato,et al.  SVG rendering of real images using data dependent triangulation , 2004, SCCG '04.

[122]  Tiow Seng Tan,et al.  Decomposing polygon meshes for interactive applications , 2001, I3D '01.

[123]  Chen Yang,et al.  Sketch-based modeling of parameterized objects , 2005, SIGGRAPH '05.

[124]  Kenneth L. Clarkson,et al.  A fast Las Vegas algorithm for triangulating a simple polygon , 1988, SCG '88.

[125]  Remco C. Veltkamp,et al.  A Survey of Content Based 3D Shape Retrieval Methods , 2004, SMI.

[126]  Rong Yang,et al.  Life-Sketch - A Framework for Sketch-Based Modelling and Animation of 3D Objects , 2010, AUIC.

[127]  Leonid M. Mestetskii.  Fat curves and representation of planar figures , 2000, Comput. Graph..

[128]  Praveen Dasigi.  Efficient Graph-based Image Matching for Recognition and Retrieval , .

[129]  Faramarz F. Samavati,et al.  Image-assisted modeling from sketches , 2010, Graphics Interface.

[130]  Rupert Paget,et al.  Texture synthesis via a noncausal nonparametric multiscale Markov random field , 1998, IEEE Trans. Image Process..

[131]  John E. Hummel,et al.  Two Roles for Attention in Shape Perception: A Structural Description Model of Visual Scrutiny , 1998 .

[132]  Takeo Igarashi,et al.  Floral diagrams and inflorescences: interactive flower modeling using botanical structural constraints , 2005, SIGGRAPH 2005.

[133]  Michel Vidal-Naquet,et al.  Visual features of intermediate complexity and their use in classification , 2002, Nature Neuroscience.

[134]  Dani Lischinski,et al.  Depixelizing pixel art , 2011 .

[135]  Paolo Cignoni,et al.  Metro: Measuring Error on Simplified Surfaces , 1998, Comput. Graph. Forum.

[136]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[137]  Benoit M. Macq,et al.  Segmentation using a region-growing thresholding , 2005, IS&T/SPIE Electronic Imaging.

[138]  Yizhou Yu,et al.  Patch-based image vectorization with automatic curvilinear feature alignment , 2009, ACM Trans. Graph..

[139]  Hongxin Zhang,et al.  Prototype Modeling from Sketched Silhouettes based on Convolution Surfaces , 2004, Comput. Graph. Forum.

[140]  Fang Liu,et al.  Stroke extraction in cartoon images using edge-enhanced isotropic nonlinear filter , 2010, VRCAI '10.

[141]  C. Steger.  SUBPIXEL-PRECISE EXTRACTION OF LINES AND EDGES , 2000 .

[142]  Matthew Stone,et al.  Visual explanations , 2010, NPAR.

[143]  I. Gauthier,et al.  Visual object understanding , 2004, Nature Reviews Neuroscience.

[144]  Xavier Granier,et al.  Improved skeleton extraction and surface generation for sketch-based modeling , 2007, GI '07.

[145]  Sankar K. Pal,et al.  A review on image segmentation techniques , 1993, Pattern Recognit..

[146]  Bartlett W. Mel.  SEEMORE: Combining Color, Shape, and Texture Histogramming in a Neurally Inspired Approach to Visual Object Recognition , 1997, Neural Computation.

[147]  Wenyu Liu,et al.  Skeleton Pruning by Contour Partitioning with Discrete Curve Evolution , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[148]  Hui Zhang,et al.  Image segmentation evaluation: A survey of unsupervised methods , 2008, Comput. Vis. Image Underst..

[149]  Andrew Nealen,et al.  RigMesh: automatic rigging for part-based shape modeling and deformation , 2012, ACM Trans. Graph..

[150]  F. Fang,et al.  3D reconstruction of polyhedral objects from single perspective projections using cubic corner , 2012 .