Stylizing 2.5-D video

In recent years considerable interest has been given to nonphotorealistic rendering of photographs, video, and 3D models for illustrative or artistic purposes. Conventional 2D inputs such as photographs and video are easy to create and capture, while 3D models allow for a wider variety of stylization techniques, such as cross-hatching. In this paper, we propose using video with depth information (2.5-D video) to combine the advantages of 2D and 3D input. 2.5-D video is becoming increasingly easy to capture, and with the additional depth information, stylization techniques that require shape information can be applied. However, because 2.5-D video contains only limited shape information and 3D correspondence over time is unknown, it is difficult to create temporally coherent stylized animations directly from raw 2.5-D video. In this paper, we present techniques for processing 2.5-D video to overcome these drawbacks, and demonstrate several styles that can be created using these techniques.

[1]  Ken Perlin,et al.  Painterly rendering for video and interaction , 2000, NPAR '00.

[2]  Takeo Kanade,et al.  Three-dimensional scene flow , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[4]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[5]  David Salesin,et al.  Orientable textures for image-based pen-and-ink illustration , 1997, SIGGRAPH.

[6]  Paul Haeberli,et al.  Paint by numbers: abstract image representations , 1990, SIGGRAPH.

[7]  Ilan Shimshoni,et al.  Estimating the principal curvatures and the Darboux frame from real 3D range data , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[8]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[9]  Zoran Popovic,et al.  The space of human body shapes: reconstruction and parameterization from range scans , 2003, ACM Trans. Graph..

[10]  Peter Litwinowicz,et al.  Processing images and video for an impressionist effect , 1997, SIGGRAPH.

[11]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, ACM Trans. Graph..

[12]  Igor Guskov,et al.  Extracting Animated Meshes with Adaptive Motion Estimation , 2004, VMV.

[13]  Bruce Gooch,et al.  Non-photorealistic rendering , 2001 .

[14]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[15]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..

[16]  Victoria Interrante,et al.  Line direction matters: an argument for the use of principal directions in 3D line drawings , 2000, NPAR '00.

[17]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, ACM Trans. Graph..

[18]  Tobias Isenberg,et al.  High Quality Hatching , 2004, Comput. Graph. Forum.

[19]  Barbara J. Meier Painterly rendering for animation , 1996, SIGGRAPH.

[20]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[21]  Harry Shum,et al.  Video tooning , 2004, ACM Trans. Graph..

[22]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Gershon Elber,et al.  Line Art Illustrations of Parametric and Implicit Forms , 1998, IEEE Trans. Vis. Comput. Graph..

[24]  Aaron Hertzmann,et al.  Illustrating smooth surfaces , 2000, SIGGRAPH.

[25]  Li Zhang,et al.  Spacetime faces: high resolution capture for modeling and animation , 2004, SIGGRAPH 2004.

[26]  Marc Levoy,et al.  The digital Michelangelo project: 3D scanning of large statues , 2000, SIGGRAPH.

[27]  Adam Finkelstein,et al.  Real-time hatching , 2001, SIGGRAPH.

[28]  Harry Shum,et al.  Video object cut and paste , 2005, ACM Trans. Graph..

[29]  Aaron Hertzmann,et al.  Painterly rendering with curved brush strokes of multiple sizes , 1998, SIGGRAPH.

[30]  Irfan A. Essa,et al.  Image and video based painterly animation , 2004, NPAR '04.