Shadow theatre

Shadow theatre is a genre of performance art in which the actors are only visible as shadows projected on the screen. The goal of this study is to generate animated characters, the shadows of which match a sequence of target silhouettes. This poses several challenges. The motion of multiple characters are carefully coordinated to form a target silhouette on the screen, and each character's pose should be stable, balanced, and plausible. The resulting character animation should be smooth and coherent spatially and temporally. We formulate the problem as nonlinear constrained optimization with objectives, which were designed to generate plausible human motions. Our optimization algorithm was primarily inspired by the heuristic strategies of professional shadow theatre actors. Their know-how was studied and then incorporated into our optimization formulation. We demonstrate the effectiveness of our approach with a variety of target silhouettes and 3D fabrication of the results.

[1]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[2]  Daniel Cohen-Or,et al.  3D collage: expressive non-realistic modeling , 2007, NPAR '07.

[3]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[4]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[5]  Niloy J. Mitra,et al.  Shadow art , 2009, SIGGRAPH 2009.

[6]  Henrik Wann Jensen,et al.  Global Illumination using Photon Maps , 1996, Rendering Techniques.

[7]  Sebastian Thrun,et al.  SCAPE: shape completion and animation of people , 2005, SIGGRAPH '05.

[8]  Jinxiang Chai,et al.  Intuitive Interactive Human-Character Posing with Millions of Example Poses , 2011, IEEE Computer Graphics and Applications.

[9]  Donald P. Greenberg,et al.  A user interface for interactive cinematic shadow design , 2002, SIGGRAPH.

[10]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[11]  Yaser Sheikh,et al.  Leveraging the talent of hand animators to create three-dimensional animation , 2009, SCA '09.

[12]  Alejo Hausner,et al.  Simulating decorative mosaics , 2001, SIGGRAPH.

[13]  Cristian Sminchisescu,et al.  Human Pose Estimation from Silhouettes - A Consistent Approach Using Distance Level Sets , 2002, WSCG.

[14]  Michael J. Black,et al.  Estimating human shape and pose from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Jehee Lee,et al.  Tiling Motion Patches , 2013, IEEE Transactions on Visualization and Computer Graphics.

[16]  Marc Alexa,et al.  ShadowPix: Multiple Images from Self Shadowing , 2012, Comput. Graph. Forum.

[17]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[18]  Michael Wimmer,et al.  Freeform Shadow Boundary Editing , 2013, Comput. Graph. Forum.

[19]  Donald P. Greenberg,et al.  Modeling the interaction of light between diffuse surfaces , 1984, SIGGRAPH.

[20]  Ying He,et al.  A Sketching Interface for Sitting Pose Design in the Virtual Environment , 2012, IEEE Transactions on Visualization and Computer Graphics.

[21]  Lance Williams,et al.  Casting curved shadows on curved surfaces , 1978, SIGGRAPH.

[22]  Eftychios Sifakis,et al.  Comprehensive biomechanical modeling and simulation of the upper body , 2009, TOGS.

[23]  Martin Guay,et al.  The line of action , 2013, ACM Trans. Graph..

[24]  Neil D. Lawrence,et al.  Gaussian Process Latent Variable Models for Human Pose Estimation , 2007, MLMI.

[25]  Ronald Poppe,et al.  Evaluating Example-based Pose Estimation: Experiments on the HumanEva Sets , 2007 .

[26]  Je-hee Lee,et al.  A Hierarchical Approach to Motion Analysis and Synthesis for Articulated Figures , 2000 .

[27]  Mannes Poel,et al.  Comparison of silhouette shape descriptors for example-based human pose recovery , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[28]  Turner Whitted,et al.  An improved illumination model for shaded display , 1979, CACM.

[29]  Fabio Pellacini,et al.  Jigsaw image mosaics , 2002, ACM Trans. Graph..

[30]  David Salesin,et al.  A sketching interface for articulated figure animation , 2006, SIGGRAPH 2006.

[31]  Ankur Agarwal,et al.  3D human pose from silhouettes by relevance vector regression , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[32]  Arthur Appel,et al.  Some techniques for shading machine renderings of solids , 1968, AFIPS Spring Joint Computing Conference.

[33]  T. Kanade,et al.  Reconstructing 3D Human Pose from 2D Image Landmarks , 2012, ECCV.

[34]  Dinesh K. Pai,et al.  EigenSkin: real time large deformation character skinning in hardware , 2002, SCA '02.

[35]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[36]  Franklin C. Crow,et al.  Shadow algorithms for computer graphics , 1977, SIGGRAPH.

[37]  Takeo Igarashi,et al.  As-rigid-as-possible shape manipulation , 2005, ACM Trans. Graph..