Action synopsis: pose selection and illustration

Illustrating motion in still imagery for the purpose of summary, abstraction and motion description is important for a diverse spectrum of fields, ranging from arts to sciences. In this paper, we introduce a method that produces an action synopsis for presenting motion in still images. The method carefully selects key poses based on an analysis of a skeletal animation sequence, to facilitate expressing complex motions in a single image or a small number of concise views. Our approach is to embed the high-dimensional motion curve in a low-dimensional Euclidean space, where the main characteristics of the skeletal action are kept. The lower complexity of the embedded motion curve allows a simple iterative method which analyzes the curve and locates significant points, associated with the key poses of the original motion. We present methods for illustrating the selected poses in an image as a means to convey the action. We applied our methods to a variety of motions of human actions given either as 3D animation sequences or as video clips, and generated images that depict their synopsis.

[1]  Alston S. Householder,et al.  A note on multidimensional psychophysical analysis , 1941 .

[2]  W. Torgerson.  Multidimensional scaling: I. Theory and method , 1952 .

[3]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. II , 1962 .

[4]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[5]  V. Mcgee.  Multidimensional Scaling Of N Sets Of Similarity Measures: A Nonmetric Individual Differences Approach. , 1968, Multivariate behavioral research.

[6]  Urs Ramer,et al.  An iterative procedure for the polygonal approximation of plane curves , 1972, Comput. Graph. Image Process..

[7]  D. F. Fisher,et al.  Perception and pictorial representation , 1979 .

[8]  E. Aronson,et al.  Theory and method , 1985 .

[9]  H. Edgerton,et al.  Stopping Time: The Photographs of Harold Edgerton , 1987 .

[10]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[11]  Aaron F. Bobick,et al.  Recognition of human body motion using phase space constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[12]  Walter Bender,et al.  Salient Stills: Process and Practice , 1996, IBM Syst. J..

[13]  Richard Szeliski,et al.  Creating full view panoramic image mosaics and environment maps , 1997, SIGGRAPH.

[14]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[15]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[16]  Daniel Thalmann,et al.  Key-posture extraction out of human motion data , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  Lihi Zelnik-Manor,et al.  Event-based analysis of video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Jonathan Foote,et al.  Summarizing video using non-negative similarity matrix factorization , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[19]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[20]  J. Cutting.  Representing Motion in a Static Image: Constraints and Parallels in Art, Science, and Popular Culture , 2002, Perception.

[21]  Patrick Pérez,et al.  Rapid Summarisation and Browsing of Video Sequences , 2002, BMVC.

[22]  Toby Howard,et al.  Interactive reconstruction of virtual environments from video sequences , 2003, Computers & graphics.

[23]  Feng Liu,et al.  3D motion retrieval with motion index tree , 2003, Comput. Vis. Image Underst..

[24]  Bobby Bodenheimer,et al.  An evaluation of a cost metric for selecting transitions between motion segments , 2003, SCA '03.

[25]  Stefan Carlsson,et al.  Pose-based clustering in action sequences , 2003, First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, 2003. HLK 2003..

[26]  Kunio Kondo,et al.  Keyframes Extraction Method for Motion Capture Data , 2004 .

[27]  Sung Yong Shin,et al.  Example‐based motion cloning , 2004, Comput. Animat. Virtual Worlds.

[28]  Ahmed M. Elgammal,et al.  Gait style and gait content: bilinear models for gait recognition using gait re-sampling , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[29]  Aaron Hertzmann,et al.  Style-based inverse kinematics , 2004, SIGGRAPH 2004.

[30]  Larry S. Davis,et al.  Gait Recognition Using Image Self-Similarity , 2004, EURASIP J. Adv. Signal Process..

[31]  Michael Gleicher,et al.  Automated extraction and parameterization of motions in large data sets , 2004, SIGGRAPH 2004.

[32]  Jessica K. Hodgins,et al.  Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces , 2004, SIGGRAPH 2004.

[33]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[34]  Patrick Gros,et al.  A Geometrical Key-Frame Selection Method Exploiting Dominant Motion Estimation in Video , 2004, CIVR.