Creating realistic animations using video

Generating realistic animations of passive dynamic systems such as rigid bodies, cloth and fluids is an important problem in computer graphics. Although several techniques for animating these phenomena have been developed, achieving the level of realism exhibited by objects in the real world has proved to be incredibly hard. This thesis presents two techniques for using video to create realistic animations of phenomena such as rigid bodies, cloth, waterfalls, streams, smoke and fire. The first technique, inverse simulation, estimates the parameters of physical simulations from video. The behavior of a physical simulator is governed by a set of parameters, typically specified by the animator. However, directly tuning the physical parameters of complex simulations like rigid bodies or cloth to achieve a desired motion is often cumbersome and nonintuitive. The inverse simulation framework uses optimization to automatically estimate the simulation parameters from video sequences obtained from simple calibration experiments (e.g., throwing a rigid body, waving a swatch of fabric). To demonstrate the power of this approach, we apply this framework to find the parameters for tumbling rigid bodies and for four different fabrics. The second technique presents a video editing framework for creating photorealistic animations of natural phenomena such as waterfalls by directly editing reference video footage. Our algorithm analyzes the dynamics and appearance of textured particles in the input video along user-specified flow lines, and synthesizes seamless infinite video by continuously generating particles along the flow lines. The user can then edit the video by manipulating flow lines from the original footage. The algorithm is simple to implement and use. We applied this technique to perform significant edits to the video. For example, we were able to change the terrain of a waterfall, add obstacles along the flow and add wind to smoke and flames, to demonstrate the editing capability of our approach. The results from these two techniques demonstrate the effectiveness of using video to improve the realism of computer animations. We hope that the methods developed in this thesis provide useful insights into how realistic animations for other domains might be designed.