Dive into the movie: an instant casting and immersive experience in the story

Our research project, Dive into Movie (DIM) aims to build a new genre of interactive entertainment which enables anyone to easily participate in a movie by assuming a role and enjoying an embodied, first-hand theater experience. This is specifically accomplished by replacing the original roles of the precreated traditional movie with user created, high-realism, 3-D CG characters. DIM movie is in some sense a hybrid entertainment form, somewhere between a game and storytelling. We hope that DIM movies might enhance interaction and offer more dramatic presence, engagement, and fun for the audience. In DIM movie, audiences can experience highrealism 3-D CG character action with individualized facial characteristics, expression, gait and voice. The DIM system has two key features: First, it can full-automatically create a CG character in a few minutes from capturing the face, body, gait and voice feature of a user and generating her/his corresponding CG animation, to inserting the individualized CG character into the movie in real-time which do not cause any discomfort to the participant; Second, the DIM system makes it possible for multiple participants to take part in a movie at the same time in different roles, such as a family, a circle of friends, etc. And also DIM project proposes a panoramic image capture/projection system and 3D sound field capture/playback system from performer, Äôs view and standing point.