Interactive Editing of Live Visuals

This paper describes novel concepts for the interactive composition of artistic real-time graphics, so-called live visuals. By establishing two fundamental techniques dealing with the structured media integration and the intrinsic design process, we significantly increase the efficiency of interactive editing in live visuals applications. First, we present a media manager that supports the user in both retrieval and utilization of automatically annotated digital media. The computer-assisted application of individual media items permits the interactive control of non-linear editing (NLE) of video in real-time. Second, we optimize the design process by introducing the design tree, which collects and organizes the artist’s work in an intuitive way. Design tree operations provide interactive high-level editing methods which allow for exploration, combination, reuse, and evolution of designs before and particularly during the performance. We examined the effectiveness of our techniques on numerous long-lasting live performances from which representative examples are demonstrated.

[1]  Masataka Goto,et al.  An Audio-based Real-time Beat Tracking System for Music With or Without Drum-sounds , 2001 .

[2]  Michael Wimmer,et al.  Instant architecture , 2003, ACM Trans. Graph..

[3]  John R. Smith,et al.  Video personalization and summarization system , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[4]  Miller Puckette,et al.  Max at Seventeen , 2002, Computer Music Journal.

[5]  Richard Taylor,et al.  S. M. Eisenstein: Selected Works. Vol. 2. Towards a Theory of Montage , 1994 .

[6]  C. Gorbman,et al.  Unheard Melodies: Narrative Film Music , 1987 .

[7]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[8]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1899, Outstanding Dissertations in the Computer Sciences.

[9]  M. G. Wagner,et al.  DeepWave: visualizing music with VRML , 2001, Proceedings Seventh International Conference on Virtual Systems and Multimedia.

[10]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[11]  Mark D. Plumbley,et al.  Real-time temporal segmentation of note objects in music signals , 2004, ICMC.

[12]  Rainer Lienhart,et al.  Comparison of automatic shot boundary detection algorithms , 1998, Electronic Imaging.

[13]  J. Alton.  Painting with Light , 1949 .

[14]  Eric D. Scheirer,et al.  Tempo and beat analysis of acoustic musical signals. , 1998, The Journal of the Acoustical Society of America.

[15]  Stefan Müller Arisona,et al.  Soundium2: An Interactive Multimedia Playground , 2003, ICMC.

[16]  Martin Fowler,et al.  Refactoring - Improving the Design of Existing Code , 1999, Addison Wesley object technology series.

[17]  Thomas S. Huang,et al.  Constructing table-of-content for videos , 1999, Multimedia Systems.

[18]  D. Janaki Ram,et al.  Modeling Design Versions , 1996, VLDB.

[19]  Tristan Jehan,et al.  Cati dance: self-edited, self-synchronized music video , 2003, SIGGRAPH '03.

[20]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[21]  Simon Schubiger-Banz,et al.  A real-time multimedia composition layer , 2006, AMCMM '06.

[22]  Paul S. Strauss,et al.  An object-oriented 3D graphics toolkit , 1992, SIGGRAPH.

[23]  Andreas Girgensohn,et al.  Creating music videos using automatic media analysis , 2002, MULTIMEDIA '02.