A system for using perceiver input to vary the quality of generative multimedia performances

Generative Multimedia (GM) applications are an increasingly popular way to implement interactive media performances. Our contributions include creating a metric for evaluating Generative Multimedia performances, designing a model for accepting perceiver preferences, and using those preferences to adapt GM performances. The metric we use is imprecision, which is the ratio of the actual computation time of a GM element to the computation time of a complete version of that GM element. By taking a perceiver's preferences into account when making adaptation decisions, applications can produce GM performances that meet soft real-time and resource constraints while allocating imprecision to the GM elements the perceiver least cares about. Compared to other approaches, perceiver-directed imprecision best allocates impreciseness while minimizing delay.

[1]  Umakishore Ramachandran,et al.  DFuse: a framework for distributed data fusion , 2003, SenSys '03.

[2]  Keld K. Jensen,et al.  Timbre Models of Musical Sounds , 1999 .

[3]  Richard Boulanger.  The Csound book: perspectives in software synthesis, sound design, signal processing, and programming , 2000 .

[4]  Sartaj Sahni,et al.  Data Structures, Algorithms, and Applications in C++ , 1997 .

[5]  Jeffrey O. Kephart,et al.  An architectural approach to autonomic computing , 2004, International Conference on Autonomic Computing, 2004. Proceedings..

[6]  Karsten Schwan,et al.  Dynamic adaptation of real-time software , 1991, TOCS.

[7]  Karla L. Homan.  Combinatorial optimization: Current successes and directions for the future , 2000 .

[8]  W. Graham Richards,et al.  Art of electronics , 1983, Nature.

[9]  John M. Chowning,et al.  The Synthesis of Complex Audio Spectra by Means of Frequency Modulation , 1973 .

[10]  Gil Weinberg,et al.  The Aesthetics, History and Future Challenges of Interconnected Music Networks , 2002, ICMC.

[11]  Liam Murphy,et al.  User-perceived quality-aware adaptive delivery of MPEG-4 content , 2003, NOSSDAV '03.

[12]  Ralf Steinmetz,et al.  Human Perception of Jitter and Media Synchronization , 1996, IEEE J. Sel. Areas Commun..

[13]  B. Jeff,et al.  PARSYNTH: a case study on implementing a real-time digital audio synthesizer , 1996, Proceedings of the 4th International Workshop on Parallel and Distributed Real-Time Systems.

[14]  George Ghinea,et al.  QoS impact on user perception and understanding of multimedia video clips , 1998, MULTIMEDIA '98.

[15]  Christian Poellabauer,et al.  Q-Fabric: System Support for Continuous Online Quality Management , 2004 .

[16]  Abraham Silberschatz,et al.  Operating Systems Concepts , 2005 .

[17]  Scott A. Brandt.  Performance analysis of dynamic soft real-time systems , 2001, Conference Proceedings of the 2001 IEEE International Performance, Computing, and Communications Conference (Cat. No.01CH37210).

[18]  Katherine Guo,et al.  Sync-MS: synchronized messaging service for real-time multi-player distributed games , 2002, 10th IEEE International Conference on Network Protocols, 2002. Proceedings..

[19]  Shlomo Zilberstein,et al.  Operational Rationality through Compilation of Anytime Algorithms , 1995, AI Mag..

[20]  David Phillips.  Csound for Linux , 1999 .

[21]  Mark Billinghurst,et al.  Augmenting the science centre and museum experience , 2004, GRAPHITE '04.

[22]  Andrew T. Campbell,et al.  A survey of QoS architectures , 1998, Multimedia Systems.

[23]  Julius O. Smith,et al.  Physical Modeling Synthesis Update , 1996 .

[24]  Joni da Silva Fraga,et al.  Specification, Mapping and Control for QoS Adaptation , 2002, Real-Time Systems.

[25]  John A. Zinky,et al.  Overview of Quality of Service for Distributed Objects , 1995 .

[26]  Karsten Schwan,et al.  FARA-a framework for adaptive resource allocation in complex real-time systems , 1998, Proceedings. Fourth IEEE Real-Time Technology and Applications Symposium (Cat. No.98TB100245).

[27]  Predicting the intensity JND from the loudness of tones and noise , 2003 .

[28]  Riccardo Bettati,et al.  Use of Imprecise Computation to Enhance Dependability of Real-Time Systems , 1994 .

[29]  Hal Chamberlin.  Musical Applications of Microprocessors , 1980 .

[30]  Valentin S. Kisimov,et al.  Video Acceptability and Frame Rate , 1995, IEEE Multim..

[31]  Mark Claypool,et al.  The effects of jitter on the peceptual quality of video , 1999, MULTIMEDIA '99.

[32]  Xavier Serra,et al.  Musical Sound Modeling with Sinusoids plus Noise , 1997 .

[33]  John A. Zinky,et al.  Architectural Support for Quality of Service for CORBA Objects , 1997, Theory Pract. Object Syst..

[34]  Mattias Östergren.  Sound Pryer: Adding Value to Traffic Encounters with Streaming Audio , 2004, ICEC.

[35]  Mrinal K. Mandal,et al.  Digital Audio Processing , 2003 .

[36]  Sheng-Tzong Cheng,et al.  Dynamic quota-based admission control with sub-rating in multimedia servers , 2000, Multimedia Systems.

[37]  John A. Zinky,et al.  Specifying and measuring quality of service in distributed object systems , 1998, Proceedings First International Symposium on Object-Oriented Real-Time Distributed Computing (ISORC '98).

[38]  Stephen R. Gulliver,et al.  Infotainment across access devices: the perceptual impact of multimedia QoS , 2004, SAC '04.

[39]  Andrew T. Campbell,et al.  A quality of service architecture , 1994, CCRV.

[40]  Ronald L. Boring,et al.  Helping users determine video quality of service settings , 2002, CHI Extended Abstracts.

[41]  Sandhya Dwarkadas,et al.  Dynamic adaptation to available resources for parallel computing in an autonomous network of workstations , 2001, PPoPP '01.

[42]  Tsuhan Chen,et al.  Realistic video avatar , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[43]  Klara Nahrstedt,et al.  The QOS Broker , 1995, IEEE Multim..

[44]  Kenneth C. Pohlmann,et al.  Principles of Digital Audio , 1986 .

[45]  Jeffrey O. Kephart,et al.  The Vision of Autonomic Computing , 2003, Computer.