Visual prototyping for audio applications

The goal of the proposed master thesis is to define an architecture that enables visual prototyping of real-time audio applications and plug-ins. Visual prototyping means that a developer can build a working application, including user interface and processing core, just by assembling elements together and changing their properties in a visual way. Specifically, this research will address the problem of having to bind interactive user interface to a real-time processing core, when both are defined dynamically, the set of components is extensible, it allows bidirectional communication of arbitrary types of data between the interface and the processing core, and, it still fulfils the real-time requirements of audio applications.

[1]  George Tzanetakis,et al.  MARSYAS3D: A PROTOTYPE AUDIO BROWSER-EDITOR USING A LARGE SCALE IMMERSIVE VISUAL AND AUDIO DISPLAY , 2001 .

[2]  Marian Petre,et al.  Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework , 1996, J. Vis. Lang. Comput..

[3]  Òscar Celma,et al.  Foafing the Music: Bridging the Semantic Gap in Music Recommendation , 2006, SEMWEB.

[4]  Alexandre R. J. François,et al.  A modular middleware flow scheduling framework (poster session) , 2000, ACM Multimedia.

[5]  Roger B. Dannenberg.  Combining Visual and Textual Representations for Flexible Interactive Audio Signal Processing , 2004, ICMC.

[6]  Ed F. Deprettere,et al.  Context-aware process networks , 2003, Proceedings IEEE International Conference on Application-Specific Systems, Architectures, and Processors. ASAP 2003.

[7]  Ralph E. Johnson.  Frameworks = (Components + Patterns) -- How frameworks compare to other object-oriented reuse techniques. , 1997 .

[8]  David García,et al.  CLAM: an OO framework for developing audio and music applications , 2002, OOPSLA '02.

[9]  Chandrasekhar Ramakrishnan,et al.  The CREATE Signal Library ("Sizzle"): Design, Issues, and Applications , 2003, ICMC.

[10]  Bruce Powel Douglass.  Real-Time Design Patterns , 1999 .

[11]  Òscar Celma,et al.  Tools for Content-based Retrieval and Transformation of Audio using MPEG-7: The SPOffline and the MDTools , 2004 .

[12]  Dominique Fober,et al.  Optimized Lock-Free FIFO Queue continued , 2005 .

[13]  Dominique Fober,et al.  Jack Audio Server: MacOSX port and multi-processor version , 2004 .

[14]  J. Michon,et al.  STUDIES ON SUBJECTIVE DURATION. I. DIFFERENTIAL SENSITIVITY IN THE PERCEPTION OF REPEATED TEMPORAL INTERVALS. , 1964, Acta psychologica.

[15]  Margaret M. Burnett,et al.  Scaling Up Visual Programming Languages , 1995, Computer.

[16]  Roger B. Dannenberg.  Real Time Control For Interactive Computer Music and Animation , 1989 .

[17]  Pau Arumí,et al.  A Data Flow Pattern Language for Audio and Music Computing , 2006 .

[18]  Òscar Celma,et al.  MUCOSA: A Music Content Semantic Annotator , 2005, ISMIR.

[19]  Roger B. Dannenberg,et al.  Toward Modular, Portable, Real-Time Software , 1995, ICMC.

[20]  L. V. Noorden.  Temporal coherence in the perception of tone sequences , 1975 .

[21]  Perry R. Cook,et al.  Synthesis toolkit in C , 2002, SIGGRAPH 1996.

[22]  Ross Bencina,et al.  PortAudio - an Open Source Cross Platform Audio API , 2001, ICMC.

[23]  Roger B. Dannenberg.  The CMU MIDI Toolkit , 1986, ICMC.

[24]  Edward A. Lee,et al.  Overview of the Ptolemy project , 2001 .

[25]  Roger B. Dannenberg,et al.  Combining Instrument and Performance Models for High-Quality Music Synthesis , 1998 .

[26]  J. Rubio.  An object-oriented metamodel for digital signal processing with a focus on audio and music , 2005 .

[27]  James M. Neighbors,et al.  The Draco Approach to Constructing Software from Reusable Components , 1984, IEEE Transactions on Software Engineering.

[28]  Miller Puckette,et al.  Max at Seventeen , 2002, Computer Music Journal.

[29]  Bruce W. Pennycook,et al.  Computer-music interfaces: a survey , 1985, CSUR.

[30]  Margaret Burnett,et al.  SOFTWARE ENGINEERING FOR VISUAL PROGRAMMING LANGUAGES , 2001 .

[31]  Arie van Deursen,et al.  Domain-specific languages: an annotated bibliography , 2000, SIGP.

[32]  S Puckette Miller,et al.  Pure Data : another integrated computer music environment , 1996 .

[33]  Xavier Amatriain,et al.  A dataflow pattern catalog for sound and music computing , 2006, PLoP '06.

[34]  Xavier Serra.  Sound hybridization techniques based on a deterministic plus stochastic decomposition model , 1994 .

[35]  Brian Foote,et al.  Designing Reusable Classes , 2001 .

[36]  Pau Arumí,et al.  Extracció d'acords amb l'Anotador de Música de CLAM , 2006 .

[37]  Xavier Amatriain,et al.  A Domain-Specific Metamodel for Multimedia Processing Systems , 2007, IEEE Transactions on Multimedia.

[38]  Philipp Ackermann,et al.  Direct manipulation of temporal structures in a multimedia application framework , 1994, MULTIMEDIA '94.

[39]  Stephen L. Burbeck,et al.  Applications programming in smalltalk-80: how to use model-view-controller (mvc) , 1987 .

[40]  Steven G. Johnson,et al.  FFTW: an adaptive software architecture for the FFT , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[41]  Lawrence A. Rowe,et al.  Design and performance of the Berkeley Continuous Media Toolkit , 1997, Electronic Imaging.

[42]  Stephen Travis Pope,et al.  The Well-Tempered Object: Musical Applications of Object-Oriented Software Technology , 1992 .

[43]  Roger B. Dannenberg,et al.  Low-latency Music Software Using Off-the-shelf Operating Systems , 1998, ICMC.

[44]  Edward A. Lee,et al.  Dataflow process networks , 1995, Proc. IEEE.

[45]  J. Christopher Ramming,et al.  Two Application languages in software production , 1994 .

[46]  Arie van Deursen,et al.  REPORT RAPPORT , 1997 .

[47]  David García,et al.  The CLAM Annotator: A Cross-Platform Audio Descriptors Editing Tool , 2005, ISMIR.

[48]  Steve Kubica,et al.  Cantata: visual programming environment for the Khoros system , 1995, COMG.

[49]  Kannan Srinivasan,et al.  New product development structures and time-to-market , 1997 .

[50]  David García,et al.  CLAM: a framework for efficient and rapid development of cross-platform audio applications , 2006, ACM Multimedia.

[51]  Erich Gamma,et al.  Design and Implementation of ET++, a Seamless Object-Oriented Application Framework , 1989, Struct. Program..

[52]  Mark Guzdial,et al.  Squeak - open personal computing and multimedia , 2001 .

[53]  Kenneth B. Kahn.  The PDMA Handbook of New Product Development , 1996 .

[54]  Tim Menzies,et al.  Evaluation Issues for Visual Programming Languages , 2000 .

[55]  Paul F. Whelan,et al.  Machine Vision Algorithms in Java: Techniques and Implementation , 2000 .

[56]  H. Lucic,et al.  Developing multimedia services using high-performance concurrent communication design patterns , 2003, Proceedings of the 7th International Conference on Telecommunications, 2003. ConTEL 2003..

[57]  Richard Cohn.  Neo-Riemannian Operations, Parsimonious Trichords, and Their "Tonnetz" Representations , 1997 .

[58]  X. Amatriain,et al.  XML as a means of control for audio processing, synthesis and analysis , 2001 .

[59]  H. W. M. LUNNEY,et al.  Time as heard in speech and music , 1974, Nature.

[60]  Xavier Serra,et al.  Sound transformations based on the SMS high level attributes , 1998 .

[61]  Pau Arumí,et al.  Developing Cross-Platform audio and Music Applications with the CLAM Framework , 2005, ICMC.

[62]  Miller Puckette,et al.  Pure Data , 1997, ICMC.

[63]  F. Adriaensen.  Using a DLL to filter time , 2005 .

[64]  K. Beck,et al.  Extreme Programming Explained , 2002 .

[65]  Gary P. Scavone.  RtAudio: A Cross-Platform C++ Class for Realtime Audio Input/Output , 2002, ICMC.

[66]  Peter Knees,et al.  An innovative three-dimensional user interface for exploring music collections enriched , 2006, MM '06.

[67]  Brad A. Myers,et al.  Past, Present and Future of User Interface Software Tools , 2000, TCHI.

[68]  Donald Roberts,et al.  Evolve Frameworks into Domain-Specific Languages , 1997 .

[69]  D.P. Skinner,et al.  The cepstrum: A guide to processing , 1977, Proceedings of the IEEE.

[70]  David L. Tennenhouse,et al.  The VuSystem: A Programming System for Compute-Intensive Multimedia , 1996, IEEE J. Sel. Areas Commun..

[71]  D. Bruce.  What Makes a Good Domain-speciic Language? Apostle, and Its Approach to Parallel Discrete Event Simulation , 1997 .

[72]  Ralph E. Johnson,et al.  An interactive environment for object-oriented music composition and sound synthesis , 1988, OOPSLA '88.

[73]  David H. Laidlaw,et al.  The application visualization system: a computational environment for scientific visualization , 1989, IEEE Computer Graphics and Applications.

[74]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[75]  Victor Lazzarini.  SOUND PROCESSING WITH THE SndObj LIBRARY: AN OVERVIEW , 2001 .

[76]  Nick Kanopoulos,et al.  Reducing the Time to Market Through Rapid Prototyping - Guest Editors' Introduction , 1995, Computer.

[77]  Amar Chaudhary,et al.  An Open Architecture for Real-Time Audio Processing Software , 1999 .

[78]  Xavier Serra,et al.  A system for sound analysis/transformation/synthesis based on a deterministic plus stochastic decomposition , 1989 .

[79]  Tino Lourens,et al.  TiViPE - Tino's Visual Programming Environment , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[80]  Stephen J. Mellor,et al.  Executable UML How to Build Class Models , 2001 .

[81]  Gérard G. Medioni,et al.  A Modular Software Architecture for Real-Time Video Processing , 2001, ICVS.

[82]  Emilia Gómez,et al.  Tonality Visualization of Polyphonic audio , 2005, ICMC.

[83]  Mark Green,et al.  The University of Alberta user interface management system , 1985, SIGGRAPH.