Operational semantics of a domain specific language for real time musician–computer interaction

With the advent and availability of powerful personal computing, the computer music research and industry have been focusing on real-time musical interactions between musicians and computers; delegating human-like actions to computers who interact with a musical environment. One common use-case of this kind is Automatic Accompaniment where the system is comprised of a real-time machine listening system that in reaction to recognition of events in a score from a human performer, launches necessary actions for the accompaniment section. While the real-time detection of score events out of live musicians’ performance has been widely addressed in the literature, score accompaniment (or the reactive part of the process) has been rarely discussed. This paper deals with this missing component in the literature from a formal language perspective. We show how language considerations would enable better authoring of time and interaction during programming/composing and how it addresses critical aspects of a musical performance (such as errors) in real-time. We sketch the real-time features required by automatic musical accompaniment seen as a reactive system. We formalize the timing strategies for musical events taking into account the various temporal scales used in music. Various strategies for the handling of synchronization constraints and the handling of errors are presented. We give a formal semantics to model the possible behaviors of the system in terms of Parametric Timed Automata.

[1]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[2]  Myriam Desainte-Catherine,et al.  Interactive scores: A model for specifying temporal relations between interactive and static events , 2005 .

[3]  Louis Mandel,et al.  Interactive Programming of Reactive Systems , 2009, Electron. Notes Theor. Comput. Sci..

[4]  Benedikt Bollig,et al.  Distributed Timed Automata with Independently Evolving Clocks , 2008, CONCUR.

[5]  F. Jacquemard,et al.  Formalisation des relations temporelles entre une partition et une performance musicale dans un contexte d’accompagnement automatique , 2011 .

[6]  Thierry Gautier,et al.  Programming real-time applications with SIGNAL , 1991, Proc. IEEE.

[7]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[8]  Gérard Berry,et al.  The Esterel Synchronous Programming Language: Design, Semantics, Implementation , 1992, Sci. Comput. Program..

[9]  Thomas A. Henzinger,et al.  Parametric real-time reasoning , 1993, STOC.

[10]  Marc Pouzet,et al.  The fundamentals of hybrid systems modelers , 2010, 49th IEEE Conference on Decision and Control (CDC).

[11]  Jakob Engblom,et al.  The worst-case execution-time problem—overview of methods and survey of tools , 2008, TECS.

[12]  Arshia Cont,et al.  On the creative use of score following and its impact on research , 2011 .

[13]  E. Large,et al.  The dynamics of attending: How people track time-varying events. , 1999 .

[14]  Miller S. Puckette,et al.  Combining Event and Signal Processing in the MAX Graphical Programming Environment , 1991 .

[15]  Marc Pouzet.  Lucid Synchrone: tutorial and reference manual , 2001 .

[16]  Roger B. Dannenberg,et al.  An On-Line Algorithm for Real-Time Accompaniment , 1984, ICMC.

[17]  Jean-Louis Giavitto,et al.  Formalisation des relations temporelles dans un contexte d'accompagnement musical automatique , 2011 .

[18]  Emmanuelle Encrenaz-Tiphène,et al.  An Inverse Method for Parametric Timed Automata , 2008, Electron. Notes Theor. Comput. Sci..

[19]  Léa Fanchon.  Formal Analysis of Scores , 2012 .

[20]  Barry Vercoe,et al.  The Synthetic Performer in The Context of Live Performance , 1984, International Conference on Mathematics and Computing.

[21]  Christopher Raphael,et al.  Music score alignment and computer accompaniment , 2006, CACM.

[22]  Laurent Fribourg,et al.  Synthèse de contraintes temporisées pour une architecture d'automatisation en réseau , 2009 .

[23]  Christopher Raphael.  The informatics philharmonic , 2014 .

[24]  Pascal Raymond,et al.  The synchronous data flow programming language LUSTRE , 1991, Proc. IEEE.

[25]  Arshia Cont,et al.  Antescofo: Anticipatory Synchronization and control of Interactive parameters in Computer Music , 2008, ICMC.

[26]  Arshia Cont,et al.  A Coupled Duration-Focused Architecture for Real-Time Music-to-Score Alignment , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.