Foundations for Reliable and Flexible Interactive Multimedia Scores

Interactive Scores (IS) is a formalism for composing and performing interactive multimedia scores with several applications in video games, live performance installations, and virtual museums. The composer defines the temporal organization of the score by asserting temporal relations (TRs) between temporal objects (TOs). At execution time, the performer may modify the start/stop times of the TOs by triggering interaction points and the system guarantees that all the TRs are satisfied. Implementations of IS and formal models of their behavior have already been proposed, but these do not provide usable means to reason about their properties. In this paper we introduce ReactiveIS, a programming language that fully captures the temporal structure of IS during both composition and execution. For that, we propose a semantics based on tree-like structures representing the execution state of the score at each point in time. The semantics captures the hierarchical aspects of IS and provides an intuitive representation of their execution. We also endow ReactiveIS with a logical semantics based on linear logic, thus widening the reasoning techniques available for IS. We show that ReactiveIS is general enough to capture the full behavior of IS and it also provides declarative ways to increase the expressivity of IS with, for instance, conditional statements and loops.