A High-Fidelity Orchestra Simulator for Individual Musicians’ Practice

We developed the Open Orchestra system to provide individual musicians with a high-fidelity experience of ensemble rehearsal or performance, combined with the convenience and flexibility of solo study. This builds on the theme of an immersive orchestral simulator that also supports the pedagogical objective of instructor feedback on individual recordings, as needed to improve one's performance. Unlike previous systems intended for musical rehearsal, Open Orchestra attempts to offer both an auditory and a visual representation of the rest of the orchestra, spatially rendered from the perspective of the practicing musician. We review the objectives and architecture of our system, describe the functions of our digital music stand, discuss the challenges of generating the media content needed for this system, and describe provisions for offering feedback during rehearsal.

[1]  Kjetil Falkenberg Hansen,et al.  IMUTUS : an interactive system for learning to play a musical instrument , 2004 .

[2]  Jeremy R. Cooperstock,et al.  Extended User Control over Multichannel Content Delivered over the Web , 2011 .

[3]  Megan A. Winget.  Annotations on musical scores by performing musicians: Collaborative models, interactive methods, and music digital library tool development , 2008 .

[4]  Pete Candeland.  Harmonix "Rock Band II" , 2009, SIGGRAPH ASIA '09.

[5]  Megan A. Winget,et al.  Annotations on musical scores by performing musicians: Collaborative models, interactive methods, and music digital library tool development , 2008, J. Assoc. Inf. Sci. Technol..

[6]  Christopher Raphael,et al.  Demonstration of Music Plus One - A Real-Time System for Automatic Orchestral Accompaniment , 2006, AAAI.

[7]  Yann Geslin,et al.  Sound and musical representation: the Acousmographe software , 2004, ICMC.

[8]  Pedro Rebelo,et al.  Networked Music Performance: State of the Art , 2007 .

[9]  Kjetil Falkenberg Hansen,et al.  Design and Implementation of Automatic Evaluation of recorder Performance in Imutus , 2005, ICMC.

[10]  Anders Friberg,et al.  pDM: An Expressive Sequencer with Real-Time Control of the KTH Music-Performance Rules , 2006, Computer Music Journal.

[11]  George Tzanetakis,et al.  Polyphonic audio matching and alignment for music retrieval , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[12]  E. Lindemann.  Music Synthesis with Reconstructive Phrase Modeling , 2007, IEEE Signal Processing Magazine.

[13]  Meinard Müller,et al.  Syncplayer - An Advanced System for Multimodal Music Access , 2005, ISMIR.

[14]  Anton Nijholt,et al.  Temporal interaction between an artificial orchestra conductor and human musicians , 2008, CIE.

[15]  Sung Yong Shin,et al.  Rhythmic-motion synthesis based on motion-beat analysis , 2003, ACM Trans. Graph..

[16]  Michael F. Schober.  Virtual environments for creative work in collaborative music-making , 2006, Virtual Reality.

[17]  Joseph Sammut.  Performance Learning System , 2012 .

[18]  Robert Joseph,et al.  Results from the Piano Tutor Project 1 , 1993 .

[19]  Dominique Fober,et al.  VEMUS - Feedback and Groupware Technologies for Music Instrument Learning , 2007 .

[20]  Elaine Chew,et al.  Distributed musical performances: Architecture and stream management , 2008, TOMCCAP.

[21]  Arshia Cont,et al.  A Coupled Duration-Focused Architecture for Real-Time Music-to-Score Alignment , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Barry Vercoe,et al.  The Synthetic Performer in The Context of Live Performance , 1984, International Conference on Mathematics and Computing.

[23]  Yuri Ivanov,et al.  The UBS Virtual Maestro: an Interactive Conducting System , 2009, NIME.

[24]  Donald Byrd,et al.  Prospects for Improving OMR with Multiple Recognizers , 2006, ISMIR.

[25]  David Hsu,et al.  Digital violin tutor: an integrated system for beginning violin learners , 2005, ACM Multimedia.

[26]  Meinard Müller,et al.  Automatic Mapping of Scanned Sheet Music to Audio Recordings , 2008, ISMIR.

[27]  Mathieu Lagrange,et al.  Analysis of Saxophone Performance for Computer-Assisted Tutoring , 2007, ICMC.

[28]  Mark B. Sandler,et al.  The Sonic Visualiser: A Visualisation Platform for Semantic Descriptors from Musical Signals , 2006, ISMIR.

[29]  Roger B. Dannenberg,et al.  An On-Line Algorithm for Real-Time Accompaniment , 1984, ICMC.

[30]  Niina Koivunen.  Organizational Music the Role of Listening in Interaction Processes , 2002 .

[31]  Christopher Raphael,et al.  Music score alignment and computer accompaniment , 2006, CACM.

[32]  R. Shepard.  Circularity in Judgments of Relative Pitch , 1964 .

[33]  Meinard Müller,et al.  Sheet Music-Audio Identification , 2009, ISMIR.

[34]  Rumi Hiraga,et al.  Visualization of music performance as an aid to listener's comprehension , 2004, AVI.

[35]  Georg Hajdu,et al.  Maxscore: Music Notation in Max/MSP , 2009, ICMC.

[36]  Stefania Serafin,et al.  Proceedings of the International Computer Music Conference , 2007 .

[37]  Gerhard Widmer,et al.  MATCH: A Music Alignment Tool Chest , 2005, ISMIR.

[38]  Stefanos Nikolaidis,et al.  REAL-TIME DETECTION AND VISUALIZATION OF CLARINET BAD SOUNDS 1 , 2008 .

[39]  William L. Martens,et al.  Shake, rattle, and roll : Getting immersed in multisensory, interactive music via broadband networks , 2005 .

[40]  Ichiro Fujinaga,et al.  Optical Music Recognition System within a Large-Scale Digitization Project , 2000, ISMIR.

[41]  George Tzanetakis,et al.  Effective use of multimedia for computer-assisted musical instrument tutoring , 2007, Emme '07.

[42]  Gerhard Widmer,et al.  Machine Discoveries: A Few Simple, Robust Local Expression Principles , 2002 .

[43]  Jeremy R. Cooperstock,et al.  Visualization feedback for musical ensemble practice: a case study on phrase articulation and dynamics , 2012, Visualization and Data Analysis.

[44]  Daniel P. W. Ellis,et al.  Improving MIDI-audio alignment with acoustic features , 2009, 2009 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics.

[45]  Steve DiPaola,et al.  Emotional remapping of music to facial animation , 2006, Sandbox '06.

[46]  Guerino Mazzola,et al.  Performance and Interpretation , 2002 .

[47]  J. Progler.  Searching for Swing: Participatory Discrepancies in the Jazz Rhythm Section , 1995 .

[48]  Joseph Psotka,et al.  Immersive training systems: Virtual reality and education and training , 1995 .

[49]  Gerhard Widmer,et al.  Computational Models of Expressive Music Performance: The State of the Art , 2004 .

[50]  Nell P. McAngusTodd,et al.  The dynamics of dynamics: A model of musical expression , 1992 .

[51]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[52]  Stefanos Nikolaidis,et al.  REAL-TIME DETECTION AND VISUALIZATION OF CLARINET BAD SOUNDS , 2008 .

[53]  Gérard Assayag,et al.  High Level Musical Control of Sound Synthesis in OpenMusic , 2000, ICMC.

[54]  Max Mühlhäuser,et al.  Personal orchestra: a real-time audio/video system for interactive conducting , 2004, Multimedia Systems.