FREISCHÜTZ DIGITAL : DEMOS OF AUDIO-RELATED CONTRIBUTIONS

The project “Freischütz Digital” 1 (funded by the German Federal Ministry of Education and Research), was a close cooperation between musicologists and computer scientists to explore new digital ways for analyzing and presenting music-related data in critical editions. The opera “Der Freischütz” by Carl Maria von Weber served as challenging example scenario in the project. It offers a large number of (historical) sources including different versions of the musical score, the libretto, and (multitrack) audio recordings. One major task of the project was to adapt music synchronization techniques to align different versions of the same piece of music [1,2,4]. Such alignments can then be used to realize extended music audio players, that display various types of music-related information while playing. In this contribution, we present three audio-related web demos providing simple navigation and interaction possibilities with the music material. 1. DEMO: SCORE FOLLOWER & INTERPRETATION SWITCHER The goal in score following [1] is to retrieve the current score position in a sheet music representation while playing back a corresponding music recording. Similarly, the goal in interpretation switching is to to retrieve the time positions in different recordings that correspond to the current playback position of a given music recording. Figure 1 shows a web-based audio player providing score following and interpretation switching functionalities. During the playback of a recording, the corresponding position in the musical score is highlighted (blue rectangle). The user can choose between scans of historical scores (facsimiles) or a rendering of a digitally encoded score in MEI. 2 To display the MEI score, we use the library Verovio. 3 Furthermore, a list of available audio ver1 http://www.freischuetz-digital.de 2 http://music-encoding.org/ 3 http://www.verovio.org/ c © Thomas Prätzlich, Meinard Müller, Benjamin W. Bohl, Joachim Veit. Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Attribution: Thomas Prätzlich, Meinard Müller, Benjamin W. Bohl, Joachim Veit. “Freischütz Digital: Demos of audio-related contributions”, Extended abstracts for the Late-Breaking Demo Session of the 16th International Society for Music Information Retrieval Conference, 2015. Figure 1. Score Follower & Interpretation Switcher Interface. Top: Score Rendering, Bottom: Autograph. sions is shown. Simultaneously to the playback of the active audio version (marked in blue on the left side), the corresponding time positions in all available audio versions is displayed. When selecting another audio version, the player switches seamlessly to the chosen version and starts the playback at the position that corresponds to the time position of the previously active audio version. The score following and interpretation switching is realized by providing offline alignments between the different audio versions and the musical score. 2. DEMO: MULTITRACK RECORDINGS Within the Freischütz Digital project, three numbers (No. 6, 8, and 9) of the opera “Der Freischütz” have been proBass Ännchen Agathe Viola Violin 2 Violin 1 Bassoon Horn Clarinet Cello Flute Violin 1 Figure 2. Multitrack Recording Interface. duced at the Erich-Thienhaus-Institute (HfM Detmold). The main purpose for the recording sessions was to produce royalty free audio material that can be used for demonstration and research purposes. Besides a professional stereo mix of the three numbers, the dataset provides the raw multitrack recordings from the individual microphones as well as individual group mixes that emphasize different voices or instrument sections. Figure 2 shows an audio player interface that sketches the microphone setup during the recording of No. 6. The interface provides the possibility to listen to the individual microphone recordings. Furthermore, a matrix visualization displays which instruments are currently active (black) or inactive (white). This information has been extracted from a digital score representation that was aligned to the audio recordings. 3. DEMO: INSTRUMENT EQUALIZER An instrument equalizer provides the possibility to adjust the volume of an individual instrument in a recording without affecting the volume of the other instruments. Figure 3 shows a multitrack audio player with an individual track for each voice/instrument of No. 6 from “Der Freischütz”. When studying a specific melody line of the violins and the flutes, for example, the instrument equalizer enables a user to raise the volume for these two voices and to lower it for the others, see Figure 3. Note that, when recording orchestra music, the microphones for capturing the different voices are usually not shielded from each other. In practice, each microphone not only records sound from its dedicated voice or instrument, but also from all others in the room. This results in recordings that do not feature isolated signals, but rather mixtures of a predominant voice with all others being audible through what is referred to as interference. In [3], we presented a method for reducing interferences in multitrack Figure 3. Instrument Equalizer Interface. recordings. A way to use multitrack recordings suffering from interference in an instrument equalizer is to apply an interference reduction first. All demos as well as the multitrack recordings discussed in this contribution will be available at: http://freischuetz-digital.de/demos/ Acknowledgments: We want to thank Laurent Pugin for integrating Verovio into the Score Follower. This work has been supported by the BMBF project Freischütz Digital (Funding Code 01UG1239A to C). The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität ErlangenNürnberg (FAU) and Fraunhofer IIS.