HReMAS: hybrid real-time musical alignment system

This paper presents a real-time audio-to-score alignment system for musical applications. The aim of these systems is to synchronize a live musical performance with its symbolic representation in a music sheet. We have used as a base our previous real-time alignment system by enhancing it with a traceback stage, a stage used in offline alignment to improve the accuracy of the aligned note. This stage introduces some delay, what forces to assume a trade-off between output delay and alignment accuracy that must be considered in the design of this type of hybrid techniques. We have also improved our former system to execute faster in order to minimize this delay. Other interesting improvements, like identification of silence frames, have also been incorporated to our proposed system.

[1]  Nicola Orio,et al.  Alignment of Monophonic and Polyphonic Music to a Score , 2001, ICMC.

[2]  José Ranilla,et al.  An efficient musical accompaniment parallel system for mobile devices , 2016, The Journal of Supercomputing.

[3]  Daniel P. W. Ellis,et al.  Handling Asynchrony in Audio-Score Alignment , 2009, ICMC.

[4]  José Ranilla,et al.  Parallel online time warping for real-time audio-to-score alignment in multi-core systems , 2016, The Journal of Supercomputing.

[5]  George Tzanetakis,et al.  Polyphonic audio matching and alignment for music retrieval , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[6]  Christopher Raphael,et al.  Evaluation of Real-Time Audio-to-Score Alignment , 2007, ISMIR.

[7]  Christopher Raphael,et al.  Music Plus One and Machine Learning , 2010, ICML.

[8]  Christopher Raphael,et al.  Music score alignment and computer accompaniment , 2006, CACM.

[9]  Simon Dixon,et al.  An On-Line Time Warping Algorithm for Tracking Musical Performances , 2005, IJCAI.

[10]  Gerhard Widmer,et al.  Automatic Page Turning for Musicians via Real-Time Machine Listening , 2008, ECAI.

[11]  Bryan Pardo,et al.  Soundprism: An Online System for Score-Informed Source Separation of Music Audio , 2011, IEEE Journal of Selected Topics in Signal Processing.

[12]  Pedro Vera-Candeas,et al.  Tempo Driven Audio-to-Score Alignment Using Spectral Decomposition and Online Dynamic Time Warping , 2016, ACM Trans. Intell. Syst. Technol..

[13]  Peter Grosche,et al.  High resolution audio synchronization using chroma onset features , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Arshia Cont.  Realtime Audio to Score Alignment for Polyphonic Music Instruments, using Sparse Non-Negative Constraints and Hierarchical HMMS , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[15]  Tapio Lokki,et al.  Anechoic recording system for symphony orchestra , 2008 .

[16]  Jordi Janer,et al.  Score-Informed Source Separation for Multichannel Orchestral Recordings , 2016, J. Electr. Comput. Eng..

[17]  Nicolás Ruiz-Reyes,et al.  An Audio to Score Alignment Framework Using Spectral Factorization and Dynamic Time Warping , 2015, ISMIR.

[18]  Xavier Rodet,et al.  Globally Optimal Short-Time Dynamic Time Warping, Application to Score to Audio Alignment , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[19]  Bochen Li,et al.  An Approach to Score Following for Piano Performances With the Sustained Effect , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.