Towards Polyphony Reconstruction Using Multidimensional Multiple Sequence Alignment

The digitization of printed music scores through the process of optical music recognition is imperfect. In polyphonic scores, with two or more simultaneous voices, errors of duration or position can lead to badly aligned and inharmonious digital transcriptions. We adapt biological sequence analysis tools as a post-processing step to correct the alignment of voices. Our multiple sequence alignment approach works on multiple musical dimensions and we investigate the contribution of each dimension to the correct alignment. Structural information, such musical phrase boundaries, is of major importance; therefore, we propose the use of the popular bioinformatics aligner Mafft which can incorporate such information while being robust to temporal noise. Our experiments show that a harmony-aware Mafft outperforms sophisticated, multidimensional alignment approaches and can achieve near-perfect polyphony reconstruction.

[1]  M. O. Dayhoff,et al.  22 A Model of Evolutionary Change in Proteins , 1978 .

[2]  Petri Toiviainen,et al.  A Matlab Toolbox for Music Information Retrieval , 2007, GfKl.

[3]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[4]  Marcelo Rodríguez López,et al.  Automatic Melody Segmentation , 2016 .

[5]  P. Hogeweg,et al.  The alignment of sets of sequences and the construction of phyletic trees: An integrated method , 2005, Journal of Molecular Evolution.

[6]  Carlos Guedes,et al.  Optical music recognition: state-of-the-art and open issues , 2012, International Journal of Multimedia Information Retrieval.

[7]  S. Henikoff,et al.  Amino acid substitution matrices from protein blocks. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Remco C. Veltkamp,et al.  A data-driven approach to chord similarity and chord mutability , 2016, 2016 IEEE Second International Conference on Multimedia Big Data (BigMM).

[9]  A. Dress,et al.  Multiple DNA and protein sequence alignment based on segment-to-segment comparison. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[10]  K. Katoh,et al.  MAFFT: a novel method for rapid multiple sequence alignment based on fast Fourier transform. , 2002, Nucleic acids research.

[11]  Perry G. Ridge,et al.  Effects of Gap Open and Gap Extension Penalties , 2006 .

[12]  Simon Dixon,et al.  Robust Joint Alignment of Multiple Versions of a Piece of Music , 2016, ISMIR.

[13]  Remco C. Veltkamp,et al.  Applying Rhythmic Similarity Based on Inner Metric Analysis to Folksong Research , 2007, ISMIR.

[14]  Tim Crawford,et al.  Evaluating OMR on the Early Music Online Collection , 2013, International Society for Music Information Retrieval Conference.

[15]  Alexander K. Hudek.  Improvements in the Accuracy of Pairwise Genomic Alignment , 2010 .

[16]  Emilios Cambouropoulos,et al.  The Local Boundary Detection Model (LBDM) and its Application in the Study of Expressive Timing , 2001, ICMC.

[17]  Yoshua Bengio,et al.  High-dimensional sequence transduction , 2012, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  P. Sanguansat.  Multiple Multidimensional Sequence Alignment Using Generalized Dynamic Time Warping , 2012 .

[19]  P. van Kranenburg,et al.  A Computational Approach to Content-Based Retrieval of Folk Song Melodies , 2010 .

[20]  Ta Theo Arentze,et al.  Activity pattern similarity : a multidimensional sequence alignment method , 2002 .

[21]  L. Polansky,et al.  Temporal Gestalt Perception in Music , 1980 .

[22]  R. Doolittle,et al.  Progressive sequence alignment as a prerequisitetto correct phylogenetic trees , 2007, Journal of Molecular Evolution.

[23]  D. Lipman,et al.  The multiple sequence alignment problem in biology , 1988 .

[24]  Jun Zhu,et al.  Modelling High-Dimensional Sequences with LSTM-RTRBM: Application to Polyphonic Music Generation , 2015, IJCAI.

[25]  Tao Jiang,et al.  On the Complexity of Multiple Sequence Alignment , 1994, J. Comput. Biol..

[26]  Yoshua Bengio,et al.  Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription , 2012, ICML.

[27]  Ray Jackendoff,et al.  An overview of hierarchical structure in music , 1983 .

[28]  Simon Whelan,et al.  Measuring the distance between multiple sequence alignments , 2012, Bioinform..

[29]  J. Thompson,et al.  CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, position-specific gap penalties and weight matrix choice. , 1994, Nucleic acids research.