MusicScore: mobile music composition for practice and fun

In this paper, we present our design of MusicScore, a professional grade application on the iOS platform for music composition and live performance tracking, used by composers and amateurs alike. As its foundation, we have designed and implemented a high-quality music engraver, capable of real-time interactive rendering on mobile devices, as well as an intuitive user interface based on multi-touch, both built from scratch using Objective-C and Cocoa Touch. To make MusicScore appealing to the general population for their practice and fun, we have introduced a unique auditory capability to MusicScore, so that it can "listen" to and analyze live instrument performance in real time. In order to compensate for the imperfect audio sensing system on mobile devices, we have proposed a collaborative sensing solution to better capture music signals in real time. To maximize the accuracy of live progress tracking and performance evaluation using a mobile device, we have designed a collection of note detection and tempo-based note matching algorithms, using a combination of microphone and accelerometer sensors. Based on our real-world implementation of MusicScore, extensive evaluation results show that MusicScore can achieve acceptably low error ratios, even for music pieces performed by highly inexperienced players.

[1]  Harvey Fletcher,et al.  Quality of Piano Tones , 1962 .

[2]  Arshia Cont,et al.  A unified approach to real time audio-to-score and audio-to-audio alignment using sequential Montecarlo inference techniques , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Hideki Kawahara,et al.  YIN, a fundamental frequency estimator for speech and music. , 2002, The Journal of the Acoustical Society of America.

[4]  Mototsugu Abe,et al.  Design Criteria for Simple Sinusoidal Parameter Estimation Based on Quadratic Interpolation of FFT Magnitude Peaks , 2004 .

[5]  A. Arzt.  SIMPLE TEMPO MODELS FOR REAL-TIME MUSIC TRACKING , 2010 .

[6]  P. Strevens.  Iii , 1985 .

[7]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[8]  Han-Wen Nienhuys,et al.  LILYPOND, A SYSTEM FOR AUTOMATED MUSIC ENGRAVING , 2003 .

[9]  DeLiang Wang,et al.  Pitch Detection in Polyphonic Music using Instrument Tone Models , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[10]  Ted Ross.  The Art of Music Engraving and Processing , 1970 .

[11]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[12]  Masataka Goto,et al.  An Audio-based Real-time Beat Tracking System for Music With or Without Drum-sounds , 2001 .

[13]  Eric D. Scheirer,et al.  Tempo and beat analysis of acoustic musical signals. , 1998, The Journal of the Acoustical Society of America.