Staff line Removal using Generative Adversarial Networks

Staff line removal is a crucial pre-processing step in Optical Music Recognition. In this paper we propose a novel approach for staff line removal, based on Generative Adversarial Networks. We convert staff line images into patches and feed them into a U-Net, used as Generator. The Generator intends to produce staff-less images at the output. Then the Discriminator does binary classification and differentiates between the generated fake staff-less image and real ground truth staff less image. For training, we use a Loss function which is a weighted combination of L2 loss and Adversarial loss. L2 loss minimizes the difference between real and fake staff-less image. Adversarial loss helps to retrieve more high quality textures in generated images. Thus our architecture supports solutions which are closer to ground truth and it reflects in our results. For evaluation we consider the ICDAR/GREC 2013 staff removal database. Our method achieves superior performance in comparison to other conventional approaches on the same dataset.

[1]  Kia Ng,et al.  Music Manuscript Tracing , 2001, GREC.

[2]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[3]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[4]  Muriel Visani,et al.  ICDAR 2013 Music Scores Competition: Staff Removal , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[5]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[7]  Umapada Pal,et al.  An Efficient Staff Removal Approach from Printed Musical Documents , 2010, 2010 20th International Conference on Pattern Recognition.

[8]  Adelaide V. Finch,et al.  September , 1867, The Hospital.

[9]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Thierry Géraud,et al.  A morphological method for music score staff removal , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[12]  Carlos Guedes,et al.  Staff Detection with Stable Paths , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Ichiro Fujinaga,et al.  A Comparative Study of Staff Removal Algorithms , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Shijian Lu,et al.  An Effective Staff Detection and Removal Technique for Musical Documents , 2012, 2012 10th IAPR International Workshop on Document Analysis Systems.

[16]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Camille Couprie,et al.  Semantic Segmentation using Adversarial Networks , 2016, NIPS 2016.

[18]  Tamara L. Berg,et al.  Learning Temporal Transformations from Time-Lapse Videos , 2016, ECCV.

[19]  Jorge Calvo-Zaragoza,et al.  Staff-line removal with selectional auto-encoders , 2017, Expert Syst. Appl..

[20]  J. Koenderink Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[21]  Nicholas P. Carter,et al.  Automatic Recognition of Printed Music , 1992 .

[22]  Abhinav Gupta,et al.  Generative Image Modeling Using Style and Structure Adversarial Networks , 2016, ECCV.

[23]  Luisa Micó,et al.  Music staff removal with supervised pixel classification , 2016, International Journal on Document Analysis and Recognition (IJDAR).

[24]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  David Bainbridge,et al.  Dealing with superimposed objects in optical music recognition , 1997 .

[26]  Frank D. Julca-Aguilar,et al.  Image Operator Learning Coupled with CNN Classification and Its Application to Staff Line Removal , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[27]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  José Oncina,et al.  Staff-line detection and removal using a convolutional neural network , 2017, Machine Vision and Applications.

[29]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.