Fog Computing Approach for Music Cognition System Based on Machine Learning Algorithm

With the wide spreading of mobile and Internet of Things (IoT) devices, music cognition as a meaningful task for music promotion has attracted a lot of attention around the world. How to automatically generate music score is an important part in music cognition, which acts as an important carrier so as to disposing huge quantity of music data in IoT networks or Internet. For the reason that the computers lack of the domain knowledge and cognitive ability, it is hard for computers to recognize the melody of music or write score while listening to the music. Therefore, a music cognition system is introduced to cognate music and automatically write score based on machine learning methods. First, considering large-scale data processing is needed by machine learning algorithms and a number of music devices are involved in the cognition system through Internet, fog computing is adopted in the proposed architecture to efficiently allocate computing resources. Then, the system can collect, preprocess, and store raw music data on the fringe nodes. Meanwhile, these data will be transmitted from fog nodes to cloud servers to form music databases. Then, machine learning algorithms, such as hidden Markov model and Gaussian mixture model, are performed in cloud servers to recognize music melody. Finally, a case study of music score generation demonstrates the proposed system. It is shown that the method provides an effective support to generate music score, and also proposed a promising way for the research and application of music cognition.

[1]  Antonio Iera,et al.  The Social Internet of Things (SIoT) - When social networks meet the Internet of Things: Concept, architecture and network characterization , 2012, Comput. Networks.

[2]  Kai Feng,et al.  The subspace Gaussian mixture model - A structured model for speech recognition , 2011, Comput. Speech Lang..

[3]  Andrew S. Cassidy,et al.  Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[4]  Jaap A. Haitsma,et al.  Robust Audio Hashing for Content Identification , 2001 .

[5]  Athanasios V. Vasilakos,et al.  IoT-Based Big Data Storage Systems in Cloud Computing: Perspectives and Challenges , 2017, IEEE Internet of Things Journal.

[6]  Frank Kurth,et al.  A unified approach to content-based and fault-tolerant music recognition , 2004, IEEE Transactions on Multimedia.

[7]  Marco Jahn,et al.  Designing a Smart City Internet of Things Platform with Microservice Architecture , 2015, 2015 3rd International Conference on Future Internet of Things and Cloud.

[8]  Wu He,et al.  Internet of Things in Industries: A Survey , 2014, IEEE Transactions on Industrial Informatics.

[9]  Karthikeyan Umapathy,et al.  Audio Signal Feature Extraction and Classification Using Local Discriminant Bases , 2004, IEEE Transactions on Audio, Speech, and Language Processing.

[10]  Vladimir Viro,et al.  Peachnote: Music Score Search and Analysis Platform , 2011, ISMIR.

[11]  M.P. Ryynanen,et al.  Polyphonic music transcription using note event modeling , 2005, IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, 2005..

[12]  Raja Lavanya,et al.  Fog Computing and Its Role in the Internet of Things , 2019, Advances in Computer and Electrical Engineering.

[13]  Songyang Lao,et al.  Feature analysis and extraction for audio automatic classification , 2005, SMC.

[14]  Chalapathy Neti,et al.  Noisy audio feature enhancement using audio-visual speech data , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[15]  Rajendra D. Kokate,et al.  An Effective Optimization-Based Neural Network for Musical Note Recognition , 2019, J. Intell. Syst..

[16]  Kyunghyun Cho,et al.  A Comparison of Audio Signal Preprocessing Methods for Deep Neural Networks on Music Tagging , 2017, 2018 26th European Signal Processing Conference (EUSIPCO).

[17]  D. S. Bormane,et al.  Dynamic time warping technique for musical instrument recognition for isolated notes , 2011, 2011 International Conference on Emerging Trends in Electrical and Computer Technology.

[18]  Aishwarya Srivastava,et al.  An automatic classification of bird species using audio feature extraction and support vector machines , 2016, 2016 International Conference on Inventive Computation Technologies (ICICT).

[19]  Juan Gabriel Aviña-Cervantes,et al.  Pitch estimation for musical note recognition using Artificial Neural Networks , 2014, 2014 International Conference on Electronics, Communications and Computers (CONIELECOMP).

[20]  Eui-nam Huh,et al.  Fog Computing and Smart Gateway Based Communication for Cloud of Things , 2014, 2014 International Conference on Future Internet of Things and Cloud.

[21]  Beth Logan,et al.  A music similarity function based on signal analysis , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[22]  Christian Bonnet,et al.  Fog Computing architecture to enable consumer centric Internet of Things services , 2015, 2015 International Symposium on Consumer Electronics (ISCE).

[23]  Lida Xu,et al.  Feature space theory - a mathematical foundation for data mining , 2001, Knowl. Based Syst..

[24]  Christopher Raphael,et al.  Music score alignment and computer accompaniment , 2006, CACM.

[25]  Hongming Cai,et al.  A process-mining-based scenarios generation method for SOA application development , 2016, Service Oriented Computing and Applications.

[26]  Marcelo M. Wanderley,et al.  Indirect Acquisition of Violin Instrumental Controls from Audio Signal with Hidden Markov Models , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[27]  György Fazekas,et al.  Automatic Ontology Generation for Musical Instruments Based on Audio Analysis , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[28]  Francis R. Bach,et al.  An online em algorithm in hidden (semi-)Markov models for audio segmentation and clustering , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[29]  Sascha Spors,et al.  Spatio-temporal signal preprocessing for multichannel acoustic echo cancellation , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[30]  Wei-Chen Chang,et al.  A novel recurrent network based pitch detection technique for quasi-periodic/pitch-varying signals , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[31]  G. Sivaranjani,et al.  EMOTION RECOGNITION FROM SPEECH WITH GAUSSIAN MIXTURE MODELS AND VIA BOOSTED GMM , 2018 .

[32]  Dharmendra S. Modha Cognitive computing: Neuroscience, supercomputing, nanotechnology , 2011, DAC 2011.