Development of an Automatic Emotional Music Accompaniment System by Fuzzy Logic and Adaptive Partition Evolutionary Genetic Algorithm

Music is everywhere in the world, and its applications in commerce are extremely versatile. Generally speaking, in order to create some music for background music, it is necessary to engage sound recordists and instrumental performers. However, the process is very time-consuming and costly. In this paper, a real-time emotion-based music accompaniment system is proposed to solve this issue. For different emotions, a fuzzy logic controller is designed to adjust the tempo of the music, and an adaptive partition evolutionary genetic algorithm is developed to create corresponding melodies. The chord progressions are generated via music theory, and the instrumentation is disposed by the conception of the probability. What is noteworthy is that all the processes can be output by Virtual Studio Technology in real time so that users can listen directly to the composing results from any emotions. From the experimental results, the proposed adaptive partition evolutionary genetic algorithm performs better than other optimal algorithms in such topics.

[1]  Chung-Hsien Wu,et al.  Speaking Effect Removal on Emotion Recognition From Facial Expressions Based on Eigenface Conversion , 2013, IEEE Transactions on Multimedia.

[2]  Marc Leman,et al.  The Cortical Topography of Tonal Structures Underlying Western Music , 2002, Science.

[3]  Tung-Kuan Liu,et al.  Developing a Multiobjective Optimization Scheduling System for a Screw Manufacturer: A Refined Genetic Algorithm Approach , 2014, IEEE Access.

[4]  Thierry Pun,et al.  Multimodal Emotion Recognition in Response to Videos , 2012, IEEE Transactions on Affective Computing.

[5]  Hatice Gunes,et al.  Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space , 2011, IEEE Transactions on Affective Computing.

[6]  Jesús Ibáñez.  Emotional Sea: Showing Valence and Arousal Through the Sharpness and Movement of Digital Cartoonish Sea Waves , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[7]  Homer H. Chen,et al.  Emotional Accompaniment Generation System Based on Harmonic Progression , 2013, IEEE Transactions on Multimedia.

[8]  Yi-Hsuan Yang,et al.  A Regression Approach to Music Emotion Recognition , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[9]  R. Zatorre,et al.  Interactions Between the Nucleus Accumbens and Auditory Cortices Predict Music Reward Value , 2013, Science.

[10]  Alan Hanjalic,et al.  Affective video content representation and modeling , 2005, IEEE Transactions on Multimedia.

[11]  Jonghwa Kim,et al.  Transsituational Individual-Specific Biopsychological Classification of Emotions , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[12]  Erwie Zahara,et al.  A hybrid genetic algorithm and particle swarm optimization for multimodal functions , 2008, Appl. Soft Comput..

[13]  Jurij F. Tasic,et al.  Affective Labeling in a Content-Based Recommender System for Images , 2013, IEEE Transactions on Multimedia.

[14]  A. Sima Uyar,et al.  Automatic music generation using evolutionary algorithms and neural networks , 2011, 2011 International Symposium on Innovations in Intelligent Systems and Applications.

[15]  Nina G. Jablonski.  The naked truth. , 2010, Scientific American.

[16]  R. C. Wong,et al.  T-Music: A melody composer based on frequent pattern mining , 2013, 2013 IEEE 29th International Conference on Data Engineering (ICDE).

[17]  Yi-Hsuan Yang,et al.  Prediction of the Distribution of Perceived Music Emotions Using Discrete Samples , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[18]  Yi-Hsuan Yang,et al.  Ranking-Based Emotion Recognition for Music Organization and Retrieval , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Chien-Hung Liu,et al.  Evolutionary composition using music theory and charts , 2013, 2013 IEEE Symposium on Computational Intelligence for Creativity and Affective Computing (CICAC).

[20]  Jeffrey O Kephart.  Learning from Nature , 2011, Science.

[21]  Tomoko Matsui,et al.  Music Genre and Emotion Recognition Using Gaussian Processes , 2014, IEEE Access.

[22]  Tung-Kuan Liu,et al.  Solving Distributed and Flexible Job-Shop Scheduling Problems for a Real-World Fastener Manufacturer , 2014, IEEE Access.

[23]  Chang-Le Zhou,et al.  A computational model on harmonizing Chinese folksong with piano accompaniment , 2008, 2008 IEEE International Conference on Granular Computing.

[24]  Carlos Guedes,et al.  A CONTOUR-BASED JAZZ WALKING BASS GENERATOR , 2013 .

[25]  Neil T. Roach,et al.  Elastic energy storage in the shoulder and the evolution of high-speed throwing in Homo , 2013, Nature.

[26]  Chuan-Kang Ting,et al.  Polyphonic accompaniment using genetic algorithm with music theory , 2012, 2012 IEEE Congress on Evolutionary Computation.

[27]  D. Bramble,et al.  Endurance running and the evolution of Homo , 2004, Nature.

[28]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[29]  J. Russell.  A circumplex model of affect. , 1980 .

[30]  M.D. Korhonen,et al.  Modeling emotional content of music using system identification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Vladimiro Miranda,et al.  EPSO-evolutionary particle swarm optimization, a new algorithm with applications in power systems , 2002, IEEE/PES Transmission and Distribution Conference and Exhibition.

[32]  Harish Katti,et al.  CAVVA: Computational Affective Video-in-Video Advertising , 2014, IEEE Transactions on Multimedia.

[33]  Tuomas Eerola,et al.  Semantic Computing of Moods Based on Tags in Social Media of Music , 2013, IEEE Transactions on Knowledge and Data Engineering.

[34]  T. Onisawa,et al.  Construction of Music Composition System with Interactive Genetic Algorithm , 2003 .

[35]  Mike Thelwall,et al.  Seeing Stars of Valence and Arousal in Blog Posts , 2013, IEEE Transactions on Affective Computing.

[36]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[37]  Charalampos Bratsas,et al.  On the Classification of Emotional Biosignals Evoked While Viewing Affective Pictures: An Integrated Data-Mining-Based Approach for Healthcare Applications , 2010, IEEE Transactions on Information Technology in Biomedicine.

[38]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.