Develop a personalized intelligent music selection system based on heart rate variability and machine learning

Music often plays an important role in people’s daily lives. Because it has the power to affect human emotion, music has gained a place in work environments and in sports training as a way to enhance the performance of particular tasks. Studies have shown that office workers perform certain jobs better and joggers run longer distances when listening to music. However, a personalized music system which can automatically recommend songs according to user’s physiological response remains absent. Therefore, this study aims to establish an intelligent music selection system for individual users to enhance their learning performance. We first created an emotional music database using data analytics classifications. During testing, innovative wearable sensing devices were used to detect heart rate variability (HRV) in experiments, which subsequently guided music selection. User emotions were then analyzed and appropriate songs were selected by using the proposed application software (App). Machine learning was used to record user preference, ensuring accurate and precise classification. Significant results generated through experimental validation indicate that this system generates high satisfaction levels, does not increase mental workload, and improves users’ performance. Under the trend of the Internet of Things (IoT) and the continuing development of wearable devices, the proposed system could stimulate innovative applications for smart factory, home, and health care.

[1]  David F. Larcker,et al.  Structural Equation Models with Unobservable Variables and Measurement Error: Algebra and Statistics: , 1981 .

[2]  P. Laukka,et al.  Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening , 2004 .

[3]  Ming-Chuan Chiu,et al.  Integrating psychological and physiological techniques to measure and improve usability an empirical study on health management sport applied product , 2014, Proceedings of the 2014 IEEE 18th International Conference on Computer Supported Cooperative Work in Design (CSCWD).

[4]  Sammy W. Pearson,et al.  Development of a Tool for Measuring and Analyzing Computer User Satisfaction , 1983 .

[5]  Qiang Ji,et al.  Multiple emotional tagging of multimedia data by exploiting dependencies among emotions , 2013, Multimedia Tools and Applications.

[6]  Yueting Zhuang,et al.  Popular music retrieval by detecting mood , 2003, SIGIR.

[7]  B. Scholl Objects and attention: the state of the art , 2001, Cognition.

[8]  Izak Benbasat,et al.  Business Competence of Information Technology Professionals: Conceptual Development and Influence on IT-Business Partnerships , 2004, MIS Q..

[9]  K. Watson The nature and measurement of musical meanings , 1942 .

[10]  J. Davitz,et al.  The communication of emotional meaning , 1964 .

[11]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[12]  J. Russell,et al.  A cross-cultural study of a circumplex model of affect. , 1989 .

[13]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[14]  Po-Hsin Huang,et al.  INTEGRATING PHYSIOLOGICAL AND PSYCHOLOGICAL TECHNIQUES TO MEASURE AND IMPROVE USABILITY: AN EMPIRICAL STUDY ON KINECT APPLYING OF HEALTH MANAGEMENT SPORT , 2014 .

[15]  D. Larcker,et al.  STRUCTURAL EQUATION MODELS WITH UNOBSERVABLE VARIABLES AND MEASUREMENT ERROR , 1981 .

[16]  C. Liégeois-Chauvel,et al.  Brain regions involved in the recognition of happiness and sadness in music , 2005, Neuroreport.

[17]  Kian-Lee Tan,et al.  A novel framework for efficient automated singer identification in large music databases , 2009, TOIS.

[18]  Yi-Hsuan Yang,et al.  A Regression Approach to Music Emotion Recognition , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Antoni Ligeza Artificial Intelligence: A Modern Approach: By Stuart Russell and Peter Norwig. ISBN# 0-13-103805-2, pp. 932 + 28. Prentice Hall, Englewood Cliffs, New Jersey, 1995 , 1995 .

[20]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[21]  Tomohiko Ise,et al.  Scenery based Kansei music selection for car audio systems , 2009, 2009 IEEE 13th International Symposium on Consumer Electronics.

[22]  Brian P. Bailey,et al.  DJogger: a mobile dynamic music device , 2006, CHI Extended Abstracts.

[23]  A. Gabrielsson,et al.  The influence of musical structure on emotional expression. , 2001 .

[24]  I. Fónagy,et al.  Emotional Patterns in Intonation and Music , 1963 .

[25]  Lie Lu,et al.  Automatic mood detection and tracking of music audio signals , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[26]  Nuria Oliver,et al.  PAPA: Physiology and Purpose-Aware Automatic Playlist Generation , 2006, ISMIR.

[27]  C. Goose,et al.  Glossary of Terms , 2004, Machine Learning.

[28]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[29]  M. Eysenck,et al.  Personality and Individual Differences: A Natural Science Approach , 1985 .

[30]  Sheue-Ling Hwang,et al.  Investigating the Relationship between Therapeutic Music and Emotion: A Pilot Study on Healthcare Service , 2014, ISPE CE.

[31]  David J. Hand,et al.  Discrimination and Classification , 1982 .

[32]  I. Peretz,et al.  A developmental study of the affective value of tempo and mode in music , 2001, Cognition.

[33]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[34]  K. Newman,et al.  Interrogating SERVQUAL: a critical assessment of service quality measurement in a high street retail bank , 2001 .

[35]  H. Kida,et al.  The Effects of Physical Exercise with Music on Cognitive Function of Elderly People: Mihama-Kiho Project , 2014, PloS one.

[36]  Shannon H Pennington Blood, sweat and fears. , 2011, JEMS : a journal of emergency medical services.

[37]  A. Parasuraman,et al.  A Conceptual Model of Service Quality and Its Implications for Future Research , 1985 .

[38]  David G. Stork,et al.  Pattern Classification , 1973 .

[39]  Qiang Li,et al.  MusicalHeart: a hearty way of listening to music , 2012, SenSys '12.

[40]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[41]  E. Glenn Schellenberg,et al.  Perceiving emotion in melody: Interactive effects of pitch and rhythm. , 2000 .

[42]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[43]  Seungmin Rho,et al.  Music emotion classification and context-based music recommendation , 2010, Multimedia Tools and Applications.

[44]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .

[45]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[46]  G. Fairbanks,et al.  An experimental study of the pitch characteristics of the voice during the expression of emotion , 1939 .

[47]  Masahiro Niitsuma,et al.  Development of an Automatic Music Selection System Based on Runner's Step Frequency , 2008, ISMIR.

[48]  J. Russell A circumplex model of affect. , 1980 .

[49]  Chia-Hung Yeh,et al.  Popular music representation: chorus detection & emotion recognition , 2013, Multimedia Tools and Applications.

[50]  Zhouyu Fu,et al.  A Survey of Audio-Based Music Classification and Annotation , 2011, IEEE Transactions on Multimedia.

[51]  Xavier Serra,et al.  Chroma Binary Similarity and Local Alignment Applied to Cover Song Identification , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[52]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[53]  Y. K. Park,et al.  SERVICE QUALITY ANALYSIS AND IMPROVEMENT: DEVELOPMENT OFA SYSTEMATIC FRAMEWORK , 2006 .

[54]  Marek Malik,et al.  Standard Measurement of Heart Rate Variability , 2007 .

[55]  Yi-Hsuan Yang,et al.  Machine Recognition of Music Emotion: A Review , 2012, TIST.

[56]  Blake Ives,et al.  The measurement of user information satisfaction , 1983, CACM.