Music Recommendation System Using Human Activity Recognition From Accelerometer Data

Music listening is a very personal and situational behavior for modern people who always carry smartphones in everyday life. Therefore, contextual information, such as user’s current activity and mood state could be used to greatly improve music recommendations. In this paper, we develop a smartphone-based mobile system that includes two core modules for recognizing human activities and then accordingly recommending music. In the proposed method, a deep residual bidirectional gated recurrent neural network is applied to obtain high activity recognition accuracy from accelerometer signals on the smartphone. In order to improve the performance of tempo-oriented music classification, an ensemble of dynamic classification using a long-term modulation spectrum and sequence classification using a short-term spectrogram is used. Music recommendation is performed using the relationship between the recognized human activities and the music files indexed by tempo-oriented music classification that reflects user preference models in order to achieve high user satisfaction. The results of comprehensive experiments on real data confirm the accuracy of the proposed activity-aware music recommendation framework.

[1]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[2]  G. C. Elvers,et al.  Background Music and Cognitive Performance , 2010, Perceptual and motor skills.

[3]  Hamed Zamani,et al.  Current challenges and visions in music recommender systems research , 2017, International Journal of Multimedia Information Retrieval.

[4]  Jun Hu,et al.  Music Playlist Recommendation Based on User Heartbeat and Music Preference , 2009, 2009 International Conference on Computer Technology and Development.

[5]  Yan Song,et al.  Robust sound event recognition using convolutional neural networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  W. Marsden.  I and J , 2012 .

[7]  Miikka Vilermo,et al.  Modified Discrete Cosine Transform: Its Implications for Audio Coding and Error Concealment , 2003 .

[8]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[9]  Kyoungro Yoon,et al.  Music recommendation system using emotion triggering low-level features , 2012, IEEE Transactions on Consumer Electronics.

[10]  Francesco Ricci,et al.  Location-adapted music recommendation using tags , 2011, UMAP'11.

[11]  Mustafa E. Kamasak,et al.  Emotion Based Music Recommendation System Using Wearable Physiological Sensors , 2018, IEEE Transactions on Consumer Electronics.

[12]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[13]  Yuqing Chen,et al.  A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[14]  Tae-Seong Kim,et al.  Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home , 2012, IEEE Transactions on Consumer Electronics.

[15]  Seungmin Rho,et al.  Music emotion classification and context-based music recommendation , 2010, Multimedia Tools and Applications.

[16]  Ricardo Chavarriaga,et al.  The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition , 2013, Pattern Recognit. Lett..

[17]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[18]  Graça Bressan,et al.  Music recommendation system based on user's sentiments extracted from social networks , 2015, IEEE Transactions on Consumer Electronics.

[19]  Masataka Goto,et al.  Hybrid Collaborative and Content-based Music Recommendation Using Probabilistic Model with Latent User Preferences , 2006, ISMIR.

[20]  Erik Duval,et al.  Context-Aware Recommender Systems for Learning: A Survey and Future Challenges , 2012, IEEE Transactions on Learning Technologies.

[21]  Zafar Ali Khan,et al.  Abnormal human activity recognition system based on R-transform and kernel discriminant technique for elderly home care , 2011, IEEE Transactions on Consumer Electronics.

[22]  Philip S. Yu,et al.  Music Recommendation Using Content and Context Information Mining , 2010, IEEE Intelligent Systems.

[23]  Zuriñe Resa,et al.  Towards time-aware contextual music recommendation : an exploration of temporal patterns of music listening using Circular Statistics , 2010 .

[24]  Bandar Saleh Mouhammed ِAlmaslukh,et al.  An effective deep autoencoder approach for online smartphone-based human activity recognition , 2017 .

[25]  Robert F. Potter,et al.  Effects of Music on Physiological Arousal: Explorations into Tempo and Genre , 2007 .

[26]  Rui Lu.  BIDIRECTIONAL GRU FOR SOUND EVENT DETECTION , 2017 .

[27]  Jürgen Schmidhuber,et al.  LSTM can Solve Hard Long Time Lag Problems , 1996, NIPS.

[28]  Yu Zhao,et al.  Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors , 2017, Mathematical Problems in Engineering.

[29]  Riyanarto Sarno,et al.  Music tempo classification using audio spectrum centroid, audio spectrum flatness, and audio spectrum spread based on MPEG-7 audio features , 2017, 2017 3rd International Conference on Science in Information Technology (ICSITech).

[30]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[31]  Bill Tomlinson,et al.  PersonalSoundtrack: context-aware playlists that adapt to user pace , 2006, CHI Extended Abstracts.

[32]  Jun Liang,et al.  Residual Recurrent Neural Networks for Learning Sequential Representations , 2018, Inf..