Visual Background Recommendation for Dance Performances Using Deep Matrix Factorization

The stage background is one of the most important features for a dance performance, as it helps to create the scene and atmosphere. In conventional dance performances, the background images are usually selected or designed by professional stage designers according to the theme and the style of the dance. In new media dance performances, the stage effects are usually generated by media editing software. Selecting or producing a dance background is quite challenging and is generally carried out by skilled technicians. The goal of the research reported in this article is to ease this process. Instead of searching for background images from the sea of available resources, dancers are recommended images that they are more likely to use. This work proposes the idea of a novel system to recommend images based on content-based social computing. The core part of the system is a probabilistic prediction model to predict a dancer’s interests in candidate images through social platforms. Different from traditional collaborative filtering or content-based models, the model proposed here effectively combines a dancer’s social behaviors (rating action, click action, etc.) with the visual content of images shared by the dancer using deep matrix factorization (DMF). With the help of such a system, dancers can select from the recommended images and set them as the backgrounds of their dance performances through a media editor. According to the experiment results, the proposed DMF model outperforms the previous methods, and when the dataset is very sparse, the proposed DMF model shows more significant results.

[1]  Abhinandan Das,et al.  Google news personalization: scalable online collaborative filtering , 2007, WWW '07.

[2]  Tat-Seng Chua,et al.  Learning Image and User Features for Recommendation in Social Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Paolo Avesani,et al.  Trust-aware recommender systems , 2007, RecSys '07.

[4]  A. M. Arymurthy,et al.  A syntactical modeling and classification for performance evaluation of Bali traditional dance , 2012, 2012 International Conference on Advanced Computer Science and Information Systems (ICACSIS).

[5]  Ye Wang,et al.  Improving Content-based and Hybrid Music Recommendation using Deep Learning , 2014, ACM Multimedia.

[6]  Michael R. Lyu,et al.  Learning to recommend with social trust ensemble , 2009, SIGIR.

[7]  Mark Claypool,et al.  Combining Content-Based and Collaborative Filters in an Online Newspaper , 1999, SIGIR 1999.

[8]  Benjamin Schrauwen,et al.  Deep content-based music recommendation , 2013, NIPS.

[9]  Alexis Clay,et al.  Integrating augmented reality to enhance expression, interaction & collaboration in live performances: A ballet dance case study , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality - Media, Art, Social Science, Humanities and Design (ISMAR-MASH'D).

[10]  David C. Wilson,et al.  Temporal integration of interactive technology in dance: creative process impacts , 2011, C&C '11.

[11]  David C. Wilson,et al.  Exploring the design space in technology-augmented dance , 2010, CHI EA '10.

[12]  Raymond J. Mooney,et al.  Content-boosted collaborative filtering for improved recommendations , 2002, AAAI/IAAI.

[13]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[14]  Hailin Jin,et al.  Collaborative feature learning from social media , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jean-Baptiste de la Rivière,et al.  Interactions and systems for augmenting a live dance performance , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH).

[16]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[17]  Pai H. Chou,et al.  A wearable wireless sensor platform for interactive dance performances , 2006, Fourth Annual IEEE International Conference on Pervasive Computing and Communications (PERCOM'06).

[18]  Deborah Estrin,et al.  Beyond Classification: Latent User Interests Profiling from Visual Contents Analysis , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[19]  James She,et al.  Visual Background Recommendation for Dance Performances Using Dancer-Shared Images , 2016, 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).

[20]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[21]  James She,et al.  Connection discovery using shared images by Gaussian relational topic model , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Jun Wang,et al.  Unifying user-based and item-based collaborative filtering approaches by similarity fusion , 2006, SIGIR.

[24]  Dit-Yan Yeung,et al.  Collaborative Deep Learning for Recommender Systems , 2014, KDD.

[25]  Ian Soboroff. Charles Nicholas Combining Content and Collaboration in Text Filtering , 1999 .

[26]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[27]  Joseph A. Paradiso,et al.  Sensemble: A Wireless, Compact, Multi-User Sensor System for Interactive Dance , 2006, NIME.

[28]  Tina Eliassi-Rad,et al.  A Probabilistic Model for Using Social Networks in Personalized Item Recommendation , 2015, RecSys.

[29]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[30]  Loren Olson,et al.  Movement-based interactive dance performance , 2006, MM '06.