6 Seconds of Sound and Vision: Creativity in Micro-videos

The notion of creativity, as opposed to related concepts such as beauty or interestingness, has not been studied from the perspective of automatic analysis of multimedia content. Meanwhile, short online videos shared on social media platforms, or micro-videos, have arisen as a new medium for creative expression. In this paper we study creative micro-videos in an effort to understand the features that make a video creative, and to address the problem of automatic detection of creative content. Defining creative videos as those that are novel and have aesthetic value, we conduct a crowdsourcing experiment to create a dataset of over 3, 800 micro-videos labelled as creative and non-creative. We propose a set of computational features that we map to the components of our definition of creativity, and conduct an analysis to determine which of these features correlate most with creative video. Finally, we evaluate a supervised approach to automatically detect creative video, with promising results, showing that it is necessary to model both aesthetic value and novelty to achieve optimal classification accuracy.

[1]  R. Weisberg Creativity: Beyond the Myth of Genius , 1993 .

[2]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[3]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[4]  Gabriela Csurka,et al.  Assessing the aesthetic quality of photographs using generic image descriptors , 2011, 2011 International Conference on Computer Vision.

[5]  Masashi Nishiyama,et al.  Aesthetic quality classification of photographs based on color harmony , 2011, CVPR 2011.

[6]  M. Mumford Where Have We Been, Where Are We Going? Taking Stock in Creativity Research , 2003 .

[7]  O. Lartillot,et al.  A MATLAB TOOLBOX FOR MUSICAL FEATURE EXTRACTION FROM AUDIO , 2007 .

[8]  Petri Toiviainen,et al.  MIR in Matlab (II): A Toolbox for Musical Feature Extraction from Audio , 2007, ISMIR.

[9]  Vicente Ordonez,et al.  High level describable attributes for predicting aesthetics and interestingness , 2011, CVPR 2011.

[10]  Brian A. Barsky,et al.  Can Motion Features Inform Video Aesthetic Preferences? , 2012 .

[11]  Petri Toiviainen,et al.  Exploring relationships between audio features and emotion in music , 2009 .

[12]  Mu Qiao,et al.  OSCAR: On-Site Composition and Aesthetics Feedback Through Exemplars for Photographers , 2012, International Journal of Computer Vision.

[13]  Dong Liu,et al.  Towards a comprehensive computational model foraesthetic assessment of videos , 2013, MM '13.

[14]  Mary Lou Maher,et al.  Evaluating creativity in humans, computers, and collectively intelligent systems , 2010, DESIRE.

[15]  David Wettergreen,et al.  Aesthetic Image Classification for Autonomous Agents , 2010, 2010 20th International Conference on Pattern Recognition.

[16]  Bernard Mérialdo,et al.  Saliency moments for image categorization , 2011, ICMR.

[17]  Jianxiong Xiao,et al.  What makes an image memorable? , 2011, CVPR 2011.

[18]  Lexis F. Higgins,et al.  Applying Principles of Creativity Management to Marketing Research Efforts in High-Technology Markets , 1999 .

[19]  H. Simon,et al.  The Processes of Creative Thinking , 1959 .

[20]  Xiaogang Wang,et al.  Content-based photo quality assessment , 2011, 2011 International Conference on Computer Vision.

[21]  I. Kant,et al.  Critique of judgment , 2000 .

[22]  Nuria Oliver,et al.  Towards Computational Models of the Visual Aesthetic Appeal of Consumer Videos , 2010, ECCV.

[23]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[24]  P. Valdez,et al.  Effects of color on emotions. , 1994, Journal of experimental psychology. General.

[25]  Yan Ke,et al.  The Design of High-Level Features for Photo Quality Assessment , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Francis Sparshott Basic Film Aesthetics. , 1971 .

[27]  Naila Murray,et al.  AVA: A large-scale database for aesthetic visual analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Mubarak Shah,et al.  A framework for photo-quality assessment and enhancement based on visual aesthetics , 2010, ACM Multimedia.

[31]  Bernard Mérialdo,et al.  Where is the beauty?: retrieving appealing VideoScenes by learning Flickr-based graded judgments , 2012, ACM Multimedia.

[32]  Mateu Sbert,et al.  Conceptualizing Birkhoff's Aesthetic Measure Using Shannon Entropy and Kolmogorov Complexity , 2007, CAe.