OmniArt

Baselines are the starting point of any quantitative multimedia research, and benchmarks are essential for pushing those baselines further. In this article, we present baselines for the artistic domain with a new benchmark dataset featuring over 2 million images with rich structured metadata dubbed OmniArt. OmniArt contains annotations for dozens of attribute types and features semantic context information through concepts, IconClass labels, color information, and (limited) object-level bounding boxes. For our dataset we establish and present baseline scores on multiple tasks such as artist attribution, creation period estimation, type, style, and school prediction. In addition to our metadata related experiments, we explore the color spaces of art through different types and evaluate a transfer learning object recognition pipeline.

[1]  Babak Saleh,et al.  Quantifying Creativity in Art Networks , 2015, ICCC.

[2]  Mohammad Soleymani,et al.  VSD, a public dataset for the detection of violent scenes in movies: design, annotation, analysis and evaluation , 2014, Multimedia Tools and Applications.

[3]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[4]  Thomas Mensink,et al.  The Rijksmuseum Challenge: Museum-Centered Visual Recognition , 2014, ICMR.

[5]  David A. Shamma,et al.  YFCC100M , 2015, Commun. ACM.

[6]  Mohamed Elhoseiny,et al.  The Shape of Art History in the Eyes of the Machine , 2018, AAAI.

[7]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[8]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[10]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[11]  Lior Wolf,et al.  Classification of Artistic Styles Using Binarized Features Derived from a Deep Neural Network , 2014, ECCV Workshops.

[12]  Jia Li,et al.  Image processing for artist identification , 2008, IEEE Signal Processing Magazine.

[13]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[14]  Marcel Worring,et al.  OmniArt: Multi-task Deep Learning for Artistic Data Analysis , 2017, ArXiv.

[15]  Eric O. Postma,et al.  Toward Discovery of the Artist's Style: Learning to recognize artists by their artworks , 2015, IEEE Signal Processing Magazine.

[16]  Sergei O. Kuznetsov,et al.  Frequent Itemset Mining for Clustering Near Duplicate Web Documents , 2009, ICCS.

[17]  Florian Metze,et al.  Query by Example Search on Speech at Mediaeval 2015 , 2014, MediaEval.

[18]  L. D. Couprie.  Iconclass: an iconographic classification system , 1983 .

[19]  Florian Yger,et al.  Recognizing Art Style Automatically in Painting with Deep Learning , 2017, ACML.

[20]  Hongping Cai,et al.  Detecting People in Artwork with CNNs , 2016, ECCV Workshops.

[21]  Trevor Darrell,et al.  Recognizing Image Style , 2013, BMVC.

[22]  Li Fei-Fei,et al.  CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Gjorgji Strezoski.  Plug-and-Play Interactive Deep Network Visualization , 2017 .

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[26]  Georges Quénot,et al.  TRECVID 2017: Evaluating Ad-hoc and Instance Video Search, Events Detection, Video Captioning and Hyperlinking , 2017, TRECVID.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  J. C. Rush.  Acquiring a Concept of Painting Style , 1979 .

[29]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[30]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Henning Müller,et al.  Div150Multi: a social image retrieval result diversification dataset with multi-topic queries , 2016, MMSys.

[32]  Andrew Zisserman,et al.  In Search of Art , 2014, ECCV Workshops.

[33]  Mohammad Soleymani,et al.  The Benchmarking Initiative for Multimedia Evaluation: MediaEval 2016 , 2017, IEEE Multim..

[34]  Paul Over,et al.  The TREC2001 Video Track: Information Retrieval on Digital Video Information , 2002, ECDL.

[35]  I Ignatov Dmitry,et al.  Frequent Itemset Mining for Clustering Near Duplicate Web Documents , 2009 .

[36]  James She,et al.  DeepArt: Learning Joint Representations of Visual Arts , 2017, ACM Multimedia.

[37]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[38]  Emmanuel Dellandréa,et al.  LIRIS-ACCEDE: A Video Database for Affective Content Analysis , 2015, IEEE Transactions on Affective Computing.

[39]  Francesco G. B. De Natale,et al.  Synchronization of Multi-User Event Media (SEM) at MediaEval 2014: Task Description, Datasets, and Evaluation , 2014, MediaEval.

[40]  Roy S. Berns,et al.  A Color Target for Museum Applications , 2010, Color Imaging Conference.

[41]  Babak Saleh,et al.  Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Feature , 2015, ArXiv.

[42]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Jitendra Malik,et al.  Detecting people in Cubist art , 2014, SIGAI.

[44]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[45]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[46]  Yan Kang,et al.  Picasso, Matisse, or a Fake? Automated Analysis of Drawings at the Stroke Level for Attribution and Authentication , 2017, AAAI.

[47]  Sergio Escalera,et al.  ChaLearn Joint Contest on Multimedia Challenges Beyond Visual Analysis: An overview , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).