The automatic annotation and retrieval of digital images of prints and tile panels using network link analysis algorithms

The study of the visual art of printmaking is fundamental for art history. Printmaking methods have been used for centuries to replicate visual art works, which have influenced generations of artists. Particularly in this work, we are interested in the influence of prints on artistic tile panel painters, who have produced an impressive body of work in Portugal. The study of such panels has gained interest by art historians, who try to understand the influence of prints on tile panels artists in order to understand the evolution of this type of visual arts. Several databases of digitized art images have been used for such end, but the use of these databases relies on manual image annotations, an effective internal organization, and an ability of the art historian to visually recognize relevant prints. We propose an automation of these tasks using statistical pattern recognition techniques that takes into account not only the manual annotations available, but also the visual characteristics of the images. Specifically, we introduce a new network link-analysis method for the automatic annotation and retrieval of digital images of prints. Using a database of 307 annotated images of prints, we show that the annotation and retrieval results produced by our approach are better than the results of state-of-the-art content-based image retrieval methods.

[1]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[2]  Siwei Lyu,et al.  A digital technique for art authentication , 2004, Proc. Natl. Acad. Sci. USA.

[3]  Eric O. Postma,et al.  Computerized visual analysis of paintings , 2005 .

[4]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[5]  Wei-Ying Ma,et al.  Imagerank : spectral techniques for structural analysis of image database , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[6]  Jing Liu,et al.  Image annotation via graph learning , 2009, Pattern Recognit..

[7]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[8]  Jianping Fan,et al.  Structured max-margin learning for multi-label image annotation , 2010, CIVR '10.

[9]  Shannon M. Hughes,et al.  Detection of forgery in paintings using supervised learning , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[10]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[11]  Gustavo Carneiro,et al.  Supervised Learning of Semantic Classes for Image Annotation and Retrieval , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[13]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[14]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  James Ze Wang,et al.  Studying digital imagery of ancient paintings by mixtures of stochastic models , 2004, IEEE Transactions on Image Processing.

[16]  Henri Maître,et al.  15 years of image processing and the fine arts , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[17]  David J. Fleet,et al.  Stochastic Image Denoising , 2009, BMVC.

[18]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  John Langford,et al.  Multi-Label Prediction via Compressed Sensing , 2009, NIPS.

[20]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[21]  Michael I. Jordan,et al.  Link Analysis, Eigenvectors and Stability , 2001, IJCAI.

[22]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Allan Borodin,et al.  Link analysis ranking: algorithms, theory, and experiments , 2005, TOIT.

[24]  Tommi S. Jaakkola,et al.  Partially labeled classification with Markov random walks , 2001, NIPS.

[25]  Jia Li,et al.  Image processing for artist identification , 2008, IEEE Signal Processing Magazine.

[26]  Ramesh C. Jain,et al.  Semi-supervised annotation of brushwork in paintings domain using serial combinations of multiple experts , 2006, ACM Multimedia.

[27]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Ohad Shamir,et al.  Multiclass-Multilabel Classification with More Classes than Examples , 2010, AISTATS.

[29]  Stefanie Nowak,et al.  Performance measures for multilabel evaluation: a case study in the area of image classification , 2010, MIR '10.

[30]  Andrew Zisserman,et al.  Multiple kernels for object detection , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[31]  Daniel J. Graham,et al.  Stylometrics of artwork: uses and limitations , 2010, Electronic Imaging.

[32]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[33]  Shannon M. Hughes,et al.  Stylistic analysis of paintings usingwavelets and machine learning , 2009, 2009 17th European Signal Processing Conference.

[34]  Ben Taskar,et al.  Learning structured prediction models: a large margin approach , 2005, ICML.

[35]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[36]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[37]  D. Field,et al.  Mapping the similarity space of paintings: Image statistics and visual perception , 2010 .