Surfing on artistic documents with visually assisted tagging

This paper describes a complete architecture for the interactive exploration and annotation of artistic collections. In particular the focus is on Renaissance illuminated manuscripts, which typically contain thousands of pictures, used to comment or embellish the manuscript Gothic text. The final aim is to create a human centered multimedia application allowing the non practitioners to enjoy these masterpieces and expert users to share their knowledge. The system is composed by a modern user interface for browsing, surfing and querying, an automatic segmentation module, to ease the initial picture extraction task, and a similarity based retrieval engine, used to provide visually assisted tagging capabilities. A relevance feedback procedure is included to further refine the results. Experiments are reported regarding the adopted visual features based on covariance matrices and the Mean Shift Feature Space Warping relevance feedback. Finally some hints on the user interface for museum installations are discussed.

[1]  Kinji Ono,et al.  Digital bleaching and content extraction for the digital archive of rare books , 2006, Second International Conference on Document Image Analysis for Libraries (DIAL'06).

[2]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[3]  Tsuhan Chen,et al.  Feature space warping: an approach to relevance feedback , 2002, Proceedings. International Conference on Image Processing.

[4]  Arnold W. M. Smeulders,et al.  Real-time bag of words, approximately , 2009, CIVR '09.

[5]  Jean-Yves Ramel,et al.  AGORA: the interactive document image analysis tool of the BVH project , 2006, Second International Conference on Document Image Analysis for Libraries (DIAL'06).

[6]  Fatih Murat Porikli,et al.  Pedestrian Detection via Classification on Riemannian Manifolds , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Rita Cucchiara,et al.  Optimized Block-Based Connected Components Labeling With Decision Trees , 2010, IEEE Transactions on Image Processing.

[8]  Falk Scholer,et al.  User performance versus precision measures for simple search tasks , 2006, SIGIR.

[9]  George Nagy,et al.  Twenty Years of Document Image Analysis in PAMI , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Rita Cucchiara,et al.  Picture extraction from digitized historical manuscripts , 2009, CIVR '09.

[11]  Meng Wang,et al.  Visual tag dictionary: interpreting tags with visual words , 2009, WSMC '09.

[12]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[13]  Ahmed Elgammal,et al.  Human-centered multimedia: representations and challenges , 2006, HCM '06.

[14]  J. Ogier,et al.  Madonne: Document Image Analysis Techniques for Cultural Heritage Documents , 2006 .

[15]  Marcel Worring,et al.  Learning tag relevance by neighbor voting for social image retrieval , 2008, MIR '08.

[16]  Thomas S. Huang,et al.  Content-based image retrieval with relevance feedback in MARS , 1997, Proceedings of International Conference on Image Processing.

[17]  Frank Lebourgeois,et al.  DEBORA: Digital AccEss to BOoks of the RenAissance , 2006, International Journal of Document Analysis and Recognition (IJDAR).

[18]  Dorothea Blostein,et al.  A survey of document image classification: problem statement, classifier architecture and performance evaluation , 2007, International Journal of Document Analysis and Recognition (IJDAR).

[19]  Shih-Fu Chang,et al.  Query-Adaptive Fusion for Multimodal Search , 2008, Proceedings of the IEEE.

[20]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[21]  Tsuhan Chen,et al.  Mean shift feature space warping for relevance feedback , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[22]  Heung-Kyu Lee,et al.  Re-ranking algorithm using post-retrieval clustering for content-based image retrieval , 2005, Inf. Process. Manag..

[23]  Jean-Yves Ramel,et al.  Document image characterization using a multiresolution analysis of the texture: application to old documents , 2008, International Journal of Document Analysis and Recognition (IJDAR).

[24]  W. Förstner,et al.  A Metric for Covariance Matrices , 2003 .

[25]  Nanning Zheng,et al.  Document Images Retrieval Based on Multiple Features Combination , 2007 .

[26]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[27]  Thierry Paquet,et al.  Document Image Segmentation Using a 2D Conditional Random Field Model , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).