Interactive, incremental and multi-level exploration of large collections of images. ( Exploration interactive, incrémentale et multi-niveau de larges collections d'images)

The research work that is presented and discussed in this thesis focuses on large and evergrowing image collections. More specifically, we aim at providing one the possibility to explore such image collections, either to extract some kind of information and knowledge, or to wander in the collections. This thesis addresses this issue from the perspective of Interactive Data Exploration and Analytics. We take advantage of the similarity-based image collection browsing paradigm and aim at meeting simultaneously the three following constraints: (i) handling large image collections, up to millions of images, (ii) handling dynamic image collections, to deal with ever-growing image collections, and (iii) providing interactive means to explore image collections. To do so, we jointly study the indexing and the interactive visualisation of large and ever-growing image collections. Our contribution is three-fold. First, we focus on the incremental construction of a proximity graph, namely the Relative Neighbourhood Graph (RNG), to structure the image collections. Second, we propose a hierarchical and graph-based hybrid viewable structure to allow an interactive exploration of large image collections. A data partitioning algorithm, namely BIRCH, is used to yield this structure. Last, we present our interactive visualisation platform for large image collections. To evaluate the platform, along with an user evaluation, several different use cases are discussed such as a recommendation system for ever-growing image collections and visual analysis system for Document Image Analysis. These use cases are based on real world large image collections, that contain up to seven millions of images.

[1]  Kevin W. Boyack,et al.  OpenOrd: an open-source toolbox for large graph layout , 2011, Electronic Imaging.

[2]  Satoru Kawai,et al.  An Algorithm for Drawing General Undirected Graphs , 1989, Inf. Process. Lett..

[3]  Kerry Rodden,et al.  Does organisation by similarity assist image browsing? , 2001, CHI.

[4]  Horst Bunke,et al.  Using a Statistical Language Model to Improve the Performance of an HMM-Based Cursive Handwriting Recognition System , 2001, Int. J. Pattern Recognit. Artif. Intell..

[5]  Gerhard Friedrich,et al.  Recommender Systems - An Introduction , 2010 .

[6]  Yifan Hu,et al.  Efficient, High-Quality Force-Directed Graph Drawing , 2006 .

[7]  James T. Enns,et al.  Attention and Visual Memory in Visualization and Computer Graphics , 2012, IEEE Transactions on Visualization and Computer Graphics.

[8]  Fernando Pereira,et al.  MPEG-7: A Standard for Multimedia Content Description , 2001, Int. J. Image Graph..

[9]  Ajay Divakaran,et al.  MPEG-7 visual motion descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[10]  Daniel A. Keim,et al.  Visual Analytics: Definition, Process, and Challenges , 2008, Information Visualization.

[11]  Sigmar-Olaf Tergan,et al.  Visualizing Knowledge and Information: An Introduction , 2005, Knowledge and Information Visualization.

[12]  Bart Thomee,et al.  New trends and ideas in visual concept detection: the MIR flickr retrieval evaluation initiative , 2010, MIR '10.

[13]  Denis Gracanin,et al.  Large Image Collections - Comprehension and Familiarization by Interactive Visual Analysis , 2009, Smart Graphics.

[14]  Mihaela Scuturici,et al.  Modèle topologique pour l'interrogation des bases d'images , 2004, EGC.

[15]  Stefan M. Rüger,et al.  NNk Networks for Content-Based Image Retrieval , 2004, ECIR.

[16]  Jun Ma,et al.  Similarity-based visualization of large image collections , 2015, Inf. Vis..

[17]  Jean-Daniel Fekete,et al.  Author Manuscript, Published in "sigchi Conference on Human Factors in Computing Systems Topology-aware Navigation in Large Networks , 2022 .

[18]  Xing Xie,et al.  Effective browsing of web image search results , 2004, MIR '04.

[19]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[20]  C. Won,et al.  Efficient Use of MPEG‐7 Edge Histogram Descriptor , 2002 .

[21]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[22]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Radford M. Neal,et al.  Multiple Alignment of Continuous Time Series , 2004, NIPS.

[24]  Mor Naaman,et al.  Generating summaries and visualization for large collections of geo-referenced photographs , 2006, MIR '06.

[25]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[26]  Jean-Yves Ramel,et al.  Flexible Sequence Matching Technique: Application to Word Spotting in Degraded Documents , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[27]  Mihaela Scuturici,et al.  Topological Query in Image Databases , 2003, CIARP.

[28]  Rong Huang,et al.  Font Distribution Observation by Network-Based Analysis , 2013, CBDAR.

[29]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[30]  W. Relative Neighborhood Graphs and Their Relatives , 2004 .

[31]  Yaokai Feng,et al.  Character Image Patterns as Big Data , 2012, 2012 International Conference on Frontiers in Handwriting Recognition.

[32]  Gerald Schaefer,et al.  Visualisation and Browsing of Image Databases , 2011, Multimedia Analysis, Processing and Communications.

[33]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[34]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..

[35]  Yuzuru Tanaka,et al.  Slit Style HOG Feature for Document Image Word Spotting , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[36]  B. S. Manjunath,et al.  Color and texture descriptors , 2001, IEEE Trans. Circuits Syst. Video Technol..

[37]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[38]  M. Jacomy,et al.  ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi Software , 2014, PloS one.