Newspaper Navigator: Open Faceted Search for 1.5 Million Images

Despite the marked improvements in image classification and recognition tasks over the past decade, affordances and interfaces for searching images have remained largely unchanged on the web. In this demo, we present Newspaper Navigator, an open faceted search system for 1.5 million historic newspaper photographs. In contrast to standard faceted search, which requires facets to be pre-selected and applied to images, Newspaper Navigator empowers users to specify their own facets in an open-domain fashion during the search process by selecting relevant examples and iteratively training a machine learner. With Newspaper Navigator, users can quickly sort 1.5 million images according to dynamically-specified facets such as "baseball player'' and "oval-shaped portrait.'' Newspaper Navigator also drives facet exploration by suggesting related keyword search queries for a user to perform. Our demo walks through examples of searching with Newspaper Navigator and highlights the facet learning and exploration affordances.

[1]  Desney S. Tan,et al.  CueFlik: interactive concept learning in image search , 2008, CHI.

[2]  Kate Zwaard,et al.  The Newspaper Navigator Dataset: Extracting And Analyzing Visual Content from 16 Million Historic Newspaper Pages in Chronicling America , 2020, ArXiv.

[3]  Marti A. Hearst,et al.  Hierarchical faceted metadata in site search interfaces , 2002, CHI Extended Abstracts.

[4]  Aniket Kittur,et al.  SearchLens: composing and capturing complex user interests for exploratory search , 2019, IUI.

[5]  Desney S. Tan,et al.  Overview based example selection in end user interactive concept learning , 2009, UIST '09.

[6]  Desney S. Tan,et al.  Examining multiple potential models in end-user interactive concept learning , 2010, CHI.

[7]  A. Azzouz 2011 , 2020, City.

[8]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[9]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Paul N. Bennett,et al.  Guidelines for Human-AI Interaction , 2019, CHI.

[12]  Desney S. Tan,et al.  Effective End-User Interaction with Machine Learning , 2011, AAAI.

[13]  Maya Cakmak,et al.  Power to the People: The Role of Humans in Interactive Machine Learning , 2014, AI Mag..

[14]  Eero Sormunen,et al.  End-User Searching Challenges Indexing Practices in the Digital Newspaper Photo Archive , 2004, Information Retrieval.

[15]  Kevin Li,et al.  Faceted metadata for image search and browsing , 2003, CHI '03.

[16]  Joemon M. Jose,et al.  Spatial querying for image retrieval: a user-oriented evaluation , 1998, SIGIR '98.