An InfoVis Tool for Interactive Component-Based Evaluation

In this paper, we present an InfoVis tool based on Sankey diagrams for the exploration of large combinatorial combinations of IR components - the Grid of Points (GoP). The goal of this tool is to ease the comprehension of the behavior of single IR components within fully functioning off-the-shelf IR systems without recurring to complex statistical tools.

[1]  Giuseppe Santucci,et al.  VIRTUE: A visual tool for information retrieval performance evaluation and failure analysis , 2014, J. Vis. Lang. Comput..

[2]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[3]  Wendy A. Lawrence-Fowler,et al.  Integrating query thesaurus, and documents through a common visual representation , 1991, SIGIR '91.

[4]  Fabio Crestani,et al.  A graphical user interface for the retrieval of hierarchically structured documents , 2004, Inf. Process. Manag..

[5]  Michael Lewis,et al.  Testing visual information retrieval methodologies case study: Comparative analysis of textual, icon, graphical, and spring displays , 2002, J. Assoc. Inf. Sci. Technol..

[6]  Sherry Koshman.  Testing user interaction with a prototype visualization-based information retrieval system , 2005, J. Assoc. Inf. Sci. Technol..

[7]  Jin Zhang.  TOFIR: A tool of facilitating information retrieval - introduce a visual retrieval model , 2001, Inf. Process. Manag..

[8]  Nicola Ferro,et al.  Toward an anatomy of IR system component performances , 2018, J. Assoc. Inf. Sci. Technol..

[9]  Giuseppe Santucci,et al.  CLAIRE: A combinatorial visual analytics system for information retrieval evaluation , 2018, Inf. Process. Manag..

[10]  Nicola Ferro,et al.  3.5K runs, 5K topics, 3M assessments and 70M measures: What trends in 10 years of Adhoc-ish CLEF? , 2017, Inf. Process. Manag..

[11]  Marti A. Hearst.  'Natural' search user interfaces , 2011, CACM.

[12]  Giuseppe Santucci,et al.  A Visual Analytics Approach for What-If Analysis of Information Retrieval Systems , 2016, SIGIR.

[13]  Nicola Ferro,et al.  A General Linear Mixed Models Approach to Study System Component Effects , 2016, SIGIR.

[14]  Allan Hanbury,et al.  Visual Pool: A Tool to Visualize and Interact with the Pooling Method , 2017, SIGIR.

[15]  Jin Zhang.  Visualization for Information Retrieval , 2018, Encyclopedia of Database Systems.