A design space approach to analysis of information retrieval adaptive filtering systems

In this paper we suggest a new approach to analysis and design of IR systems. We argue for design space exploration in constructing IR systems and in analyzing the effects of individual modules and parameters. We present results of experiments with parametric interpolation, or "homotopy", between two systems, and show, incidentally, that the best results are not achieved at the endpoints, and may lie outside the bounding hypercube defined by our choice of parameterization. Three distinct classes of interpolation are introduced to deal with the complexities of the specific example.

[1]  Stephen E. Robertson,et al.  The TREC 2002 Filtering Track Report , 2002, TREC.

[2]  Andrew Gelsey,et al.  High-Performance Supersonic Missile Inlet Design Using Automated Optimization , 1997 .

[3]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[4]  R. Kellogg,et al.  Pathways to solutions, fixed points, and equilibria , 1983 .

[5]  Bin Liu,et al.  TREC 11 Experiments at CAS-ICT: Filtering and Web , 2002, TREC.

[6]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[7]  Benjamin Bishop,et al.  An evolutionary approach for VLSI standard cell design , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[8]  Carol Peters,et al.  Cross-Language Evaluation Forum: Objectives, Results, Achievements , 2004, Information Retrieval.

[9]  Noriko Kando,et al.  Evaluation of Information Access Technologies at NTCIR Workshop , 2003, CLEF.

[10]  Andrew Gelsey,et al.  Intelligent gradient-based search of incompletely defined design spaces , 1997, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[11]  L. Steinberg,et al.  Optimization by searching a tree of populations , 1999 .

[12]  Andrew Gelsey,et al.  Using Modeling Knowledge to Guide Design Space Search , 1998, Artif. Intell..

[13]  Chris Buckley,et al.  OHSUMED: an interactive retrieval evaluation and new large test collection for research , 1994, SIGIR '94.

[14]  Paul B. Kantor,et al.  Rutgers Filtering Work at TREC 2002: Adaptive and Batch , 2002, TREC.

[15]  Simon Szykman.  Optimization as a Driver for Design Space Exploration , 1997 .

[16]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .