References and citations in automatic indexing and retrieval systems - experiments with the boomerang effect

The objective of the present dissertation work is to investigate the use of references and citations as an integrated part of automatic indexing and retrieval techniques operating on scientific full text documents. There are two main motivations behind the dissertation: Firstly, the few scattered studies from both the system-driven and the useroriented tradition that have investigated references and citations for information retrieval (IR) purposes, have generally shown promising results. Secondly, scientific documents are for the first time becoming available in large quantities in electronic form. This offers new possibilities for combining conventional automatic indexing and retrieval techniques with the exploitation of references and citations in IR. Analytical and empirical investigations are carried out with the aim of investigating which factors that affect the behaviour and performance of automatic indexing and retrieval techniques given that references and citations are an integrated part of the document representation of scientific full text documents in the IR system. It is investigated why references and citations might be useful in IR in an analytical review of the literature on citer motivations and citation behaviour. A normative and a social constructivist position on the motives of authors in citing other work is identified. The most unifying theoretical explanation is found to Small’s (1978) notion of references as ‘concept symbols’ that stand for an idea or concept that is being used in the course of an argument. This explanation is found attractive from the point of exploiting references and citations in IR, because regardless of whether or not some references are omitted, forgotten, biased etc. the references actually given function as symbols for a concept. This can explain why references and citations are useful in IR: Because the references represent concepts when used for document representation, they are well-suited for IR purposes as long as the user’s information need can be expressed in the same concepts, i.e., as seed documents. A review of earlier studies which have employed references and citations showed that the most common use of them for IR purposes was as seed documents in a forward chaining, where documents that refer to the seed document are retrieved. The ability of these seed documents to express the user’s information need was found to be the main factor affecting the behaviour and performance of references and citations in IR.

[1]  Marcia J. Bates,et al.  Information search tactics , 1979, J. Am. Soc. Inf. Sci..

[2]  Peter Ingwersen,et al.  Applying diachronic citation analysis to ongoing research program evaluations , 2000 .

[3]  Donald Michie,et al.  On Machine Intelligence , 1974 .

[4]  Stephen Cole,et al.  Testing the Ortega hypothesis: Milestone or millstone? , 1987, Scientometrics.

[5]  Nicholas J. Belkin,et al.  Retrieval techniques , 1987 .

[6]  Birger Larsen,et al.  Synchronous and diachronous citation analysis for information retrieval - generating a boomerang effect from the network of scientific papers , 2001 .

[7]  R. Merton Social Theory and Social Structure , 1958 .

[8]  Peter Ingwersen,et al.  The Turn - Integration of Information Seeking and Retrieval in Context , 2005, The Kluwer International Series on Information Retrieval.

[9]  C. Bazerman Shaping Written Knowledge: The Genre and Activity of the Experimental Article in Science , 1989 .

[10]  P. Ingwersen Cognitive Information Retrieval. , 1999 .

[11]  Eero Sormunen,et al.  Liberal relevance criteria of TREC -: counting on negligible documents? , 2002, SIGIR '02.

[12]  Ben-Ami Lipetz,et al.  Improvement of the selectivity of citation indexes to science literature through inclusion of citation relationship indicators , 1965 .

[13]  E. A. Fox,et al.  Combining the Evidence of Multiple Query Representations for Information Retrieval , 1995, Inf. Process. Manag..

[14]  Irene Wormell SAP — a new way to produce subject descriptions of books , 1981 .

[15]  William H. Mischo,et al.  Federating Diverse Collections of Scientific Literature , 1996, Computer.

[16]  J. Venn,et al.  . On the diagrammatic and mechanical representation of propositions and reasonings , 2022 .

[17]  Nicholas J. Belkin,et al.  Ask for Information Retrieval: Part I. Background and Theory , 1997, J. Documentation.

[18]  Helen M. Brooks,et al.  Expert systems and intelligent information retrieval , 1987, Inf. Process. Manag..

[19]  F. W. Lancaster,et al.  MEDLARS: Report on the Evaluation of Its Operating Efficiency. , 1997 .

[20]  F. C. Thorne The citation index: Another case of spurious validity , 1977 .

[21]  Blaise Cronin,et al.  The citation process: The role and significance of citations in scientific communication , 1984 .

[22]  Ellen M. Voorhees,et al.  The Tenth Text REtrieval Conference, TREC 2001 | NIST , 2002 .

[23]  C. Lee Giles,et al.  Autonomous citation matching , 1999, AGENTS '99.

[24]  B. C. Brookes The foundations of information science. Part I. Philosophical aspects , 1980 .

[25]  Ying-Hsang Liu,et al.  Intermediary's information seeking, inquiring minds, and elicitation styles , 2003, J. Assoc. Inf. Sci. Technol..

[26]  B. C. Griffith,et al.  The Structure of Scientific Literatures II: Toward a Macro- and Microstructure for Science , 1974 .

[27]  N. Fuhr PAN-Uncovering Plagiarism , Authorship , and Social Software Misuse ImageCLEF 2013-Cross Language Image Annotation and Retrieval INEX-INitiative for the Evaluation of XML retrieval , 2002 .

[28]  David Hung-Chang Du,et al.  Experimental study of extended HIPPI connections over ATM networks , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[29]  Miranda Lee Pao,et al.  Term and Citation Retrieval: A Field Study , 1993, Inf. Process. Manag..

[30]  Peter Ingwersen,et al.  A cognitive view of three selected online search facilities , 1984 .

[31]  Paul Over,et al.  TREC-6 Interactive Report , 1997, TREC.

[32]  Mounia Lalmas,et al.  Representing and retrieving structured documents using the Dempster-Shafer theory of evidence: modelling and evaluation , 1998, J. Documentation.

[33]  Stephen Robertson,et al.  The methodology of information retrieval experiment , 1981 .

[34]  Gerard Salton,et al.  AUTOMATIC INDEXING USING BIBLIOGRAPHIC CITATIONS , 1971 .

[35]  Tom Wilson,et al.  The cognitive approach to information-seeking behaviour and information use , 1984 .

[36]  B. Cronin,et al.  The web of knowledge: a festschrift in honor of Eugene Garfield , 2000 .

[37]  M. Moravcsik,et al.  Some Results on the Function and Quality of Citations , 1975 .

[38]  Diane Henderson ASIS '90 : proceedings of the 53rd ASIS Annual Meeting, Toronto, Ontario, November 4-8, 1990 , 1990 .

[39]  Anthony E. Cawkell Methods of information retrieval using Web of Science: pulmonary hypertension as a subject example , 2000, J. Inf. Sci..

[40]  Peter Ingwersen,et al.  The boomerang effect: retrieving scientific documents via the network of references and citations , 2002, SIGIR '02.

[41]  A. F. J. VAN RAAN,et al.  In matters of quantitative studies of science the fault of theorists is offering too little and asking too much , 1998, Scientometrics.

[42]  T. Brooks Evidence of complex citer motivations , 1986, J. Am. Soc. Inf. Sci..

[43]  M L Pao Titles retrieved from medline and from citation relations , 1985 .

[44]  M. M. Kessler Bibliographic coupling between scientific papers , 1963 .

[45]  Anthony E. Cawkell,et al.  Search strategy, construction and use of citation networks, with a socio-scientific example: "Amorphous semi-conductors and S.R. ovshinsky" , 1974, J. Am. Soc. Inf. Sci..

[46]  Jan W. Buzydlowski,et al.  Real-time author co-citation mapping for online searching , 2003, Inf. Process. Manag..

[47]  David A. Hull Using statistical testing in the evaluation of retrieval experiments , 1993, SIGIR.

[48]  Peter Ingwersen,et al.  Information Retrieval Interaction , 1992 .

[49]  Amanda Spink Study of interactive feedback during mediated information retrieval , 1997 .

[50]  E. B. Duncan,et al.  Qualified Citation Indexing: Its Relevance to Educational Technology. , 1981 .

[51]  Katherine W. McCain,et al.  Descriptor and citation retrieval in the medical behavioral sciences literature:retrieval overlaps and novelty distribution , 1989 .

[52]  Gerard Salton,et al.  Some experiments in the generation of word and document associations , 1899, AFIPS '62 (Fall).

[53]  Gabriella Kazai,et al.  Construction of a Test Collection for the Focussed Retrieval of Structured Documents , 2003, ECIR.

[54]  K. Sparck Jones,et al.  INFORMATION RETRIEVAL TEST COLLECTIONS , 1976 .

[55]  Harry Bruce Emerging frameworks and methods : CoLIS4 : proceedings of the Fourth International Conference on Conceptions of Library and Information Science, Seattle, WA, USA, July 21-25, 2002 , 2002 .

[56]  I. H. Witten,et al.  Digital Libraries 98 : the third ACM Conference on Digital Libraries, June 23-26, 1998, Pittsburgh, PA, , 1998 .

[57]  Stephen E. Robertson,et al.  Relevance weighting of search terms , 1976, J. Am. Soc. Inf. Sci..

[58]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[59]  Norman M Kaplan,et al.  The norms of citation behavior: Prolegomena to the footnote , 1965 .

[60]  Cyril W. Cleverdon,et al.  Factors determining the performance of indexing systems , 1966 .

[61]  N. Mohaghegh,et al.  WHY THE IMPACT FACTOR OF JOURNALS SHOULD NOT BE USED FOR EVALUATING RESEARCH , 2005 .

[62]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[63]  Jonathan C. L. Liu,et al.  Design and Evaluation of a Generic Software Architecture for On-Demand Video Servers , 1999, IEEE Trans. Knowl. Data Eng..

[64]  Stephen P. Harter,et al.  Online Information Retrieval: Concepts, Principles and Techniques , 1986 .

[65]  Kristian J. Hammond,et al.  Using citations to facilitate precise indexing and automatic index creation in collections of research papers , 2001, Knowl. Based Syst..

[66]  M. E. Maron,et al.  An evaluation of retrieval effectiveness for a full-text document-retrieval system , 1985, CACM.

[67]  Kalervo Järvelin,et al.  Targeted s-gram matching: a novel n-gram matching technique for cross- and mono-lingual word form variants , 2002, Inf. Res..

[68]  Richard M. Everson,et al.  When Are Links Useful? Experiments in Text Classification , 2003, ECIR.

[69]  Paul B. Kantor,et al.  A study of information seeking and retrieving. III. Searchers, searches, and overlap , 1988, J. Am. Soc. Inf. Sci..

[70]  Cyril W. Cleverdon,et al.  Report on the first stage of an investigation into the comparative efficiency of indexing systems , 1960 .

[71]  W. Bruce Croft,et al.  I 3 R: a new approach to the design of document retrieval systems , 1987 .

[72]  Jeff Conklin,et al.  Hypertext: An Introduction and Survey , 1987, Computer.

[73]  Paul B. Kantor,et al.  A Study of Information Seeking and Retrieving. III. Searchers, Searches, and Overlap* , 1988 .

[74]  Paul B. Kantor,et al.  A study of information seeking and retrieving. I. background and methodology , 1988 .

[75]  David Ellis,et al.  The Physical and Cognitive Paradigms in Information Retrieval Research , 1992, J. Documentation.

[76]  Mary Elizabeth Stevens,et al.  Statistical Association Methods for Mechanized Documentation. , 1967 .

[77]  Donald M MacKay,et al.  What makes the question , 1960 .

[78]  Peter Ingwersen,et al.  Online citation analysis , 1996, Scientometrics.

[79]  Gabriella Kazai,et al.  The Accessibility Dimension for Structured Document Retrieval , 2002, ECIR.

[80]  Ellen M. Voorhees,et al.  The fifth text REtrieval conference (TREC-5) , 1997 .

[81]  W. S. Cooper Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems , 1968 .

[82]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[83]  Kui-Lam Kwok The use of title and cited titles as document representation for automatic classification , 1975, Inf. Process. Manag..

[84]  Kristian J. Hammond,et al.  Automatically indexing documents: content vs. reference , 2002, IUI '02.

[85]  Louise T. Su Evaluation Measures for Interactive Information Retrieval , 1992, Inf. Process. Manag..

[86]  Gerard Salton,et al.  Automatic indexing , 1980, ACM '80.

[87]  W. Bruce Croft,et al.  A Comparison of Text Retrieval Models , 1992, Comput. J..

[88]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[89]  Malcolm Otter,et al.  Lost in hyperspace: metrics and mental models , 2000, Interact. Comput..

[90]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[91]  Terrence A. Brooks,et al.  Private acts and public objects: An investigation of citer motivations , 1985, J. Am. Soc. Inf. Sci..

[92]  W. Bruce Croft,et al.  SIGIR '94 : proceedings of the seventeenth annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, organised by Dublin City University : 3-6 July 1994, Dublin, Ireland , 1994 .

[93]  Y. Zhang,et al.  Enhancement of text representations using related document titles , 1986, Inf. Process. Manag..

[94]  Paul Wouters,et al.  Beyond the holy grail: From citation theory to indicator theories , 1999, Scientometrics.

[95]  J. P. Martino Citation indexing for research and development management , 1971 .

[96]  Marcia J. Bates,et al.  Subject access in online catalogs: A design model , 1986 .

[97]  William M. Shaw,et al.  Subject and citation indexing, Part II: The optimal, cluster-based retrieval performance of composite representations , 1991, J. Am. Soc. Inf. Sci..

[98]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[99]  Peter Ingwersen,et al.  Polyrepresentation of information needs and semantic entities: elements of a cognitive theory for information retrieval interaction , 1994, SIGIR '94.

[100]  Nicholas J. Belkin,et al.  The cognitive viewpoint in information science , 1990, J. Inf. Sci..

[101]  Gabriella Kazai,et al.  Proceedings of the First Workshop of the INitiative for the Evaluation of XML Retrieval (INEX), Schloss Dagstuhl, Germany, December 9-11, 2002 , 2002, INEX.

[102]  J. Rehfeld,et al.  Four basic characteristics of the gastrin-cholecystokinin system. , 1981, The American journal of physiology.

[103]  Katherine W. McCain,et al.  Visualizing a discipline: an author co-citation analysis of information science, 1972–1995 , 1998 .

[104]  Gregory B. Newby,et al.  ASIS '89 : proceedings of the 52nd ASIS Annual Meeting, Washington, DC, October 30-November 2, 1989 , 1989 .

[105]  E. Michael Keen,et al.  Presenting Results of Experimental Retrieval Comparisons , 1997, Inf. Process. Manag..

[106]  Ellen M. Voorhees,et al.  Evaluation by highly relevant documents , 2001, SIGIR '01.

[107]  H. Zuckerman The sociology of science. , 1988 .

[108]  Pia Borlund,et al.  Evaluation of interactive information retrieval systems , 2000 .

[109]  Peter Ingwersen,et al.  Perspective of webometrics , 2004, Scientometrics.

[110]  Jens-Erik Mai Emerging Frameworks and Methods: The Fouth International Conference on Conceptions of Library and Information Science , 2005 .

[111]  Harold Borko,et al.  Automatic indexing , 1981, ACM '81.

[112]  Jian Qin Semantic similarities between a keyword database and a controlled vocabulary database: an investigation in the antibiotic resistance literature , 2000 .

[113]  Leo Egghe,et al.  Co-citation, bibliographic coupling and a characterization of lattice citation networks , 2002, Scientometrics.

[114]  B. C. Griffith,et al.  The Structure of Scientific Literatures I: Identifying and Graphing Specialties , 1974 .

[115]  Nicholas J. Belkin,et al.  Cognitive models and information transfer , 1984 .

[116]  Michael B. Eisenberg,et al.  A re-examination of relevance: toward a dynamic, situational definition , 1990, Inf. Process. Manag..

[117]  Manfred Kochen,et al.  Principles of information retrieval , 1974 .

[118]  Leo Egghe,et al.  Introduction to Informetrics: Quantitative Methods in Library, Documentation and Information Science , 1990 .

[119]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR '00.

[120]  Beng Chin Ooi,et al.  Disk striping strategies for large video-on-demand servers , 1997, MULTIMEDIA '96.

[121]  C. Lee Giles,et al.  CiteSeer: an automatic citation indexing system , 1998, DL '98.

[122]  Bernice W. Polemis Nonparametric Statistics for the Behavioral Sciences , 1959 .

[123]  John M. Swales,et al.  Genre Analysis: English in Academic and Research Settings , 1993 .

[124]  C. Lee Giles,et al.  Scholarly publishing in the Internet age: a citation analysis of computer science literature , 2001, Inf. Process. Manag..

[125]  A. E. CAWKELL SEARCH STRATEGIES USING THE SCIENCECITATION INDEX , .

[126]  W. J. Hutchins The concept of “aboutness” in subject indexing , 1997 .

[127]  Peter Ingwersen,et al.  Measures of relative relevance and ranked half-life: performance indicators for interactive IR , 1998, SIGIR '98.

[128]  Paul B. Kantor,et al.  A study of information seeking and retrieving. II. Users, questions, and effectiveness , 1988, J. Am. Soc. Inf. Sci..

[129]  Ben Shneiderman,et al.  Navigating in hyperspace: designing a structure-based toolbox , 1994, CACM.

[130]  Karen Spärck Jones Document Retrieval: Shallow Data, Deep Theories; Historical Reflections, Potential Directions , 2003, ECIR.

[131]  R. Merton The Matthew Effect in Science, II: Cumulative Advantage and the Symbolism of Intellectual Property , 1988, Isis.

[132]  Pia Borlund,et al.  Experimental components for the evaluation of interactive information retrieval systems , 2000, J. Documentation.

[133]  Daryl E. Chubin,et al.  Content Analysis of References: Adjunct or Alternative to Citation Counting? , 1975 .

[134]  Amanda Spink,et al.  Multiple Search Sessions Model of End-User Behavior: An Exploratory Study , 1996, J. Am. Soc. Inf. Sci..

[135]  Ole Harbo,et al.  Theory and application of information research : proceedings of the second International Research Forum on Information Science, 3-6 August 1977, Royal School of Librarianship, Copenhagen , 1980 .

[136]  Pia Borlund,et al.  The concept of relevance in IR , 2003, J. Assoc. Inf. Sci. Technol..

[137]  Edward A. Fox,et al.  Coefficients of combining concept classes in a collection , 1988, SIGIR '88.

[138]  Susan E. Cozzens,et al.  What do citations count? the rhetoric-first model , 1989, Scientometrics.

[139]  Anthony E. Cawkell Checking research progress on 'image retrieval by shape-matching' using the Web of Science , 1998, Aslib Proc..

[140]  S. Pollock Measures for the comparison of information retrieval systems , 1968 .

[141]  Thed N. van Leeuwen,et al.  New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications , 1995, Scientometrics.

[142]  Henry Small,et al.  Cited Documents as Concept Symbols , 1978 .

[143]  Annelise Mark Pejtersen A Library System for Information Retrieval Based on a Cognitive Task Analysis and Supported by an Icon-Based Interface , 1989, SIGIR.

[144]  Cyril W. Cleverdon,et al.  Aslib Cranfield research project: report on the testing and analysis of an investigation into the comparative efficiency of indexing systems , 1962 .

[145]  Ramesh Govindan,et al.  A file system for continuous media , 1992, TOCS.

[146]  Barry Cipra,et al.  Diagram Masters Cry 'Venn-i, Vidi, Vici' , 2003, Science.

[147]  S. Shapin Laboratory life. The social construction of scientific facts , 1981, Medical History.

[148]  Eugene Garfield,et al.  KeyWords Plus™—algorithmic derivative indexing , 1993 .

[149]  E GARFIELD,et al.  Citation indexes for science; a new dimension in documentation through association of ideas. , 2006, Science.

[150]  R. Merton The Normative Structure of Science , 1973 .

[151]  M. M. Kessler Comparison of the results of bibliographic coupling and analytic subject indexing , 1965 .

[152]  B. Dervin,et al.  Information needs and uses. , 1986 .

[153]  Louise T. Su The Relevance of Recall and Precision in User Evaluation , 1994, J. Am. Soc. Inf. Sci..

[154]  Amanda Spink,et al.  Information seeking and mediated searching. Part 5. User-intermediary interaction , 2002, J. Assoc. Inf. Sci. Technol..

[155]  William M. Shaw,et al.  Subject and citation indexing, Part I: The clustering structure of composite representations in the Cystic Fibrosis Document Collection , 1991, J. Am. Soc. Inf. Sci..

[156]  Fred J. Damerau,et al.  A technique for computer detection and correction of spelling errors , 1964, CACM.

[157]  J. Rehfeld,et al.  Immunochemical studies on cholecystokinin. II. Distribution and molecular heterogeneity in the central nervous system and small intestine of man and hog. , 1978, The Journal of biological chemistry.

[158]  Gabriella Kazai,et al.  Overview of the Initiative for the Evaluation of XML retrieval (INEX) 2002 , 2002, INEX Workshop.

[159]  Mike Thelwall,et al.  What is this link doing here? Beginning a fine-grained process of identifying reasons for academic hyperlink creation , 2003, Inf. Res..

[160]  J. Rehfeld,et al.  Immunochemical studies on cholecystokinin. I. Development of sequence-specific radioimmunoassays for porcine triacontatriapeptide cholecystokinin. , 1978, The Journal of biological chemistry.

[161]  H. P. Luhn Key word‐in‐context index for technical literature (kwic index) , 1960 .

[162]  Henry G. Small,et al.  Co-citation in the scientific literature: A new measure of the relationship between two documents , 1973, J. Am. Soc. Inf. Sci..

[163]  Peter Ingwersen,et al.  Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory , 1996, J. Documentation.

[164]  Gerard Salton,et al.  Associative Document Retrieval Techniques Using Bibliographic Information , 1963, JACM.

[165]  Gobinda G. Chowdhury,et al.  Bibliometric information retrieval system (BIRS): A web search interface utilizing bibliometric research results , 2000, J. Am. Soc. Inf. Sci..

[166]  Yves Chiaramella,et al.  Information Retrieval and Structured Documents , 2000, ESSIR.

[167]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[168]  Howard D. White,et al.  Author cocitation: A literature measure of intellectual structure , 1981, J. Am. Soc. Inf. Sci..

[169]  C. Lee Giles,et al.  CiteSeer: an autonomous Web agent for automatic retrieval and identification of interesting publications , 1998, AGENTS '98.

[170]  Kristian J. Hammond,et al.  Constructing indices from citations in collections of research papers , 1999 .

[171]  Ellen M. Voorhees,et al.  The Sixth Text REtrieval Conference (TREC-6) , 2000, Inf. Process. Manag..

[172]  Fabio Crestani,et al.  Lectures on information retrieval : third European Summer -School, ESSIR 2000, Varenna, Italy, September 11-15, 2000 : revised lectures , 2001 .

[173]  B. Houghton Computer based information retrieval systems , 1968 .

[174]  Helen R. Tibbo,et al.  The Cystic Fibrosis Database: Content and Research Opportunities. , 1991 .

[175]  Barry O'Sullivan,et al.  Retrieval through explanation: an abductive inference approach to relevance feedback , 1999 .

[176]  Peter Ingwersen,et al.  Data set isolation for bibliometric online analyses of research publications: fundamental methodological issues , 1997 .

[177]  Donna K. Harman,et al.  Overview of the first TREC conference , 1993, SIGIR.

[178]  Gertrud Herlach,et al.  Can retrieval of information from citation indexes be simplified? Multiple mention of a reference as a characteristic of the link between cited and citing article , 1978, J. Am. Soc. Inf. Sci..

[179]  C. J. Van Rijsbergen Research and development in information retrieval : proceedings of the third joint BCS and ACM symposium, King's College, Cambridge, 2-6 July 1984 , 1984 .

[180]  Ellen M. Voorhees,et al.  Evaluating Evaluation Measure Stability , 2000, SIGIR 2000.

[181]  Carnot E. Nelson,et al.  Communication among scientists and engineers , 1970 .

[182]  E. Garfield Announcing the SCI Compact Disc Edition: CD-ROM Gigabyte Storage Technology, Novel Software, and Bibliographic Coupling Make Desktop Research and Discovery a Reality , 1988 .

[183]  K. Järvelin,et al.  EVALUATING INFORMATION RETRIEVAL SYSTEMS UNDER THE CHALLENGES OF INTERACTION AND MULTIDIMENSIONAL DYNAMIC RELEVANCE , 2002 .

[184]  Miranda Lee Pao,et al.  Retrieval effectiveness by semantic and citation searching , 1989, JASIS.

[185]  Paul A. Studer,et al.  Information science and the phenomenon of information , 1978, J. Am. Soc. Inf. Sci..

[186]  Peter Ingwersen,et al.  Dimensions of relevance , 2000, Inf. Process. Manag..

[187]  A. Cawkell Science Citation Index , 1970, Nature.

[188]  W. Bruce Croft,et al.  The INQUERY Retrieval System , 1992, DEXA.

[189]  Peter Ingwersen,et al.  Mapping national research profiles in social science disciplines , 2001, J. Documentation.

[190]  Kui-Lam Kwok,et al.  A Document-Document Similarity Measure Based on Cited Titles and Probability Theory, and Its Application to Relevance Feedback Retrieval , 1984, SIGIR.

[191]  Michael Keen,et al.  ASLIB CRANFIELD RESEARCH PROJECT FACTORS DETERMINING THE PERFORMANCE OF INDEXING SYSTEMS VOLUME 2 , 1966 .

[192]  David Ellis,et al.  A Behavioural Approach to Information Retrieval System Design , 1989, J. Documentation.

[193]  David Ellis,et al.  Progress and problems in information retrieval , 1996 .

[194]  Jaana Kekäläinen,et al.  The effects of query complexity, expansion and structure on retrieval performance in probabilistic text retrieval , 1999 .

[195]  Kui-Lam Kwok,et al.  Experiments with cited titles for automatic document indexing and similarity measure in a probabilistic context , 1985, SIGIR '85.

[196]  Barry Cipra,et al.  Joint mathematics meetings. Diagram masters cry 'Venn-i, vidi, vici'. , 2003, Science.

[197]  Jaana Kekäläinen,et al.  Using graded relevance assessments in IR evaluation , 2002, J. Assoc. Inf. Sci. Technol..

[198]  Peter Ingwersen,et al.  Search Procedures in the Library - Analysed from the Cognitive Point of View , 1982, J. Documentation.

[199]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[200]  Harry W. Bruce,et al.  A Cognitive View of the Situational Dynamism of User-Centered Relevance Estimation , 1994, J. Am. Soc. Inf. Sci..

[201]  W. Bruce Croft,et al.  Combining automatic and manual index representations in probabilistic retrieval , 1995 .

[202]  David Ellis,et al.  Paradigms and research traditions in information retrieval research , 1998 .

[203]  Michael H. MacRoberts,et al.  Quantitative Measures of Communication in Science: A Study of the Formal Level , 1986 .

[204]  Birger Larsen,et al.  Exploiting citation overlaps for Information Retrieval: Generating a boomerang effect from the network of scientific papers , 2002, Scientometrics.

[205]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[206]  Timo Niemi,et al.  A user-oriented interface for generalised informetric analysis based on applying advanced data modelling techniques , 2000, J. Documentation.

[207]  James Allan,et al.  Approaches to passage retrieval in full text information systems , 1993, SIGIR.

[208]  Christine L. Borgman,et al.  All users of information retrieval systems are not created equal: An exploration into individual differences , 1989, Inf. Process. Manag..

[209]  Michael H. MacRoberts,et al.  Problems of citation analysis: A critical review , 1989, JASIS.

[210]  Carol Collier Kuhlthau Inside the Search Process: Information Seeking from the User's Perspective. , 1991 .

[211]  W. Bruce Croft,et al.  Evaluation of an inference network-based retrieval model , 1991, TOIS.

[212]  Jr. Robert G. Sumner,et al.  Utilizing the Age of References to Control the Exhaustivity of the Reference Representation in Information Retrieval , 1995, Inf. Process. Manag..

[213]  E. Garfield From Citation Indexes to Informetrics: Is the Tail Now Wagging the Dog ? , 1998 .

[214]  G. Gilbert Referencing as Persuasion , 1977 .

[215]  James Allan,et al.  INQUERY Does Battle With TREC-6 , 1997, TREC.

[216]  R. Merton,et al.  The Sociology of Science: Theoretical and Empirical Investigations , 1975, Journal for the Scientific Study of Religion.

[217]  Kui-Lam Kwok,et al.  A probabilistic theory of indexing and similarity measure based on cited and citing documents , 1985, J. Am. Soc. Inf. Sci..

[218]  W. Bruce Croft,et al.  Inference networks for document retrieval , 1989, SIGIR '90.

[219]  Timo Lahtinen,et al.  Automatic indexing: an approach using an index term corpus and combining linguistic and statistical methods , 2000 .

[220]  E Garfield,et al.  "Science Citation Index"--A New Dimension in Indexing. , 1964, Science.

[221]  Ellen M. Voorhees,et al.  The seventh text REtrieval conference (TREC-7) , 1999 .