Bibliomining for automated collection development in a digital library setting: Using data mining to discover Web-based scholarly research works

This research creates an intelligent agent for automated collection development in a digital library setting. It uses a predictive model based on facets of each Web page to select scholarly works. The criteria came from the academic library selection literature, and a Delphi study was used to refine the list to 41 criteria. A Perl program was designed to analyze a Web page for each criterion and applied to a large collection of scholarly and nonscholarly Web pages. Bibliomining, or data mining for libraries, was then used to create different classification models. Four techniques were used: logistic regression, non-parametric discriminant analysis, classification trees, and neural networks. Accuracy and return were used to judge the effectiveness of each model on test datasets. In addition, a set of problematic pages that were difficult to classify because of their similarity to scholarly research was gathered and classified using the models. The resulting models could be used in the selection process to automatically create a digital library of Web-based scholarly research works. In addition, the technique can be extended to create a digital library of any type of structured electronic information.

[1]  Carol H. Fenichel,et al.  Developing Library and Information Center Collections , 1988 .

[2]  Reva Basch.  Databank software for the 1990s and beyond—part 2: the Online services respond , 1990 .

[3]  Limsoon Wong,et al.  DATA MINING TECHNIQUES , 2003 .

[4]  Hamid R. Nemati,et al.  Organizational Data Mining: Leveraging Enterprise Data Resources for Optimal Performance , 2003 .

[5]  Sherry Piontek,et al.  Creating a World Wide Web resource collection , 1996, Internet Res..

[6]  Anthony Beavers.  Evaluating Search Engine Models for Scholarly Purposes: A Report from the Internet Applications Laboratory , 1998, D Lib Mag..

[7]  Yulan He,et al.  Mining a web citation database for the retrieval of scientific publications over the www. , 2000 .

[8]  Michael Nentwich.  The European Research Papers Archive: Quality Filters in Electronic Publishing , 1999 .

[9]  May Y. Chau.  Web Mining Technology and Academic Librarianship: Human-Machine Connections for the Twenty-First Century , 1999, First Monday.

[10]  Eric Lease Morgan.  Communication is the Key to Our Success , 1998 .

[11]  S. D. Neill,et al.  The information analyst as a quality filter in the scientific communication process , 1989, J. Inf. Sci..

[12]  Patrick Flannery,et al.  Guidelines for Internet resource selection , 1996 .

[13]  Cyril W. Cleverdon,et al.  Aslib Cranfield research project: report on the testing and analysis of an investigation into the comparative efficiency of indexing systems , 1962 .

[14]  C. Lee Giles,et al.  Accessibility of information on the web , 1999, Nature.

[15]  Elizabeth Futas,et al.  Library acquisition policies and procedures , 1977 .

[16]  Kyle Banerjee,et al.  Is data mining right for your library , 1998 .

[17]  Boyd R. Collins.  Beyond Cruising: Reviewing. , 1996 .

[18]  John P. Dickinson,et al.  Science and scientific researchers in modern society , 1984 .

[19]  C. Lee Giles,et al.  Digital Libraries and Autonomous Citation Indexing , 1999, Computer.

[20]  Michael J. A. Berry,et al.  Data mining techniques - for marketing, sales, and customer support , 1997, Wiley computer publishing.

[21]  Geoffrey E. Hinton,et al.  How neural networks learn from experience. , 1992, Scientific American.

[22]  Rachel Cassel.  Selection criteria for Internet resources , 1995 .

[23]  Alastair G. Smith,et al.  Testing the Surf: Criteria for Evaluating Internet Information Resources , 1997 .

[24]  C. Cleverdon.  Report on the testing and analysis of an investigation into comparative efficiency of indexing systems , 1962 .

[25]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[26]  Andrew McCallum,et al.  Building Domain-Specific Search Engines with Machine Learning Techniques , 1999 .

[27]  A. K. Pujari,et al.  Data Mining Techniques , 2006 .

[28]  Mark Johnston,et al.  Selection advisor : an expert system for collection development , 1990 .

[29]  Reva Basch.  Databank software for the 1990s and beyond—part 1: the users' wish list , 1989 .

[30]  Scott Nicholson,et al.  Gaining Strategic Advantage Through Bibliomining: Data Mining for Management Decisions in Corporate, Special, Digital, and Traditional Libraries , 2003 .