A Scalable Architecture for Harvest-Based Digital

This paper discusses the requirements of current and emerging applications based on the Open Archives Initiative (OAI) and emphasizes the need for a common infrastructure to support them. Inspired by HTTP proxy, cache, gateway and web service concepts, a design for a scalable and reliable infrastructure that aims at satisfying these requirements is presented. Moreover it is shown how various applications can exploit the services included in the proposed infrastructure. The paper concludes by discussing the current status of several prototype implementations.

[1]  C. Lee Giles,et al.  CiteSeer: an automatic citation indexing system , 1998, DL '98.

[2]  Les Carr,et al.  Developing services for open eprint archives: globalisation, integration and the impact of links , 2000, DL '00.

[3]  Peter B. Danzig,et al.  The Harvest Information Discovery and Access System , 1995, Comput. Networks ISDN Syst..

[4]  Peter B. Danzig,et al.  A Hierarchical Internet Object Cache , 1996, USENIX ATC.

[5]  Herbert Van de Sompel,et al.  Generalizing the OpenURL Framework beyond References to Scholarly Works: The Bison-Futé Model , 2001, D Lib Mag..

[6]  Darrell D. E. Long,et al.  Exploring the Bounds of Web Latency Reduction from Caching and Prefetching , 1997, USENIX Symposium on Internet Technologies and Systems.

[7]  Kurt Maly,et al.  Kepler - An OAI Data/Service Provider for the Individual , 2001, D Lib Mag..

[8]  Kurt Maly,et al.  The UPS Prototype: An Experimental End-User Service across E-Print Archives , 2000 .

[9]  Edward A. Fox,et al.  A Framework for Building Open Digital Libraries , 2001, D Lib Mag..

[10]  Herbert Van de Sompel,et al.  Open Linking in the Scholarly Information Environment Using the OpenURL Framework , 2001, D Lib Mag..

[11]  Michael L. Nelson,et al.  Object Persistence and Availability in Digital Libraries , 2002, D Lib Mag..

[12]  William Y. Arms,et al.  Reference Linking for Journal Articles , 1999, D Lib Mag..

[13]  Kurt Maly,et al.  Arc - An OAI Service Provider for Digital Library Federation , 2001, D Lib Mag..

[14]  Les Carr,et al.  Trailblazing the literature of hypertext: author co-citation analysis (1989–1998) , 1999, HYPERTEXT '99.

[15]  Kurt Maly,et al.  DP9: an OAI gateway service for web crawlers , 2002, JCDL '02.

[16]  Stevan Harnad,et al.  How and Why To Free All Refereed Research From Access- and Impact-Barriers Online, Now , 2001 .

[17]  Carl Lagoze,et al.  NCSTRL: design and deployment of a globally distributed digital library , 2000 .