The aDORe federation architecture: digital repositories at scale

The need to federate repositories emerges in two distinctive scenarios. In one scenario, scalability-related problems in the operation of a repository reach a point beyond which continued service requires parallelization and hence federation of the repository infrastructure. In the other scenario, multiple distributed repositories manage collections of interest to certain communities or applications, and federation is an approach to present a unified perspective across these repositories. The high-level, 3-Tier aDORe federation architecture can be used as a guideline to federate repositories in both cases. This paper describes the architecture, consisting of core interfaces for federated repositories in Tier-1, two shared infrastructure components in Tier-2, and a single-point of access to the federation in Tier-3. The paper also illustrates two large-scale deployments of the aDORe federation architecture: the aDORe Archive repository (over 100,000,000 digital objects) at the Los Alamos National Laboratory and the Ghent University Image Repository federation (multiple terabytes of image files).

[1]  Herbert Van de Sompel,et al.  An Interoperable Fabric for Scholarly Value Chains , 2006, D-Lib Magazine.

[2]  Herbert Van de Sompel,et al.  Resource Harvesting within the OAI-PMH Framework , 2004, D Lib Mag..

[3]  MacKenzie Smith,et al.  The DSpace institutional digital repository system: current functionality , 2003, 2003 Joint Conference on Digital Libraries, 2003. Proceedings..

[4]  Herbert Van de Sompel,et al.  Rethinking Scholarly Communication: Building the System that Scholars Deserve , 2004, D Lib Mag..

[5]  Daniel R. Rehak,et al.  A Model and Infrastructure for Federated Learning Content Repositories , 2005 .

[6]  Rich Salz,et al.  A Universally Unique IDentifier (UUID) URN Namespace , 2005, RFC.

[7]  Jeroen Bekaert,et al.  Standards-Based Interfaces for Harvesting and Obtaining Assets from Digital Repositories , 2006 .

[8]  H.N. Jerez,et al.  The multi-faceted use of the OAI-PMH in the LANL repositor , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[9]  Herbert Van de Sompel,et al.  aDORe: a modular, standards-based Digital Object Repository , 2005, Comput. J..

[10]  Niso.  ANSI/NISO Z39.88-2004 The OpenURL Framework for Context-Sensitive Services , 2008 .

[11]  Ann Apps.  The JISC Information Environment Service Registry , 2005 .

[12]  Robert Tansley.  Building a Distributed, Standards-based Repository Federation , 2006 .

[13]  Herbert Van de Sompel,et al.  Representing digital assets usingMPEG-21 Digital Item Declaration , 2005, International Journal on Digital Libraries.

[14]  J. M. Stack.  Los Alamos National Laboratory Research Library: Integrating the present with the future , 1997 .

[15]  Giridhar Manepalli,et al.  ADL-R: The First Instance of a CORDRA Registry , 2006, D Lib Mag..

[16]  Wang Jun.  Open Archives Initiative Protocol for Metadata Harvesting , 2005 .

[17]  Robert Wilensky,et al.  A framework for distributed digital object services , 2006, International Journal on Digital Libraries.

[18]  Jerome McDonough,et al.  METS: standardized encoding for digital library objects , 2006, International Journal on Digital Libraries.

[19]  Herbert Van de Sompel,et al.  A Standards-based Solution for the Accurate Transfer of Digital Assets , 2005, D Lib Mag..

[20]  Herbert Van de Sompel,et al.  IJDL special issue on complex digital objects: Guest editors' introduction , 2005, International Journal on Digital Libraries.

[21]  Sandra Payette,et al.  Fedora: an architecture for complex objects and their relationships , 2005, International Journal on Digital Libraries.

[22]  Herbert Van de Sompel,et al.  Using MPEG-21 DIDL to Represent Complex Digital Objects in the Los Alamos National Laboratory Digital Library , 2003, D Lib Mag..

[23]  Rebecca S. Guenther,et al.  Practical Preservation: The PREMIS Experience , 2005, Libr. Trends.

[24]  Herbert Van de Sompel,et al.  Using MPEG-21 DIP and NISO OpenURL for the Dynamic Dissemination of Complex Digital Objects in the Los Alamos National Laboratory Digital Library , 2004, D Lib Mag..

[25]  M. Robinson.  DRIVER - Digital Repository Infrastructure Vision for European Research , 2007, ELPUB.

[26]  Paolo Manghi,et al.  Digital Repository Infrastructure Vision for European Research , 2009, IRCDL.

[27]  Sandra Payette,et al.  Pathways: augmenting interoperability across scholarly repositories , 2007, International Journal on Digital Libraries.

[28]  Herbert Van de Sompel,et al.  Open Archives Initiative - Protocol for Metadata Harvesting - XML Schema to describe content and policies of repositories in the e-print community , 2003 .

[29]  Carl Lagoze,et al.  Dienst: an architecture for distributed document libraries , 1995, CACM.

[30]  Herbert Van de Sompel,et al.  The open archives initiative: building a low-barrier interoperability framework , 2001, JCDL '01.

[31]  Robert Tansley.  Building a Distributed, Standards-based Repository Federation: The China Digital Museum Project , 2006, D Lib Mag..

[32]  Herbert Van de Sompel,et al.  The "info" URI Scheme for Information Assets with Identifiers in Public Namespaces , 2006, RFC.

[33]  Giridhar Manepalli,et al.  FeDCOR: An Institutional CORDRA Registry , 2006, D Lib Mag..

[34]  Carl Lagoze,et al.  NCSTRL: Design and deployment of a globally distributed digital library , 2000, J. Am. Soc. Inf. Sci..

[35]  Herbert Van de Sompel,et al.  File-Based Storage of Digital Objects and Constituent Datastreams: XMLtapes and Internet Archive ARC Files , 2005, ECDL.

[36]  Herbert Van de Sompel,et al.  The multi-faceted use of the OAI-PMH in the LANL repositor , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..