The Significance of Storage in the "Cost of Risk" of Digital Preservation

As storage costs drop, storage is becoming the lowest cost in a digital repository – and the biggest risk. We examine current modelling of costs and risks in digital preservation, concentrating on the Total Cost of Risk when using digital storage systems for preserving audiovisual material. We present a managed approach to preservation, and the vital role of storage and show how planning for long-term preservation of data should consider the risks involved in using digital storage technology. Gaps in information necessary for accurate modeling – and planning – are presented. We call for new functionality to support recovery of files with errors, to eliminate the all-or-nothing approach of current IT systems, reduce the impact of failures of digital storage technology and mitigate against loss of digital data.

[1]  Paul B. Watry Digital Preservation Theory and Application: Transcontinental Persistent Archives Testbed Activity , 2007, Int. J. Digit. Curation.

[2]  Helen Shenton Life Cycle Collection Management , 2003 .

[3]  Stephen Chapman Counting the Costs of Digital Preservation: Is Repository Storage Affordable? , 2003, J. Digit. Inf..

[4]  John Kubiatowicz,et al.  Erasure Coding Vs. Replication: A Quantitative Comparison , 2002, IPTPS.

[5]  Tony Hendley Comparison of Methods & Costs of Digital Preservation , 1998 .

[6]  Eduardo Pinheiro,et al.  Failure Trends in a Large Disk Drive Population , 2007, FAST.

[7]  Neil Beagrie,et al.  A Strategic Policy Framework for Creating and Preserving Digital Collections: A Report to the Digital Archiving Working Group , 1998 .

[8]  Mary Baker,et al.  A fresh look at the reliability of long-term digital storage , 2005, EuroSys.

[9]  David S. H. Rosenthal,et al.  Bit Preservation: A Solved Problem? , 2010, Int. J. Digit. Curation.

[10]  Luiz André Barroso,et al.  The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines , 2009, The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines.

[11]  Robert H. McDonald,et al.  Disk and Tape Storage Cost Models , 2007, Archiving Conference.

[12]  Matthew Addis,et al.  SUSTAINABLE ARCHIVING AND STORAGE MANAGEMENT OF AUDIOVISUAL DIGITAL ASSETS , 2009 .

[13]  Michael Gill,et al.  An Evaluation of Motion JPEG 2000 for Video Archiving , 2005 .

[14]  Richard Wright,et al.  Broadcast Archives: Preserving the Future , 2001, ICHIM.

[15]  Arkady Kanevsky,et al.  Are disks the dominant contributor for storage failures?: A comprehensive study of storage subsystem failure characteristics , 2008, TOS.

[16]  Andrea C. Arpaci-Dusseau,et al.  Parity Lost and Parity Regained , 2008, FAST.

[17]  Bianca Schroeder,et al.  Disk Failures in the Real World: What Does an MTTF of 1, 000, 000 Hours Mean to You? , 2007, FAST.

[18]  John R. Turner Comparison of Methods and Costs of Digital Preservation , 1999 .

[19]  Matthew Addis,et al.  A NEW APPROACH TO AUDIOVISUAL DIGITAL ARCHIVING , 2009 .

[20]  D. DeMets,et al.  Data integrity. , 2020, Controlled clinical trials.

[21]  Manfred Thaller,et al.  A generic XML language for characterising objects to support digital preservation , 2008, SAC '08.