Toward Democratizing Access to Facilities Data: A Framework for Intelligent Data Discovery and Delivery

Data collected by large-scale instruments, observatories, and sensor networks are key enablers of scientific discoveries in many disciplines. However, ensuring that these data can be accessed, integrated, and analyzed in a democratized and timely manner remains a challenge. In this article, we explore how state-of-the-art techniques for data discovery and access can be adapted to facility data and develop a conceptual framework for intelligent data access and discovery. ¢ Science in the 21st century is being transformed by our unprecedented ability to collect and process data from a variety of sources. For example, large-scale multiuser scientific observatories, instruments, and experimental platforms provide a broad community of researchers and educators with open access to shared-use infrastructure and data products generated from geodistributed instruments and equipment [1]. These large facilities (LF) have recently enabled significant scientific discoveries such as the detection of gravitational waves [2] and the imaging of the event horizon of a black hole [3]. However, as the number and scale of such LF increases along with corresponding growth in the number, distribution, and diversity of users, ensuring that LF data can be discovered, accessed, integrated, and analyzed in a timely manner is a growing challenge that is resulting in significant demands on LF cyberinfrastructure (CI) [4]. For example, the Ocean Observatory Initiative (OOI) [5] integrates over 1,250 instruments, producing over 25,000 data items and over 100,000 data products. Similarly, each antenna of the Square Kilometre Array (SKA), the world’s largest radio telescope project, produces raw data at the rate of approximately 0.5-1TB per second and approximately 300PB of data after pre-processing per telescope per year1. 1 https://www.skatelescope.org/the-skaproject/