Enabling scientific data sharing and re-use

Higher sensor throughput has increased the demand for cyberinfrastructure, requiring those unfamiliar with large database management to acquire new skills or outsource. Some have called this shift from sensor-limited data collection the “data deluge.” As an alternative, we propose that the deluge is the result of sensor control software failing to keep pace with hardware capabilities. Rather than exploit the potential of powerful embedded operating systems and construct intelligent sensor networks that harvest higher quality data, the old paradigm (i.e. collect everything) is still dominant. To mitigate the deluge, we present an adaptive sampling algorithm based on the Nyquist-Shannon sampling theorem. We calibrate the algorithm for both data reduction and increased sampling over “hot moments,” which we define as periods of elevated signal activity, deviating from previous works which have emphasized adaptive sampling for data compression via minimization of signal reconstruction error. Under the feature extraction concept, samples drawn from user-defined events carry greater importance and effective control requires the researcher to describe the context of events in the form of both an identification heuristic (for calibration) and a real-time sampling model. This event-driven approach is important when observation is focused on intermittent dynamics. In our case study application, we develop a heuristic to identify hot moments from historical data and use it to train and evaluate the adaptive model in an offline analysis using soil moisture data. Results indicate the adaptive model is superior to uniform sampling, capable of extracting 20% to 100% more samples during hot moments at equivalent levels of overall efficiency. Research data sharing is one of the key challenges in the e-science era. IT technologies facilitate an enhanced management and sharing of research data. It is crucial to understand the current status of research data sharing in order to facilitate enhanced data sharing in the future. In this study, a conceptual model has been developed to characterize the process of data sharing and the factors which give rise to variations in data re-use. The study goes beyond a solely technical analysis and includes also psychological, social, organizational, legal and political components. The model was developed based on the literature and 21 face to face interviews with research, funding, data centre and publishing experts. It was validated by both a vigorous workshop and a further 55 structured telephone interviews. The overall model identifies sub-models of process, of context, and of drivers, barriers and enablers. These provide a comprehensive description of the factors that enable or inhibit the sharing of research data. They affect whether data are shared, how they are shared, and how successfully they are shared. Implementing the enablers will help the research community overcome the barriers to data re-use to facilitate future e-science endeavors.