Rapid Data Integration and Analysis for Upstream Oil and Gas Applications

The increasingly large number of sensors and instruments in the oil and gas industry, along with novel means of communication in the enterprise has led to a corresponding increase in the volume of data that is recorded in various information repositories. The variety of information sources is also expanding: from traditional relational databases to time series data, social network communications, collections of unsorted text reports, and linked data available on the Web. Enabling end-to-end optimization considering these diverse types of information requires creating semantic links between them. Though integration of data across silo-ed databases has been recognized as a problem for a long time, it has proven to be difficult to accomplish due to the complexity of the data arrangement within databases, scarcity of metadata that describe the content, lack of a direct mapping between related entities across databases, and the several types of data represented within a database. In addition, there are large amounts of unstructured text data such as text entries in databases and document repositories. These contain valuable information on processes from the field but there is currently no method to convert this raw data to useable information. The Center for Interactive Smart Oilfield Technologies (CiSoft) is a USC-Chevron Center of Excellence for Research and Academic Training on Smart Oilfield Technologies. We describe the Integrated Optimization project at CiSoft which has the goal of developing a framework for automated linking of heterogeneous data sources and analysis of the integrated data in the context of upstream applications.