Integration of relational databases and record-based legacy systems for populating data warehouses

The number of data sources that an organization has to deal with continues to be nontrivial. Integrating this data is a growing problem. A great deal of research has been done to solve the general problem. Work on topics like multi-databases, mediators and ontologies has been directed at solving the general data integration problem. While all of this activity has been useful, the general problem of integrating heterogeneous data sources remains only partially solved. This paper looks at the solution of a sub-problem of the general problem where the data sources are restricted to relational databases and record-based legacy systems owned by the same organization. For many organizations, this restriction precisely defines their integration problem. For example, data warehouses typically have a tuple as their storage format, whether they are table- or cube-oriented. The task of defining the tuple requires integrating the organization's existing relational and/or record-based legacy systems. Populating the data warehouse can be accomplished by querying the integrated data sources. Specifically, we provide a mechanism for developing a relational model for the set of data sources, provide a method for generating correct queries over the model, and create an architecture for executing the queries based on the mobile agent paradigm. A prototype of the system has been designed and implemented.