论文信息 - Optimizing ETL processes in data warehouses

Optimizing ETL processes in data warehouses

Extraction-transformation-loading (ETL) tools are pieces of software responsible for the extraction of data from several sources, their cleansing, customization and insertion into a data warehouse. Usually, these processes must be completed in a certain time window; thus, it is necessary to optimize their execution time. In this paper, we delve into the logical optimization of ETL processes, modeling it as a state-space search problem. We consider each ETL workflow as a state and fabricate the state space through a set of correct state transitions. Moreover, we provide algorithms towards the minimization of the execution cost of an ETL workflow.

[1] Panos Vassiliadis,et al. Optimizing ETL processes in data warehouse environments , 2005, ICDE 2005.

[2] Goetz Graefe,et al. Query evaluation techniques for large databases , 1993, CSUR.

[3] Hector Garcia-Molina,et al. Efficient resumption of interrupted warehouse loads , 2000, SIGMOD '00.

[4] Elke A. Rundensteiner,et al. A Transactional Model for Data Warehouse Maintenance , 2002, ER.

[5] Erhard Rahm,et al. Data Cleaning: Problems and Current Approaches , 2000, IEEE Data Eng. Bull..

[6] Panos Vassiliadis,et al. A Framework for the Design of ETL Scenarios , 2003, CAiSE.

[7] Dennis Shasha,et al. AJAX: an extensible data cleaning tool , 2000, SIGMOD '00.

[8] Hector Garcia-Molina,et al. Efficient resumption of interrupted warehouse loads , 2000, SIGMOD 2000.

[9] Joseph M. Hellerstein,et al. Potter's Wheel: An Interactive Data Cleaning System , 2001, VLDB.

[10] Matthias Jarke,et al. Query Optimization in Database Systems , 1984, CSUR.

[11] Patrick Valduriez,et al. Principles of Distributed Database Systems , 1990 .

[12] Jennifer Widom,et al. Lineage tracing for general data warehouse transformations , 2003, The VLDB Journal.

[13] Qiang Chen,et al. Aurora : a new model and architecture for data stream management ) , 2006 .

[14] Tony Bain,et al. Data Transformation Services , 2004 .

[15] Jennifer Widom,et al. Continuous queries over data streams , 2001, SGMD.