Data warehouse process management

Abstract Previous research has provided metadata models that enable the capturing of the static components of a data warehouse architecture, along with information on different quality factors over these components. This paper complements this work with the modeling of the dynamic parts of the data warehouse. The proposed metamodel of data warehouse operational processes is capable of modeling complex activities, their interrelationships, and the relationship of activities with data sources and execution details. Moreover, the metamodel complements the existing architecture and quality models in a coherent fashion, resulting in a full framework for quality-oriented data warehouse management, capable of supporting the design, administration and especially evolution of a data warehouse. Finally, we exploit our framework to revert the widespread belief that data warehouses can be treated as collections of materialized views. We have implemented this metamodel using the language Telos and the metadata repository system ConceptBase.

[1]  Panos Vassiliadis,et al.  Gulliver in the land of data warehousing: practical experiences and observations of a researcher , 2000, DMDW.

[2]  ShethAmit,et al.  An overview of workflow management , 1995 .

[3]  Matthias Jarke,et al.  Information Systems Quality and Quality Informations Systems , 1992, The Impact of Computer Supported Technologies in Information Systems Development.

[4]  Kenneth A. Ross,et al.  Adapting materialized views after redefinitions , 1995, SIGMOD '95.

[5]  Fabio Casati,et al.  Workflow Evolution , 1996, ER.

[6]  Maria E. Orlowska,et al.  A general model for nested transactional workflows , 1996 .

[7]  Panos Vassiliadis,et al.  Towards Quality-oriented Data Warehouse Usage and Evolution , 2000, Inf. Syst..

[8]  Dennis Shasha,et al.  AJAX: an extensible data cleaning tool , 2000, SIGMOD '00.

[9]  Won Kim,et al.  On optimizing an SQL-like nested query , 1982, TODS.

[10]  Matthias Jarke,et al.  Design and Analysis of Quality Information for Data Warehouses , 1998, ER.

[11]  Hamid Pirahesh,et al.  Magic is relevant , 1990, SIGMOD '90.

[12]  M. Muralikrishna,et al.  Improved Unnesting Algorithms for Join Aggregate SQL Queries , 1992, VLDB.

[13]  Richard Y. Wang,et al.  Anchoring data quality dimensions in ontological foundations , 1996, CACM.

[14]  Kalle Lyytinen,et al.  The Impact of Computer Supported Technologies in Information Systems Development, Proceedings of the IFIP WG8.2 Working Conference on The Impact of Computer Supported Technologies in Information Systems Development, Minneapolis, Minnesota, USA, 14-17 June, 1992 , 1992, The Impact of Computer Supported Technologies in Information Systems Development.

[15]  Matthias Jarke,et al.  Data warehouse architecture and quality model , 1997 .

[16]  Divesh Srivastava,et al.  Answering Queries Using Views. , 1999, PODS 1995.

[17]  Ashish Gupta,et al.  Aggregate-Query Processing in Data Warehousing Environments , 1995, VLDB.

[18]  Harry K. T. Wong,et al.  Optimization of nested SQL queries revisited , 1987, SIGMOD '87.

[19]  Kyuseok Shim,et al.  Optimizing queries with materialized views , 1995, Proceedings of the Eleventh International Conference on Data Engineering.

[20]  Diego Calvanese,et al.  A Principled Approach to Data Integration and Reconciliation in Data Warehousing , 1999, DMDW.

[21]  Maria E. Orlowska,et al.  Applying Graph Reduction Techniques for Identifying Structural Conflicts in Process Models , 1999, CAiSE.

[22]  Philip A. Bernstein,et al.  Meta-Data Support for Data Transformations Using Microsoft Repository , 1999, IEEE Data Eng. Bull..

[23]  M. Muralikrishna.  Optimization and Dataflow Algorithms for Nested Tree Queries , 1989, VLDB.

[24]  Matthias Jarke,et al.  Telos: representing knowledge about information systems , 1990, TOIS.

[25]  Kyuseok Shim,et al.  Optimizing Queries with Aggregate Views , 1996, EDBT.

[26]  Matthias Jarke,et al.  Architecture and Quality in Data Warehouses: An Extended Repository Approach , 1999, Information Systems.

[27]  Vasant Dhar,et al.  Supporting Systems Development by Capturing Deliberations During Requirements Engineering , 1992, IEEE Trans. Software Eng..

[28]  Stuart E. Madnick,et al.  Data quality requirements analysis and modeling , 2011, Proceedings of IEEE 9th International Conference on Data Engineering.

[29]  Matthias Jarke,et al.  DB-Prism: Integrated Data Warehouses and Knowledge Net- works for Bank Controlling , 2000 .

[30]  Veda C. Storey,et al.  A Framework for Analysis of Data Quality Research , 1995, IEEE Trans. Knowl. Data Eng..

[31]  Giri Kumar Tayi,et al.  Examining data quality , 1998, CACM.

[32]  Umeshwar Dayal,et al.  Of Nests and Trees: A Unified Approach to Processing Queries That Contain Nested Subqueries, Aggregates, and Quantifiers , 1987, VLDB.

[33]  Matthias Jarke,et al.  A software process data model for knowledge engineering in information systems , 1990, Inf. Syst..

[34]  Werner Nutt,et al.  Deciding equivalences among aggregate queries , 1998, PODS '98.

[35]  Panos Vassiliadis,et al.  Modeling multidimensional databases, cubes and cube operations , 1998, Proceedings. Tenth International Conference on Scientific and Statistical Database Management (Cat. No.98TB100243).

[36]  John Mylopoulos,et al.  Understanding "why" in software process modelling, analysis, and design , 1994, Proceedings of 16th International Conference on Software Engineering.

[37]  Colette Rolland,et al.  A Comprehensive View of Process Engineering , 1998, CAiSE.

[38]  Mokrane Bouzeghoub,et al.  Modeling the Data Warehouse Refreshment Process as a Workflow Application , 1999, DMDW.

[39]  Fabio Casati,et al.  An Environment for Designing Exceptions in Workflows , 1998, Inf. Syst..

[40]  Zohra Bellahsene.  Structural view maintenance in data warehousing systems , 1998, BDA.

[41]  Linda H. Rosenberg,et al.  A Software Quality Model and Metrics for Identifying Project Risks and Assessing Software Quality , 1996 .

[42]  Martin White,et al.  Enterprise information portals , 2000, Electron. Libr..

[43]  Alon Y. Halevy,et al.  Query Optimization by Predicate Move-Around , 1994, VLDB.

[44]  Divesh Srivastava,et al.  Answering Queries with Aggregation Using Views , 1996, VLDB.

[45]  Manfred Reichert,et al.  Enterprise-wide and Cross-enterprise Workflow Management:Concepts, Systems, Applications , 1999 .

[46]  Richard Y. Wang,et al.  A product perspective on total data quality management , 1998, CACM.

[47]  Hamid Pirahesh,et al.  Extensible/rule based query rewrite optimization in Starburst , 1992, SIGMOD '92.

[48]  Franz Baader,et al.  Description Logics with Concrete Domains and Aggregation , 1998, ECAI.

[49]  Fabio Casati,et al.  Conceptual Modeling of Workflows , 1995, Advances in Object-Oriented Data Modeling.

[50]  Panos Vassiliadis,et al.  Data Warehouse Modeling and Quality Issues , 2000 .

[51]  Manfred A. Jeusfeld,et al.  Proceedings of the Intl. Workshop on Design and Management of Data Warehouses, DMDW'99, Heidelberg, Germany, June 14-15, 1999 , 1999, Design and Management of Data Warehouses.

[52]  Victor R. Basili,et al.  Representing Software Engineering Models: The TAME Goal Oriented Approach , 1992, IEEE Trans. Software Eng..

[53]  Matthias Jarke,et al.  Fundamentals of Data Warehouses , 2000, Springer Berlin Heidelberg.

[54]  Barry W. Boehm,et al.  Software Risk Management , 1989, ESEC.