BPMN Patterns for ETL Conceptual Modelling and Validation

ETL systems continue to suffer from a lack of a simple and rigorous approach for modelling and validation of populating processes for data warehouses. In spite of the efforts that researchers have been done, there is not yet a convinced and simply approach for modelling (conceptual and logical views), validating and testing ETL processes before conduct them to implementation and roll out. In this paper we explored the use of BPMN for ETL conceptual modelling and validation. Basically, we intended to provide a set of BPMN meta-models (patterns) especially designed to map standard data warehousing ETL processes and testing them before constructing the final system. We think this is a practical approach to reduce significantly the inadequacy of an ETL system in its real world scenario. A surrogate key pipelining process was selected as a case study to demonstrate the use and utility of the ETL modelling approach presented here.

[1]  Matjaz B. Juric,et al.  Business Process Execution Language for Web Services BPEL and BPEL4WS 2nd Edition , 2006 .

[2]  Panos Vassiliadis,et al.  A Methodology for the Conceptual Modeling of ETL Processes , 2003, CAiSE Workshops.

[3]  Ralph Kimball,et al.  The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning, Conforming, and Delivering Data , 2004 .

[4]  Juan Trujillo,et al.  A UML Based Approach for Modeling ETL Processes in Data Warehouses , 2003, ER.

[5]  Panos Vassiliadis,et al.  Modeling ETL activities as graphs , 2002, DMDW.

[6]  Walt Scacchi,et al.  Process Models in Software Engineering , 2001 .

[7]  Panos Vassiliadis,et al.  On the Logical Modeling of ETL Processes , 2002, CAiSE.

[8]  Peretz Shoval,et al.  Conceptual Modeling - ER 2010, 29th International Conference on Conceptual Modeling, Vancouver, BC, Canada, November 1-4, 2010. Proceedings , 2010, ER.

[9]  Frank Leymann.  BPEL vs. BPMN 2.0: Should You Care? , 2010, BPMN.

[10]  Mathias Weske,et al.  Advances in business process management , 2004, Data Knowl. Eng..

[11]  Kevin Wilkinson,et al.  Leveraging Business Process Models for ETL Design , 2010, ER.

[12]  Esteban Zimányi,et al.  A model-driven framework for ETL process development , 2011, DOLAP '11.

[13]  Torben Bach Pedersen,et al.  Proceedings of the ACM tenth international workshop on Data warehousing and OLAP , 2007, CIKM 2007.

[14]  Kevin Wilkinson,et al.  Business Processes Meet Operational Business Intelligence , 2009, IEEE Data Eng. Bull..

[15]  Matjaz B. Juric,et al.  Business process execution language for web services , 2004 .

[16]  Tok Wang Ling,et al.  Conceptual Modeling - ER 2003 , 2003, Lecture Notes in Computer Science.

[17]  John J. Marciniak,et al.  Encyclopedia of Software Engineering , 1994, Encyclopedia of Software Engineering.

[18]  Esteban Zimányi,et al.  Defining ETL worfklows using BPMN and BPEL , 2009, DOLAP.

[19]  Zahir Tari,et al.  On the Move to Meaningful Internet Systems: OTM 2008 Workshops OTM Confederated International Workshops and Posters, ADI, AWeSoMe, COMBEK, EI2N, IWSSA, ... Applications, incl. Internet/Web, and HCI) , 2008 .

[20]  Peter Fettke,et al.  Business Process Modeling Notation , 2008, Wirtschaftsinf..

[21]  Jose-Norberto Mazón,et al.  Modelling ETL Processes of Data Warehouses with UML Activity Diagrams , 2008, OTM Workshops.