Improving correctness and failure handling in workflow management systems

A workflow management system (WFMS) facilitates the coordinated execution of applications (steps) that comprise a business process (workflow) across distributed nodes. State-of-the-art WFMSs do not have adequate support for handling various correctness and failure handling requirements of workflows. Different correctness requirements arise due to the dependencies between steps that access shared resources. Failure of steps in a workflow and system failures can cause data inconsistencies if handled improperly. Scalability is also a problem in current WFMSs since they use a centralized workflow control architecture that can easily become a performance bottleneck. In this thesis, we have developed the concepts and infrastructure necessary to address the above issues. To handle dependencies across workflows we have developed techniques for expressing and handling mutual-exclusion, relative ordering and rollback dependency requirements across workflow instances. To handle failure of steps within a workflow, we have proposed a new opportunistic scheme that avoids unnecessary compensations and re-executions when workflows are rolled back partially and re-executed. To handle system failures we have designed suitable logging schemes and protocols. To achieve scalability while satisfying the different correctness and failure handling requirements, we have enhanced our techniques to work on parallel and distributed workflow control architectures. To realize the above concepts, we have designed a workflow specification language, a two stage compiler and a rule-based run-time system. A workflow designer specifies the workflow schema and the resources accessed by the steps from a global database of resources. The two stage workflow compiler determines data dependencies and translates the high level schema into a uniform set of rules. The run-time system interprets these rules and executes the workflows in accordance with their requirements under central, parallel and distributed workflow control. To demonstrate the usefulness and practicality of our approach, we have implemented a prototype system that can offer better correctness, performance and functionality than state-of-the-art WFMSs.

[1]  Umeshwar Dayal,et al.  Organizing long-running activities with triggers and transactions , 1990, SIGMOD '90.

[2]  Amit P. Sheth,et al.  Supporting State-Wide Immunisation Tracking Using Multi-Paradigm Workflow Technology , 1996, VLDB.

[3]  Fabio Casati,et al.  Conceptual Modeling of Workflows , 1995, Advances in Object-Oriented Data Modeling.

[4]  Umeshwar Dayal,et al.  A transactional nested process management system , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[5]  Umeshwar Dayal,et al.  A Transactional Model for Long-Running Activities , 1991, VLDB.

[6]  Christoph Bussler,et al.  A client/server architecture for distributed workflow management systems , 1994, Proceedings of 3rd International Conference on Parallel and Distributed Information Systems.

[7]  Leon J. Osterweil,et al.  An Analysis of Process Languages , 1995 .

[8]  Patrick Valduriez,et al.  Principles of Distributed Database Systems , 1990 .

[9]  M. Tamer Özsu,et al.  Using semantic knowledge of transactions to increase concurrency , 1989, TODS.

[10]  Frank Manola,et al.  Specification and management of extended transactions in a programmable transaction environment , 1994, Proceedings of 1994 IEEE 10th International Conference on Data Engineering.

[11]  Surajit Chaudhuri,et al.  An overview of data warehousing and OLAP technology , 1997, SGMD.

[12]  Johannes Klein,et al.  Coordinating multi-transaction activities , 1990 .

[13]  Krzysztof Kozminski,et al.  Executable Workflows: A Paradigm For Collaborative Design On The Internet , 1997, Proceedings of the 34th Design Automation Conference.

[14]  Frank Leymann,et al.  Supporting Business Transactions Via Partial Backward Recovery In Workflow Management Systems , 1995, BTW.

[15]  Hamid Pirahesh,et al.  ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging , 1998 .

[16]  Frank Leymann,et al.  Business process management with FlowMark , 1994, Proceedings of COMPCON '94.

[17]  Timothy W. Finin,et al.  Exotica: a Research Perspective on Workkow Management Systems. Data Engineering Bulletin, Special Issue on Infrastructure for Acknowledgements Special Thanks to 5.1 Updating Integrated Views 3 Issues in Data Representation 2.2 Architectures for Database Interoperation Managing Semantic Heterogeneity , 1997 .

[18]  Dimitri P. Bertsekas,et al.  Data Networks , 1986 .

[19]  Panos K. Chrysanthis,et al.  A Formalism for Extended Transaction Model , 1991, VLDB.

[20]  Betty Salzberg,et al.  DSDT: Durable Scripts containing Database Transactions , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[21]  Gunter Schlageter,et al.  A transaction manager development facility for non standard database systems , 1992 .

[22]  Amit P. Sheth,et al.  On Transactional Workflows , 1993, IEEE Data Eng. Bull..

[23]  Reidar Conradi,et al.  EPOS: object-oriented cooperative process modelling , 1994 .

[24]  Lori A. Clarke,et al.  PLEIADES: an object management system for software engineering environments , 1993, SIGSOFT '93.

[25]  Dennis Heimbigner,et al.  APPL/A: a language for software process programming , 1995, TSEM.

[26]  Sharad Mehrotra,et al.  INCAs: Managing Dynamic Workflows in Distributed Environments , 1996 .

[27]  Hans-Jörg Schek,et al.  Concepts and Applications of Multilevel Transactions and Open Nested Transactions , 1992, Database Transaction Models for Advanced Applications.

[28]  Amit P. Sheth,et al.  Using Flexible Transactions to Support Multi-System Telecommunication Applications , 1992, VLDB.

[29]  Munindar P. Singh.  Distributed Scheduling of Workflow Computations , 1996 .

[30]  Ahmed K. Elmagarmid,et al.  Transaction Models for Advanced Database Applications , 1991 .

[31]  Sandra Heiler,et al.  Distributed Object Management , 1992, Int. J. Cooperative Inf. Syst..

[32]  Abraham Silberschatz,et al.  A Formal Approach to Recovery by Compensating Transactions , 1990, VLDB.

[33]  Gerhard Weikum,et al.  Principles and realization strategies of multilevel transaction management , 1991, TODS.

[34]  Jennifer Widom,et al.  Production Rules in Parallel and Distributed Database Environments , 1992, VLDB.

[35]  Gustavo Alonso,et al.  Advanced transaction models in workflow contexts , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[36]  E. B. Moss,et al.  Nested Transactions: An Approach to Reliable Distributed Computing , 1985 .

[37]  Weimin Du,et al.  Quasi Serializability: a Correctness Criterion for Global Concurrency Control in InterBase , 1989, VLDB.

[38]  Gustavo Alonso,et al.  Workflow Management Systems: The Next Generation Of Distributed Processing Tools , 1997 .

[39]  Alexander Borgida,et al.  Language features for flexible handling of exceptions in information systems , 1985, TODS.

[40]  Amit P. Sheth,et al.  CORBA-Based Run Time Architectures for Workflow Management Systems , 1996 .

[41]  Willy Zwaenepoel,et al.  Manetho: Transparent Rollback-Recovery with Low Overhead, Limited Rollback, and Fast Output Commit , 1992, IEEE Trans. Computers.

[42]  M. Suzuki,et al.  Meta-Operations in the Process Model Hfsp for the Dynamics and Flexibility of Software Processes , 1991, Proceedings. First International Conference on the Software Process,.

[43]  Johannes Klein.  Advanced rule driven transaction management , 1991, COMPCON Spring '91 Digest of Papers.

[44]  Rajeev Rastogi,et al.  The concurrency control problem in multidatabases: characteristics and solutions , 1992, SIGMOD '92.

[45]  Qian Cui,et al.  Data-Oriented Exception Handling , 1992, IEEE Trans. Software Eng..

[46]  Andreas Reuter,et al.  Transaction Processing: Concepts and Techniques , 1992 .

[47]  Calton Pu,et al.  Split-Transactions for Open-Ended Activities , 1988, VLDB.

[48]  Sharma Chakravarthy,et al.  Snoop: An Expressive Event Specification Language for Active Databases , 1994, Data Knowl. Eng..

[49]  T. Kaneko,et al.  A Method for Software Process Modeling and Description Using Lotos , 1991, Proceedings. First International Conference on the Software Process,.

[50]  Calton Pu,et al.  A Practical and Modular Implementation of Extended Transaction Models , 1995, VLDB.

[51]  Gustavo Alonso,et al.  Exotica/FMQM: A Persistent Message-Based Architecture for Distributed Workflow Management , 1995 .

[52]  Donald D. Chamberlin,et al.  Dynamic Data Distribution (D3) in a Shared-Nothing Multiprocessor Data Store , 1992, VLDB.

[53]  M. Rusinkiewicz.  Speciication and Execution of Transactional Workkows , 1994 .

[54]  Paul W. P. J. Grefen,et al.  WIDE-a distributed architecture for workflow management , 1997, Proceedings Seventh International Workshop on Research Issues in Data Engineering. High Performance Database Management for Large-Scale Applications.

[55]  Wilhelm Schäfer,et al.  MERLIN: supporting cooperation in software development through a knowledge-based environment , 1994 .

[56]  Narain H. Gehani,et al.  ASSET: a system for supporting extended transactions , 1994, SIGMOD '94.

[57]  Layna Fischer,et al.  The workflow paradigm : the impact of information technology on business process reengineering , 1994 .

[58]  Gail E. Kaiser,et al.  Experience with Process Modeling in the Marvel Software Development Environment Kernel , 1989 .

[59]  Bharat K. Bhargava,et al.  Ensuring relaxed atomicity for flexible transactions in multidatabase systems , 1994, SIGMOD '94.

[60]  Gail E. Kaiser,et al.  The CORD approach to extensible concurrency control , 1997, Proceedings 13th International Conference on Data Engineering.

[61]  Philip A. Bernstein.  Transaction processing monitors , 1990, CACM.

[62]  Mark F. Hornick,et al.  Customizing Transaction Models and Mechanisms in a Programmable Environment Supporting Reliable Workflow Automation , 1996, IEEE Trans. Knowl. Data Eng..

[63]  Friedemann Schwenkreis.  APRICOTS a prototype implementation of a ConTract system: management of the control flow and the communication system , 1993, Proceedings of 1993 IEEE 12th Symposium on Reliable Distributed Systems.

[64]  Krithi Ramamritham,et al.  Failure handling and coordinated execution of concurrent workflows , 1998, Proceedings 14th International Conference on Data Engineering.

[65]  Johann Eder,et al.  The Workflow Activity Model WAMO , 1995, CoopIS.

[66]  Gerhard Weikum,et al.  Efficient transparent application recovery in client-server information systems , 1998, SIGMOD '98.

[67]  James A. Gosling,et al.  The java language environment: a white paper , 1995 .

[68]  Th. Kreifelts.  Woetzel: distribution and error handling in an office procedure system , 1986 .

[69]  Leon J. Osterweil,et al.  The design of a next-generation process language , 1997, ESEC '97/FSE-5.

[70]  Fabio Casati,et al.  Deriving Active Rules for Workflow Enactment , 1996, DEXA.

[71]  Munindar P. Singh.  Synthesizing distributed constrained events from transactional workflow specifications , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[72]  Panos K. Chrysanthis,et al.  ACTA: a framework for specifying and reasoning about transaction structure and behavior , 1990, SIGMOD '90.

[73]  Panos K. Chrysanthis,et al.  In Search of Acceptability Citeria: Database Consistency Requirements and Transaction Correctness properties , 1992, IWDOM.

[74]  Santosh K. Shrivastava,et al.  A System for Specifying and Coordinating the Execution of Reliable Distributed Applications , 1997, DAIS.

[75]  Andreas Reuter,et al.  The ConTract Model , 1991, Database Transaction Models for Advanced Applications.

[76]  Ming-Chien Shan,et al.  . " . . HEWLETT a : ~ PACKARD Flexible Compensation ofWorkflow Processes , 2000 .

[77]  Ming-Chien Shan,et al.  Issues in Operation Flow Management for Long-Running Acivities , 1993, IEEE Data Eng. Bull..

[78]  Hector Garcia-Molina,et al.  Modeling long-running activities as nested sagas , 1991 .

[79]  K. Ramamritham,et al.  Modeling, Correctness and Systems Issues in Supporting Advanced Database Applications using Workflow Management Systems , 1995 .

[80]  Charles T. Davies,et al.  Data Processing Spheres of Control , 1978, IBM Syst. J..

[81]  Layna Fischer.  New Tools for New Times: The Workflow Paradigm , 1994 .

[82]  Gail E. Kaiser,et al.  Concurrency control in advanced database applications , 1991, CSUR.

[83]  Panos K. Chrysanthis,et al.  Advances in concurrency control and transaction processing - executive briefing , 1996 .

[84]  G. Alonso,et al.  Providing High Availability in Very Large Workkow Management Systems 1 , 1996 .

[85]  Hector Garcia-Molina,et al.  Altruistic locking , 1994, TODS.

[86]  Krithi Ramamritham,et al.  Performance characteristics of epsilon serializability with hierarchical inconsistency bounds , 1993, Proceedings of IEEE 9th International Conference on Data Engineering.

[87]  Panos K. Chrysanthis,et al.  A taxonomy of correctness criteria in database applications , 1996, The VLDB Journal.