Method towards reconstructing collaborative business processes with cloud services using evolutionary deep Q-learning

Abstract Service-oriented architecture (SOA) is a significant framework that enables intelligent information systems to offer business process-based services, namely Business Process as a Service (BPaaS). Cloud service-based business model reconstruction within and across enterprises has become an important issue to obtain competitive advantages. Finding appropriate service component is a critical phase in enterprise collaboration, to improve the quality and correlations among collaborative service-providers. Existed methods for business processes reconstruction have not systematically and fully considered the quality correlations in finer-grain, i.e. task-level or activity-level, and temporal performance in a process model optimization. Moreover, the existed approaches might fail to work in an uncertain cloud environment where the quality parameters are unknown in advance. Q-learning has proven its worth in an uncertain cloud environment. However, advances in Q-learning are challenges to leverage in collaborative business process reconstruction. The reconstruction algorithms based on Q-learning suffered from the two core drawbacks: lack of effective exploration and extremely slow convergence property. A hybrid Evolutionary Deep Q-Learning-based BPaaS reconstruction algorithm, named as EDQL-BPR, is proposed by leveraging Particle Swarm Optimization to improve Deep Q-Learning algorithm for systematically optimizing collaborative business processes reconstruction. In order to verify the effectiveness of the proposed algorithm, an annotated transition system has been developed supporting for all possible behaviors state convention from heterogeneous initial states to the target with several collaborative information at activity level, and encoding annotated BP representation matrix automatically. Then, an autonomous three-layer framework has been built to facilitate business process discovery and service reconstruction. In this framework, our optimal collaborative services composition is treated as a multi-objective constraint optimization problem via Evolutionary Deep Q-learning with updated strategy of Q-value, and each annotated component launch a service discovery by service agent. Extensive evaluation results show that EDQL-BPR can outperform several representative business process reconstructions in terms of model optimal convergence, QoS optimality, effectiveness and efficiency under heterogeneous service selection workloads in an uncertain Cloud environment.