Using BPM Frameworks for Identifying Customer Feedback About Process Performance

Every organization has business processes, however, there are numerous organizations in which execution logs of processes are not available. Consequently, these organizations do not have the opportunity to exploit the potential of execution logs for analyzing the performance of their processes. As a first step towards facilitating these organizations, in this paper, we argue that customer feedback is a valuable source of information that can provide important insights about process performance. However, a key challenge to this approach is that the feedback includes a significant amount of comments that are not related to process performance. Therefore, utilizing the complete feedback without omitting the irrelevant comments may generate misleading results. To that end, firstly, we have generated a customer feedback corpus of 3356 comments. Secondly, we have used two well-established BPM frameworks, Devil’s Quadrangle and Business Process Redesign Implementation framework, to manually classify the comments as relevant and irrelevant to process performance. Finally, we have used five supervised learning techniques to evaluate the effectiveness of the two frameworks for their ability to automatically identify performance relevant comments. The results show that Devil’s Quadrangle is more suitable framework than Business Process Redesign Implementation framework.

[1]  Wil M. P. van der Aalst,et al.  Beyond Process Mining: From the Past to Present and Future , 2010, CAiSE.

[2]  Mariska Netjes,et al.  Quantifying the Performance of Workflows , 2008, Inf. Syst. Manag..

[3]  Jorge Munoz-Gama,et al.  Conformance Checking and Diagnosis in Process Mining , 2016, Lecture Notes in Business Information Processing.

[4]  Cha Zhang,et al.  Ensemble Machine Learning: Methods and Applications , 2012 .

[5]  Ronghua Liang,et al.  The Design and Implementation of a Process-Driven Higher Educational Administrative System , 2012 .

[6]  Hajo A. Reijers,et al.  Best practices in business process redesign: an overview and qualitative evaluation of successful redesign heuristics , 2005 .

[7]  Cha Zhang,et al.  Ensemble Machine Learning , 2012 .

[8]  Johannes Gehrke,et al.  Data Mining with Decision Trees , 2000, ICDE.

[9]  Manoj K. Malhotra,et al.  Operations Management: Processes and Value Chains , 2004 .

[10]  Boudewijn F. van Dongen,et al.  Replaying history on process models for conformance checking and performance analysis , 2012, WIREs Data Mining Knowl. Discov..

[11]  E. Hippel,et al.  Customers As Innovators: A New Way to Create Value , 2002 .

[12]  P. Danaher,et al.  Customer Satisfaction during the Service Delivery Process , 1994 .

[13]  J. Neter,et al.  Applied Linear Regression Models , 1983 .

[14]  Karen Corral,et al.  The impact of alternative diagrams on the accuracy of recall: A comparison of star-schema diagrams and entity-relationship diagrams , 2006, Decis. Support Syst..

[15]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[16]  Sari Kujala,et al.  User involvement: A review of the benefits and challenges , 2003, Behav. Inf. Technol..

[17]  Hajo A. Reijers,et al.  Best practices in business process redesign: validation of a redesign framework , 2005, Comput. Ind..

[18]  Jorge Munoz-Gama,et al.  Conformance Checking and Diagnosis in Process Mining: Comparing Observed and Modeled Processes , 2016 .