The Imprecisions of Precision Measures in Process Mining

Abstract In process mining, precision measures are used to quantify how much a process model overapproximates the behavior seen in an event log. Although several measures have been proposed throughout the years, no research has been done to validate whether these measures achieve the intended aim of quantifying over-approximation in a consistent way for all models and logs. This paper fills this gap by postulating a number of axioms for quantifying precision consistently for any log and any model. Further, we show through counter-examples that none of the existing measures consistently quantifies precision.

[1]  Boudewijn F. van Dongen,et al.  Measuring precision of modeled behavior , 2015, Inf. Syst. E Bus. Manag..

[2]  Welch Bl.  THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[3]  Tadao Murata,et al.  Petri nets: Properties, analysis and applications , 1989, Proc. IEEE.

[4]  Bart Baesens,et al.  A robust F-measure for evaluating discovered process models , 2011, 2011 IEEE Symposium on Computational Intelligence and Data Mining (CIDM).

[5]  Wil M. P. van der Aalst,et al.  Conformance checking of processes based on monitoring real behavior , 2008, Inf. Syst..

[6]  Wil M.P. van der Aalst,et al.  YAWL: yet another workflow language , 2005, Inf. Syst..

[7]  Luigi Pontieri,et al.  Discovering expressive process models by clustering log traces , 2006, IEEE Transactions on Knowledge and Data Engineering.

[8]  Sander J. J. Leemans,et al.  Scalable process discovery and conformance checking , 2016, Software & Systems Modeling.

[9]  Sander J. J. Leemans,et al.  Discovering Block-Structured Process Models from Event Logs - A Constructive Approach , 2013, Petri Nets.

[10]  Boudewijn F. van Dongen,et al.  Replaying history on process models for conformance checking and performance analysis , 2012, WIREs Data Mining Knowl. Discov..

[11]  Boudewijn F. van Dongen,et al.  Quality Dimensions in Process Discovery: The Importance of Fitness, Precision, Generalization and Simplicity , 2014, Int. J. Cooperative Inf. Syst..

[12]  Bart Baesens,et al.  Robust Process Discovery with Artificial Negative Events , 2009, J. Mach. Learn. Res..

[13]  Boudewijn F. van Dongen,et al.  Avoiding Over-Fitting in ILP-Based Process Discovery , 2015, BPM.

[14]  Josep Carmona,et al.  A Unified Approach for Measuring Precision and Generalization Based on Anti-alignments , 2016, BPM.

[15]  Boudewijn F. van Dongen,et al.  Conformance Checking Using Cost-Based Fitness Analysis , 2011, 2011 IEEE 15th International Enterprise Distributed Object Computing Conference.

[16]  Boudewijn F. van Dongen,et al.  A genetic algorithm for discovering process trees , 2012, 2012 IEEE Congress on Evolutionary Computation.

[17]  Ricardo Seguel,et al.  Process Mining Manifesto , 2011, Business Process Management Workshops.

[18]  Boudewijn F. van Dongen,et al.  The ProM Framework: A New Era in Process Mining Tool Support , 2005, ICATPN.

[19]  Marlon Dumas,et al.  BPMN Miner: Automated discovery of BPMN process models with hierarchical structure , 2016, Inf. Syst..

[20]  Josep Carmona,et al.  Anti-alignments in Conformance Checking - The Dark Side of Process Models , 2016, Petri Nets.

[21]  W. M. P. V. D. Aalsta,et al.  YAWL : yet another workflow language , 2015 .

[22]  Boudewijn F. van Dongen,et al.  Alignment Based Precision Checking , 2012, Business Process Management Workshops.

[23]  Wil M. P. van der Aalst,et al.  Process Mining , 2016, Springer Berlin Heidelberg.

[24]  Bart Baesens,et al.  Improved Artificial Negative Event Generation to Enhance Process Event Logs , 2012, CAiSE.

[25]  Wil M. P. van der Aalst,et al.  Verification of Workflow Nets , 1997, ICATPN.

[26]  Josep Carmona,et al.  A Fresh Look at Precision in Process Conformance , 2010, BPM.

[27]  Bart Baesens,et al.  Determining Process Model Precision and Generalization with Weighted Artificial Negative Events , 2014, IEEE Transactions on Knowledge and Data Engineering.