Evaluating Coding Behavior in Software Development Processes: A Process Mining Approach

Process mining is a family of techniques that aim at analyzing business process execution data recorded in event logs. Conformance checking is a branch of this discipline embracing approaches for verifying whether the behavior of a process, as recorded in a log, is in line with some expected behavior provided in the form of a process model. In the literature, process mining techniques have already been used to study software development processes starting from logs derived from version management systems or from document management systems. In this paper, we use conformance checking to test coding behaviors starting from event logs generated from IDE usage. Understanding how developers carry out coding activities and what hurdles they usually face should provide useful tips for improving and supporting software development processes. In particular, through conformance checking, we can compare different process executions, and identify behavioral similarities and differences. In our experimentation, we evaluated the activities performed by 40 novice developers performing coding activities in 5 development sessions. We assessed the developers to distinguish the ones obtaining the best performance. We then compared the behavior extracted from this group of developers with the others. The results show different IDE usage patterns for developers with different skills and performance.

[1]  Wil M. P. van der Aalst,et al.  Process mining in software systems: Discovering real-life business transactions and process models from distributed systems , 2015, 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS).

[2]  Boudewijn F. van Dongen,et al.  Component behavior discovery from software execution data , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[3]  Fabrizio Maria Maggi,et al.  Lights, Camera, Action! Business Process Movies for Online Process Discovery , 2014, Business Process Management Workshops.

[4]  Sander J. J. Leemans,et al.  Scalable process discovery and conformance checking , 2016, Software & Systems Modeling.

[5]  Fabrizio Maria Maggi,et al.  M3D: a tool for the model driven development of web applications , 2012, WIDM '12.

[6]  Sander J. J. Leemans,et al.  Discovering Block-Structured Process Models from Event Logs - A Constructive Approach , 2013, Petri Nets.

[7]  Les Gasser,et al.  Bug Report Networks: Varieties, Strategies, and Impacts in a F/OSS Development Community , 2004, MSR.

[8]  Tom DeMarco,et al.  Peopleware: Productive Projects and Teams , 1987 .

[9]  Wil M. P. van der Aalst,et al.  Process Mining , 2016, Springer Berlin Heidelberg.

[10]  Fabio Martinelli,et al.  Dynamic malware detection and phylogeny analysis using process mining , 2018, International Journal of Information Security.

[11]  Boudewijn F. van Dongen,et al.  Process Mining Framework for Software Processes , 2007, ICSP.

[12]  Boudewijn F. van Dongen,et al.  A genetic algorithm for discovering process trees , 2012, 2012 IEEE Congress on Evolutionary Computation.

[13]  Wil M.P. van der Aalst,et al.  Fuzzy Mining - Adaptive Process Simplification Based on Multi-perspective Metrics , 2007, BPM.

[14]  Ruven E. Brooks.  Towards a theory of the cognitive processes in computer programming , 1999, Int. J. Hum. Comput. Stud..

[15]  Abreu Fernando Brito e,et al.  Software Development Process Mining: Discovery, Conformance Checking and Enhancement , 2016 .

[16]  Bob Rehder,et al.  Cognitive Activities and Levels of Abstraction in Procedural and Object-Oriented Design , 1995, Hum. Comput. Interact..

[17]  Ricardo Massa Ferreira Lima,et al.  Conformance Checking of Software Development Processes Through Process Mining , 2011, SEKE.

[18]  J. Herbsleb,et al.  Two case studies of open source software development: Apache and Mozilla , 2002, TSEM.

[19]  Ricardo Seguel,et al.  Process Mining Manifesto , 2011, Business Process Management Workshops.

[20]  Fabrizio Maria Maggi,et al.  Do activity lifecycles affect the validity of a business rule in a business process? , 2016, Inf. Syst..

[21]  R. Guindon,et al.  Control of cognitive processes during software design: what tools are needed? , 1988, CHI '88.

[22]  Alexander L. Wolf,et al.  Discovering models of software processes from event-based data , 1998, TSEM.

[23]  Walt Scacchi,et al.  Understanding Requirements for Open Source Software , 2009 .

[24]  Wil M. P. van der Aalst,et al.  Big Software on the Run In Vivo Software Analytics Based on Process Mining (Keynote) , 2015 .