Architectural Design Decisions for Systems Supporting Model-Based Analysis of Runtime Events: A Qualitative Multi-method Study

Models are popular artefacts in the software development process which promise to improve stakeholder communication and the overall quality of a software system under construction. Recent research proposes that the usefulness of models is not limited only to the software design phase: Empirical evidence indicates that manual analysis of a running system is improved when models are linked to recorded runtime events. However, software architects are confronted with various design decisions when designing a system that yields the required runtime events and correlates them to the model elements they originate from. The contribution of this paper is a taxonomy of architectural design decisions distilled from a series of qualitative studies following a multi-method research study design: We utilized coding techniques from Grounded Theory to build an initial taxonomy from architectural concepts found in the literature, and verified and extended the taxonomy independently by five novice software architects in a practical scenario. The resulting taxonomy captures essential architectural decisions when implementing a system that supports the analysis of its runtime behaviour using models. We then performed initial steps towards a architectural guidance model by applying the taxonomy to another realistic scenario, following the Design Science Research method, in order to analyze the properties and deepen our own technical understanding of the captured concepts in the taxonomy.

[1]  Brice Morin,et al.  Models@ Run.time to Support Dynamic Adaptation , 2009, Computer.

[2]  Harald C. Gall,et al.  Runtime metric meets developer: building better cloud applications using feedback , 2015, Onward!.

[3]  Nelly Bencomo,et al.  Requirements reflection: requirements as runtime entities , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[4]  M. Patton,et al.  Enhancing the quality and credibility of qualitative analysis. , 1999, Health services research.

[5]  Nelly Bencomo,et al.  Models@run.time , 2014, Lecture Notes in Computer Science.

[6]  Uwe Zdun,et al.  Controlled experiment on the comprehension of runtime phenomena using models created at design time , 2016, MoDELS.

[7]  Wil M. P. van der Aalst,et al.  Conformance checking of processes based on monitoring real behavior , 2008, Inf. Syst..

[8]  D. Jayathilake,et al.  Towards structured log analysis , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).

[9]  A. Strauss,et al.  The discovery of grounded theory: strategies for qualitative research aldine de gruyter , 1968 .

[10]  Mark Rouncefield,et al.  Empirical assessment of MDE in industry , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[11]  Frank Leymann,et al.  Managing architectural decision models with dependency relations, integrity constraints, and production rules , 2009, J. Syst. Softw..

[12]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[13]  Uwe Zdun,et al.  Systematic literature review of the objectives, techniques, kinds, and architectures of models at runtime , 2016, Software & Systems Modeling.

[14]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[15]  Olaf Zimmermann,et al.  Reference architecture, metamodel, and modeling principles for architectural knowledge management in information technology services , 2012, J. Syst. Softw..

[16]  Marian Petre,et al.  UML in practice , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[17]  Frank Eliassen,et al.  Using architecture models for runtime adaptability , 2006, IEEE Software.

[18]  Nelly Bencomo,et al.  On the use of software models during software execution , 2009, 2009 ICSE Workshop on Modeling in Software Engineering.

[19]  Hsiu-Fang Hsieh,et al.  Three Approaches to Qualitative Content Analysis , 2005, Qualitative health research.

[20]  Manfred Reichert,et al.  Supporting workflow schema evolution by efficient compliance checks , 2005 .

[21]  Uwe Zdun,et al.  Reusable event types for models at runtime to support the examination of runtime phenomena , 2015, 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS).

[22]  Frank Leymann,et al.  Reusable Architectural Decision Models for Enterprise Application Development , 2007, QoSA.

[23]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[25]  K. Perreault,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2011 .