Enterprise architecture framework evaluation criteria: a literature review and artifact development

Selecting an enterprise architecture framework (EAF) that will best address the needs of their organization is a difficult task for enterprise architecture practitioners. The objective of this study is to make this difficult task easier. To do so, this research first conducts a review of the literature on EAF evaluation criteria. Findings from this review show the shortcomings of this literature, most importantly the lack of a comprehensive set of EAF criteria and adequate measures for their operationalization. Based on these findings, and using the design science research approach, this study then designs and tests an EAF evaluation artifact that identifies, elaborates and operationalizes a comprehensive set of 14 criteria. Results of the experiment that followed show that: (i) 90.87% of the criteria were perceived usable, (ii) 97.62% of them were perceived to be applicable and relevant, and (iii) 90.48% were perceived as correct. This study makes several contributions. First, it provides a much-needed and timely overview of the literature on EAF evaluation criteria. Second, this study is the first to present a comprehensive set of criteria that not only synthesizes previously proposed criteria but also includes new criteria (e.g., SOA models and usability). Third, it answers EA practitioners’ requests for a tool that is both theoretically sound and practical. Finally, and most importantly, this study is the first to propose objective measures to operationalize EAF evaluation criteria.

[1]  Babak Darvish Rouhani,et al.  A Comparison Enterprise Architecture Implementation Methodologies , 2013, 2013 International Conference on Informatics and Creative Multimedia.

[2]  Robert Winter,et al.  Essential Layers, Artifacts, and Dependencies of Enterprise Architecture , 2006, 2006 10th IEEE International Enterprise Distributed Object Computing Conference Workshops (EDOCW'06).

[3]  Susanne Leist,et al.  Evaluation of current architecture frameworks , 2006, SAC.

[4]  William J. Kettinger,et al.  Business Process Change: A Study of Methodologies, Techniques, and Tools , 1997, MIS Q..

[5]  Sang-gun Park,et al.  A Comparative Analysis of Enterprise Architecture Frameworks Based on EA Quality Attributes , 2009, 2009 10th ACIS International Conference on Software Engineering, Artificial Intelligences, Networking and Parallel/Distributed Computing.

[6]  John F. Sowa,et al.  Extending and Formalizing the Framework for Information Systems Architecture , 1992, IBM Syst. J..

[7]  Stevan Mrdalj,et al.  A comparison of enterprise architecture frameworks , 2006 .

[8]  Abderrahmane Leshob,et al.  A Model-Driven Service Specification Approach from BPMN Models , 2017, 2017 IEEE 14th International Conference on e-Business Engineering (ICEBE).

[9]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[10]  Stephen H. Kaisler,et al.  Enterprise Architecting: Critical Problems , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[11]  Robert Winter,et al.  Enterprise architecture governance: the need for a business-to-IT approach , 2008, SAC '08.

[12]  Henk Plessius,et al.  On the Categorization and Measurability of Enterprise Architecture Benefits with the Enterprise Architecture Value Framework , 2012, TEAR/PRET.

[13]  K VaishnaviVijay,et al.  Analysis and modeling in software development , 1992 .

[14]  Galal Hassan Galal-Edeen,et al.  Towards a Framework for Enterprise Architecture Frameworks Comparison and Selection , 2006 .

[15]  Jorg Desel,et al.  Enterprise Modelling and Information Systems Architectures , 2005 .

[16]  A Comparison of the Top Four Enterprise-Architecture Methodologies , 2010 .

[17]  Marc M. Lankhorst.  Enterprise Architecture at Work - Modelling, Communication and Analysis, 3rd Edition , 2005, The Enterprise Engineering Series.

[18]  Farhad Arbab,et al.  Enterprise architecture: Management tool and blueprint for the organisation , 2006, Inf. Syst. Frontiers.

[19]  P. Rumrill,et al.  Using scoping literature reviews as a means of understanding and interpreting existing literature. , 2010, Work.

[20]  Li Da Xu,et al.  Enterprise Architecture: A Literature Review , 2017 .

[21]  Daniel M. Yellin,et al.  Using Enterprise Architecture Standards in Managing Information Technology , 2006, J. Manag. Inf. Syst..

[22]  Brian H. Cameron,et al.  Analyzing the Current Trends in Enterprise Architecture Frameworks , 2013 .

[23]  Mengmeng Zhang,et al.  A Systematic Review of Business-IT Alignment Research With Enterprise Architecture , 2018, IEEE Access.

[24]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[25]  Rodina Binti Ahmad,et al.  A systematic literature review on Enterprise Architecture Implementation Methodologies , 2015, Inf. Softw. Technol..

[26]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[27]  Paul P. Tallon,et al.  Competing Perspectives on the Link Between Strategic Information Technology Alignment and Organizational Agility: Insights from a Mediation Model , 2011, MIS Q..

[28]  Jeff Sauro,et al.  A method to standardize usability metrics into a single score , 2005, CHI.

[29]  Guy Doumeingts,et al.  Architectures for enterprise integration and interoperability: Past, present and future , 2008, Comput. Ind..

[30]  Andrzej Sobczak.  Methods of the Assessment of Enterprise Architecture Practice Maturity in an Organization , 2013, BIR.

[31]  L. Delbeke.  Quasi-experimentation - design and analysis issues for field settings - cook,td, campbell,dt , 1980 .

[32]  Mathias Ekstedt,et al.  EAF2- A Framework for Categorizing Enterprise Architecture Frameworks , 2009, 2009 10th ACIS International Conference on Software Engineering, Artificial Intelligences, Networking and Parallel/Distributed Computing.

[33]  Robert Winter,et al.  A Comprehensive Enterprise Architecture Metamodel and Its Implementation Using a Metamodeling Platform , 2005 .

[34]  H. Arksey,et al.  Scoping studies: towards a methodological framework , 2005 .

[35]  Mario Piattini,et al.  Empirical validation of class diagram metrics , 2002, Proceedings International Symposium on Empirical Software Engineering.

[36]  Antony Tang,et al.  A comparative analysis of architecture frameworks , 2004, 11th Asia-Pacific Software Engineering Conference.

[37]  Sabine Buckl,et al.  A Pattern based Approach for constructing Enterprise Architecture Management Information Models , 2007, Wirtschaftsinformatik.

[38]  Marc M. Lankhorst,et al.  Supporting viewpoint-oriented enterprise architecture , 2004, Proceedings. Eighth IEEE International Enterprise Distributed Object Computing Conference, 2004. EDOC 2004..

[39]  Robert Winter,et al.  How are Enterprise Architecture Design Principles Used? , 2011, 2011 IEEE 15th International Enterprise Distributed Object Computing Conference Workshops.

[40]  Monique Snoeck,et al.  Enterprise architecture for small and medium-sized enterprises: a starting point for bringing EA to SMEs, based on adoption models , 2012, CENTERIS 2012.

[41]  Bill Curtis,et al.  Process modeling , 1992, CACM.

[42]  Samir Chatterjee,et al.  A Design Science Research Methodology for Information Systems Research , 2008 .

[43]  Susumu Saito,et al.  Ergonomics of human-system interaction , 1994 .

[44]  Hafedh Mili,et al.  From inter-organizational business process models to service-oriented architecture models , 2018, Service Oriented Computing and Applications.

[45]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[46]  Victor R. Basili,et al.  The TAME Project: Towards Improvement-Oriented Software Environments , 1988, IEEE Trans. Software Eng..

[47]  Juhani Iivari,et al.  Distinguishing and contrasting two strategies for design science research , 2015, Eur. J. Inf. Syst..

[48]  Jaap Schekkerman,et al.  How to Survive in the Jungle of Enterprise Architecture Framework: Creating or Choosing an Enterprise Architecture Framework , 2003 .