Comparison of scenario-based software architecture evaluation methods

Software engineering community has proposed several methods to evaluate software architectures with respect to desired quality attributes such as maintainability, performance, and so on. There is, however, little effort on systematically comparing such methods to discover similarities and differences between existing approaches. In this paper, we compare four well known scenario-based SA evaluation methods using an evaluation framework. The framework considers each method from the point of view of method context, stakeholders, structure, and reliability. The comparison reveals that most of the studied methods are structurally similar but there are a number of differences among their activities and techniques. Therefore, some methods overlap, which guides us to identify five common activities that can form a generic process model.

[1]  Richard N. Taylor,et al.  A Classification and Comparison Framework for Software Architecture Description Languages , 2000, IEEE Trans. Software Eng..

[2]  李幼升,et al.  Ph , 1989 .

[3]  Gregory D. Abowd,et al.  Analyzing the Properties of User Interface Software , 1993 .

[4]  Thomas J. Mowbray,et al.  AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis , 1998 .

[5]  C.U. Smith.  Performance engineering for software architectures , 1997, Proceedings Twenty-First Annual International Computer Software and Applications Conference (COMPSAC'97).

[6]  Connie U. Smith,et al.  PASASM: a method for the performance assessment of software architectures , 2002, WOSP '02.

[7]  Klaus Kronlöf,et al.  Method integration: concepts and case studies , 1993 .

[8]  Ivar Jacobson,et al.  The Unified Modeling Language User Guide , 1998, J. Database Manag..

[9]  Claes Wohlin,et al.  A method for understanding quality attributes in software architecture structures , 2002, SEKE '02.

[10]  Felix Bachmann,et al.  Preliminary Design of ArchE: A Software Architecture Design Assistant , 2003 .

[11]  Rami Bahsoon,et al.  Evaluating software architectures: development, stability, and evolution , 2003 .

[12]  Philippe Kruchten,et al.  The 4+1 View Model of Architecture , 1995, IEEE Softw..

[13]  Eila Niemelä,et al.  A Survey on Software Architecture Analysis Methods , 2002, IEEE Trans. Software Eng..

[14]  Leonard J. Bass,et al.  SAAM: a method for analyzing the properties of software architectures , 1994, Proceedings of 16th International Conference on Software Engineering.

[15]  KruchtenPhilippe.  The 4+1 View Model of Architecture , 1995 .

[16]  Philippe Kruchten,et al.  The Rational Unified Process: An Introduction , 1998 .

[17]  Rick Kazman.  Tool support for architecture analysis and design , 1996, ISAW '96.

[18]  Mary Shaw,et al.  The coming-of-age of software architecture research , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[19]  Rick Kazman,et al.  The architecture tradeoff analysis method , 1998, Proceedings. Fourth IEEE International Conference on Engineering of Complex Computer Systems (Cat. No.98EX193).

[20]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[21]  Rick Kazman,et al.  Workshop on methods and techniques for software architecture review and assessment (SARA) , 2002, Proceedings of the 24th International Conference on Software Engineering. ICSE 2002.

[22]  Mark Klein,et al.  Attribute-Based Architectural Styles , 1999 .

[23]  Mari Matinlassi,et al.  Comparison of software product line architecture design methods: COPA, FAST, FORM, KobrA and QADA , 2004, Proceedings. 26th International Conference on Software Engineering.

[24]  Elaine J. Weyuker,et al.  Investigating metrics for architectural assessment , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[25]  Meliha Handzic,et al.  Managing Software Engineering Knowledge , 2010, Springer Berlin Heidelberg.

[26]  Hans van Vliet,et al.  How well can we predict changes at architecture design time? , 2003, J. Syst. Softw..

[27]  Chung-Horng Lung,et al.  A stakeholder-centric software architecture analysis approach , 1996, ISAW '96.

[28]  PerOlof Bengtsson,et al.  Architecture-Level Modifiability Analysis , 2002 .

[29]  Connie U. Smith,et al.  Performance evaluation of software architectures , 1998, WOSP '98.

[30]  Hans van Vliet,et al.  Towards a broader view on software architecture analysis of flexibility , 1999, Proceedings Sixth Asia Pacific Software Engineering Conference (ASPEC'99) (Cat. No.PR00509).

[31]  Rick Kazman,et al.  Evaluating Software Architectures: Methods and Case Studies , 2001 .

[32]  Jan Bosch,et al.  Architecture level prediction of software maintenance , 1999, Proceedings of the Third European Conference on Software Maintenance and Reengineering (Cat. No. PR00090).

[33]  Connie U. Smith,et al.  PASA(SM): An Architectural Approach to Fixing Software Performance Problems , 2002, Int. CMG Conference.

[34]  Jan Bosch,et al.  Architecture-level modifiability analysis (ALMA) , 2004, J. Syst. Softw..

[35]  P. Krutchen,et al.  The Rational Unified Process: An Introduction , 2000 .

[36]  Connie U. Smith,et al.  Software Performance Engineering: A Case Study Including Performance Comparison with Design Alternatives , 1993, IEEE Trans. Software Eng..

[37]  Nimal Jayaratna,et al.  Understanding and Evaluating Methodologies: NIMSAD, a Systematic Framework , 1994 .

[38]  N. H. Lassing,et al.  Using UML in Architecture-Level Modifiability Analysis , 2001 .

[39]  Paul Clements,et al.  Software architecture in practice , 1999, SEI series in software engineering.

[40]  Liming Zhu,et al.  A framework for classifying and comparing software architecture evaluation methods , 2004, 2004 Australian Software Engineering Conference. Proceedings..

[41]  Juan Antonio de la Puente,et al.  A Software Architecture Evaluation Model , 1998, ESPRIT ARES Workshop.

[42]  Leonard J. Bass,et al.  Attribute-Based Architecture Styles , 1999, WICSA.

[43]  Paul Clements.  Active Reviews for Intermediate Designs , 2000 .

[44]  Robert L. Nord,et al.  Applied Software Architecture , 1999, Addison Wesley object technology series.