KPIs for Software Ecosystems: A Systematic Mapping Study

To create value with a software ecosystem (SECO), a platform owner has to ensure that the SECO is healthy and sustainable. Key Performance Indicators (KPI) are used to assess whether and how well such objectives are met and what the platform owner can do to improve. This paper gives an overview of existing research on KPI-based SECO assessment using a systematic mapping of research publications. The study identified 34 relevant publications for which KPI research and KPI practice were extracted and mapped. It describes the strengths and gaps of the research published so far, and describes what KPI are measured, analyzed, and used for decision-making from the researcher’s point of view. For the researcher, the maps thus capture state-of-knowledge and can be used to plan further research. For practitioners, the generated map points to studies that describe how to use KPI for managing of a SECO.

[1]  Olavo Barbosa,et al.  A Systematic Mapping Study on Software Ecosystems , 2011, IWSECO@ICSOB.

[2]  Klaus Marius Hansen,et al.  Reviewing the Health of Software Ecosystems - A Conceptual Framework Proposal , 2013, IWSECO@ICSOB.

[3]  David Parmenter,et al.  Key Performance Indicators: Developing, Implementing,and Using Winning KPIs , 2007 .

[4]  H. Boley,et al.  Digital Ecosystems: Principles and Semantics , 2007, 2007 Inaugural IEEE-IES Digital EcoSystems and Technologies Conference.

[5]  Anthony Plummer,et al.  Towards Optimal Cooperative Caching in Social Wireless Networks , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[6]  Rukshan Athauda,et al.  TIDE: measuring and evaluating trustworthiness and credibility of enterprises in digital ecosystem , 2011, MEDES.

[7]  L. Telesca,et al.  A peer-to-peer multidimensional trust model for digital ecosystems , 2008, 2008 2nd IEEE International Conference on Digital Ecosystems and Technologies.

[8]  Tom Mens,et al.  On the variation and specialisation of workload—A case study of the Gnome ecosystem community , 2014, Empirical Software Engineering.

[9]  Detmar W. Straub,et al.  Measuring Firm Performance at the Network Level: A Nomology of the Business Impact of Digital Supply Networks , 2004, J. Manag. Inf. Syst..

[10]  F. Stuart Chapin,et al.  Principles of Ecosystem Sustainability , 1996, The American Naturalist.

[11]  Slinger Jansen,et al.  Software ecosystems: a software ecosystem strategy assessment model , 2010, ECSA '10.

[12]  Slinger Jansen,et al.  A sense of community: A research agenda for software ecosystems , 2009, 2009 31st International Conference on Software Engineering - Companion Volume.

[13]  Klaus Marius Hansen,et al.  Software ecosystems - A systematic literature review , 2013, J. Syst. Softw..

[14]  Adriano M. Pereira,et al.  Assessing success factors of selling practices in electronic marketplaces , 2009, MEDES.

[15]  Tomoya Enokido,et al.  An Integrated Power Consumption Model for Communication and Transaction Based Applications , 2011, 2011 IEEE International Conference on Advanced Information Networking and Applications.

[16]  Xiaowei Li,et al.  A Loss Inference Algorithm for Wireless Sensor Networks to Improve Data Reliability of Digital Ecosystems , 2011, IEEE Transactions on Industrial Electronics.

[17]  Gregory L. Richards,et al.  The Information Technology Ecosystem: Structure, Health, and Performance , 2006 .

[18]  Soo Dong Kim,et al.  A model of quality-in-use for service-based mobile ecosystem , 2013, 2013 1st International Workshop on the Engineering of Mobile-Enabled Systems (MOBS).

[19]  Robert Costanza,et al.  What is a healthy ecosystem? , 1999, Aquatic Ecology.

[20]  Miles McQueen,et al.  Analyses of two end-user software vulnerability exposure metrics (extended version) , 2013, Inf. Secur. Tech. Rep..

[21]  Carles Sierra,et al.  Supplier performance in a digital ecosystem , 2009, 2009 3rd IEEE International Conference on Digital Ecosystems and Technologies.

[22]  Jan Bosch,et al.  From software product lines to software ecosystems , 2009, SPLC.

[23]  Tom Mens,et al.  A framework for analysing and visualising open source software ecosystems , 2010, IWPSE-EVOL '10.

[24]  Elizabeth Chang,et al.  A service concept recommendation system for enhancing the dependability of semantic service matchmakers in the service ecosystem environment , 2011, J. Netw. Comput. Appl..

[25]  Amel Ben Hadj Salem Mhamdia Performance measurement practices in software ecosystem , 2013 .

[26]  Elizabeth Chang,et al.  Semantic Service Retrieval and QoS Measurement in the Digital Ecosystem Environment , 2010, 2010 International Conference on Complex, Intelligent and Software Intensive Systems.

[27]  Eric van Heck,et al.  Network transparency and the performance of dynamic business networks , 2010, 4th IEEE International Conference on Digital Ecosystems and Technologies.

[28]  Louise Young,et al.  Enterprise university as a digital ecosystem: Visual analysis of academic collaboration , 2009, IEEE International Conference on Digital Ecosystems and Technologies.

[29]  Farookh Khadeer Hussain,et al.  A Methodology for Maintaining Trust in Industrial Digital Ecosystems , 2013, IEEE Transactions on Industrial Electronics.

[30]  Slinger Jansen,et al.  Relationship intimacy in software ecosystems: a survey of the Dutch software industry , 2011, MEDES.

[31]  Roel Wieringa,et al.  Requirements engineering paper classification and evaluation criteria: a proposal and a discussion , 2005, Requirements Engineering.

[32]  Antonio Krüger,et al.  AppFunnel: a framework for usage-centric evaluation of recommender systems that suggest mobile applications , 2013, IUI '13.

[33]  A. McMichael,et al.  Assessing ecosystem health. , 1998, Trends in ecology & evolution.

[34]  Reiner R. Dumke,et al.  Growth- and Entropy-Based SOA Measurement: Vision and Approach in a Large Scale Environment , 2011, 2011 Joint Conference of the 21st International Workshop on Software Measurement and the 6th International Conference on Software Process and Product Measurement.

[35]  Elizabeth Chang,et al.  A QoS-based service retrieval methodology for digital ecosystems , 2009, Int. J. Web Grid Serv..

[36]  Andrea Giessmann,et al.  Leveraging the Software Ecosystem - Towards a Business Model Framework for Marketplaces , 2012, DCNET/ICE-B/OPTICS.

[37]  Slinger Jansen How quality attributes of software platform architectures influence software ecosystems , 2013, WEA 2013.

[38]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[39]  John Murray,et al.  Impact of plugins on the security of web applications , 2010, MetriSec '10.

[40]  Yi Liu,et al.  Service Ecosystem Evolution and Controlling: A Research Framework for the Effects of Dynamic Services , 2013, 2013 International Conference on Service Sciences (ICSS).

[41]  Jan Bosch,et al.  Architecture for embedded open software ecosystems , 2014, J. Syst. Softw..

[42]  Chen Wu,et al.  A Method for Service Quality Assessment in a Service Ecosystem , 2007, 2007 Inaugural IEEE-IES Digital EcoSystems and Technologies Conference.

[43]  Reijo Savola,et al.  Metrics driven security management framework for e-health digital ecosystem focusing on chronic diseases , 2012, MEDES.

[44]  Rodrigo Pereira dos Santos,et al.  Software Ecosystems: Trends and Impacts on Software Engineering , 2012, 2012 26th Brazilian Symposium on Software Engineering.

[45]  Robert Costanza,et al.  Ecosystem Health New Goals for Environmental Management , 1992 .

[46]  Paul Krause,et al.  A digital ecosystem view on cloud computing , 2012, 2012 6th IEEE International Conference on Digital Ecosystems and Technologies (DEST).

[47]  Jing,et al.  Business Ecosystem Strategies of Mobile Network Operators in the 3G Era , 2011 .

[48]  Fatos Xhafa,et al.  A simulation system for WSNs as a Digital Eco-System approach considering goodput metric , 2010, 4th IEEE International Conference on Digital Ecosystems and Technologies.

[49]  Chunhua Tian,et al.  Service analytics framework for web-delivered services , 2008, 2008 IEEE International Conference on Service Operations and Logistics, and Informatics.

[50]  Neeraj Suri,et al.  Quantitative Assessment of Cloud Security Level Agreements - A Case Study , 2012, SECRYPT.

[51]  Gary Cokins,et al.  Performance Management: Integrating Strategy Execution, Methodologies, Risk, and Analytics , 2009 .