Systematic performance evaluation using component-in-the-loop approach

Timeliness and throughput critical applications require a framework offering predictable temporal characteristics. The best practice for estimating a prediction of the system dynamics relies on benchmarking. Each novel middleware solution needs such an evaluation as part of the development process to assure an appropriate throughput in the future use. General purpose blockchain frameworks are viable replacements for many current systems in several sectors by providing a fully distributed, secure, and non-repudiable ledger as a service. Blockchain technologies target domains with a large number of interactions, thus demanding strict performance guarantees. Engineering for performance targets in a trustworthy manner requires performance models. However, performance characteristics of blockchain systems are highly unexplored due to the novelty of the technology. This paper proposes a general-purpose, systematic methodology for the performance analysis of complex systems, aided by a component-in-the-loop approach. The Linux foundation-hosted hyperledger fabric serves as a case study for the presented methodology.