Blockchain Evaluation Approaches: State-of-the-Art and Future Perspective

The present increase of attention toward blockchain-based systems is currently reaching a tipping point with the corporate focus shifting from exploring the technology potential to creating Distributed Ledger Technology (DLT)-based systems. In light of a significant number of already existing blockchain applications driven by the Internet of Things (IoT) evolution, the developers are still facing a lack of tools and instruments for appropriate and efficient performance evaluation and behavior observation of different blockchain architectures. This paper aims at providing a systematic review of current blockchain evaluation approaches and at identifying the corresponding utilization challenges and limitations. First, we outline the main metrics related to the blockchain evaluation. Second, we propose the blockchain modeling and analysis classification based on the critical literature review. Third, we extend the review with publicly accessible industrial tools. Next, we analyze the selected results for each of the proposed classes and outline the corresponding limitations. Finally, we identify current challenges of the blockchain analysis from the system evaluation perspective, as well as provide future perspectives.

[1]  Omer Levy,et al.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding , 2018, BlackboxNLP@EMNLP.

[2]  Friedrich Recknagel,et al.  Applications of machine learning to ecological modelling , 2001 .

[3]  D. Woolley The White Paper. , 1972, British medical journal.

[4]  Suporn Pongnumkul,et al.  Performance Analysis of Private Blockchain Platforms in Varying Workloads , 2017, 2017 26th International Conference on Computer Communication and Networks (ICCCN).

[5]  Aleksandr Ometov,et al.  Time-Dependent SIR Analysis in Shopping Malls Using Fractal-Based Mobility Models , 2017, WWIC.

[6]  Yuanyuan Yang,et al.  A Survey of IoT Applications in Blockchain Systems , 2020, ACM Comput. Surv..

[7]  Aleksandr Ometov,et al.  Safe, Secure Executions at the Network Edge: Coordinating Cloud, Edge, and Fog Computing , 2017, IEEE Software.

[8]  Zibin Zheng,et al.  An Overview of Blockchain Technology: Architecture, Consensus, and Future Trends , 2017, 2017 IEEE International Congress on Big Data (BigData Congress).

[9]  Angelika Mueller,et al.  Principles Of Random Walk , 2016 .

[10]  Myron Hlynka,et al.  Queueing Networks and Markov Chains (Modeling and Performance Evaluation With Computer Science Applications) , 2007, Technometrics.

[11]  Aleksandr Ometov,et al.  Blockchain Technology for Smartphones and Constrained IoT Devices: A Future Perspective and Implementation , 2019, 2019 IEEE 21st Conference on Business Informatics (CBI).

[12]  Kartik Nayak,et al.  Stubborn Mining: Generalizing Selfish Mining and Combining with an Eclipse Attack , 2016, 2016 IEEE European Symposium on Security and Privacy (EuroS&P).

[13]  Abhi Shelat,et al.  Analysis of the Blockchain Protocol in Asynchronous Networks , 2017, EUROCRYPT.

[14]  Satoshi Nakamoto Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[15]  Abhi Shelat,et al.  A Better Method to Analyze Blockchain Consistency , 2018, CCS.

[16]  C. Coulson Elements of the Theory of Markov Processes and Their Applications , 1961 .

[17]  Kaiwen Zhang,et al.  VIBES: fast blockchain simulations for large-scale peer-to-peer networks: demo , 2017, Middleware Posters and Demos.

[18]  Cyril Grunspan,et al.  ON PROFITABILITY OF NAKAMOTO DOUBLE SPEND , 2019, Probability in the Engineering and Informational Sciences.

[19]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[20]  Dongfang Zhao,et al.  Toward Accurate and Efficient Emulation of Public Blockchains in the Cloud , 2019, CLOUD.

[21]  Balaji Viswanathan,et al.  Performance Benchmarking and Optimizing Hyperledger Fabric Blockchain Platform , 2018, 2018 IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS).

[22]  Mahdi H. Miraz,et al.  Blockchain Enabled Smart Contract Based Applications: Deficiencies with the Software Development Life Cycle Models , 2020, ArXiv.

[23]  Dimitar Dimitrov,et al.  VerX: Safety Verification of Smart Contracts , 2020, 2020 IEEE Symposium on Security and Privacy (SP).

[24]  Aleksandr Ometov,et al.  An Overview on Blockchain for Smartphones: State-of-the-Art, Consensus, Implementation, Challenges and Future Trends , 2020, IEEE Access.

[25]  Steffen Udluft,et al.  The Markov Decision Process Extraction Network , 2010, ESANN.

[26]  Blockchain in Europe : Closing the Strategy Gap , 2018 .

[27]  Alex Borges Vieira,et al.  Learning Blockchain Delays , 2019, SIGMETRICS Perform. Evaluation Rev..

[28]  Heung-No Lee,et al.  Profitable Double-Spending Attacks , 2019, Applied Sciences.

[29]  Chen Chen,et al.  Using Virtualization for Blockchain Testing , 2017, SmartCom.

[30]  Nishant Rodrigues,et al.  KEVM: A Complete Semantics of the Ethereum Virtual Machine , 2017 .

[31]  Vishanth Weerakkody,et al.  A framework for analysing blockchain technology adoption: Integrating institutional, market and technical factors , 2020, Int. J. Inf. Manag..

[32]  Xing Liu,et al.  Distributed Ledger Technology , 2024, Communications in Computer and Information Science.

[33]  Quan-Lin Li,et al.  Markov processes in blockchain systems , 2019, ArXiv.

[34]  Christopher King,et al.  The fluid limit of a random graph model for a shared ledger , 2019, Advances in Applied Probability.

[35]  J. Banks,et al.  Discrete-Event System Simulation , 1995 .

[36]  Shoji Kasahara,et al.  Effect of Bitcoin fee on transaction-confirmation process , 2016, Journal of Industrial & Management Optimization.

[37]  Trevor Clohessy,et al.  Investigating the influence of organizational factors on blockchain adoption , 2019, Ind. Manag. Data Syst..

[38]  Celso Leandro Palma,et al.  Principles of Modeling and Simulation: A Multidisciplinary Approach , 2009 .

[39]  Jianping Li,et al.  Modeling of Blockchain Based Systems Using Queuing Theory Simulation , 2018, 2018 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP).

[40]  Leandros Tassiulas,et al.  Stochastic Models and Wide-Area Network Measurements for Blockchain Design and Analysis , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[41]  Hubert Ritzdorf,et al.  On the Security and Performance of Proof of Work Blockchains , 2016, IACR Cryptol. ePrint Arch..

[42]  Luca Veltri,et al.  IoTChain: A blockchain security architecture for the Internet of Things , 2018, 2018 IEEE Wireless Communications and Networking Conference (WCNC).

[43]  Beng Chin Ooi,et al.  BLOCKBENCH: A Framework for Analyzing Private Blockchains , 2017, SIGMOD Conference.

[44]  Kevin Fiedler,et al.  Elements of the theory of Markov processes and their applications , 1960 .

[45]  David Metcalf,et al.  The Hyperledger Project , 2017 .

[46]  Aleksandr Ometov,et al.  Applying Blockchain Technology for User Incentivization in mmWave-Based Mesh Networks , 2020, IEEE Access.

[47]  Aggelos Kiayias,et al.  The Bitcoin Backbone Protocol with Chains of Variable Difficulty , 2017, CRYPTO.

[48]  Noureddine Lasla,et al.  Local Bitcoin Network Simulator for Performance Evaluation using Lightweight Virtualization , 2020, 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT).

[49]  Oliver Hinz,et al.  Blockchain , 2020, Bus. Inf. Syst. Eng..

[50]  Yusuke Aoki,et al.  SimBlock: A Blockchain Network Simulator , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[51]  Xing Wang,et al.  A Deep Dive Into Blockchain Selfish Mining , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[52]  Michel Mandjes,et al.  A Bitcoin-inspired infinite-server model with a random fluid limit , 2019, Stochastic Models.

[53]  Aviv Zohar,et al.  Optimal Selfish Mining Strategies in Bitcoin , 2015, Financial Cryptography.

[54]  Alex Groce,et al.  Manticore: A User-Friendly Symbolic Execution Framework for Binaries and Smart Contracts , 2019, 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[55]  P. Goffard Fraud risk assessment within blockchain transactions , 2019, Advances in Applied Probability.

[56]  Debajani Mohanty Frameworks: Truffle and Embark , 2018 .

[57]  Bart Preneel,et al.  Publish or Perish: A Backward-Compatible Defense Against Selfish Mining in Bitcoin , 2017, CT-RSA.

[58]  A.W.G. de Vries Bitcoin's Growing Energy Problem , 2018 .

[59]  Youssef Iraqi,et al.  Blockchain-Based Distributed Trust and Reputation Management Systems: A Survey , 2020, IEEE Access.

[60]  Shoji Kasahara,et al.  Transaction-Confirmation Time for Bitcoin: A Queueing Analytical Approach to Blockchain Mechanism , 2017, QTNA.

[61]  Anita Gehlot,et al.  Adoption of blockchain technology in various realms: Opportunities and challenges , 2020, Secur. Priv..

[62]  Jian-Ping Li,et al.  Simulation Model for Blockchain Systems Using Queuing Theory , 2019, Electronics.

[63]  Peter Bauer,et al.  Challenges and design choices for global weather and climate models based on machine learning , 2018, Geoscientific Model Development.

[64]  Luca Veltri,et al.  A Sidecar Object for the Optimized Communication Between Edge and Cloud in Internet of Things Applications , 2019, Future Internet.

[65]  Ian Welch,et al.  Modelling and Prediction of Resource Utilization of Hadoop Clusters: A Machine Learning Approach , 2019, UCC.

[66]  Shahaboddin Shamshirband,et al.  State of the Art of Machine Learning Models in Energy Systems, a Systematic Review , 2019, Energies.

[67]  Miguel Correia,et al.  BlockSim: Blockchain Simulator , 2019, 2019 IEEE International Conference on Blockchain (Blockchain).

[68]  Quan-Lin Li,et al.  Blockchain Queue Theory , 2018, CSoNet.

[69]  Khaled Salah,et al.  IoT security: Review, blockchain solutions, and open challenges , 2017, Future Gener. Comput. Syst..

[70]  Bodo Rosenhahn,et al.  Markov Chain Neural Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[71]  Ali Sunyaev,et al.  Trade-offs between Distributed Ledger Technology Characteristics , 2019, ACM Comput. Surv..

[72]  Emin Gün Sirer,et al.  Majority Is Not Enough: Bitcoin Mining Is Vulnerable , 2013, Financial Cryptography.

[73]  B. Maurer,et al.  “When perhaps the real problem is money itself!”: the practical materiality of Bitcoin , 2013 .