Reputation-based Q-Routing for Robust Inter-committee Routing in the Sharding-based Blockchain

The fault-tolerant consensus problem has been extensively studied in the distributed systems era since the late 1970s. It has regained popularity with the rise of blockchain. However, the scalability and efficiency issues remain to be the challenges in the consensus. One of the solutions is using sharding algorithms to partition networks into several committees. However, previous work does not consider Byzantine problem during the inter-committee routing. In this paper, we propose a Reputation-based Q-routing algorithm using reinforcement learning to assist the communication procedure between different shards. Through the modification of Q-routing protocol, our algorithm can efficiently withstand some of the Byzantine failures. Moreover, to resist routing information tampering, we design a hierarchical structure that uses BFT to validate the routing tables. The experiments show the robustness of routing algorithm brought by reinforcement learning in the verification procedure of sharding-based consensus.

[1]  Elaine Shi,et al.  Thunderella: Blockchains with Optimistic Instant Confirmation , 2018, IACR Cryptol. ePrint Arch..

[2]  Gabriel Bracha,et al.  An O(log n) expected rounds randomized byzantine generals protocol , 1987, JACM.

[3]  Philip C. Treleaven,et al.  Blockchain Technology in Finance , 2017, Computer.

[4]  Baruch Awerbuch,et al.  A quantitative approach to dynamic networks , 1990, PODC '90.

[5]  Yang Xu,et al.  A Blockchain-Based Nonrepudiation Network Computing Service Scheme for Industrial IoT , 2019, IEEE Transactions on Industrial Informatics.

[6]  Raphael Rom,et al.  Failsafe End-to-End Protocols in Computer Networks with Changing Topology , 1987, IEEE Trans. Commun..

[7]  Changsheng Xu,et al.  HAPGN: Hierarchical Attentive Pooling Graph Network for Point Cloud Segmentation , 2021, IEEE Transactions on Multimedia.

[8]  Baojiang Cui,et al.  A Method of Information Protection for Collaborative Deep Learning under GAN Model Attack , 2019, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[9]  Abhi Shelat,et al.  Analysis of the Blockchain Protocol in Asynchronous Networks , 2017, EUROCRYPT.

[10]  Ying-Chang Liang,et al.  Applications of Deep Reinforcement Learning in Communications and Networking: A Survey , 2018, IEEE Communications Surveys & Tutorials.

[11]  Michael L. Littman,et al.  A Distributed Reinforcement Learning Scheme for Network Routing , 1993 .

[12]  K. V. S. Ramarao,et al.  On the diagnosis of Byzantine faults , 1988, Proceedings [1988] Seventh Symposium on Reliable Distributed Systems.

[13]  George Danezis,et al.  Chainspace: A Sharded Smart Contracts Platform , 2017, NDSS.

[14]  Zygmunt J. Haas,et al.  Securing the Internet routing infrastructure , 2002, IEEE Commun. Mag..

[15]  S. Finn.  Resynch Procedures and a Fail-Safe Network Protocol , 1979, IEEE Trans. Commun..

[16]  Neeraj Suri,et al.  On-Line Diagnosis and Recovery: On the Choice and Impact of Tuning Parameters , 2007, IEEE Transactions on Dependable and Secure Computing.

[17]  C. Karlof,et al.  Secure routing in wireless sensor networks: attacks and countermeasures , 2003, Proceedings of the First IEEE International Workshop on Sensor Network Protocols and Applications, 2003..

[18]  Douglas R. Heisterkamp,et al.  Simulated Annealing Based Hierarchical Q-Routing: A Dynamic Routing Protocol , 2011, 2011 Eighth International Conference on Information Technology: New Generations.

[19]  Jim Dowling,et al.  Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[20]  Wei-Pang Yang,et al.  Reaching Fault Diagnosis Agreement under a Hybrid Fault Model , 2000, IEEE Trans. Computers.

[21]  GERNOT METZE,et al.  On the Connection Assignment Problem of Diagnosable Systems , 1967, IEEE Trans. Electron. Comput..

[22]  Baruch Awerbuch,et al.  Reliable broadcast protocols in unreliable networks , 1986, Networks.

[23]  Sreeram Kannan,et al.  PolyShard: Coded Sharding Achieves Linearly Scaling Efficiency and Security Simultaneously , 2018, IEEE Transactions on Information Forensics and Security.

[24]  Aggelos Kiayias,et al.  The Bitcoin Backbone Protocol: Analysis and Applications , 2015, EUROCRYPT.

[25]  Michael L. Littman,et al.  Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach , 1993, NIPS.

[26]  Risto Miikkulainen,et al.  Confidence-based Q-Routing: An on-line adaptive network routing algorithm , 1998 .

[27]  Yehuda Afek,et al.  End-to-end communication in unreliable networks , 1988, PODC '88.

[28]  Kevin I-Kai Wang,et al.  Multi-Modality Behavioral Influence Analysis for Personalized Recommendations in Health Social Media Environment , 2019, IEEE Transactions on Computational Social Systems.

[29]  Prateek Saxena,et al.  A Secure Sharding Protocol For Open Blockchains , 2016, CCS.

[30]  A. El Saddik,et al.  Ant Colony-Based Reinforcement Learning Algorithm for Routing in Wireless Sensor Networks , 2007, 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007.

[31]  Song Han,et al.  SoK: Sharding on Blockchain , 2019, IACR Cryptol. ePrint Arch..

[32]  Frederick Ducatelle,et al.  Adaptive routing in ad hoc wireless multi-hop networks , 2007 .

[33]  Qun Jin,et al.  Academic Influence Aware and Multidimensional Network Analysis for Research Collaboration Navigation Based on Scholarly Big Data , 2021, IEEE Transactions on Emerging Topics in Computing.

[34]  Baruch Awerbuch,et al.  Applying static network protocols to dynamic networks , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[35]  Dit-Yan Yeung,et al.  Predictive Q-Routing: A Memory-based Reinforcement Learning Approach to Adaptive Traffic Control , 1995, NIPS.

[36]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[37]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[38]  Leslie Lamport,et al.  The Byzantine Generals Problem , 1982, TOPL.

[39]  Philipp Jovanovic,et al.  OmniLedger: A Secure, Scale-Out, Decentralized Ledger via Sharding , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[40]  Amir Herzberg,et al.  Blockchain Access Privacy: Challenges and Directions , 2018, IEEE Security & Privacy.

[41]  Miguel Oom Temudo de Castro,et al.  Practical Byzantine fault tolerance , 1999, OSDI '99.

[42]  Shailesh Kumar and Risto Miikkulainen.  Dual Reinforcement Q-Routing: An On-Line Adaptive Routing Algorithm , 1997 .

[43]  Xiaofei Xing,et al.  Trustworthy Network Anomaly Detection Based on an Adaptive Learning Rate and Momentum in IIoT , 2020, IEEE Transactions on Industrial Informatics.

[44]  Emin Gün Sirer,et al.  Scalable and Probabilistic Leaderless BFT Consensus through Metastability , 2019, ArXiv.

[45]  Jiwei Zhang,et al.  Lightweight Attention Pyramid Network for Object Detection and Instance Segmentation , 2020 .

[46]  George Danezis,et al.  Centrally Banked Cryptocurrencies , 2015, NDSS.

[47]  Cheng Zhang,et al.  Blockchain-Enabled Accountability Mechanism Against Information Leakage in Vertical Industry Services , 2020, IEEE Transactions on Network Science and Engineering.

[48]  Iddo Bentov,et al.  Tortoise and Hares Consensus: the Meshcash Framework for Incentive-Compatible, Scalable Cryptocurrencies , 2017, IACR Cryptol. ePrint Arch..

[49]  Cheng Zhang,et al.  Blockchain Empowered Arbitrable Data Auditing Scheme for Network Storage as a Service , 2020, IEEE Transactions on Services Computing.

[50]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[51]  Satoshi Nakamoto.  Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[52]  Risto Miikkulainen,et al.  Confidence Based Dual Reinforcement Q-Routing: An adaptive online network routing algorithm , 1999, IJCAI.

[53]  Albert Cabellos-Aparicio,et al.  A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization , 2017, ArXiv.

[54]  Aggelos Kiayias,et al.  SoK: Communication Across Distributed Ledgers , 2019, IACR Cryptol. ePrint Arch..

[55]  K. V. S. Ramarao,et al.  Distributed diagnosis of Byzantine processors and links , 1989, [1989] Proceedings. The 9th International Conference on Distributed Computing Systems.

[56]  Mariana Raykova,et al.  RapidChain: Scaling Blockchain via Full Sharding , 2018, CCS.

[57]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.