Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus

Wepropose separating the task of reliable transaction dissemination from transaction ordering, to enable high-performance Byzantine fault-tolerant quorum-based consensus. We design and evaluate a mempool protocol, Narwhal, specializing in high-throughput reliable dissemination and storage of causal histories of transactions. Narwhal tolerates an asynchronous network and maintains high performance despite failures. Narwhal is designed to easily scale-out using multiple workers at each validator, and we demonstrate that there is no foreseeable limit to the throughput we can achieve. Composing Narwhal with a partially synchronous consensus protocol (Narwhal-HotStuff) yields significantly better throughput even in the presence of faults or intermittent loss of liveness due to asynchrony. However, loss of liveness can result in higher latency. To achieve overall good performance when faults occur we design Tusk, a zero-message overhead asynchronous consensus protocol, to work with Narwhal. We demonstrate its high performance under a variety of configurations and faults. As a summary of results, on a WAN, Narwhal-Hotstuff achieves over 130,000 tx/sec at less than 2-sec latency compared with 1,800 tx/sec at 1-sec latency for Hotstuff. Additional workers increase throughput linearly to 600,000 tx/sec without any latency increase. Tusk achieves 160,000 tx/sec with about 3 seconds latency. Under faults, both protocols maintain high throughput, but Narwhal-HotStuff suffers from increased latency.

[1]  Bryan Ford,et al.  Enhancing Bitcoin Security and Performance with Strong Consistency via Collective Signing , 2016, USENIX Security Symposium.

[2]  Roger Wattenhofer,et al.  Divide and Scale: Formalization of Distributed Ledger Sharding Protocols , 2019, ArXiv.

[3]  Nancy A. Lynch,et al.  An Efficient Algorithm for Byzantine Agreement without Authentication , 1982, Inf. Control..

[4]  Philipp Jovanovic,et al.  OmniLedger: A Secure, Scale-Out, Decentralized Ledger via Sharding , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[5]  Marko Vukolic,et al.  Hyperledger fabric: a distributed operating system for permissioned blockchains , 2018, EuroSys.

[6]  Aggelos Kiayias,et al.  The Bitcoin Backbone Protocol: Analysis and Applications , 2015, EUROCRYPT.

[7]  Idit Keidar,et al.  All You Need is DAG , 2021, PODC.

[8]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[9]  Alexander Spiegelman,et al.  ACE: Abstract Consensus Encapsulation for Liveness Boosting of State Machine Replication , 2019, OPODIS.

[10]  Maria Gradinariu Potop-Butucaru,et al.  Impact of network delays on Hyperledger Fabric , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[11]  Dahlia Malkhi,et al.  Asynchronous Distributed Key Generation for Computationally-Secure Randomness, Consensus, and Threshold Signatures. , 2020, CCS.

[12]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1983, PODS '83.

[13]  Sam Toueg,et al.  Asynchronous consensus and broadcast protocols , 1985, JACM.

[14]  Michael K. Reiter,et al.  Fault-scalable Byzantine fault-tolerant services , 2005, SOSP '05.

[15]  George Danezis,et al.  Blockmania: from Block DAGs to Consensus , 2018, ArXiv.

[16]  Jianbo Gao,et al.  Performance Analysis of the Libra Blockchain: An Experimental Study , 2019, 2019 2nd International Conference on Hot Information-Centric Networking (HotICN).

[17]  Gavin Andresen,et al.  Graphene: efficient interactive set reconciliation applied to blockchain propagation , 2019, SIGCOMM.

[18]  Sreeram Kannan,et al.  Prism: Deconstructing the Blockchain to Approach Physical Limits , 2019, CCS.

[19]  Ittai Abraham,et al.  HotStuff: BFT Consensus with Linearity and Responsiveness , 2019, PODC.

[20]  Mariana Raykova,et al.  RapidChain: Scaling Blockchain via Full Sharding , 2018, CCS.

[21]  Gabriel Bracha,et al.  Asynchronous Byzantine Agreement Protocols , 1987, Inf. Comput..

[22]  Hovav Shacham,et al.  Short Signatures from the Weil Pairing , 2001, J. Cryptol..

[23]  Ittai Abraham,et al.  Asymptotically Optimal Validated Asynchronous Byzantine Agreement , 2019, PODC.

[24]  John Lane,et al.  Prime: Byzantine Replication under Attack , 2011, IEEE Transactions on Dependable and Secure Computing.

[25]  Miguel Castro,et al.  Practical byzantine fault tolerance and proactive recovery , 2002, TOCS.

[26]  Haibin Zhang,et al.  BEAT: Asynchronous BFT Made Practical , 2018, CCS.

[27]  A. Sonnino,et al.  State Machine Replication in the Libra Blockchain , 2019 .

[28]  Emin Gün Sirer,et al.  Bitcoin-NG: A Scalable Blockchain Protocol , 2015, NSDI.

[29]  Alysson Neves Bessani,et al.  State Machine Replication for the Masses with BFT-SMART , 2014, 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[30]  Elaine Shi,et al.  The Honey Badger of BFT Protocols , 2016, CCS.

[31]  Daniel J. Abadi,et al.  An overview of deterministic database systems , 2018, Commun. ACM.

[32]  C. Stathakopoulou,et al.  Mir-BFT: High-Throughput BFT for Blockchains , 2019, ArXiv.

[33]  Jing Xu,et al.  Dumbo: Faster Asynchronous BFT Protocols , 2020, IACR Cryptol. ePrint Arch..

[34]  Rachid Guerraoui,et al.  Introduction to Reliable and Secure Distributed Programming , 2011 .

[35]  George Danezis,et al.  Chainspace: A Sharded Smart Contracts Platform , 2017, NDSS.

[36]  Bryan Ford,et al.  Threshold Logical Clocks for Asynchronous Distributed Coordination and Consensus , 2019, ArXiv.

[37]  Ethan Buchman,et al.  Tendermint: Byzantine Fault Tolerance in the Age of Blockchains , 2016 .

[38]  Dahlia Malkhi,et al.  Twins: White-Glove Approach for BFT Testing , 2020, ArXiv.

[39]  Xiwei Xu,et al.  On the performance of distributed ledgers for Internet of Things , 2020, Internet Things.