Lockless hash tables with low false negatives

Hash tables are efficient storage data structures widely used in many types of high-performance computer-related problems. In their design, optimal trade-offs must be made to accommodate for the specific characteristics of the application. In this paper we present lock-free low-false-negative (LFN) tables, a family of hash tables designed to address one such type of trade-off. LFN tables sacrifice a low probability of false negatives and a very low (or negligible) probability of false positives to achieve higher performance access time in concurrent shared memory applications. LFM tables are structurally biased towards false negatives and therefore are more suitable for applications that tolerate better false negatives than positives. In this paper we provide a mathematical analysis of their performance and provide use cases where they can be deployed to eliminate shared memory access bottlenecks in the context of very high-speed computer networks.

[1]  Patrick Crowley,et al.  Segmented hash , 2005, 2005 Symposium on Architectures for Networking and Communications Systems (ANCS).

[2]  Nir Shavit,et al.  Split-ordered lists: lock-free extensible hash tables , 2003, PODC '03.

[3]  A Saritha,et al.  A system for detecting network intruders in real-time , 2016 .

[4]  Andrei Z. Broder,et al.  Using multiple hash functions to improve IP lookups , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[5]  Haoyu Song,et al.  Fast hash table lookup using extended bloom filter: an aid to network processing , 2005, SIGCOMM '05.

[6]  Eli Upfal,et al.  Balanced Allocations , 1999, SIAM J. Comput..

[7]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[8]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[9]  Berthold Vöcking,et al.  How asymmetry helps load balancing , 1999, JACM.

[10]  Jordi Ros-Giralt,et al.  Overcoming performance collapse for 100Gbps cyber security , 2013, CLHS '13.

[11]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[12]  V. Srinivasan,et al.  Fast address lookups using controlled prefix expansion , 1999, TOCS.

[13]  Timothy L. Harris,et al.  A Pragmatic Implementation of Non-blocking Linked-Lists , 2001, DISC.

[14]  Maged M. Michael,et al.  High performance dynamic lock-free hash tables and list-based sets , 2002, SPAA '02.