The Design and Implementation of Multi-dimensional Bloom Filter Storage Matrix

Bloom filter is a bit array (a one-dimensional storage structure) that provides a compact representation for a set of data, which can be used to answer the membership query in an efficient manner with possible false positives. It has a lot of applications in many areas. In this paper, we further improve Bloom filter by proposing the use of multi-dimensional matrix to replace the one-dimensional structure. Based on our N-dimensional matrix structure, we propose four kinds of filter implementation, namely OFFF, ZFFF, WOFF, FFF (we refer it as Feng Filter). We prove that the false positive rate of our method is lower than the traditional one-dimensional Bloom filter. We also present the detailed implementation of our proposed filter. The traditional Bloom filter can be regarded as a special case of the Feng Filter.

[1]  Udi Manber,et al.  An Algorithm for Approximate Membership checking with Application to Password Security , 1994, Inf. Process. Lett..

[2]  James K. Mullin Estimating the size of a relational join , 1993, Inf. Syst..

[3]  James K. Mullin,et al.  Optimal Semijoins for Distributed Database Systems , 1990, IEEE Trans. Software Eng..

[4]  Margo I. Seltzer,et al.  Self-organization in peer-to-peer systems , 2002, EW 10.

[5]  M. D. McIlroy,et al.  Development of a Spelling List , 1982, IEEE Trans. Commun..

[6]  Lee L. Gremillion Designing a Bloom filter for differential file access , 1982, CACM.

[7]  Andrei Broder,et al.  Network Applications of Bloom Filters: A Survey , 2004, Internet Math..

[8]  George Varghese,et al.  Beyond bloom filters: from approximate membership checks to approximate state machines , 2006, SIGCOMM 2006.

[9]  John Kubiatowicz,et al.  Probabilistic location and routing , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[10]  Richard P. Martin,et al.  PlanetP: using gossiping to build content addressable peer-to-peer information sharing communities , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.