TMBF: Bloom filter algorithms of time-dependent multi bit-strings for incremental set

Set is widely used as a kind of basic data structure. However, when it is used for large scale data set the cost of storage, search and transport is overhead. The bloom filter uses a fixed size bit string to represent elements in a static set, which can reduce storage space and search cost that is a fixed constant. The time-space efficiency is achieved at the cost of a small probability of false positive in membership query. However, for many applications the space savings and locating time constantly outweigh this drawback. Dynamic bloom filter (DBF) can support concisely representation and approximate membership queries of dynamic set instead of static set. It has been proved that DBF not only possess the advantage of standard bloom filter, but also has better features when dealing with dynamic set. This paper proposes a time-dependent multiple bit-strings bloom filter (TMBF) which roots in the DBF and targets on dynamic incremental set. TMBF uses multiple bit-strings in time order to present a dynamic increasing set and uses backward searching to test whether an element is in a set. Based on the system logs from a real P2P file sharing system, the evaluation shows a 20% reduction in searching cost compared to DBF.

[1]  Edith Cohen,et al.  Search and replication in unstructured peer-to-peer networks , 2002 .

[2]  Mingzhong Xiao,et al.  i-DBF: an Improved Bloom Filter Representation Method on Dynamic Set , 2006, 2006 Fifth International Conference on Grid and Cooperative Computing Workshops.

[3]  Yafei Dai,et al.  MBF: a Real Matrix Bloom Filter Representation Method on Dynamic Set , 2007, 2007 IFIP International Conference on Network and Parallel Computing Workshops (NPC 2007).

[4]  Edith Cohen,et al.  Search and replication in unstructured peer-to-peer networks , 2002, ICS '02.

[5]  Michael Mitzenmacher,et al.  Compressed bloom filters , 2002, TNET.

[6]  Li Fan,et al.  Summary cache: a scalable wide-area web cache sharing protocol , 2000, TNET.

[7]  Jie Wu,et al.  Theory and Network Applications of Dynamic Bloom Filters , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[8]  Xiao Ming Split Bloom Filter , 2004 .

[9]  Yossi Matias,et al.  Spectral bloom filters , 2003, SIGMOD '03.

[10]  Fang Hao,et al.  Incremental Bloom Filters , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[11]  Abhishek Kumar,et al.  Space-code bloom filter for efficient per-flow traffic measurement , 2004, IEEE INFOCOM 2004.

[12]  A. Kumar,et al.  Space-code bloom filter for efficient per-flow traffic measurement , 2004, IEEE INFOCOM 2004.

[13]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.