Hardware Accelerator for Full-Text Search (HAFTS) with Succinct Data Structure

Efficient utilization of massive data, such as full-text search has become important in view of the growing needs for Web search and genome analysis. High-speed search and small storage space are required to handle massive amounts of data. For high-speed search, generally, a data structure such as index which needs additional storage space is required. Recently, compressed suffix array, which is a data structure with an indexable dictionary that can be used to compress data to its information-theoretic lower bound, has been proposed. The distinctive feature of this array is that it enables direct data retrieval without decompression from the compressed data. Further, theoretically, the computational complexity of data retrieval is the same for both compressed and uncompressed data when we assume that rank operation involving the bit vector can be executed in constant time; this rank operation returns the number of occurrences of smaller elements. Practically, rank operation involves many bit-manipulations and random access to the memory. Hence, this constant time is not negligible, and as a result, data retrieval using compressed suffix array is relatively slower than that using plain suffix array. Although compression to create an indexable dictionary is performed only once, data retrieval queries occur repeatedly. Hence, high speed rank operations involving bit vectors are essential for compressed suffix arrays. We propose a FPGA-based hardware accelerator for full-text search (HAFTS) with compressed suffix array. FPGA helps speedup rank operation for compressed suffix array by enabling many bit calculations performed simultaneously and controlling the order of memory accesses. We conduct performance simulations of HAFTS. We consider a development board on which FPGA is connected to DDR2-800 SDRAM by a 64-bit bus as our model. We evaluate the performance of HAFTS by comparing it with that of software implementation. As a result, we conclude that the search speed of FPGA-based HAFTS is seven times faster than that of software implementation.