An FPGA-based Search Engine for Unstructured Database

We present an FPGA-based search engine that implements the Smith-Waterman local sequence alignment algorithm to search a stream of input data for patterns up to 38 bytes long. This engine supports the sequence gap, match, and mismatch weightings necessary for genome sequence alignment, and it can perform inexact string matching on generic text. As the engine simply processes input data streaming through it, it can readily operate on data that are unindexed and unsorted, i.e., on unstructured databases. Furthermore, the engine can sustain a search throughput of 100 MB/sec , making it capable of processing input data at the maximum sustained throughput of the hard drive (or array of drives) storing the data. For a performance demonstration we compare the throughput of the engine embedded in a prototype system with the execution times of a direct software implementation of the search engine’s kernel. Although the prototype system limits the input data to 40.5MB/sec, we show how parallelism and pipelining allow this FPGA-based search engine to sustain a search throughput of 100 MB/sec, given a fast enough method for delivering input data, and thereby yield a performance gain over the software implementation of two orders of magnitude.