Novel Table Lookup-Based Algorithms for High-Performance CRC Generation

A framework for designing a family of novel fast cyclic redundancy code (CRC) generation algorithms is presented. Our algorithms can ideally read arbitrarily large amounts of data at a time, while optimizing their memory requirement to meet the constraints of specific computer architectures. In addition, our algorithms can be implemented in software using commodity processors instead of specialized parallel circuits. We use this framework to design two efficient algorithms that run in the popular Intel IA32 processor architecture. First, a "slicing-by-4" algorithm doubles the performance of existing software-based table-driven CRC implementations based on the Sarwate [12] algorithm while using a 4-Kbyte cache footprint. Second, a "slicing-by-8" algorithm triples the performance of existing software-based CRC implementations while using an 8-Kbyte cache footprint. Whereas well-known software-based CRC implementations compute the current CRC value from a bitstream reading 8 bits at a time, our algorithms read 32 and 64 bits at a time, respectively (this is an extended version of a paper that appeared at the 10th IEEE International Symposium on Computers and Communications (ISCC '05) in Cartagena, Spain, in June 2005). The slicing-by-8 source code is freely available for experimentation and can be found at http://sourceforge.net/projects/slicing-by-8.