Exploring Prefetching, Pre-Execution and Branch Outcome Streaming for In-Memory Database Lookups

Lookup operations for in-memory databases are heavily memory-bound because they often rely on pointer-chasing linked data structure traversals. They are also branch heavy with branches that are hard-to-predict due to random key lookups. In this study, we show that although cache misses are the primary bottleneck for these applications, without a method for eliminating the branch mispredictions, only a small fraction of the performance benefit is achieved through prefetching. We propose the Node Tracker (NT), a novel programmable prefetcher/pre-execution unit that is highly effective in exploiting inter key-lookup parallelism to improve single-thread performance. We extend the NT with branch outcome streaming (BOS) to reduce branch mispredictions and show that NT with BOS can achieve an extra 3x speedup. Finally, we evaluated the NT as a pre-execution unit and show that we can further improve the performance in both single- and multi-threaded execution modes.