Operating system support for application-specific speculation

Speculative execution is a technique that allows serial tasks to execute in parallel. An implementation of speculative execution can be divided into two parts: (1) a policy that specifies what operations and values to predict, what actions to allow during speculation, and how to compare results; and (2) the mechanisms that support speculative execution, such as checkpointing, rollback, causality tracking, and output buffering. In this paper, we show how to separate policy from mechanism. We implement a speculation mechanism in the operating system, where it can coordinate speculations across all applications and kernel state. Policy decisions are delegated to applications, which have the most semantic information available to direct speculation. We demonstrate how custom policies can be used in existing applications to add new features that would otherwise be difficult to implement. Using custom policies in our separated speculation system, we can hide 85% of program load time by predicting the program's launch, decrease SSL connection latency by 15% in Firefox, and increase a BFT client's request rate by 82%. Despite the complexity of the applications, small modifications can implement these features since they only specify policy choices and rely on the system to realize those policies. We provide this increased programmability with a modest performance trade-off, executing only 8% slower than an optimized, application-implemented speculation system.