Operating system support for low-latency streaming

Streaming applications such as voice-over-IP and soft modems running on commodity operating systems (OSs) are becoming common today. These applications are characterized by tight timing constraints that must be satisfied for correct operation. For example, voice-over-IP systems have one-way delay requirements of 150–200 ms where only a third of that time can be spent in the operating system and in the application layers. Similarly, soft modems require periodic execution with low jitter every 12.5 ms from the operating system. To satisfy the timing requirements of these low-latency streaming applications, the operating system (OS) must allow fine-grained scheduling so that applications can be scheduled at precisely the time they require execution. Unfortunately, the metric traditionally used to evaluate current general-purpose OSs focuses on application throughput rather than latency. Consequently, current operating systems use mechanisms such as coarse-grained timers and schedulers, and large packets to amortize operating system “overhead” over long periods of application-level work. This approach has the effect of improving throughput at the expense of latency. Similarly, current OSs have large nonpreemptible sections and use large kernel buffers to improve CPU throughput. All of these mechanisms result in increased latency in the kernel, which conflicts with the timing requirements of low-latency applications because it reduces control over the precise times at which applications can be scheduled. This dissertation shows that general-propose OSs can be designed to support low-latency applications without significantly affecting the performance of traditional throughput-oriented applications. We identify and experimentally evaluate the major sources of latency in current OSs and show that these latencies have three main causes: timing mechanisms, non preemptible kernel sections and output buffering. We propose three techniques, firm timers, fine-grained kernel preemptibility, and adaptive send-buffer tuning for reducing each of these sources of latency. The firm timers mechanism provides accurate timing with low overhead. We use fine-grained kernel preemptibility to obtain a responsive kernel. Adaptive send-buffer tuning helps to minimize buffering delay in the kernel for TCP-based streaming applications. These techniques have been integrated in our extended version of the Linux kernel, which we call Time-Sensitive Linux (TSL). Our evaluation shows that TSL can be used by streaming applications with much tighter timing constraints than standard Linux can support, and that it does not significantly degrade the performance of throughput-oriented applications. Low kernel latency in TSL enables fine-grained, feedback-based scheduling for supporting the needs of low-latency applications. Traditionally, real-time scheduling mechanisms have been used to provide predictable scheduling latency. However, these mechanisms are difficult to use in general-purpose OSs because they require precise specification of application requirements in terms of low-level resources. Such a specification may not be statically available in this environment. Hence, this thesis presents the design, implementation and evaluation of a novel feedback-based real-rate scheduler that automatically infers application requirements and thus makes it easier to use real-time scheduling mechanisms on general-purpose OSs. To be effective, feedback scheduling requires fine-grained resource monitoring and actuation. TSL, unlike conventional OSs, supports both these requirements because it provides precise timing control. Such control has enabled us to implement time-sensitive applications such as gscope, a software oscilloscope, and TSPivo, a software network traffic generator.