From Lone Dwarfs to Giant Superclusters: Rethinking Operating System Abstractions for the Cloud

Unix took a rich smorgasbord of operating system features from its predecessors and pared it down to a small but powerful set of abstractions: files, processes, pipes, and the shell to glue the system together. In the intervening forty years, the common-case computational substrate has evolved from a lone PDP-11 minicomputer to vast clouds of virtualized computational resources. Contemporary distributed systems are being built by adding layer upon layer atop the foundation established by Unix's chosen abstractions. Unfortunately, the resulting mess has lost the "simplicity, elegance, and ease of use" that was a hallmark of the original Unix design [24]. To cope with distribution at astronomic scale, we must take our operating systems back to the drawing board. We are living in a new world, and it is time to be brave.

[1]  Andrew R. Cherenson,et al.  The Sprite network operating system , 1988, Computer.

[2]  Robbert van Renesse,et al.  Amoeba A Distributed Operating System for the 1990 s Sape , 1990 .

[3]  Douglas Crockford,et al.  The application/json Media Type for JavaScript Object Notation (JSON) , 2006, RFC.

[4]  Steven Hand,et al.  New wine in old skins: the case for distributed operating systems in the data center , 2013, APSys.

[5]  Bruce Walker,et al.  The LOCUS distributed operating system , 1983, SOSP '83.

[6]  M. Abadi,et al.  Naiad: a timely dataflow system , 2013, SOSP.

[7]  Adrian Schüpbach,et al.  Your computer is already a distributed system. Why isn't your OS? , 2009, HotOS.

[8]  Bryan Ford,et al.  CPU inheritance scheduling , 1996, OSDI '96.

[9]  Daniel J. Bernstein,et al.  Some thoughts on security after ten years of qmail 1.0 , 2007, CSAW '07.

[10]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.

[11]  Jon Crowcroft,et al.  Unikernels: library operating systems for the cloud , 2013, ASPLOS '13.

[12]  Ken Thompson,et al.  Plan 9 from Bell Labs , 1995 .

[13]  Michael Isard,et al.  DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language , 2008, OSDI.

[14]  Ken Thompson,et al.  The UNIX time-sharing system , 1974, CACM.

[15]  Roberto Ierusalimschy,et al.  Programming in Lua , 2003 .

[16]  Margo I. Seltzer,et al.  Multicore OSes: Looking Forward from 1991, er, 2011 , 2011, HotOS.

[17]  Michael Stonebraker,et al.  The Case for Shared Nothing , 1985, HPTS.

[18]  Noah Evans,et al.  Message passing and scheduling in Osprey , 2013 .

[19]  Margo I. Seltzer,et al.  Hierarchical File Systems Are Dead , 2009, HotOS.

[20]  Martin Burtscher,et al.  JSZap: Compressing JavaScript Code , 2010, WebApps.

[21]  Emin Gün Sirer,et al.  HyperDex: a distributed, searchable key-value store , 2012, SIGCOMM '12.

[22]  Andrew P. Black,et al.  Fine-grained mobility in the Emerald system , 1987, TOCS.

[23]  R. Fielding,et al.  Architectural Styles and the Design of Network-based Software Architectures (CHAPTER 5) , 2000 .

[24]  Carlos A. Varela.  Programming Distributed Computing Systems: A Foundational Approach , 2013 .

[25]  Jon Louis Bentley,et al.  Programming pearls: a literate program , 1986, CACM.

[26]  David E. Culler,et al.  Hierarchical scheduling for diverse datacenter workloads , 2013, SoCC.