Revisiting actor programming in C++

The actor model of computation has gained significant popularity over the last decade. Its high level of abstraction makes it appealing for concurrent applications in parallel and distributed systems. However, designing a real-world actor framework that subsumes full scalability, strong reliability, and high resource efficiency requires many conceptual and algorithmic additives to the original model.In this paper, we report on designing and building CAF, the C++ Actor Framework. CAF targets at providing a concurrent and distributed native environment for scaling up to very large, high-performance applications, and equally well down to small constrained systems. We present the key specifications and design concepts-in particular a message-transparent architecture, type-safe message interfaces, and pattern matching facilities-that make native actors a viable approach for many robust, elastic, and highly distributed developments. We demonstrate the feasibility of CAF in three scenarios: first for elastic, upscaling environments, second for including heterogeneous hardware like GPUs, and third for distributed runtime systems. Extensive performance evaluations indicate ideal runtime at very low memory footprint for up to 64 CPU cores, or when offloading work to a GPU. In these tests, CAF continuously outperforms the competing actor environments Erlang, Charm++, SalsaLite, Scala, ActorFoundry, and even the raw message passing framework OpenMPI. HighlightsWe analyze the problem and design space for actor programming in C++.We find that type-safe message passing interfaces are important for the robustness of actor programs.Pattern Matching as a DSL in C++ eases the definition of message handlers.We introduce an according framework for actor programming in C++ (CAF).Core algorithms and the scalable architecture are thoroughly discussed.Benchmarking CAF against many other frameworks reveals overall performance benefits.

[1]  Thomas C. Schmidt,et al.  Native actors: how to scale network forensics , 2015, SIGCOMM 2015.

[2]  Scott Hauck,et al.  Reconfigurable computing: a survey of systems and software , 2002, CSUR.

[3]  Vivek Sarkar,et al.  Savina - An Actor Benchmark Suite: Enabling Empirical Evaluation of Actor Libraries , 2014, AGERE!@SPLASH.

[4]  Yi Guo,et al.  Work-first and help-first scheduling policies for async-finish task parallelism , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[5]  Vivek Sarkar,et al.  Deadlock-free scheduling of X10 computations with bounded resources , 2007, SPAA '07.

[6]  Carolyn L. Talcott,et al.  Towards a Theory of Actor Computation , 1992, CONCUR.

[7]  Bjarne Stroustrup,et al.  Open pattern matching for C , 2014 .

[8]  Maurice Herlihy,et al.  Transactional Memory: Architectural Support For Lock-free Data Structures , 1993, Proceedings of the 20th Annual International Symposium on Computer Architecture.

[9]  Jim Waldo Remote procedure calls and Java Remote Method Invocation , 1998, IEEE Concurr..

[10]  Martin Odersky,et al.  Event-Based Programming Without Inversion of Control , 2006, JMLC.

[11]  Allan Porterfield,et al.  OpenMP task scheduling strategies for multicore NUMA systems , 2012, Int. J. High Perform. Comput. Appl..

[12]  Martin Odersky,et al.  Scala Actors: Unifying thread-based and event-based programming , 2009, Theor. Comput. Sci..

[13]  Gul A. Agha,et al.  Actor frameworks for the JVM platform: a comparative analysis , 2009, PPPJ '09.

[14]  Alan Mycroft,et al.  Kilim: Isolation-Typed Actors for Java , 2008, ECOOP.

[15]  Doug Lea,et al.  A Java fork/join framework , 2000, JAVA '00.

[16]  Carlos A. Varela,et al.  Programming dynamically reconfigurable open systems with SALSA , 2001, SIGP.

[17]  Carl Hewitt,et al.  A Universal Modular ACTOR Formalism for Artificial Intelligence , 1973, IJCAI.

[18]  James R. Larus,et al.  Software and the Concurrency Revolution , 2005, ACM Queue.

[19]  Nir Shavit,et al.  Software transactional memory , 1995, PODC '95.

[20]  Bradley C. Kuszmaul,et al.  Cilk: an efficient multithreaded runtime system , 1995, PPOPP '95.

[21]  Vivek Sarkar,et al.  Integrating task parallelism with actors , 2012, OOPSLA '12.

[22]  Maurice Herlihy,et al.  Wait-free synchronization , 1991, TOPL.

[23]  Roy T. Fielding,et al.  Principled design of the modern Web architecture , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[24]  Ralph E. Johnson,et al.  Why Do Scala Developers Mix the Actor Model with other Concurrency Models? , 2013, ECOOP.

[25]  Dean M. Tullsen,et al.  Interconnections in multi-core architectures: understanding mechanisms, overheads and scaling , 2005, 32nd International Symposium on Computer Architecture (ISCA'05).

[26]  Robert D. Blumofe,et al.  Scheduling multithreaded computations by work stealing , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[27]  Charles E. Leiserson,et al.  The Cilk++ concurrency platform , 2009, 2009 46th ACM/IEEE Design Automation Conference.

[28]  Dennis Gannon,et al.  Proceedings of the ACM 2000 conference on Java Grande , 2000 .

[29]  M.M. Lehman,et al.  Programs, life cycles, and laws of software evolution , 1980, Proceedings of the IEEE.

[30]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[31]  Thomas C. Schmidt,et al.  Native actors: a scalable software platform for distributed, heterogeneous environments , 2013, AGERE! 2013.

[32]  Philipp Haller,et al.  On the integration of the actor model in mainstream technologies: the scala perspective , 2012, AGERE! 2012.

[33]  D. Bhatia,et al.  Reconfigurable computing , 1997, Proceedings Tenth International Conference on VLSI Design.

[34]  Thomas C. Schmidt,et al.  Embedded Actors - Towards distributed programming in the IoT , 2014, 2014 IEEE Fourth International Conference on Consumer Electronics Berlin (ICCE-Berlin).

[35]  Simon Josefsson,et al.  The Base16, Base32, and Base64 Data Encodings , 2003, RFC.

[36]  David D. McDonald,et al.  Programs , 1984, CL.

[37]  Carlos A. Varela,et al.  SALSA Lite: A Hash-Based Actor Runtime for Efficient Local Concurrency , 2014, Concurrent Objects and Beyond.

[38]  Joe Armstrong,et al.  Making reliable distributed systems in the presence of software errors , 2003 .

[39]  Thomas C. Schmidt,et al.  CAF - the C++ Actor Framework for Scalable and Resource-Efficient Applications , 2014, AGERE! '14.

[40]  C. Greg Plaxton,et al.  Thread Scheduling for Multiprogrammed Multiprocessors , 1998, SPAA.

[41]  Johannes Singler,et al.  Parallelization of Bulk Operations for STL Dictionaries , 2007, Euro-Par Workshops.

[42]  Lars-Åke Fredlund,et al.  McErlang: a model checker for a distributed functional programming language , 2007, ICFP '07.

[43]  Scott Meyers,et al.  C++ and the Perils of Double-Checked Locking ∗ , 2004 .

[44]  Maurice Herlihy,et al.  Contention in shared memory algorithms , 1997, J. ACM.

[45]  Thomas C. Schmidt,et al.  RIOT OS: Towards an OS for the Internet of Things , 2013, 2013 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[46]  Todd L. Veldhuizen,et al.  Expression templates , 1996 .

[47]  Éric Tanter,et al.  Parallel actor monitors: Disentangling task-level parallelism from data partitioning in the actor model , 2014, Sci. Comput. Program..

[48]  Maurice Herlihy,et al.  Transactional Memory: Architectural Support For Lock-free Data Structures , 1993, Proceedings of the 20th Annual International Symposium on Computer Architecture.

[49]  Kevin Skadron,et al.  Scalable parallel programming , 2008, 2008 IEEE Hot Chips 20 Symposium (HCS).

[50]  Laxmikant V. Kalé,et al.  Work stealing and persistence-based load balancers for iterative overdecomposed applications , 2012, HPDC '12.

[51]  Edward A. Lee The problem with threads , 2006, Computer.

[52]  Bjarne Stroustrup,et al.  Open pattern matching for C++ , 2013, GPCE '13.

[53]  Mohammad Mahdi Jaghoori,et al.  Ten Years of Analyzing Actors: Rebeca Experience , 2011, Formal Modeling: Actors, Open Systems, Biological Systems.

[54]  Gul A. Agha,et al.  Concurrent object-oriented programming , 1993, CACM.

[55]  Thomas C. Schmidt,et al.  Manyfold actors: extending the C++ actor framework to heterogeneous many-core machines using OpenCL , 2015, AGERE!@SPLASH.

[56]  James Reinders,et al.  Intel threading building blocks - outfitting C++ for multi-core processor parallelism , 2007 .

[57]  John E. Stone,et al.  OpenCL: A Parallel Programming Standard for Heterogeneous Computing Systems , 2010, Computing in Science & Engineering.

[58]  Josep Torrellas,et al.  False Sharing ans Spatial Locality in Multiprocessor Caches , 1994, IEEE Trans. Computers.

[59]  Konstantinos Sagonas,et al.  Detecting Software Defects in Telecom Applications Through Lightweight Static Analysis: A War Story , 2004, APLAS.

[60]  Gul A. Agha,et al.  ACTORS - a model of concurrent computation in distributed systems , 1985, MIT Press series in artificial intelligence.

[61]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[62]  Brian Jeff Advances in big.LITTLE Technology for Power and Energy Savings Improving Energy Efficiency in High-Performance Mobile Platforms , 2012 .