Using Lock Servers to Scale Real-Time Locking Protocols: Chasing Ever-Increasing Core Counts

During the past decade, parallelism-related issues have been at the forefront of real-time systems research due to the advent of multicore technologies. In the coming years, such issues will loom ever larger due to increasing core counts. Having more cores means a greater potential exists for platform capacity loss when the available parallelism cannot be fully exploited. In this paper, such capacity loss is considered in the context of real-time locking protocols. In this context, lock nesting becomes a key concern as it can result in transitive blocking chains that force tasks to execute sequentially unnecessarily. Such chains can be quite long on a larger machine. Contention-sensitive real-time locking protocols have been proposed as a means of “breaking” transitive blocking chains, but such protocols tend to have high overhead due to more complicated lock/unlock logic. To ease such overhead, the usage of lock servers is considered herein. In particular, four specific lock-server paradigms are proposed and many nuances concerning their deployment are explored. Experiments are presented that show that, by executing cache hot, lock servers can enable reductions in lock/unlock overhead of up to 86%. Such reductions make contention-sensitive protocols a viable approach in practice. 2012 ACM Subject Classification Computer systems organization → Real-time systems, Computer systems organization → Embedded and cyber-physical systems, Software and its engineering → Mutual exclusion, Software and its engineering → Real-time systems software, Software and its engineering → Synchronization, Software and its engineering → Process synchronization

[1]  Jan Reineke,et al.  CAMA: A Predictable Cache-Aware Memory Allocator , 2011, 2011 23rd Euromicro Conference on Real-Time Systems.

[2]  James H. Anderson,et al.  Cache Sharing and Isolation Tradeoffs in Multicore Mixed-Criticality Systems , 2015, 2015 IEEE Real-Time Systems Symposium.

[3]  Alan Burns,et al.  New schedulability analysis for MrsP , 2017, 2017 IEEE 23rd International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA).

[4]  Tommaso Cucinotta,et al.  Analysis and implementation of the multiprocessor bandwidth inheritance protocol , 2012, Real-Time Systems.

[5]  Michael L. Scott,et al.  Scheduler-conscious synchronization , 1997, TOCS.

[6]  R. Rajkumar Real-time synchronization protocols for shared memory multiprocessors , 1990, Proceedings.,10th International Conference on Distributed Computing Systems.

[7]  Hiroaki Takada,et al.  Real-time scalability of nested spin locks , 1995, Proceedings Second International Workshop on Real-Time Computing Systems and Applications.

[8]  Thomas Nolte,et al.  Independently-Developed Real-Time Systems on Multi-cores with Shared Resources , 2011, 2011 23rd Euromicro Conference on Real-Time Systems.

[9]  Hennadiy Leontyev,et al.  Real-Time Synchronization on Multiprocessors: To Block or Not to Block, to Suspend or Spin? , 2008, 2008 IEEE Real-Time and Embedded Technology and Applications Symposium.

[10]  Tei-Wei Kuo,et al.  Task synchronization and allocation for many-core real-time systems , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[11]  Maurice Herlihy,et al.  Wait-free synchronization , 1991, TOPL.

[12]  Thomas Nolte,et al.  Per Processor Spin-Based Protocols for Multiprocessor Real-Time Systems , 2017, Leibniz Trans. Embed. Syst..

[13]  Yuh-Jzer Joung,et al.  Asynchronous group mutual exclusion , 2000, Distributed Computing.

[14]  Satish K. Tripathi,et al.  Multiprocessor priority ceiling based protocols , 1994 .

[15]  James H. Anderson,et al.  Spin-based reader-writer synchronization for multiprocessor real-time systems , 2010, Real-Time Systems.

[16]  James H. Anderson,et al.  An optimal k-exclusion real-time locking protocol motivated by multi-GPU systems , 2012, Real-Time Systems.

[17]  James H. Anderson,et al.  The OMLP family of optimal multiprocessor real-time locking protocols , 2013, Des. Autom. Embed. Syst..

[18]  Tommaso Cucinotta,et al.  The Multiprocessor Bandwidth Inheritance Protocol , 2010, 2010 22nd Euromicro Conference on Real-Time Systems.

[19]  James H. Anderson,et al.  Real-time multiprocessor locks with nesting: optimizing the common case , 2019, Real-Time Systems.

[20]  Ragunathan Rajkumar,et al.  A Coordinated Approach for Practical OS-Level Cache Management in Multi-core Real-Time Systems , 2013, 2013 25th Euromicro Conference on Real-Time Systems.

[21]  Julia L. Lawall,et al.  Remote Core Locking: Migrating Critical-Section Execution to Improve the Performance of Multithreaded Applications , 2012, USENIX Annual Technical Conference.

[22]  Thomas Nolte,et al.  Towards resource sharing under multiprocessor semi-partitioned scheduling , 2012, 7th IEEE International Symposium on Industrial Embedded Systems (SIES'12).

[23]  Björn Andersson,et al.  Resource Sharing in Global Fixed-Priority Preemptive Multiprocessor Scheduling , 2009, 2009 30th IEEE Real-Time Systems Symposium.

[24]  Bryan C. Ward Sharing Non-Processor Resources in Multiprocessor Real-Time Systems , 2016 .

[25]  Vladimir Cretu,et al.  Limited Blocking Resource Sharing for Global Multiprocessor Scheduling , 2011, 2011 23rd Euromicro Conference on Real-Time Systems.

[26]  Alexander Wieder,et al.  On the Complexity of Worst-Case Blocking Analysis of Nested Critical Sections , 2014, 2014 IEEE Real-Time Systems Symposium.

[27]  Jian-Jia Chen,et al.  Resource-Oriented Partitioned Scheduling in Multiprocessor Systems: How to Partition and How to Share? , 2016, 2016 IEEE Real-Time Systems Symposium (RTSS).

[28]  James H. Anderson,et al.  Fine-grained multiprocessor real-time locking with improved blocking , 2013, RTNS '13.

[29]  Mark Moir,et al.  A Simple Local-Spin Group Mutual Exclusion Algorithm , 2001, IEEE Trans. Parallel Distributed Syst..

[30]  Björn B. Brandenburg,et al.  Feather-Trace : A Light-Weight Event Tracing Toolkit ∗ , 2007 .

[31]  James H. Anderson,et al.  Scheduling and locking in multiprocessor real-time operating systems , 2011 .

[32]  James W. Havender Avoiding Deadlock in Multitasking Systems , 1968, IBM Syst. J..

[33]  Robert I. Davis,et al.  Reducing Queue Lock Pessimism in Multiprocessor Schedulability Analysis , 2010 .

[34]  Ragunathan Rajkumar,et al.  Coordinated Task Scheduling, Allocation and Synchronization on Multiprocessors , 2009, 2009 30th IEEE Real-Time Systems Symposium.

[35]  Lui Sha,et al.  Real-time synchronization protocols for multiprocessors , 1988, Proceedings. Real-Time Systems Symposium.

[36]  Xiaodong Wu,et al.  Multiprocessor Real-Time Systems with Shared Resources: Utilization Bound and Mapping , 2014, IEEE Transactions on Parallel and Distributed Systems.

[37]  Michael L. Scott,et al.  Algorithms for scalable synchronization on shared-memory multiprocessors , 1991, TOCS.

[38]  Thomas Nolte,et al.  Flexible spin-lock model for resource sharing in multiprocessor real-time systems , 2014, Proceedings of the 9th IEEE International Symposium on Industrial Embedded Systems (SIES 2014).

[39]  Alexander Wieder,et al.  Global Real-Time Semaphore Protocols: A Survey, Unified Analysis, and Comparison , 2015, 2015 IEEE Real-Time Systems Symposium.

[40]  Mark Moir,et al.  A simple local-spin group mutual exclusion algorithm , 1999, PODC '99.

[41]  Giuseppe Lipari,et al.  A comparison of MPCP and MSRP when sharing resources in the Janus multiple-processor on a chip platform , 2003, The 9th IEEE Real-Time and Embedded Technology and Applications Symposium, 2003. Proceedings..

[42]  Thomas Nolte,et al.  Partitioning Real-Time Systems on Multiprocessors with Shared Resources , 2010, OPODIS.

[43]  Alessandro Biondi,et al.  Lightweight Real-Time Synchronization under P-EDF on Symmetric and Asymmetric Multiprocessors , 2016, 2016 28th Euromicro Conference on Real-Time Systems (ECRTS).

[44]  Björn Andersson,et al.  Provably good multiprocessor scheduling with resource sharing , 2010, Real-Time Systems.

[45]  Giuseppe Lipari,et al.  Minimizing memory utilization of real-time task sets in single and multi-processor systems-on-a-chip , 2001, Proceedings 22nd IEEE Real-Time Systems Symposium (RTSS 2001) (Cat. No.01PR1420).

[46]  Ragunathan Rajkumar,et al.  Synchronization in Real-Time Systems: A Priority Inheritance Approach , 1991 .

[47]  James H. Anderson,et al.  Supporting Nested Locking in Multiprocessor Real-Time Systems , 2012, 2012 24th Euromicro Conference on Real-Time Systems.

[48]  Insup Lee,et al.  vCAT: Dynamic Cache Management Using CAT Virtualization , 2017, 2017 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS).

[49]  Insup Lee,et al.  Analysis and Implementation of Global Preemptive Fixed-Priority Scheduling with Dynamic Cache Allocation , 2016, 2016 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS).