The Case for Migratory Priority Inheritance in Linux: Bounded Priority Inversions on Multiprocessors

Linux’s real-time performance crucially depends on priority inheritance because—on uniprocessors—it limits the maximum duration of priority inversion to one critical section per lock while ensuring that tasks remain fully preemptable even when holding locks. Unfortunately, priority inheritance is ineffective on multiprocessors under non-global scheduling (i.e., if some tasks may not execute on every processor) in the sense that it does not prevent unbounded priority inversion in all cases. In fact, as shown in the paper, in a multiprocessor system with priority inheritance, it is possible for a task to suffer from priority inversion as long as in a uniprocessor system without priority inheritance. There is thus currently no predictable locking mechanism suitable for non-global scheduling available in Linux, short of resorting to non-preemptive sections or other latency-increasing mechanisms such as priority boosting. As multicore platforms are becoming more widespread in the embedded domain, this “predictability gap” on multiprocessors limits Linux’s viability as a real-time platform. In this position paper, it is argued that a simple tweak to the priority inheritance algorithm will restore its efficacy on multiprocessors, without breaking POSIX compliance, increasing scheduling latency, or requiring large changes to Linux’s scheduling and locking code. In a nutshell, under the proposed migratory priority inheritance mechanism, inheritance is applied not only to scheduling priorities, but also to processor affinity masks. The need for migratory priority inheritance and its benefits are explained in detail with a sequence of simple examples. Additionally, a prototype implementation within the Linux kernel is described and potential challenges and simplifications are discussed.

[1]  Sanjoy K. Baruah,et al.  Schedulability Analysis of Multiprocessor Sporadic Task Systems , 2007, Handbook of Real-Time and Embedded Systems.

[2]  Hiroaki Takada,et al.  Predictable spin lock algorithms with preemption , 1994, Proceedings of 11th IEEE Workshop on Real-Time Operating Systems and Software.

[3]  Cucinotta Tommaso,et al.  An efficient and scalable implementation of global EDF in Linux , 2011 .

[4]  Aloysius Ka-Lau Mok,et al.  Fundamental design problems of distributed systems for the hard-real-time environment , 1983 .

[5]  Claudio Scordino,et al.  An EDF scheduling class for the Linux kernel ∗ , 2009 .

[6]  Giuseppe Lipari,et al.  A bandwidth inheritance algorithm for real-time task synchronization in open systems , 2001, Proceedings 22nd IEEE Real-Time Systems Symposium (RTSS 2001) (Cat. No.01PR1420).

[7]  Ragunathan Rajkumar,et al.  Coordinated Task Scheduling, Allocation and Synchronization on Multiprocessors , 2009, 2009 30th IEEE Real-Time Systems Symposium.

[8]  James H. Anderson,et al.  Real-time resource-sharing under clustered scheduling: mutex, reader-writer, and k-exclusion locks , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[9]  Theodore P. Baker,et al.  Stack-based scheduling of realtime processes , 1991, Real-Time Systems.

[10]  Michael Hohmuth,et al.  Helping in a multiprocessor environment , 2001 .

[11]  Joseph Y.-T. Leung,et al.  On the complexity of fixed-priority scheduling of periodic, real-time tasks , 1982, Perform. Evaluation.

[12]  James H. Anderson,et al.  Efficient object sharing in quantum-based real-time systems , 1998, Proceedings 19th IEEE Real-Time Systems Symposium (Cat. No.98CB36279).

[13]  Lui Sha,et al.  Priority Inheritance Protocols: An Approach to Real-Time Synchronization , 1990, IEEE Trans. Computers.

[14]  Björn Andersson,et al.  Resource Sharing in Global Fixed-Priority Preemptive Multiprocessor Scheduling , 2009, 2009 30th IEEE Real-Time Systems Symposium.

[15]  Vladimir Cretu,et al.  Limited Blocking Resource Sharing for Global Multiprocessor Scheduling , 2011, 2011 23rd Euromicro Conference on Real-Time Systems.

[16]  Hennadiy Leontyev,et al.  A Flexible Real-Time Locking Protocol for Multiprocessors , 2007, 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA 2007).

[17]  James H. Anderson,et al.  The OMLP family of optimal multiprocessor real-time locking protocols , 2013, Des. Autom. Embed. Syst..

[18]  Tommaso Cucinotta,et al.  The Multiprocessor Bandwidth Inheritance Protocol , 2010, 2010 22nd Euromicro Conference on Real-Time Systems.

[19]  Lui Sha,et al.  Real-time synchronization protocols for multiprocessors , 1988, Proceedings. Real-Time Systems Symposium.

[20]  James H. Anderson,et al.  Reader-Writer Synchronization for Shared-Memory Multiprocessor Real-Time Systems , 2009, 2009 21st Euromicro Conference on Real-Time Systems.

[21]  James H. Anderson,et al.  Replica-Request Priority Donation: A Real-Time Progress Mechanism for Global Locking Protocols , 2012, 2012 IEEE International Conference on Embedded and Real-Time Computing Systems and Applications.

[22]  James H. Anderson,et al.  Spin-based reader-writer synchronization for multiprocessor real-time systems , 2010, Real-Time Systems.

[23]  James H. Anderson,et al.  An optimal k-exclusion real-time locking protocol motivated by multi-GPU systems , 2012, Real-Time Systems.

[24]  Sébastien Tixeuil,et al.  Proceedings of the 12th International Conference on Principles of Distributed Systems , 2008 .

[25]  James H. Anderson,et al.  A Hybrid Real-Time Scheduling Approach for Large-Scale Multicore Platforms , 2007, 19th Euromicro Conference on Real-Time Systems (ECRTS'07).

[26]  Giuseppe Lipari,et al.  A comparison of MPCP and MSRP when sharing resources in the Janus multiple-processor on a chip platform , 2003, The 9th IEEE Real-Time and Embedded Technology and Applications Symposium, 2003. Proceedings..

[27]  Hennadiy Leontyev,et al.  Real-Time Synchronization on Multiprocessors: To Block or Not to Block, to Suspend or Spin? , 2008, 2008 IEEE Real-Time and Embedded Technology and Applications Symposium.

[28]  T. J. Watson,et al.  Fuss , Futexes and Furwocks : Fast Userlevel Locking in Linux Hubertus Franke IBM , 2005 .

[29]  James H. Anderson,et al.  A Comparison of the M-PCP, D-PCP, and FMLPon LITMUSRT , 2008, OPODIS.

[30]  James H. Anderson,et al.  Optimality Results for Multiprocessor Real-Time Locking , 2010, 2010 31st IEEE Real-Time Systems Symposium.

[31]  James W. Layland,et al.  Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment , 1989, JACM.

[32]  James H. Anderson,et al.  Supporting Nested Locking in Multiprocessor Real-Time Systems , 2012, 2012 24th Euromicro Conference on Real-Time Systems.

[33]  Hermann Härtig,et al.  Pragmatic Nonblocking Synchronization for Real-Time Systems , 2001, USENIX Annual Technical Conference, General Track.

[34]  Thomas Nolte,et al.  Independently-Developed Real-Time Systems on Multi-cores with Shared Resources , 2011, 2011 23rd Euromicro Conference on Real-Time Systems.

[35]  Hennadiy Leontyev,et al.  Efficient synchronization under global EDF scheduling on multiprocessors , 2006, 18th Euromicro Conference on Real-Time Systems (ECRTS'06).

[36]  Alan Burns,et al.  Applying new scheduling theory to static priority pre-emptive scheduling , 1993, Softw. Eng. J..

[37]  Ken Sakamura,et al.  A novel approach to multiprogrammed multiprocessor synchronization for real-time kernels , 1997, Proceedings Real-Time Systems Symposium.

[38]  R. Rajkumar Real-time synchronization protocols for shared memory multiprocessors , 1990, Proceedings.,10th International Conference on Distributed Computing Systems.

[39]  Mathai Joseph,et al.  Finding Response Times in a Real-Time System , 1986, Comput. J..

[40]  James H. Anderson,et al.  Implementing wait-free objects on priority-based systems , 1997, PODC '97.

[41]  James H. Anderson,et al.  Scheduling and locking in multiprocessor real-time operating systems , 2011 .

[42]  Hiroaki Takada,et al.  Priority inheritance spin locks for multiprocessor real-time systems , 1996, Proceedings Second International Symposium on Parallel Architectures, Algorithms, and Networks (I-SPAN'96).

[43]  Ragunathan Rajkumar,et al.  Synchronization in Real-Time Systems: A Priority Inheritance Approach , 1991 .