Dynamic software randomisation: Lessons learnec from an aerospace case study

Timing Validation and Verification (V&V) is an important step in real-time system design, in which a system's timing behaviour is assessed via Worst Case Execution Time (WCET) estimation and scheduling analysis. For WCET estimation, measurement-based timing analysis (MBTA) techniques are widely-used and well-established in industrial environments. However, the advent of complex processors makes it more difficult for the user to provide evidence that the software is tested under stress conditions representative of those at system operation. Measurement-Based Probabilistic Timing Analysis (MBPTA) is a variant of MBTA followed by the PROXIMA European Project that facilitates formulating this representativeness argument. MBPTA requires certain properties to be applicable, which can be obtained by selectively injecting randomisation in platform's timing behaviour via hardware or software means. In this paper, we assess the effectiveness of the PROXIMA's dynamic software randomisation (DSR) with a space industrial case study executed on a real unmodified hardware platform and an industrial operating system. We present the challenges faced in its development, in order to achieve MBPTA compliance and the lessons learned from this process. Our results, obtained using a commercial timing analysis tool, indicate that DSR does not impact the average performance of the application, while it enables the use of MBPTA. This results in tighter pWCET estimates compared to current industrial practice.

[1]  Tullio Vardanega,et al.  A rapid cache-aware procedure positioning optimization to favor incremental development , 2013, 2013 IEEE 19th Real-Time and Embedded Technology and Applications Symposium (RTAS).

[2]  Francisco J. Cazorla,et al.  Probabilistic timing analysis on conventional cache designs , 2013, 2013 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[3]  Tullio Vardanega,et al.  Measurement-Based Timing Analysis of the AURIX Caches , 2016, WCET.

[4]  Tullio Vardanega,et al.  EPC: Extended Path Coverage for Measurement-Based Probabilistic Timing Analysis , 2015, 2015 IEEE Real-Time Systems Symposium.

[5]  Liliana Cucu-Grosjean,et al.  Timing analysis of an avionics case study on complex hardware/software platforms , 2015, 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[6]  Tullio Vardanega,et al.  Measurement-Based Probabilistic Timing Analysis and Its Impact on Processor Architecture , 2014, 2014 17th Euromicro Conference on Digital System Design.

[7]  Francisco J. Cazorla,et al.  TASA: Toolchain-Agnostic Static Software randomisation for critical real-time systems , 2016, 2016 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[8]  Francisco J. Cazorla,et al.  On the Comparison of Deterministic and Probabilistic WCET Estimation Techniques , 2014, 2014 26th Euromicro Conference on Real-Time Systems.

[9]  Tullio Vardanega,et al.  A Time-composable Operating System , 2012, WCET.

[10]  Kathryn S. McKinley,et al.  Building HighPerformance Custom and GeneralPurpose Memory Allocators , 2001 .

[11]  Emery D. Berger,et al.  DieHard: probabilistic memory safety for unsafe languages , 2006, PLDI '06.

[12]  S. Nadarajah,et al.  Extreme Value Distributions: Theory and Applications , 2000 .

[13]  Emery D. Berger,et al.  STABILIZER: statistically sound performance evaluation , 2013, ASPLOS '13.

[14]  Liliana Cucu-Grosjean,et al.  PROXIMA: Improving Measurement-Based Timing Analysis through Randomisation and Probabilistic Analysis , 2016, 2016 Euromicro Conference on Digital System Design (DSD).

[15]  Jakob Engblom,et al.  The worst-case execution-time problem—overview of methods and survey of tools , 2008, TECS.

[16]  G. Box,et al.  Distribution of Residual Autocorrelations in Autoregressive-Integrated Moving Average Time Series Models , 1970 .

[17]  Kathryn S. McKinley,et al.  Composing high-performance memory allocators , 2001, PLDI '01.

[18]  Jérôme Morio,et al.  On the Sustainability of the Extreme Value Theory for WCET Estimation , 2014, WCET.

[19]  Liliana Cucu-Grosjean,et al.  Measurement-Based Probabilistic Timing Analysis for Multi-path Programs , 2012, 2012 24th Euromicro Conference on Real-Time Systems.

[20]  Irune Agirre,et al.  IEC-61508 SIL 3 Compliant Pseudo-Random Number Generators for Probabilistic Timing Analysis , 2015, 2015 Euromicro Conference on Digital System Design.

[21]  Liliana Cucu-Grosjean,et al.  Measurement-based probabilistic timing analysis: Lessons from an integrated-modular avionics case study , 2013, 2013 8th IEEE International Symposium on Industrial Embedded Systems (SIES).

[22]  Francisco J. Cazorla,et al.  Containing timing-related certification cost in automotive systems deploying complex hardware , 2014, 2014 51st ACM/EDAC/IEEE Design Automation Conference (DAC).

[23]  Francisco J. Cazorla,et al.  Multi-level Unified Caches for Probabilistically Time Analysable Real-Time Systems , 2013, 2013 IEEE 34th Real-Time Systems Symposium.

[24]  Mikel Azkarate-askasua,et al.  WCET analysis methods: Pitfalls and challenges on their trustworthiness , 2015, 10th IEEE International Symposium on Industrial Embedded Systems (SIES).

[25]  Tullio Vardanega,et al.  Upper-bounding Program Execution Time with Extreme Value Theory , 2013, WCET.

[26]  Paul A. Watters,et al.  Statistics in a nutshell , 2008 .

[27]  G. Marsaglia,et al.  A New Class of Random Number Generators , 1991 .