A Search Based Approach for Stress-Testing Integrated Circuits

In order to reduce software complexity and be power efficient, hardware platforms are increasingly incorporating functionality that was traditionally administered at a software-level (such as cache management). This functionality is often complex, incorporating multiple processors along with a multitude of design parameters. Such devices can only be reliably tested at a ‘system’ level, which presents various testing challenges; behaviour is often non-deterministic (from a software perspective), and finding suitable test sets to ‘stress’ the system adequately is often an inefficient, manual activity that yields fixed test sets that can rarely be reused. In this paper we investigate this problem with respect to ARM’s Cache Coherent Interconnect (CCI) Unit. We present an automated search-based testing approach that combines a parameterised test-generation framework with the hill-climbing heuristic to find test sets that maximally ‘stress’ the CCI by producing much larger numbers of data stall cycles than the corresponding manual test sets.

[1]  Mark Harman,et al.  A theoretical & empirical analysis of evolutionary testing and hill climbing for structural test data generation , 2007, ISSTA '07.

[2]  Myra B. Cohen,et al.  SimLatte: A Framework to Support Testing for Worst-Case Interrupt Latencies in Embedded Software , 2014, 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation.

[3]  Giuliano Antoniol,et al.  Improving network applications security: a new heuristic to generate stress testing data , 2005, GECCO '05.

[4]  Qi Luo,et al.  Automating performance bottleneck detection using search-based application profiling , 2015, ISSTA.

[5]  Ralf Tönjes,et al.  Generating test data for black-box testing using genetic algorithms , 2012, Proceedings of 2012 IEEE 17th International Conference on Emerging Technologies & Factory Automation (ETFA 2012).

[6]  Dimitris Gizopoulos,et al.  Software-Based Self Test Methodology for On-Line Testing of L1 Caches in Multithreaded Multicore Architectures , 2013, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[7]  Prabhat Mishra,et al.  Automated generation of directed tests for transition coverage in cache coherence protocols , 2012, 2012 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[8]  Andreas Windisch,et al.  A Search-Based Approach to Functional Hardware-in-the-Loop Testing , 2010, 2nd International Symposium on Search Based Software Engineering.

[9]  Alessandro Savino,et al.  Software-Based Self-Test of Set-Associative Cache Memories , 2011, IEEE Transactions on Computers.

[10]  Petru Eles,et al.  Automated software testing of memory performance in embedded GPUs , 2014, 2014 International Conference on Embedded Software (EMSOFT).

[11]  Matteo Sonza Reorda,et al.  On the functional test of L2 caches , 2012, 2012 IEEE 18th International On-Line Testing Symposium (IOLTS).

[12]  Vijay Nagarajan,et al.  McVerSi: A test generation framework for fast memory consistency verification in simulation , 2016, 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA).

[13]  Phil McMinn,et al.  Search‐based software test data generation: a survey , 2004, Softw. Test. Verification Reliab..

[14]  Matteo Sonza Reorda,et al.  On the functional test of the cache coherency logic in multi-core systems , 2015, 2015 IEEE 6th Latin American Symposium on Circuits & Systems (LASCAS).

[15]  Lionel C. Briand,et al.  Stress testing real-time systems with genetic algorithms , 2005, GECCO '05.

[16]  Joachim Wegener,et al.  Search-Based Testing with in-the-loop Systems , 2009, 2009 1st International Symposium on Search Based Software Engineering.

[17]  Maria Luisa Villani,et al.  An approach for QoS-aware service composition based on genetic algorithms , 2005, GECCO '05.

[18]  Dimitris Gizopoulos,et al.  Power-aware optimization of software-based self-test for L1 caches in microprocessors , 2014, 2014 IEEE 20th International On-Line Testing Symposium (IOLTS).

[19]  André Baresel,et al.  Structural and Functional Sequence Test of Dynamic and State-Based Software with Evolutionary Algorithms , 2003, GECCO.

[20]  Iain Bate,et al.  WCET Analysis of Modern Processors Using Multi-Criteria Optimisation , 2009, SSBSE 2009.

[21]  Dimitris Gizopoulos,et al.  Software-Based Self-Test for Small Caches in Microprocessors , 2014, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[22]  Stamatis Vassiliadis,et al.  Test Set Development for Cache Memory in Modern Microprocessors , 2008, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[23]  Dimitris Gizopoulos,et al.  A Software-Based Self-Test methodology for on-line testing of data TLBs , 2012, 2012 17th IEEE European Test Symposium (ETS).

[24]  Lionel C. Briand,et al.  Black-Box System Testing of Real-Time Embedded Systems Using Random and Search-Based Testing , 2010, ICTSS.

[25]  Wasif Afzal,et al.  A systematic review of search-based testing for non-functional system properties , 2009, Inf. Softw. Technol..