Model-Driven System-Performance Engineering for Cyber-Physical Systems : Industry Session Paper

System-Performance Engineering (SysPE) encompasses modeling formalisms, methods, techniques, and industrial practices to design systems for performance, where performance is taken integrally into account during the whole system life cycle. Industrial SysPE state of practice is generally model-based. Due to the rapidly increasing complexity of systems, there is a need to develop and establish model-driven methods and techniques. To structure the field of SysPE, we identify (1) industrial challenges motivating the importance of SysPE, (2) scientific challenges that need to be addressed to establish model-driven SysPE, (3) important focus areas for SysPE and (4) best practices. We conducted a survey to collect feedback on our views. The responses were used to update and validate the identified challenges, focus areas, and best practices. The final result is presented in this paper. Interesting observations are that industry sees a need for better design-space exploration support, more than for additional performance modeling and analysis techniques. Also tools and integral methods for SysPE need attention. From the identified focus areas, scheduling and supervisory control is seen as lacking established best practices.

[1]  C.J.H. Mann,et al.  A Practical Guide to SysML: The Systems Modeling Language , 2009 .

[2]  Didier Dubois,et al.  Knowledge-Driven versus Data-Driven Logics , 2000, J. Log. Lang. Inf..

[3]  Piotr Dziurzanski,et al.  A Survey and Comparative Study of Hard and So Real-time Dynamic Resource Allocation Strategies for Multi / Many-core Systems , 2017 .

[4]  Rahul Rai,et al.  Driven by Data or Derived Through Physics? A Review of Hybrid Physics Guided Machine Learning Techniques With Cyber-Physical System (CPS) Focus , 2020, IEEE Access.

[5]  Robert I. Davis,et al.  An Empirical Survey-based Study into Industry Practice in Real-time Systems , 2020, 2020 IEEE Real-Time Systems Symposium (RTSS).

[6]  Lothar Thiele,et al.  Y-chart based system design: a discussion on approaches , 2009 .

[7]  J. S. Khamba,et al.  Total productive maintenance: literature review and directions , 2008 .

[8]  H. Vangheluwe,et al.  Foundations of Multi-Paradigm Modelling for Cyber-Physical Systems , 2020 .

[9]  Twan Basten,et al.  Interface Modeling for Quality and Resource Management , 2020, ArXiv.

[10]  Andy D. Pimentel,et al.  Exploring Exploration: A Tutorial Introduction to Embedded Systems Design Space Exploration , 2017, IEEE Design & Test.

[11]  Rong Su,et al.  The Synthesis of Time Optimal Supervisors by Using Heaps-of-Pieces , 2012, IEEE Transactions on Automatic Control.

[12]  Manuel Wimmer,et al.  Towards a Model-Integrated Runtime Monitoring Infrastructure for Cyber-Physical Systems , 2021, 2021 IEEE/ACM 43rd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER).

[13]  Jeroen Voeten,et al.  Calibration Error Bound Estimation in Performance Modeling , 2013, 2013 Euromicro Conference on Digital System Design.

[14]  Jiming Chen,et al.  Privacy and performance trade-off in cyber-physical systems , 2016, IEEE Network.

[15]  Aruna Raja,et al.  Domain Specific Languages , 2010 .

[16]  Stephan Merz,et al.  Model Checking , 2000 .

[17]  Nan Yang,et al.  Improving Model Inference in Industry by Combining Active and Passive Learning , 2019, 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[18]  Martin Wirsing,et al.  On methods and tools for rigorous system design , 2021, Int. J. Softw. Tools Technol. Transf..

[19]  Malgorzata Sterna,et al.  Handbook on Scheduling , 2007 .

[20]  Andy D. Pimentel,et al.  Software Passports for Automated Performance Anomaly Detection of Cyber-Physical Systems , 2019, SAMOS.

[21]  Edward A. Lee,et al.  Modeling Cyber–Physical Systems , 2012, Proceedings of the IEEE.

[22]  Twan Basten,et al.  A blueprint for system-level performance modeling of software-intensive embedded systems , 2016, International Journal on Software Tools for Technology Transfer.

[23]  Wayne Wolf High-Performance Embedded Computing: Architectures, Applications, and Methodologies , 2006 .

[24]  Gerrit Muller,et al.  The Concept of Reference Architectures , 2010, Syst. Eng..

[25]  B. De Schutter,et al.  Max-plus algebra and max-plus linear discrete event systems: An introduction , 2008, 2008 9th International Workshop on Discrete Event Systems.

[26]  He Zhang,et al.  Digital Twin in Industry: State-of-the-Art , 2019, IEEE Transactions on Industrial Informatics.

[27]  George S. Fishman,et al.  Discrete-Event Simulation : Modeling, Programming, and Analysis , 2001 .

[28]  Twan Basten,et al.  Performance Engineering for Industrial Embedded Data-Processing Systems , 2015, PROFES.

[29]  Ed F. Deprettere,et al.  An Approach for Quantitative Analysis of Application-Specific Dataflow Architectures , 1997, ASAP.

[30]  Matthias Gries,et al.  Methods for evaluating and covering the design space during early design development , 2004, Integr..

[31]  Alberto L. Sangiovanni-Vincentelli,et al.  Benefits and challenges for platform-based design , 2004, Proceedings. 41st Design Automation Conference, 2004..

[32]  Twan Basten,et al.  Compositional specification of functionality and timing of manufacturing systems , 2016, 2016 Forum on Specification and Design Languages (FDL).

[33]  Connie U. Smith,et al.  The Evolution of Software Performance Engineering: A Survey , 1986, FJCC.

[34]  Twan Basten,et al.  Checking Metric Temporal Logic with TRACE , 2016, 2016 16th International Conference on Application of Concurrency to System Design (ACSD).

[35]  Gregory S. Parnell,et al.  Set‐based design: The state‐of‐practice and research opportunities , 2020, Syst. Eng..

[36]  Marc Geilen,et al.  Software/Hardware Engineering with the Parallel Object-Oriented Specification Language , 2007, 2007 5th IEEE/ACM International Conference on Formal Methods and Models for Codesign (MEMOCODE 2007).

[37]  Fei Tao,et al.  Digital twin-driven product design, manufacturing and service with big data , 2017, The International Journal of Advanced Manufacturing Technology.

[38]  Jeroen Voeten,et al.  Performance evaluation with temporal rewards , 2002, Perform. Evaluation.

[39]  Twan Basten,et al.  Online Scheduling of 2-Re-entrant Flexible Manufacturing Systems , 2017, ACM Trans. Embed. Comput. Syst..

[40]  R. Bellman A Markovian Decision Process , 1957 .

[41]  BD Bart Theelen,et al.  Performance modelling for system-level design , 2004 .

[42]  George S. Fishman,et al.  Discrete-event simulation , 2001 .

[43]  Mark W. Maier Architecting Principles for Systems‐of‐Systems , 1996 .

[44]  A. Platzer,et al.  ModelPlex: verified runtime validation of verified cyber-physical system models , 2016, Formal Methods Syst. Des..

[45]  Robert I. Davis,et al.  A Survey of Timing Verification Techniques for Multi-Core Real-Time Systems , 2019, ACM Comput. Surv..

[46]  Kevin Forsberg,et al.  The Relationship of System Engineering to the Project Cycle , 1991 .

[47]  Peter W. Glynn,et al.  Dynamic modeling of the tradeoff between productivity and safety in critical engineering systems , 2004, Reliab. Eng. Syst. Saf..

[48]  Connie U. Smith,et al.  Performance Engineering of Software Systems , 1990, SIGMETRICS Perform. Evaluation Rev..

[49]  Kevin Forsberg,et al.  The Relationship of Systems Engineering to the Project Cycle , 1992 .

[50]  Connie U. Smith,et al.  Software Performance Antipatterns in Cyber-Physical Systems , 2020, ICPE.

[51]  Martin Leucker,et al.  A brief account of runtime verification , 2009, J. Log. Algebraic Methods Program..

[52]  J. Amerongen,et al.  Boderc : Model-based design of high-tech systems , 2006 .

[53]  Benno Stein,et al.  Model-Based Diagnosis for Cyber-Physical Production Systems Based on Machine Learning and Residual-Based Diagnosis Models , 2019, AAAI.

[54]  Eelco Visser,et al.  DSL Engineering - Designing, Implementing and Using Domain-Specific Languages , 2013 .

[55]  Philipp Reinecke,et al.  Performance and Security Tradeoff , 2010, SFM.

[56]  Y. C. Tay,et al.  Analytical Performance Modeling for Computer Systems , 2010, Analytical Performance Modeling for Computer Systems.

[57]  Jörgen Hansson,et al.  Model-based engineering in the embedded systems domain: an industrial survey on the state-of-practice , 2018, Software & Systems Modeling.

[58]  Kirk Pruhs,et al.  Online scheduling , 2003 .

[59]  David Harel,et al.  Statecharts: A Visual Formalism for Complex Systems , 1987, Sci. Comput. Program..

[60]  Alejandro Salado,et al.  Value and benefits of model‐based systems engineering (MBSE): Evidence from the literature , 2020, Syst. Eng..

[61]  Claudia Szabo,et al.  Model-driven performance prediction of systems of systems , 2016, Software & Systems Modeling.

[62]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[63]  Frits W. Vaandrager,et al.  Model learning , 2017, Commun. ACM.

[64]  Joseph Sifakis,et al.  A framework for scheduler synthesis , 1999, Proceedings 20th IEEE Real-Time Systems Symposium (Cat. No.99CB37054).

[65]  David B. Stewart Measuring Execution Time and Real-Time Performance , 2001 .

[66]  Andy D. Pimentel,et al.  Calibration of Abstract Performance Models for System-Level Design Space Exploration , 2006, 2006 International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation.

[67]  P. Ramadge,et al.  Supervisory control of a class of discrete event processes , 1987 .

[68]  James C. Browne,et al.  Performance engineering of software systems: a case study , 1982, AFIPS '82.

[69]  Jan Bosch,et al.  Speed, Data, and Ecosystems - Excelling in a Software-Driven World , 2016, Chapman and Hall / CRC innovations in software engineering and software development.

[70]  Sebastian Fischmeister Mining Traces of Embedded Software Systems for Insights , 2020, ICPE.

[71]  Wang Yi,et al.  UPPAAL 4.0 , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[72]  Ronald L. Graham,et al.  Performance Guarantees for Scheduling Algorithms , 1978, Oper. Res..

[73]  Dimitris Mourtzis,et al.  Simulation in the design and operation of manufacturing systems: state of the art and new trends , 2019, Int. J. Prod. Res..

[74]  Yonghui Li,et al.  Decision support methodology for evolutionary embedded system design , 2019, 2019 International Symposium on Systems Engineering (ISSE).

[75]  Lida Xu,et al.  Big data for cyber physical systems in industry 4.0: a survey , 2019, Enterp. Inf. Syst..

[76]  Ramon R. H. Schiffelers,et al.  Towards Automated Analysis of Model-Driven Artifacts in Industry , 2018, MODELSWARD.

[77]  Wpmh Maurice Heemels,et al.  5.3.2 On the Systematic Use of Budget‐Based Design , 2006 .