Predicting software performance in symmetric multi-core and multiprocessor environments

Mit der breiten Markteinfuhrung von Mehrkernprozessoren wird Nebenlaufigkeit zu einer allgegenwartigen Herausforderung fur die Software-Entwicklung. Indem zusatzliche Prozessorkerne genutzt werden, kann Nebenlaufigkeit die Leistungsfahigkeit (auch Performanz) eines Software-Systems verbessern. Performanz-Vorhersageverfahren, die die Einflusse von Mehrkernprozessorumgebungen berucksichtigen, konnen Software-Architekten bei der Entwicklung leistungsfahiger Software Systeme bereits in fruhen Entwicklungsphasen unterstutzen. In dieser Arbeit wird ein Performanz-Vorhersageverfahren fur symmetrische Mehrprozessorumgebungen entwickelt und validiert. Der Schwerpunkt liegt dabei auf dem Einfluss des Betriebssystem-Schedulers. Weiterhin wird ein Ansatz zur Modellierung und Vorhersage nachrichtenbasierter Systeme eingefuhrt. In einer Serie von Fallstudien wird gezeigt, dass beide Ansatze in den meisten Fallen den Vorhersagefehler auf 5% bis 10% reduzieren konnen.

[1]  Giovanni Denaro,et al.  Early performance testing of distributed software applications , 2004, WOSP '04.

[2]  Samuel Kounev,et al.  SimQPN - A tool and methodology for analyzing queueing Petri net models by means of simulation , 2006, Perform. Evaluation.

[3]  M. Bravetti,et al.  From EMPA to GSMPA : Allowing for General DistributionsMario Bravetti , 2007 .

[4]  Sem C. Borst,et al.  Beyond processor sharing , 2007, PERV.

[5]  Jakob Engblom,et al.  Modeling complex flows for worst-case execution time analysis , 2000, Proceedings 21st IEEE Real-Time Systems Symposium.

[6]  Alan Fekete,et al.  Design-level performance prediction of component-based applications , 2005, IEEE Transactions on Software Engineering.

[7]  S. Mohan,et al.  Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software [Book Review] , 2003, IEEE Software.

[8]  Heiko Koziolek,et al.  Parameter dependencies for reusable performance specifications of software components , 2010 .

[9]  Kurt Jensen.  An Introduction to the Practical Use of Coloured Petri Nets , 1996, Petri Nets.

[10]  Joost-Pieter Katoen,et al.  Process algebra for performance evaluation , 2002, Theor. Comput. Sci..

[11]  Paola Inverardi,et al.  Model-based performance prediction in software development: a survey , 2004, IEEE Transactions on Software Engineering.

[12]  Samuel Kounev,et al.  Performance Modeling and Evaluation of Distributed Component-Based Systems Using Queueing Petri Nets , 2006, IEEE Transactions on Software Engineering.

[13]  Christos G. Cassandras,et al.  Optimal routing and buffer allocation for a class of finite capacity queueing systems , 1992 .

[14]  Thomas Kunz,et al.  The Influence of Different Workload Descriptions on a Heuristic Load Balancing Scheme , 1991, IEEE Trans. Software Eng..

[15]  Yan Solihin,et al.  QoS policies and architecture for cache/memory in CMP platforms , 2007, SIGMETRICS '07.

[16]  Josh Aas.  Understanding the Linux 2.6.8.1 CPU Scheduler , 2005 .

[17]  Steffen Becker,et al.  Parameter Dependent Performance Specifications of Software Components , 2006, QoSA.

[18]  Connie U. Smith,et al.  Performance Engineering of Software Systems , 1990, SIGMETRICS Perform. Evaluation Rev..

[19]  Mukesh Singhal,et al.  Load distributing for locally distributed systems , 1992, Computer.

[20]  Gianfranco Ciardo,et al.  A Characterization of the Stochastic Process Underlying a Stochastic Petri Net , 1994, IEEE Trans. Software Eng..

[21]  Arif Ghafoor,et al.  Performance modeling of load-balancing algorithms using neural networks , 1994, Concurr. Pract. Exp..

[22]  Jens Happe,et al.  Predicting Mean Service Execution Times of Software Components Based on Markov Models , 2005, QoSA/SOQUA.

[23]  Murray Woodside.  Tutorial introduction to layered modeling of software performance , 2002 .

[24]  Jörg Rech,et al.  Model-Driven Software Development - Integrating Quality Assurance , 2008 .

[25]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[26]  C. Murray Woodside,et al.  Performance analysis of distributed server systems , 2000 .

[27]  Steffen Becker,et al.  Predicting the Performance of Component-Based Software Architectures with Different Usage Profiles , 2007, QoSA.

[28]  Mikael Sjödin,et al.  Predicting execution-time for variable behaviour embedded real-time components , 2006, RTSS 2006.

[30]  Gunter Bolch,et al.  WinPEPSY-QNS - Performance evaluation and prediction system for queueing networks , 2006, MMB.

[31]  Jane Hillston,et al.  A compositional approach to performance modelling , 1996 .

[32]  Mario Bravetti,et al.  Tutte le Algebre Insieme: Concepts, Discussions and Relations of Stochastic Process Algebras with General Distributions , 2004, Validation of Stochastic Systems.

[33]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .

[34]  C. U. Smith,et al.  The prediction and evaluation of the performance of software from extended design specifications , 1980 .

[35]  Steffen Becker,et al.  Parametric Performance Contracts for QML-specified Software Components , 2005, FESCA@ETAPS.

[36]  Günter Hommel,et al.  Discrete time stochastic petri nets for modeling and evaluation of real-time systems , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[37]  C. Murray Woodside,et al.  Performance-related completions for software specifications , 2002, ICSE '02.

[38]  Sanjay K. Bose,et al.  An Introduction to Queueing Systems , 2002, Springer US.

[39]  Guillaume Urvoy-Keller,et al.  Performance analysis of LAS-based scheduling disciplines in a packet switched network , 2004, SIGMETRICS '04/Performance '04.

[40]  Egon Berghout,et al.  Integrating goal-oriented measurement in industrial software engineering: industrial experiences with and additions to the Goal/Question/Metric method (GQM) , 2001, Proceedings Seventh International Software Metrics Symposium.

[41]  Luiz Affonso Guedes,et al.  A Markovian Performance Model for Resource Allocation Scheduling on GNU/Linux , 2006, ISPA Workshops.

[42]  Linus Schrage,et al.  Letter to the Editor - A Proof of the Optimality of the Shortest Remaining Processing Time Discipline , 1968, Oper. Res..

[43]  P. Schatte.  The M/GI/1 queue as limit of closed queueing systems , 1984 .

[44]  C. Murray Woodside,et al.  Performance modeling from software components , 2004, WOSP '04.

[45]  Richard Monson-Haefel,et al.  Java message service , 2000 .

[46]  Allan Clark,et al.  Formal Methods for Performance Evaluation , 2007 .

[47]  Richard F. Serfozo,et al.  Optimality of routing and servicing in dependent parallel processing systems , 1991, Queueing Syst. Theory Appl..

[48]  Bruce Powel Douglass.  Real-Time Design Patterns , 1999 .

[49]  David Holmes,et al.  Java Concurrency in Practice , 2006 .

[50]  Guillaume Urvoy-Keller,et al.  Scheduling in practice , 2007, PERV.

[51]  Anoop Gupta,et al.  The impact of operating system scheduling policies and synchronization methods of performance of parallel applications , 1991, SIGMETRICS '91.

[52]  Paolo Ciancarini,et al.  Stochastic Process Algebra: From an Algebraic Formalism to an Architectural Description Language , 2002, Performance.

[53]  Wolfgang E. Nagel,et al.  Analysis of Linux Scheduling with VAMPIR , 2007, International Conference on Computational Science.

[54]  Stephen S. Lavenberg,et al.  Mean-Value Analysis of Closed Multichain Queuing Networks , 1980, JACM.

[55]  Mor Harchol-Balter,et al.  Evaluation of Task Assignment Policies for Supercomputing Servers: The Case for Load Unbalancing and Fairness , 2004, Cluster Computing.

[56]  Avelino Francisco Zorzo,et al.  Operating system multilevel load balancing , 2006, SAC '06.

[57]  Steffen Becker,et al.  The palladio component model , 2010, WOSP/SIPEW '10.

[58]  Mor Harchol-Balter,et al.  Size-based scheduling to improve web performance , 2003, TOCS.

[59]  Adam Wierman,et al.  Classifying scheduling policies with respect to unfairness in an M/GI/1 , 2003, SIGMETRICS '03.

[60]  Dorina C. Petriu,et al.  Software Performance Models from System Scenarios in Use Case Maps , 2002, Computer Performance Evaluation / TOOLS.

[61]  Liming Zhu,et al.  MDABench: Customized benchmark generation using MDA , 2007, J. Syst. Softw..

[62]  Mark S. Squillante,et al.  Stochastic analysis of multiserver systems , 2007, PERV.

[63]  Jean Paul Calvez,et al.  A generic RTOS model for real-time systems simulation with systemC , 2004, Proceedings Design, Automation and Test in Europe Conference and Exhibition.

[64]  Robin Milner,et al.  Definition of standard ML , 1990 .

[65]  Ronald W. Wolff,et al.  Stochastic Modeling and the Theory of Queues , 1989 .

[66]  Mark S. Squillante,et al.  The impact of I/O on program behavior and parallel scheduling , 1998, SIGMETRICS '98/PERFORMANCE '98.

[67]  Vincenzo Grassi,et al.  A Model Transformation Approach for the Early Performance and Reliability Analysis of Component-Based Systems , 2006, CBSE.

[68]  Mor Harchol-Balter,et al.  Analysis of multi-server systems via dimensionality reduction of markov chains , 2005 .

[69]  Mor Harchol-Balter,et al.  On Choosing a Task Assignment Policy for a Distributed Server System , 1998, J. Parallel Distributed Comput..

[70]  Vittorio Cortellessa,et al.  Integrating Software Models and Platform Models for Performance Analysis , 2007, IEEE Transactions on Software Engineering.

[71]  Willy Zwaenepoel,et al.  Performance and scalability of EJB applications , 2002, OOPSLA '02.

[72]  David P. Rodgers,et al.  Improvements in multiprocessor system design , 1985, ISCA '85.

[73]  Heiko Koziolek.  Introduction to Performance Metrics , 2005, Dependability Metrics.

[74]  Samuel Kounev,et al.  Workload Characterization of the SPECjms2007 Benchmark , 2007, EPEW.

[75]  Adam Duguid,et al.  Stronger Computational Modelling of Signalling Pathways Using Both Continuous and Discrete-State Methods , 2006, CMSB.

[76]  Alan Scheller-Wolf,et al.  Exploring Threshold-based Policies for Load Sharing , 2004 .

[77]  Edward D. Lazowska,et al.  Quantitative system performance - computer system analysis using queueing network models , 1983, Int. CMG Conference.

[78]  Egon Berghout,et al.  The Goal/Question/Metric method: a practical guide for quality improvement of software development , 1999 .

[79]  Heiko Koziolek,et al.  Parametric Performance Contracts for Software Components with Concurrent Behaviour , 2007, Electron. Notes Theor. Comput. Sci..

[80]  Giuseppe Serazzi,et al.  Java Modelling Tools: an Open Source Suite for Queueing Network Modelling andWorkload Analysis , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[81]  P.K. Saraswat,et al.  Design and Implementation of a Process Scheduler Simulator and an Improved Process Scheduling Algorithm for Multimedia Operating Systems , 2006, 2006 International Conference on Advanced Computing and Communications.

[82]  Vincenzo Grassi,et al.  From design to analysis models: a kernel language for performance and reliability analysis of component-based systems , 2005, WOSP '05.

[83]  Mor Harchol-Balter,et al.  Web servers under overload: How scheduling can help , 2006, TOIT.

[84]  Krzysztof Czarnecki,et al.  Generative programming - methods, tools and applications , 2000 .

[85]  Abraham Silberschatz,et al.  Operating System Concepts , 1983 .

[86]  Myron Hlynka,et al.  Queueing Networks and Markov Chains (Modeling and Performance Evaluation With Computer Science Applications) , 2007, Technometrics.

[87]  Marco Ajmone Marsan,et al.  Modelling with Generalized Stochastic Petri Nets , 1995, PERV.

[88]  Piet Demeester,et al.  Automatic inclusion of middleware performance attributes into architectural UML software models , 2005, IEEE Transactions on Software Engineering.

[89]  Kishor S. Trivedi,et al.  Reliability and Performability Modeling Using SHARPE 2000 , 2000, Computer Performance Evaluation / TOOLS.

[90]  Jean G. Vaucher,et al.  SSJ: a framework for stochastic simulation in Java , 2002, Proceedings of the Winter Simulation Conference.

[91]  Wolfgang Reisig,et al.  Lectures on Petri Nets II: Applications, Advances in Petri Nets, the volumes are based on the Advanced Course on Petri Nets , 1996 .

[92]  Pierre L'Ecuyer,et al.  Simulation in Java with SSJ , 2005, Proceedings of the Winter Simulation Conference, 2005..

[93]  Peter J. Denning,et al.  The Operational Analysis of Queueing Network Models , 1978, CSUR.

[94]  David A. Freedman,et al.  Statistical Models: Theory and Practice: References , 2005 .

[95]  Brigitte Plateau,et al.  Stochastic Automata Network For Modeling Parallel Systems , 1991, IEEE Trans. Software Eng..

[96]  Heiko Koziolek,et al.  A QoS Driven Development Process Model for Component-Based Software Systems , 2006, CBSE.

[97]  Stefano Leonardi,et al.  Approximating total flow time on parallel machines , 1997, STOC '97.

[98]  Ian Gorton,et al.  Performance prediction of J2EE applications using messaging protocols , 2005, CBSE'05.

[99]  Ajay D. Kshemkalyani,et al.  SWIFT: scheduling in web servers for fast response time , 2003, Second IEEE International Symposium on Network Computing and Applications, 2003. NCA 2003..

[100]  Adam Wierman,et al.  Multi-Server Queueing Systems with Multiple Priority Classes , 2005, Queueing Syst. Theory Appl..

[101]  S. Majumdar.  The performance of local and global scheduling strategies in multiprogrammed parallel systems , 1992, Eleventh Annual International Phoenix Conference on Computers and Communication [1992 Conference Proceedings].

[102]  Adam Wierman,et al.  A note on comparing response times in the M/GI/1/FB and M/GI/1/PS queues , 2004, Oper. Res. Lett..

[103]  Gregor Hohpe,et al.  Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions , 2003 .

[104]  Avelino Francisco Zorzo,et al.  Analytical Modeling for Operating System Schedulers on NUMA Systems , 2006, Electron. Notes Theor. Comput. Sci..

[105]  Edward A. Lee.  The problem with threads , 2006, Computer.

[106]  Falko Bause,et al.  Stochastic Petri Nets: An Introduction to the Theory , 2012, PERV.

[107]  Giuseppe Serazzi,et al.  The JMT Simulator for Performance Evaluation of Non-Product-Form Queueing Networks , 2007, 40th Annual Simulation Symposium (ANSS'07).

[108]  Virgílio A. F. Almeida,et al.  Performance by Design - Computer Capacity Planning By Example , 2004 .

[109]  Steffen Becker,et al.  A pattern-based performance completion for Message-oriented Middleware , 2008, WOSP '08.

[110]  Marco Ajmone Marsan,et al.  The Effect of Execution Policies on the Semantics and Analysis of Stochastic Petri Nets , 1989, IEEE Trans. Software Eng..

[111]  C. Murray Woodside,et al.  Resource Function Capture for Performance Aspects of Software Components and Sub-Systems , 2001, Performance Engineering.

[112]  Ward Whitt,et al.  Deciding Which Queue to Join: Some Counterexamples , 1986, Oper. Res..

[113]  C. Murray Woodside,et al.  Using regression splines for software performance analysis , 2000, WOSP '00.

[114]  Steffen Becker,et al.  Coupled model transformations for QoS enabled component-based software design , 2010 .

[115]  Alberto Policriti,et al.  Stochastic Concurrent Constraint Programming and Differential Equations , 2007, QAPL.

[116]  Steffen Becker,et al.  Predicting Software Component Performance : On the Relevance of Parameters for Benchmarking Bytecode and APIs , 2007 .

[117]  Johan J. Lukkien,et al.  Predicting real-time properties of component assemblies: a scenario-simulation approach , 2004, Proceedings. 30th Euromicro Conference, 2004..

[118]  van der Wmp Wil Aalst,et al.  PATTERNS IN COLORED PETRI NETS , 2005 .

[119]  Mache Creeger,et al.  Multicore CPUs for the Masses , 2005, QUEUE.

[120]  Sandro Morasca,et al.  Applying GQM in an industrial software factory , 1998, TSEM.

[121]  Barton P. Miller,et al.  A comparison of interactivity in the Linux 2.6 scheduler and an MLFQ scheduler , 2007 .

[122]  Jerome A. Rolia,et al.  The Method of Layers , 1995, IEEE Trans. Software Eng..

[123]  Andrei Borshchev,et al.  COVERS - A Tool for the Design of Real-time Concurrent Systems , 1995, PaCT.

[124]  John L. Henning.  SPEC CPU2000: Measuring CPU Performance in the New Millennium , 2000, Computer.

[125]  Steffen Becker,et al.  Putting Components into Context Supporting QoS-Predictions with an explicit Context Model , 2006 .

[126]  C. Petri.  Kommunikation mit Automaten , 1962 .

[127]  Samuel Kounev,et al.  Designing a Workload Scenario for Benchmarking Message-Oriented Middleware , 2007 .

[128]  Hadas Shachnai,et al.  Scheduling memory accesses through a shared bus , 2001, Perform. Evaluation.

[129]  W. Whitt,et al.  The Queueing Network Analyzer , 1983, The Bell System Technical Journal.

[130]  Anthony Ephremides,et al.  A simple dynamic routing problem , 1980 .

[131]  Jens Happe.  Performance Prediction for Embedded Systems , 2005 .

[132]  Kurt Jensen,et al.  An Introduction to the Theoretical Aspects of Coloured Petri Nets , 1993, REX School/Symposium.

[133]  John B. Nagle,et al.  On Packet Switches with Infinite Storage , 1987, IEEE Trans. Commun..

[134]  Adam Wierman,et al.  Nearly insensitive bounds on SMART scheduling , 2005, SIGMETRICS '05.

[135]  Mary K. Vernon,et al.  The performance of multiprogrammed multiprocessor scheduling algorithms , 1990, SIGMETRICS '90.

[136]  Kurt Jensen,et al.  Coloured Petri Nets: Basic Concepts, Analysis Methods and Practical Use. Vol. 2, Analysis Methods , 1992 .

[137]  Sharad Malik,et al.  Performance analysis of real-time embedded software , 1997 .

[138]  Tapani Lehtonen,et al.  On the optimality of the shortest line discipline , 1984 .

[139]  P. Sparaggis,et al.  Minimizing response times and queue lengths in systems of parallel queues , 1999 .

[140]  Adam Wierman,et al.  Tail asymptotics for policies favoring short jobs in a many-flows regime , 2006, SIGMETRICS '06/Performance '06.

[141]  Steffen Becker,et al.  Model-Driven Generation of Performance Prototypes , 2008, SIPEW.

[142]  Lars Michael Kristensen,et al.  Coloured Petri Nets and CPN Tools for modelling and validation of concurrent systems , 2007, International Journal on Software Tools for Technology Transfer.

[143]  Lisa Wells,et al.  Performance Analysis using Coloured Petri Nets , 2002 .

[144]  Jane Hillston,et al.  Fluid flow approximation of PEPA models , 2005, Second International Conference on the Quantitative Evaluation of Systems (QEST'05).

[145]  Hideyuki Tokuda,et al.  A Time-Driven Scheduling Model for Real-Time Operating Systems , 1985, RTSS.

[146]  Alan Scheller-Wolf,et al.  Analysis of cycle stealing with switching times and thresholds , 2005, Perform. Evaluation.

[147]  Connie U. Smith,et al.  Making the Business Case for Software Performance Engineering , 2003, Int. CMG Conference.

[148]  Samuel Kounev,et al.  Performance modelling of distributed e-business applications using Queuing Petri Nets , 2003, 2003 IEEE International Symposium on Performance Analysis of Systems and Software. ISPASS 2003..

[149]  Shikharesh Majumdar,et al.  Scheduling in multiprogrammed parallel systems , 1988, SIGMETRICS 1988.

[150]  Daniel Pierre Bovet,et al.  Understanding the Linux Kernel , 2000 .

[151]  Steffen Becker,et al.  The Palladio component model for model-driven performance prediction , 2009, J. Syst. Softw..

[152]  Ian Pratt,et al.  Multiprogramming Performance of the Pentium 4 with Hyper-Threading , 2004 .

[153]  Adam Wierman,et al.  Open Versus Closed: A Cautionary Tale , 2006, NSDI.

[154]  Steffen Becker,et al.  Life-Cycle Aware Modelling of Software Components , 2008, CBSE.

[155]  Heiko Koziolek,et al.  Performance Metrics for Specific Domains , 2005, Dependability Metrics.

[156]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[157]  Edwin Hsing-Mean Sha,et al.  Estimating probabilistic timing performance for real-time embedded systems , 2001, IEEE Trans. Very Large Scale Integr. Syst..

[158]  Steffen Becker,et al.  Model-Based performance prediction with the palladio component model , 2007, WOSP '07.

[159]  Wayne H. Wolf,et al.  Performance estimation for real-time distributed embedded systems , 1995, Proceedings of ICCD '95 International Conference on Computer Design. VLSI in Computers and Processors.