Session Reliability of Web Systems under Heavy-Tailed Workloads: An Approach Based on Design and Analysis of Experiments

While workload characterization and performance of web systems have been studied extensively, reliability has received much less attention. In this paper, we propose a framework for session reliability modeling which integrates the user view represented by the session layer and the system view represented by the service layer. A unique characteristic of the session layer is that, in addition to the user navigation patterns, it incorporates the session length in number of requests and allows us to account for heavy-tailed workloads shown to exist in real web systems. The service layer is focused on the request reliability as it is observed at the service provider side. It considers the multifier web server architecture and the way components interact in serving each request. Within this framework, we develop a session reliability model and solve it using simulation. Instead of the traditional one-factor-at-a-time sensitivity analysis, we use statistical design and analysis of experiments, which allow us to identify the factors and interactions that have statistically significant effect on session reliability. Our findings indicate that session reliability, which accounts for the distribution of failed requests within sessions, provides better representation of the user perceived quality than the request-based reliability.

[1]  Katerina Goseva-Popstojanova,et al.  Empirical study of session-based workload and reliability for Web servers , 2004, 15th International Symposium on Software Reliability Engineering.

[2]  B. M. Hill,et al.  A Simple General Approach to Inference About the Tail of a Distribution , 1975 .

[3]  Carlo Ghezzi,et al.  Reliability Analysis of Component-Based Systems with Multiple Failure Modes , 2010, CBSE.

[4]  Swapna S. Gokhale,et al.  Performance and Availability Analysis of an E-Commerce Site , 2006, 30th Annual International Computer Software and Applications Conference (COMPSAC'06).

[5]  Zhao Li,et al.  Testing the suitability of Markov chains as Web usage models , 2003, Proceedings 27th Annual International Computer Software and Applications Conference. COMPAC 2003.

[6]  Adam Wierman,et al.  Open Versus Closed: A Cautionary Tale , 2006, NSDI.

[7]  Virgílio A. F. Almeida,et al.  Capacity Planning for Web Services: Metrics, Models, and Methods , 2001 .

[8]  Xuan Wang,et al.  Adequacy, Accuracy, Scalability, and Uncertainty of Architecture-based Software Reliability: Lessons Learned from Large Empirical Case Studies , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[9]  Heiko Koziolek,et al.  A Large-Scale Industrial Case Study on Architecture-Based Software Reliability Analysis , 2010, 2010 IEEE 21st International Symposium on Software Reliability Engineering.

[10]  Wei Xie,et al.  Modeling of user perceived webserver availability , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[11]  Kishor S. Trivedi,et al.  Accurate and efficient stochastic reliability analysis of composite services using their compact Markov reward model representations , 2007, IEEE International Conference on Services Computing (SCC 2007).

[12]  Katerina Goseva-Popstojanova,et al.  Empirical Characterization of Session–Based Workload and Reliability for Web Servers , 2006, Empirical Software Engineering.

[13]  Robert L. Mason,et al.  Statistical Design and Analysis of Experiments , 2003 .

[14]  Heiko Koziolek,et al.  Parameterized Reliability Prediction for Component-Based Software Architectures , 2010, QoSA.

[15]  Ralf H. Reussner,et al.  Reliability prediction for component-based software architectures , 2003, J. Syst. Softw..

[16]  Zibin Zheng,et al.  Distributed QoS Evaluation for Real-World Web Services , 2010, 2010 IEEE International Conference on Web Services.

[17]  Swapna S. Gokhale SOFTWARE RELIABILITY ANALYSIS INCORPORATING SECOND-ORDER ARCHITECTURAL STATISTICS , 2005 .

[18]  Daniel A. Menascé,et al.  TPC-W: A Benchmark for E-Commerce , 2002, IEEE Internet Comput..

[19]  Roger C. Cheung,et al.  A User-Oriented Software Reliability Model , 1978, IEEE Transactions on Software Engineering.

[20]  Karama Kanoun,et al.  A framework for modeling availability of E-business systems , 2001, Proceedings Tenth International Conference on Computer Communications and Networks (Cat. No.01EX495).

[21]  Bojan Cukic,et al.  Log-Based Reliability Analysis of Software as a Service (SaaS) , 2010, 2010 IEEE 21st International Symposium on Software Reliability Engineering.

[22]  Katerina Goseva-Popstojanova,et al.  Accounting for characteristics of session workloads: A study based on partly-open queue , 2012, 2012 IEEE International Conference on Communications (ICC).

[23]  Virgílio A. F. Almeida,et al.  Business-oriented resource management policies for e-commerce servers , 2000, Perform. Evaluation.

[24]  Zhao Li,et al.  Evaluating Web software reliability based on workload and failure data extracted from server logs , 2004, IEEE Transactions on Software Engineering.

[25]  Virgílio A. F. Almeida,et al.  In search of invariants for e-business workloads , 2000, EC '00.

[26]  Zibin Zheng,et al.  Collaborative reliability prediction of service-oriented systems , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[27]  Jeff Tian,et al.  Measuring and Modeling Usage and Reliability for Statistical Web Testing , 2001, IEEE Trans. Software Eng..

[28]  Jerome A. Rolia,et al.  WAM—The Weighted Average Method for Predicting the Performance of Systems with Bursts of Customer Sessions , 2011, IEEE Transactions on Software Engineering.

[29]  O. L. Davies,et al.  Design and analysis of industrial experiments , 1954 .

[30]  Magnos Martinello,et al.  A user-perceived availability evaluation of a web based travel agency , 2003, 2003 International Conference on Dependable Systems and Networks, 2003. Proceedings..

[31]  Dai Pan,et al.  Architecture-based software reliability modeling , 2006, J. Syst. Softw..

[32]  B. Bowerman Statistical Design and Analysis of Experiments, with Applications to Engineering and Science , 1989 .

[33]  Xuan Wang,et al.  A Contribution Towards Solving the Web Workload Puzzle , 2006, International Conference on Dependable Systems and Networks (DSN'06).

[34]  Bojan Cukic,et al.  Error propagation in the reliability analysis of component based systems , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[35]  Nenad Medvidovic,et al.  Early prediction of software component reliability , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[36]  Nenad Medvidovic,et al.  A Bayesian Model for Predicting Reliability of Software Systems at the Architectural Level , 2007, QoSA.

[37]  Matthew Merzbacher,et al.  Measuring end-user availability on the Web: practical experience , 2002, Proceedings International Conference on Dependable Systems and Networks.

[38]  Eila Niemelä,et al.  Survey of reliability and availability prediction methods from the viewpoint of software architecture , 2007, Software & Systems Modeling.

[39]  Katerina Goseva-Popstojanova,et al.  Architecture-based approach to reliability assessment of software systems , 2001, Perform. Evaluation.

[40]  O. L. Davies,et al.  Design and analysis of industrial experiments , 1954 .

[41]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[42]  Hany H. Ammar,et al.  Architectural-Level Risk Analysis Using UML , 2003, IEEE Trans. Software Eng..

[43]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[44]  Jerome A. Rolia,et al.  BURN: Enabling Workload Burstiness in Customized Service Benchmarks , 2012, IEEE Transactions on Software Engineering.

[45]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[46]  Holger Dette,et al.  Box-Type Approximations in Nonparametric Factorial Designs , 1997 .

[47]  Katerina Goseva-Popstojanova,et al.  Large empirical case study of architecture-based software reliability , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[48]  Hany H. Ammar,et al.  A scenario-based reliability analysis approach for component-based software , 2004, IEEE Transactions on Reliability.

[49]  Aditya P. Mathur,et al.  Comparison of architecture-based software reliability models , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[50]  Katerina Goseva-Popstojanova,et al.  Measurement-based performance analysis of e-commerce applications with Web services components , 2005, IEEE International Conference on e-Business Engineering (ICEBE'05).

[51]  Daniel A. Menascé,et al.  Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning , 2000 .

[52]  Vincenzo Grassi,et al.  A Modeling Approach to Analyze the Impact of Error Propagation on Reliability of Component-Based Systems , 2007, CBSE.