Log-Based Reliability Analysis of Software as a Service (SaaS)

Software as a Service (SaaS) has gained momentum in the past few years and businesses have been increasingly moving to SaaS model for their IT solutions. SaaS is a newer and transformed model where software is delivered to customers as a service over the web. With the SaaS model, there is a need for service providers to ensure that the services are available and reliable for end users at all times, which introduces significant pressure on the service provider to ensure right test processes and methodologies to minimize any impact to the provisions in Service Level Agreements (SLA). There is lack of research on the unique approaches to reliability analysis of SaaS suites. In this paper, we expand traditional approaches to reliability analysis of traditional web servers and propose methods tailored towards assessing the workload and reliability of SaaS applications. In addition we show the importance of data filtration when assessing SaaS reliability from log files. Finally, we discuss the suitability of reliability measures with respect to their relevance in the context of SLAs.

[1]  Xuan Wang,et al.  A Contribution Towards Solving the Web Workload Puzzle , 2006, International Conference on Dependable Systems and Networks (DSN'06).

[2]  Eldred Nelson,et al.  Estimating software reliability from test data , 1978 .

[3]  Carey L. Williamson,et al.  Internet Web servers: workload characterization and performance implications , 1997, TNET.

[4]  Zibin Zheng,et al.  Collaborative reliability prediction of service-oriented systems , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[5]  ChoudharyVidyanand.  Comparison of Software Quality Under Perpetual Licensing and Software as a Service , 2007 .

[6]  Swapna S. Gokhale,et al.  Architecture-Based Software Reliability Analysis: Overview and Limitations , 2007, IEEE Transactions on Dependable and Secure Computing.

[7]  Minaxi Gupta,et al.  Revisiting Web Server Workload Invariants in the Context of Scientific Web Sites , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[8]  Katerina Goseva-Popstojanova,et al.  Empirical Characterization of Session–Based Workload and Reliability for Web Servers , 2006, Empirical Software Engineering.

[9]  Lorenzo Strigini,et al.  A Contribution to the Evaluation of the Reliability of Iterative-Execution Software , 1999, Softw. Test. Verification Reliab..

[10]  Vidyanand Choudhary,et al.  Comparison of Software Quality Under Perpetual Licensing and Software as a Service , 2007, J. Manag. Inf. Syst..

[11]  Katerina Goseva-Popstojanova,et al.  Empirical study of session-based workload and reliability for Web servers , 2004, 15th International Symposium on Software Reliability Engineering.

[12]  Zhao Li,et al.  Testing the suitability of Markov chains as Web usage models , 2003, Proceedings 27th Annual International Computer Software and Applications Conference. COMPAC 2003.

[13]  Wolfgang Emmerich,et al.  Service-Level Agreements for Electronic Services , 2010, IEEE Transactions on Software Engineering.

[14]  John D. Musa,et al.  Software Reliability Engineering: More Reliable Software Faster and Cheaper , 2004 .

[15]  Bojan Cukic,et al.  A Bayesian approach to reliability prediction and assessment of component based systems , 2001, Proceedings 12th International Symposium on Software Reliability Engineering.

[16]  Vincenzo Grassi,et al.  Reliability Modeling and Analysis of Service-Oriented Architectures , 2007, Test and Analysis of Web Services.

[17]  Katerina Goseva-Popstojanova,et al.  Architecture-based approach to reliability assessment of software systems , 2001, Perform. Evaluation.

[18]  Li Ma,et al.  Web error classification and analysis for reliability improvement , 2007, J. Syst. Softw..

[19]  Toine Hurkmans,et al.  Zero downtime for multi tenant SaaS systems , 2009, SINTER '09.

[20]  Dan R. Herrick.  Google this!: using Google apps for collaboration and productivity , 2009, SIGUCCS '09.

[21]  Zhao Li,et al.  Evaluating Web software reliability based on workload and failure data extracted from server logs , 2004, IEEE Transactions on Software Engineering.

[22]  Anand V. Hudli,et al.  Level-4 SaaS applications for healthcare industry , 2009, COMPUTE '09.