Automated anomaly detection and performance modeling of enterprise applications

Automated tools for understanding application behavior and its changes during the application lifecycle are essential for many performance analysis and debugging tasks. Application performance issues have an immediate impact on customer experience and satisfaction. A sudden slowdown of enterprise-wide application can effect a large population of customers, lead to delayed projects, and ultimately can result in company financial loss. Significantly shortened time between new software releases further exacerbates the problem of thoroughly evaluating the performance of an updated application. Our thesis is that online performance modeling should be a part of routine application monitoring. Early, informative warnings on significant changes in application performance should help service providers to timely identify and prevent performance problems and their negative impact on the service. We propose a novel framework for automated anomaly detection and application change analysis. It is based on integration of two complementary techniques: (i) a regression-based transaction model that reflects a resource consumption model of the application, and (ii) an application performance signature that provides a compact model of runtime behavior of the application. The proposed integrated framework provides a simple and powerful solution for anomaly detection and analysis of essential performance changes in application behavior. An additional benefit of the proposed approach is its simplicity: It is not intrusive and is based on monitoring data that is typically available in enterprise production environments. The introduced solution further enables the automation of capacity planning and resource provisioning tasks of multitier applications in rapidly evolving IT environments.

[1]  Armando Fox,et al.  Capturing, indexing, clustering, and retrieving system history , 2005, SOSP '05.

[2]  David A. Patterson,et al.  Path-Based Failure and Evolution Management , 2004, NSDI.

[3]  R. E. Cunningham,et al.  Analysis of Applications , 1980 .

[4]  E. N. Elnozahy,et al.  Measuring Client-Perceived Response Time on the WWW , 2001, USITS.

[5]  Richard Mortier,et al.  Using Magpie for Request Extraction and Workload Modelling , 2004, OSDI.

[6]  Anja Feldmann,et al.  Rate of Change and other Metrics: a Live Study of the World Wide Web , 1997, USENIX Symposium on Internet Technologies and Systems.

[7]  N. Draper,et al.  Applied Regression Analysis , 1967 .

[8]  Marcos K. Aguilera,et al.  Performance debugging for distributed systems of black boxes , 2003, SOSP '03.

[9]  Wirtschaft Application Performance Management , 2010 .

[10]  Magnus Karlsson,et al.  Dynamics and evolution of Web sites: analysis, metrics and design issues , 2001, Proceedings. Sixth IEEE Symposium on Computers and Communications.

[11]  Amin Vahdat,et al.  Measuring and characterizing end-to-end Internet service performance , 2003, TOIT.

[12]  Evgenia Smirni,et al.  Analysis of application performance and its change via representative application signatures , 2008, NOMS 2008 - 2008 IEEE Network Operations and Management Symposium.

[13]  N. Draper,et al.  Applied Regression Analysis: Draper/Applied Regression Analysis , 1998 .

[14]  Martin Arlitt The Case for Data Assurance , 2005 .

[15]  Qi Zhang,et al.  R-Capriccio: A Capacity Planning and Anomaly Detection Tool for Enterprise Services with Live Workloads , 2007, Middleware.

[16]  Virgílio A. F. Almeida,et al.  Capacity Planning and Performance Modeling: From Mainframes to Client-Server Systems , 1994 .

[17]  Ozalp Babaoglu,et al.  ACM Transactions on Computer Systems , 2007 .

[18]  Qi Zhang,et al.  A Regression-Based Analytic Model for Dynamic Resource Provisioning of Multi-Tier Applications , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).

[19]  Asser N. Tantawi,et al.  An analytical model for multi-tier internet services and its applications , 2005, SIGMETRICS '05.

[20]  Christopher Stewart,et al.  Exploiting nonstationarity for performance prediction , 2007, EuroSys '07.