Monitoring-aware IDEs

Engineering modern large-scale software requires software developers to not solely focus on writing code, but also to continuously examine monitoring data to reason about the dynamic behavior of their systems. These additional monitoring responsibilities for developers have only emerged recently, in the light of DevOps culture. Interestingly, software development activities happen mainly in the IDE, while reasoning about production monitoring happens in separate monitoring tools. We propose an approach that integrates monitoring signals into the development environment and workflow. We conjecture that an IDE with such capability improves the performance of developers as time spent continuously context switching from development to monitoring would be eliminated. This paper takes a first step towards understanding the benefits of a possible monitoring-aware IDE. We implemented a prototype of a Monitoring-Aware IDE, connected to the monitoring systems of Adyen, a large-scale payment company that performs intense monitoring in their software systems. Given our results, we firmly believe that monitoring-aware IDEs can play an essential role in improving how developers perform monitoring.

[1]  Arie van Deursen,et al.  An Exploratory Study on Faults inWeb API Integration in a Large-Scale Payment Company , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP).

[2]  Ivan Porres,et al.  DevOps: A Definition and Perceived Adoption Impediments , 2015, XP.

[3]  Brian Fitzgerald,et al.  The ABC of Software Engineering Research , 2018, ACM Trans. Softw. Eng. Methodol..

[4]  Liming Zhu,et al.  DevOps - A Software Architect's Perspective , 2015, SEI series in software engineering.

[5]  Qiang Fu,et al.  Where do developers log? an empirical study on logging practices in industry , 2014, ICSE Companion.

[6]  Andy Zaidman,et al.  Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[7]  André van der Hoek,et al.  The code orb: supporting contextualized coding via at-a-glance views (NIER track) , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[8]  Wang-Chien Lee,et al.  Personalized ranking for digital libraries based on log analysis , 2008, WIDM '08.

[9]  Felix Klaedtke,et al.  Scalable Offline Monitoring , 2014, RV.

[10]  Alastair Smith,et al.  So You Say You Want a Revolution , 1999 .

[11]  Akhil Kumar,et al.  Process mining on noisy logs - Can log sanitization help to improve performance? , 2015, Decis. Support Syst..

[12]  Ling Wu,et al.  CEclipse: An Online IDE for Programing in the Cloud , 2011, 2011 IEEE World Congress on Services.

[13]  Yu Luo,et al.  lprof: A Non-intrusive Request Flow Profiler for Distributed Systems , 2014, OSDI.

[14]  Wil M. P. van der Aalst,et al.  Fuzzy Mining - Adaptive Process Simplification Based on Multi-perspective Metrics , 2007, BPM.

[15]  P. Chandler,et al.  THE SPLIT‐ATTENTION EFFECT AS A FACTOR IN THE DESIGN OF INSTRUCTION , 1992 .

[16]  David Carrera,et al.  Business Process Mining from E-Commerce Web Logs , 2013, BPM.

[17]  Wil M. P. van der Aalst,et al.  Applications and Theory of Petri Nets , 1983, Informatik-Fachberichte.

[18]  Hai Jin,et al.  UiLog: Improving Log-Based Fault Diagnosis by Log Analysis , 2016, Journal of Computer Science and Technology.

[19]  Alexandre Bergel,et al.  Performance evolution blueprint: Understanding the impact of software evolution on performance , 2013, 2013 First IEEE Working Conference on Software Visualization (VISSOFT).

[20]  Arie van Deursen,et al.  An Experience Report on Applying Passive Learning in a Large-Scale Payment Company , 2017, ICSME.

[21]  Michael I. Jordan,et al.  Detecting large-scale system problems by mining console logs , 2009, SOSP '09.

[22]  Thomas Reidemeister,et al.  Mining unstructured log files for recurrent fault diagnosis , 2011, 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops.

[23]  Keun Ho Ryu,et al.  Prediction of Web User Behavior by Discovering Temporal Relational Rules from Web Log Data , 2012, DEXA.

[24]  Michael Httermann,et al.  DevOps for Developers , 2012 .

[25]  Rob Miller,et al.  Addressing misconceptions about code with always-on programming visualizations , 2014, CHI.

[26]  M. Di Penta,et al.  Why and How JavaScript Developers Use Linters , 2017 .

[27]  Chris North,et al.  Exploring context switching and cognition in dual-view coordinated visualizations , 2003, Proceedings International Conference on Coordinated and Multiple Views in Exploratory Visualization - CMV 2003 -.

[28]  Zhen Ming Jiang,et al.  Characterizing and Detecting Anti-Patterns in the Logging Code , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[29]  Boudewijn F. van Dongen,et al.  Process Discovery using Integer Linear Programming , 2009, Fundam. Informaticae.

[30]  Marijn J. H. Heule,et al.  Exact DFA Identification Using SAT Solvers , 2010, ICGI.

[31]  Qiang Fu,et al.  Learning to Log: Helping Developers Make Informed Logging Decisions , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[32]  Michael Hüttermann.  DevOps for Developers , 2012, Apress.

[33]  Johan A. Pouwelse,et al.  Understanding software performance regressions using differential flame graphs , 2015, 2015 IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[34]  Gustavo Pinto,et al.  Building a collaborative culture: a grounded theory of well succeeded devops adoption in practice , 2018, ESEM.

[35]  Yu Zhang,et al.  Log Clustering Based Problem Identification for Online Service Systems , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[36]  Ahmed E. Hassan,et al.  Leveraging Performance Counters and Execution Logs to Diagnose Memory-Related Performance Issues , 2013, 2013 IEEE International Conference on Software Maintenance.

[37]  Arie van Deursen,et al.  Adinda: a knowledgeable, browser-based IDE , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[38]  Alberto Sillitti,et al.  Failure prediction based on log files using Random Indexing and Support Vector Machines , 2013, J. Syst. Softw..

[39]  I. Maglogiannis,et al.  An overview of platforms for cloud based development , 2016, SpringerPlus.

[40]  Alexander Aiken,et al.  Using correlated surprise to infer shared influence , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[41]  Qiang Fu,et al.  Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[42]  Yuriy Brun,et al.  Leveraging existing instrumentation to automatically infer invariant-constrained models , 2011, ESEC/FSE '11.

[43]  Jennifer Neville,et al.  Structured Comparative Analysis of Systems Logs to Diagnose Performance Problems , 2012, NSDI.

[44]  Yuriy Brun,et al.  Shedding light on distributed system executions , 2014, ICSE Companion.

[45]  Wei Xu,et al.  Advances and challenges in log analysis , 2011, Commun. ACM.

[46]  Lakshmi M. Gadhikar,et al.  Browser Based IDE to Code in the Cloud , 2013 .

[47]  Timo Aho,et al.  Designing IDE as a Service , 2013 .

[48]  Harald C. Gall,et al.  Runtime metric meets developer: building better cloud applications using feedback , 2015, Onward!.

[49]  Gilbert Hamann,et al.  Automated performance analysis of load tests , 2009, 2009 IEEE International Conference on Software Maintenance.

[50]  Mike Loukides,et al.  What is DevOps , 2012 .

[51]  Bhavani M. Thuraisingham,et al.  Effective Software Fault Localization Using an RBF Neural Network , 2012, IEEE Transactions on Reliability.

[52]  Arvind Satyanarayan,et al.  Augmenting Code with In Situ Visualizations to Aid Program Understanding , 2018, CHI.

[53]  Harald C. Gall,et al.  Interactive Production Performance Feedback in the IDE , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[54]  Horst Lichter,et al.  Towards Definitions for Release Engineering and DevOps , 2015, 2015 IEEE/ACM 3rd International Workshop on Release Engineering.

[55]  Wil M. P. van der Aalst,et al.  Discovering signature patterns from event logs , 2013, 2013 IEEE Symposium on Computational Intelligence and Data Mining (CIDM).

[56]  James Roche,et al.  Adopting DevOps practices in quality assurance , 2013, CACM.

[57]  Xiaohui Gu,et al.  ELT: Efficient Log-based Troubleshooting System for Cloud Computing Infrastructures , 2011, 2011 IEEE 30th International Symposium on Reliable Distributed Systems.

[58]  Matthieu Roy,et al.  Experience Report: Log Mining Using Natural Language Processing and Application to Anomaly Detection , 2017, 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE).

[59]  Wilhelm Hasselbring,et al.  Software landscape and application visualization for system comprehension with ExplorViz , 2017, Inf. Softw. Technol..

[60]  Ling Huang,et al.  Online System Problem Detection by Mining Patterns of Console Logs , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[61]  Fabian Beck,et al.  In situ understanding of performance bottlenecks through visually augmented code , 2013, 2013 21st International Conference on Program Comprehension (ICPC).

[62]  David F. Redmiles,et al.  New opportunities for extracting insights from cloud based IDEs , 2014, ICSE Companion.

[63]  Maurício Aniche,et al.  The Adoption of JavaScript Linters in Practice: A Case Study on ESLint , 2020, IEEE Transactions on Software Engineering.

[64]  Danny Raz,et al.  Experience Report: Log-Based Behavioral Differencing , 2017, 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE).