A Survey of the Selenium Ecosystem

Selenium is often considered the de-facto standard framework for end-to-end web testing nowadays. It allows practitioners to drive web browsers (such as Chrome, Firefox, Edge, or Opera) in an automated fashion using different language bindings (such as Java, Python, or JavaScript, among others). The term ecosystem, referring to the open-source software domain, includes various components, tools, and other interrelated elements sharing the same technological background. This article presents a descriptive survey aimed to understand how the community uses Selenium and its ecosystem. This survey is structured in seven categories: Selenium foundations, test development, system under test, test infrastructure, other frameworks, community, and personal experience. In light of the current state of Selenium, we analyze future challenges and opportunities around it.

[1]  S. Jagannatha,et al.  Functional Software Testing for Web Applications in the Context of Industry , 2018, Journal of Computational and Theoretical Nanoscience.

[2]  Steven J. DeRose,et al.  XML Path Language (XPath) Version 1.0 , 1999 .

[3]  Ousmane Sall,et al.  Web Scraping: State-of-the-Art and Areas of Application , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[4]  Coen De Roover,et al.  Prevalence and Maintenance of Automated Functional Tests for Web Applications , 2014, 2014 IEEE International Conference on Software Maintenance and Evolution.

[5]  Paolo Tonella,et al.  Pesto: Automated migration of DOM‐based Web tests towards the visual approach , 2018, Softw. Test. Verification Reliab..

[6]  Miroslav Bures,et al.  Identification of Potential Reusable Subroutines in Recorded Automated Test Scripts , 2018, Int. J. Softw. Eng. Knowl. Eng..

[7]  Miroslav Bures,et al.  SmartDriver: Extension of Selenium WebDriver to Create More Efficient Automated Tests , 2016, 2016 6th International Conference on IT Convergence and Security (ICITCS).

[8]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[9]  Venkataraman Ramesh,et al.  Research in software engineering: an analysis of the literature , 2002, Inf. Softw. Technol..

[10]  Andrew Hines,et al.  Assessment of QoE for Video and Audio in WebRTC Applications Using Full-Reference Models , 2020, Electronics.

[11]  Galia Novakova,et al.  Automation Testing Framework for Web Applications with Selenium WebDriver: Opportunities and Threats , 2017 .

[12]  Helena Holmström Olsson,et al.  JAutomate: A Tool for System- and Acceptance-test Automation , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[13]  Slinger Jansen,et al.  Software Ecosystems: Analyzing and Managing Business Networks in the Software Industry , 2013 .

[14]  Gregg Rothermel,et al.  Software testing: a research travelogue (2000–2014) , 2014, FOSE.

[15]  Tanupriya Choudhury,et al.  Continuous and Integrated Software Development using DevOps , 2018, 2018 International Conference on Advances in Computing and Communication Engineering (ICACCE).

[16]  Filippo Ricca,et al.  What 5 million job advertisements tell us about testing: a preliminary empirical investigation , 2020, SAC.

[17]  Foutse Khomh,et al.  Software Engineering for Machine-Learning Applications: The Road Ahead , 2018, IEEE Software.

[18]  Paolo Tonella,et al.  Reducing Web Test Cases Aging by Means of Robust XPath Locators , 2014, 2014 IEEE International Symposium on Software Reliability Engineering Workshops.

[19]  Boni García,et al.  Practical Evaluation of VMAF Perceptual Video Quality for WebRTC Applications , 2019, Electronics.

[20]  Paolo Tonella,et al.  APOGEN: automatic page object generator for web testing , 2016, Software Quality Journal.

[21]  Darko Marinov,et al.  An empirical analysis of flaky tests , 2014, SIGSOFT FSE.

[22]  Kai Petersen,et al.  Benefits and limitations of automated software testing: Systematic literature review and practitioner survey , 2012, 2012 7th International Workshop on Automation of Software Test (AST).

[23]  Vahid Garousi,et al.  Developing, Verifying, and Maintaining High-Quality Automated Test Scripts , 2016, IEEE Software.

[24]  Filippo Ricca,et al.  Comparing the maintainability of selenium WebDriver test suites employing different locators: a case study , 2013, JAMAICA 2013.

[25]  Paolo Tonella,et al.  Using Multi-Locators to Increase the Robustness of Web Test Cases , 2015, 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST).

[26]  Robert Feldt,et al.  Maintenance of automated test suites in industry: An empirical study on Visual GUI Testing , 2016, Inf. Softw. Technol..

[27]  Venkataraman Ramesh,et al.  An analysis of research in computing disciplines , 2004, CACM.

[28]  Filippo Ricca,et al.  Improving Test Suites Maintainability with the Page Object Pattern: An Industrial Case Study , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[29]  Vasja Vehovar,et al.  Open-ended vs. close-ended questions in Web questionnaires , 2003 .

[30]  Yachai Limpiyakorn,et al.  Transforming JavaScript-Based Web Application to Cross-Platform Desktop with Electron , 2018, ICISA.

[31]  Antonello Calabrò,et al.  When the Testing Gets Tough, the Tough Get ElasTest , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering: Companion (ICSE-Companion).

[32]  Miikka Kuutila Benchmarking configurations for web-testing : Selenium versus Watir , 2016 .

[33]  S. Downing Validity: on the meaningful interpretation of assessment data , 2003, Medical education.

[34]  Mark Harman,et al.  The Oracle Problem in Software Testing: A Survey , 2015, IEEE Transactions on Software Engineering.

[35]  Sarah Smith Heckman,et al.  Wait, Wait. No, Tell Me. Analyzing Selenium Configuration Effects on Test Flakiness , 2019, 2019 IEEE/ACM 14th International Workshop on Automation of Software Test (AST).

[36]  Miroslav Bures Automated testing in the Czech Republic: the current situation and issues , 2014, CompSysTech.

[37]  Harpreet Kaur,et al.  Comparative Study of Automated Testing Tools: Selenium, Quick Test Professional and Testcomplete , 2013 .

[38]  Ossi Taipale,et al.  Software Test Automation in Practice: Empirical Observations , 2010, Adv. Softw. Eng..

[39]  Gerard Meszaros,et al.  xUnit Test Patterns: Refactoring Test Code , 2007 .

[40]  Boni García,et al.  Understanding and estimating quality of experience in WebRTC applications , 2018, Computing.