Combinatorial Testing of Browsers’ Support for Multimedia

The fifth version of the Hypertext Markup Language (HTML) standard is widely adopted in the diverse landscape of browser vendors and their continuously upgrading releases. One primary feature of HTML5 is native multimedia playback. The browsers’ native implementations of multimedia support bring lots of benefits such as improved security but require thorough testing, especially on the web page of complex factor combinations according to our manual checking of publicly visible existing tests. This article employs the combinatorial testing technique to trigger failure-inducing factor combinations effectively and to locate them, guided by some extracted properties that cover browsers’ major workflow of processing multimedia. Results are analyzed to give objective suggestions for browser developers to cast light on the places that implementation enhancement could be made, and for web developers to avoid undesirable effects.

[1]  Sarfraz Khurshid,et al.  Combinatorial generation of structurally complex test inputs for commercial software applications , 2016, SIGSOFT FSE.

[2]  Hareton K. N. Leung,et al.  The Minimal Failure-Causing Schema of Combinatorial Testing , 2011, TSEM.

[3]  Jian Zhang,et al.  Characterizing failure-causing parameter interactions by adaptive testing , 2011, ISSTA '11.

[4]  Alireza Sadeghi,et al.  Reducing Combinatorics in GUI Testing of Android Applications , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[5]  Jeff Yu Lei,et al.  An Input Space Modeling Methodology for Combinatorial Testing , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops.

[6]  Y. Raghu Reddy,et al.  A study on Migrating Flash files to HTML5/JavaScript , 2017, ISEC.

[7]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[8]  R. Krishnan,et al.  Combinatorial testing: learnings from our experience , 2007, SOEN.

[9]  Britta Meixner,et al.  Download and Cache Management for HTML5 Hypervideo Players , 2016, HT.

[10]  Myra B. Cohen,et al.  Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection , 2015, IEEE Transactions on Software Engineering.

[11]  Yu Lei,et al.  Introduction to Combinatorial Testing , 2013 .

[12]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[13]  Tao Guo,et al.  GramFuzz: Fuzzing testing of web browsers based on grammar analysis and structural mutation , 2013, 2013 Second International Conference on Informatics & Applications (ICIA).

[14]  Matthias Grochtmann,et al.  Classification trees for partition testing , 1993, Softw. Test. Verification Reliab..

[15]  Soojin Yoon,et al.  Attacks on Web browsers with HTML5 , 2015, 2015 10th International Conference for Internet Technology and Secured Transactions (ICITST).

[16]  Irma Syarlina Hj Che Ilias,et al.  A study of video performance analysis between Flash video and HTML 5 video , 2013, ICUIMC '13.

[17]  Lionel C. Briand,et al.  Formal Analysis of the Probability of Interaction Fault Detection Using Random Testing , 2012, IEEE Transactions on Software Engineering.

[18]  Myra B. Cohen,et al.  Covering arrays for efficient fault characterization in complex configuration spaces , 2004, IEEE Transactions on Software Engineering.

[19]  Myra B. Cohen,et al.  Feedback driven adaptive combinatorial testing , 2011, ISSTA '11.

[20]  Myra B. Cohen,et al.  Reducing Masking Effects in CombinatorialInteraction Testing: A Feedback DrivenAdaptive Approach , 2014, IEEE Transactions on Software Engineering.

[21]  Charles J. Colbourn,et al.  Locating and detecting arrays for interaction faults , 2008, J. Comb. Optim..

[22]  Xi Deng,et al.  Combinatorial Testing on Implementations of HTML5 Support , 2017, 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[23]  Wei Chen,et al.  DangerNeighbor attack: Information leakage via postMessage mechanism in HTML5 , 2019, Comput. Secur..

[24]  Yu Lei,et al.  Applying combinatorial test data generation to big data applications , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[25]  Jeff Offutt,et al.  Input parameter modeling for combination strategies , 2007, ICSE 2007.

[26]  Jun Yan,et al.  Generating combinatorial test suite using combinatorial optimization , 2014, J. Syst. Softw..

[27]  Lucia Moura,et al.  Locating Errors Using ELAs, Covering Arrays, and Adaptive Testing Algorithms , 2009, SIAM J. Discret. Math..

[28]  Robert Mandl,et al.  Orthogonal Latin squares: an application of experiment design to compiler testing , 1985, CACM.

[29]  Hareton K. N. Leung,et al.  Combinatorial testing, random testing, and adaptive random testing for detecting interaction triggered failures , 2015, Inf. Softw. Technol..

[30]  Mathieu Acher,et al.  A variability-based testing approach for synthesizing video sequences , 2014, ISSTA 2014.