Traffic diversity and code coverage: a preliminary analysis

It is generally assumed that using more diverse traffic to test network devices could achieve larger code coverage. However, how to describe the diversity of traffic traces and the relationship between the traffic diversity and code coverage is still an issue. In this paper, the traffic diversity is defined using the number of packets and the size of the subnets involved, and traces having various diversity are used to evaluate the corresponding code coverage for the programs in a network device. Experiment results show that more number of packets or larger size of network segments can generate larger diversity indices and thus larger code coverage. For Snort, as the number of packets increases from 1 to 10,000,000, representative diversity index and the code coverage can increase from 0 to 0.95 on the basis of Simpson's index and from 19.1% to 32.2%, respectively. As the size of network segments increases, representative diversity index and the code coverage can increase from 0.41 to 0.82 and from 28.2% to 32.2%, respectively. Similar results can be obtained in the case of Linux kernel. If the mappings among different diversity indices and the corresponding code coverage can be built beforehand, the quality of the tests can improved. Copyright © 2014 John Wiley & Sons, Ltd.

[1]  Shun-Ren Yang,et al.  A conformance test tool for next generation network applications and systems , 2010, Int. J. Commun. Syst..

[2]  Clifford J. Maloney,et al.  Systematic mistake analysis of digital computer programs , 1963, CACM.

[3]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[4]  A. Rényi.  On Measures of Entropy and Information , 1961 .

[5]  Ying-Dar Lin,et al.  On campus beta site: architecture designs, operational experience, and top product defects , 2010, IEEE Communications Magazine.

[6]  Yuan-Cheng Lai,et al.  Test Coverage Optimization for Large Code Problems , 2012, 2012 26th International Conference on Advanced Information Networking and Applications Workshops.

[7]  E. H. Simpson.  Measurement of Diversity , 1949, Nature.

[8]  Fred G. Evenden,et al.  A Different Kind of Country , 1968 .

[9]  Le Cam,et al.  On some asymptotic properties of maximum likelihood estimates and related Bayes' estimates , 1953 .

[10]  Adam Kolawa,et al.  Automated Defect Prevention , 2007 .

[11]  G. Crooks.  On Measures of Entropy and Information , 2015 .

[12]  Martin R. Woodward,et al.  On the relationship between two control-flow coverage criteria: all JJ-paths and MCDC , 2006, Inf. Softw. Technol..

[13]  Mohamed Ould-Khaoua,et al.  Performance comparison of end‐to‐end and on‐the‐spot traffic‐aware techniques , 2013, Int. J. Commun. Syst..

[14]  Mark Harman,et al.  Regression testing minimization, selection and prioritization: a survey , 2012, Softw. Test. Verification Reliab..

[15]  Shun-Ren Yang,et al.  A conformance test tool for next generation network applications and systems , 2010 .

[16]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[17]  Adam Kolawa,et al.  Automated Defect Prevention , 2007 .

[18]  Yuan-Cheng Lai,et al.  Test Coverage Optimization for Large Code Problems , 2012, 2012 26th International Conference on Advanced Information Networking and Applications Workshops.

[19]  Masood Khosroshahy.  UARA in edge routers: an effective approach to user fairness and traffic shaping , 2012, Int. J. Commun. Syst..

[20]  E. F. Menhinick,et al.  A Comparison of Some Species‐Individuals Diversity Indices Applied to Samples of Field Insects , 1964 .

[21]  Der-Jiunn Deng,et al.  Efficient uplink scheduling policy for variable bit rate traffic in IEEE 802.16 BWA systems , 2012, Int. J. Commun. Syst..

[22]  Christian Callegari,et al.  Skype-Hunter: A real-time system for the detection and classification of Skype traffic , 2012, Int. J. Commun. Syst..

[23]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .