Evaluation of CAVM, Austin, and CodeScroll for Test Data Generation for C

요 약 본 연구는 새로운 검색 기반 C 언어 테스트 데이터 생성 도구인 CAVM 을 제시한다. CAVM 은 상용화된 테스트 데이터 도구인 CodeScroll 을 향상하기 위해서 개발되었다. CodeScroll 은 정적 분석과 입력 파티셔닝 기법을 사용하여 테스트 데이터를 생성한다. 현재의 최첨단 C 언어용 자동 테스트 데이터 생성 도구인 Austin 과 다르게 CAVM 은 순수하게 검색 기반 기법 만을 사용하여 동적 자료 구조 데이터를 생성한다. 본 연구는 CAVM 을 49 개의 실험 대상 함수를 이용해 Austin 및 CodeScroll 과 비교하였다. 대상 함수는 소규모의 안티 패턴 케이스 및 실제 프로덕트 레벨의 오픈 소스 코드와 상용 코드로부터 수집하였다. 비교 결과 CAVM 은 CodeScroll 과 Austin 이 커버하지 못하는 분기들을 커버할 수 있었으며, 20 개의 대상 함수에서 가장 높은 분기 커버리지를 달성하였다.

[1]  Claire Le Goues,et al.  A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[2]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[3]  Ellis Horowitz,et al.  Fundamentals of Data Structures , 1984 .

[4]  Dirk Sudholt,et al.  Design and analysis of different alternating variable searches for search-based software testing , 2015, Theor. Comput. Sci..

[5]  David L. Spooner,et al.  Automatic Generation of Floating-Point Test Data , 1976, IEEE Transactions on Software Engineering.

[6]  Gordon Fraser,et al.  Does automated white-box test generation really help software testers? , 2013, ISSTA.

[7]  Mark Harman,et al.  AUSTIN: A Tool for Search Based Software Testing for the C Language and Its Evaluation on Deployed Automotive Systems , 2010, 2nd International Symposium on Search Based Software Engineering.

[8]  Shin Yoo,et al.  Evolving Human Competitive Spectra-Based Fault Localisation Techniques , 2012, SSBSE.

[9]  Mark Harman,et al.  Automated software transplantation , 2015, ISSTA.

[10]  John A. Clark,et al.  The GISMOE challenge: constructing the pareto program surface using genetic programming to find better programs (keynote paper) , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[11]  Gordon Fraser,et al.  Whole Test Suite Generation , 2013, IEEE Transactions on Software Engineering.

[12]  Name M. Lastname Automatically Finding Patches Using Genetic Programming , 2013 .

[13]  George C. Necula,et al.  CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs , 2002, CC.

[14]  Phil McMinn,et al.  AVMf: An Open-Source Framework and Implementation of the Alternating Variable Method , 2016, SSBSE.

[15]  Bogdan Korel,et al.  Automated Software Test Data Generation , 1990, IEEE Trans. Software Eng..

[16]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[17]  Paolo Tonella,et al.  Evolutionary testing of classes , 2004, ISSTA '04.

[18]  Myra B. Cohen,et al.  Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection , 2015, IEEE Transactions on Software Engineering.

[19]  Phil McMinn,et al.  IGUANA: Input Generation Using Automated Novel Algorithms. A Plug and Play Research Tool , 2007 .

[20]  Mark Harman,et al.  Genetic and Evolutionary Computation (GECCO) , 2003 .

[21]  Gordon Fraser,et al.  A Memetic Algorithm for whole test suite generation , 2015, J. Syst. Softw..

[22]  Mark Harman,et al.  Testability transformation , 2004, IEEE Transactions on Software Engineering.

[23]  Mark Harman,et al.  A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search , 2010, IEEE Transactions on Software Engineering.

[24]  Alessandro Orso,et al.  F3: fault localization for field failures , 2013, ISSTA.

[25]  Mark Harman,et al.  Optimizing for the Number of Tests Generated in Search Based Test Data Generation with an Application to the Oracle Cost Problem , 2010, 2010 Third International Conference on Software Testing, Verification, and Validation Workshops.