Automated pseudo-live testing of firewall configuration enforcement

Network security devices such as firewalls and intrusion detection systems are constantly updated in their implementation to accommodate new features, performance standards and to utilize new hardware optimization. Reliable, yet practical, testing techniques for validating the configuration enforcement after every new software and firmware update become necessary to assure correct configuration realization. Generating random traffic to test the firewall configuration enforcement is not only inaccurate but also impractical as it requires an infeasible number of test cases for a reasonable testing coverage. In addition, in most cases the policies used during testing are manually generated or have limited configuration profiles. We present a framework for automatic testing of the firewall configuration enforcement using efficient and flexible policy and traffic generation. In a typical test session, a large set of different policies are generated based on the access-control list (ACL) grammar and according to custom profiles. Test packets are generated to particularly consider critical segments of the tested policies and to achieve high coverage of the testing space. We also describe our implementation of a fully-automated framework, which includes ACL grammar modeling, the policy generation, test cases generation, capturing and analyzing firewall output, and creating detailed test reports. Our evaluation results show that our security configuration testing is not only achievable but it also offers high coverage with significant degree of confidence.

[1]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[2]  Sally Floyd,et al.  Difficulties in simulating the internet , 2001, TNET.

[3]  Jan Jürjens,et al.  Specification-Based Testing of Firewalls , 2001, Ershov Memorial Conference.

[4]  Amin Vahdat,et al.  Realistic and responsive network traffic generation , 2006, SIGCOMM 2006.

[5]  Avishai Wool.  Architecting the Lumeta Firewall Analyzer , 2001, USENIX Security Symposium.

[6]  Ehab Al-Shaer,et al.  Modeling and verification of IPSec and VPN security policies , 2005, 13TH IEEE International Conference on Network Protocols (ICNP'05).

[7]  Marianne Winslett,et al.  On the Safety and Efficiency of Firewall Policy Deployment , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[8]  Jonathan S. Turner,et al.  ClassBench: A Packet Classification Benchmark , 2005, IEEE/ACM Transactions on Networking.

[9]  Reto E. Haeni.  Firewall Penetration Testing , 1997 .

[10]  Ehab Al-Shaer,et al.  Discovery of policy anomalies in distributed firewalls , 2004, IEEE INFOCOM 2004.

[11]  Paul Barford,et al.  Self-configuring network traffic generation , 2004, IMC '04.

[12]  Ehab Al-Shaer,et al.  An Automated Framework for Validating Firewall Policy Enforcement , 2007, Eighth IEEE International Workshop on Policies for Distributed Systems and Networks (POLICY'07).

[13]  Jin Cao,et al.  Stochastic models for generating synthetic HTTP source traffic , 2004, IEEE INFOCOM 2004.

[14]  Edmund M. Clarke,et al.  Symbolic Model Checking: 10^20 States and Beyond , 1990, Inf. Comput..

[15]  John C. Cherniavsky,et al.  Validation, Verification, and Testing of Computer Software , 1982, CSUR.

[16]  Khalid Al-Tawil,et al.  Evaluation and testing of internet firewalls , 1999, Int. J. Netw. Manag..

[17]  Avishai Wool,et al.  Fang: a firewall analysis engine , 2000, Proceeding 2000 IEEE Symposium on Security and Privacy. S&P 2000.

[18]  Giovanni Vigna,et al.  A Formal Model for Firewall Testing , 2007 .