Testing for software vulnerability using environment perturbation

Describes a methodology for testing a software system for possible security flaws. Based on the observation that most security flaws are caused by a program's inappropriate interactions with the environment and are triggered by a user's malicious perturbation on the environment (which we call an "environment fault"), we view the security testing problem as the problem of testing for the fault-tolerance properties of a software system. We consider each environment perturbation as a fault, and the resulting security compromise as a failure in the toleration of such faults. Our approach is based on the well-known technique of fault injection. Environment faults are injected into the system under test, and the system's behavior is observed. A failure to tolerate faults is an indicator of a potential security flaw in the system. An environment-application interaction (EAI) fault model is proposed which guides us to decide what faults to inject. Based on EAI, we have developed a security testing methodology, and we have applied it to several applications. We have successfully identified a number of vulnerabilities, including vulnerabilities in the Windows NT operating system.

[1]  Wenliang Du,et al.  Categorization of Software Errors that led to Security Breaches , 1998 .

[2]  Barton P. Miller,et al.  Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services , 1995 .

[3]  Wenliang Du,et al.  Security relevancy analysis on the registry of Windows NT 4.0 , 1999, Proceedings 15th Annual Computer Security Applications Conference (ACSAC'99).

[4]  Matt Bishop,et al.  Checking for Race Conditions in File Accesses , 1996, Comput. Syst..

[5]  Farnam Jahanian,et al.  ORCHESTRA: A Fault Injection Environment for Distributed Systems , 1996 .

[6]  Virgil D. Gligor,et al.  A New Security Testing Method and Its Application to the Secure Xenix Kernel , 1987, IEEE Transactions on Software Engineering.

[7]  HsuehMei-Chen,et al.  Fault Injection Techniques and Tools , 1997 .

[8]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[9]  Farnam Jahanian,et al.  Testing of fault-tolerant and real-time distributed systems via protocol fault injection , 1996, Proceedings of Annual Symposium on Fault Tolerant Computing.

[10]  Carl E. Landwehr,et al.  A taxonomy of computer program security flaws , 1993, CSUR.

[11]  Jeffrey M. Voas,et al.  Predicting How Badly "Good" Software Can Behave , 1997, IEEE Softw..

[12]  Jeffrey M. Voas Testing Software For Characteristics Other Than Correctness: Safety, Failure Tolerance, And Securi , 1996 .

[13]  Jacob A. Abraham,et al.  FERRARI: a tool for the validation of system dependability properties , 1992, [1992] Digest of Papers. FTCS-22: The Twenty-Second International Symposium on Fault-Tolerant Computing.

[14]  Shari Lawrence Pfleeger,et al.  A methodology for penetration testing , 1989, Comput. Secur..

[15]  Barton P. Miller,et al.  An empirical study of the reliability of UNIX utilities , 1990, Commun. ACM.

[16]  Kang G. Shin,et al.  DOCTOR: an integrated software fault injection environment for distributed real-time systems , 1995, Proceedings of 1995 IEEE International Computer Performance and Dependability Symposium.

[17]  Matt Bishop,et al.  A Taxonomy of UNIX System and Network Vulnerabilities , 1997 .

[18]  Karl N. Levitt,et al.  Property-based testing of privileged programs , 1994, Tenth Annual Computer Security Applications Conference.

[19]  Gary McGraw,et al.  An automated approach for identifying potential vulnerabilities in software , 1998, Proceedings. 1998 IEEE Symposium on Security and Privacy (Cat. No.98CB36186).

[20]  Hong Zhu,et al.  Software unit test coverage and adequacy , 1997, ACM Comput. Surv..

[21]  Ravishankar K. Iyer,et al.  FINE: A Fault Injection and Monitoring Environment for Tracing the UNIX System Behavior under Faults , 1993, IEEE Trans. Software Eng..

[22]  Stuart Reid Software fault injection: inoculating programs against errors. By Jeffrey Voas and Gary McGraw. Published by John Wiley & Sons Ltd, New York, 1998. ISBN: 0‐471‐18381‐4, 353 pages. Price: U.K. £39.95, U.S.A. $49.99, including one CD‐ROM, Hard Cover , 1999 .

[23]  Daniel P. Siewiorek,et al.  Automated robustness testing of off-the-shelf software components , 1998, Digest of Papers. Twenty-Eighth Annual International Symposium on Fault-Tolerant Computing (Cat. No.98CB36224).

[24]  Richard R. Linde,et al.  Operating system penetration , 1975, AFIPS '75.

[25]  Dhiraj K. Pradhan,et al.  Fault Injection: A Method for Validating Computer-System Dependability , 1995, Computer.

[26]  E. J. McCAULEY,et al.  KSOS—The design of a secure operating system* , 1979, 1979 International Workshop on Managing Requirements Knowledge (MARK).

[27]  Eugene H. Spafford,et al.  Software vulnerability analysis , 1998 .