WHEN JAVA WAS ONE: THREATS FROM HOSTILE BYTE CODE

In Java’s first year it has become clear that many of the problems posed by executable content have not been solved. The almost exclusive focus of the Java community on executable content has left numerous avenues unexplored for threats. It has been observed that there is no one-to-one correspondence between Java source code (programs) and Java byte code (class files). While every program written in Java can be compiled to byte code by a Java compiler, it is possible to create class files which no Java compiler can produce, and yet, which pass the Java Verifier with flying colors. This fact has one very serious implication No matter what claims are made, and even formally demonstrated, for the security of the Java language, all bets are off when it comes to byte code running in the Java Virtual Machine. This paper will explore some of the implications of this curious lack of coherence between Java source code and byte code. It will also illustrate how easy it is to alter Java class files for malicious purposes. 1. THE STATE OF JAVA SECURITY The Java programming language has recently turned one year old. In its first year, Java has had a number of spectacular holes punched in its security model by Ed Felten and the Safe Internet Programming Team at Princeton University [McGF]. Since February of 1996 the Hostile Applets Home Page at the Georgia Institute of Technology’s School of Mathematics has featured a collection of evil applets (with complete source code), and yet Sun Microsystems and its corporate partners have shown little progress in combatting hostile applets [LaD1, LaD2]. A year ago, when Java was first gaining notoriety, few people imagined that so many serious flaws would surface so quickly, and even fewer believed that threats from hostile applets would persist. In Java’s first year it has become clear that many of the problems posed by executable content have not been solved. The power and complexity of the language make it extremely likely that security holes will continue to appear in years to come. It has been observed that there is no one-to-one correspondence between Java source code (programs) and Java byte code (class files) [McGF, LaD4]. While every program written in Java can be compiled to byte code by a Java compiler, it is possible to create class files which no Java compiler WHEN JAVA WAS ONE: THREATS FROM HOSTILE BYTE CODE can produce, and yet, which pass the Java Verifier with flying colors. Such class files are said to be deviant. Not only is it possible to create deviant class files, it is a simple matter to do so, and the number of these noncompiler class files greatly exceeds the number of those producible by Java compilers. This fact has one very serious implication No matter what claims are made, and even formally demonstrated, for the security of the Java language, all bets are off when it comes to byte code running in the Java Virtual Machine. Deviant class files that pass the Verifier and exploit unenforced, or improperly implemented, Verifier rules have the potential to reduce Java Security to rubble. Note that this applies as well to the most untrusted of applets (which are Java programs downloaded and run automatically by most browsers) as it does to applications (which are programs set up and run in more traditional ways). While inadvertently trusting a hostile application can lead to ruin, so can accidentally downloading a hostile applet that exploits the increased power of Java byte code over Java source code. Thus the distinction between applications and applets is unimportant in the present context. Until this new threat posed by Java applets is more fully understood and explored, it is wise to regard applets with more suspicion than ever before. This paper will explore some of the implications of this curious lack of coherence between Java source code and byte code. It will also illustrate how easy it is to alter Java class files for malicious purposes. Section 2 contains an overview of some salient facts about the Java class file format. It highlights the ease with which class files can be altered to become deviant and do things beyond the power of Java source code. Section 3 describes the problem of incoherence between Java source code and byte code. It points out several surprising properties of byte code as well as several rules unenforced by the Java Verifier, all of which could lead to security breaches. Section 4 then introduces a number of examples in order to illustrate the threats. One particularly interesting example that will be considered at length is the application HoseMocha.java, which can be applied to applications and applets, making them impervious to the celebrated Mocha decompiler. Finally, Section 5 recounts recent experience with some rudimentary Java Platform viruses, and it assesses the possibility of more virulent threats from hostile byte code. 2. AN OVERVIEW OF THE JAVA CLASS FILE FORMAT When Java source code is compiled, the result is a class file, having a .class extension and containing platform-independent byte code in a very specific format. A class file should be regarded as a stream of 8-bit bytes, with 16-bit, 32-bit, and 64-bit quantities being constructed in big-endian order from two, four, and eight consecutive 8-bit bytes, respectively. The WHEN JAVA WAS ONE: THREATS FROM HOSTILE BYTE CODE Java Virtual Machine (JVM) Specification represents a class file in a C-like structure notation as follows [Lind]: