Catching Remote Administration Trojans (RATs)

A Remote Administration Trojan (RAT ) allows an attacker to remotely control a computing system and typically consists of a server invisibly running and listening to specific TCP -UDP ports on a victim machine as well as a client acting as the interface between the server and the attacker. The accuracy of host and-or network-based methods often employed to identify RATs highly depends on the quality of Trojan signatures derived from static patterns appearing in RAT programs and-or their communications. Attackers may also obfuscate such patterns by having RATs use dynamic ports, encrypted messages, and even changing Trojan banners. In this paper, we propose a comprehensive framework termed RAT Catcher, which reliably detects and ultimately blocks RAT malicious activities even when Trojans use multiple evasion techniques. Employing network-based methods and functioning in inline mode to inspect passing packets in real time, our RAT Catcher collects and maintains status information for every connection and conducts session correlation to greatly improve detection accuracy. The RAT Catcher re-assembles packets in each data stream and dissects the resulting aggregation according to known Trojan communication protocols, further enhancing its traffic classification. By scanning not only protocol headers but also payloads, RAT Catcher is a truly application-layer inspector that performs a range of corrective actions on identified traffic including alerting, packet dropping, and connection termination. We show the effectiveness and efficiency of RAT Catcher with experimentation in both laboratory and real-world settings. Copyright © 2007 John Wiley & Sons, Ltd.

[1]  Bruce Schneier,et al.  Applied cryptography : protocols, algorithms, and source codein C , 1996 .

[2]  Leonard M. Adleman,et al.  An Abstract Theory of Computer Viruses , 1988, CRYPTO.

[3]  Alex Delis,et al.  A Pragmatic Methodology for Testing Intrusion Prevention Systems , 2009, Comput. J..

[4]  Seth Kulakow NetBus 2.1, is it still a Trojan horse or an actual valid remote control administration tool? , 2001 .

[5]  Harry B. DeMaio Viruses-a management issue , 1989, Comput. Secur..

[6]  R. Jagannathan,et al.  A prototype real-time intrusion-detection expert system , 1988, Proceedings. 1988 IEEE Symposium on Security and Privacy.

[7]  Douglas E. Comer,et al.  Internetworking with TCP/IP - Principles, Protocols, and Architectures, Fourth Edition , 1988 .

[8]  Steven B. Lipner,et al.  Non-Discretionery Controls for Commercial Applications , 1982, 1982 IEEE Symposium on Security and Privacy.

[9]  Dorothy E. Denning,et al.  An Intrusion-Detection Model , 1987, IEEE Transactions on Software Engineering.

[10]  D. Elliott Bell,et al.  Secure Computer System: Unified Exposition and Multics Interpretation , 1976 .

[11]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[12]  David D. Clark,et al.  A Comparison of Commercial and Military Computer Security Policies , 1987, 1987 IEEE Symposium on Security and Privacy.

[13]  Winfried Gleissner,et al.  A mathematical theory for the spread of computer viruses , 1989, Comput. Secur..

[14]  H. S. Teng,et al.  Adaptive real-time anomaly detection using inductively generated sequential patterns , 1990, Proceedings. 1990 IEEE Computer Society Symposium on Research in Security and Privacy.

[15]  Fred Cohen Models of practical defenses against computer viruses , 1989, Comput. Secur..

[16]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[17]  Dug Song,et al.  Nidsbench - a Network Intrusion Detection Test Suite , 1999, Recent Advances in Intrusion Detection.

[18]  Fred Cohen,et al.  Computational aspects of computer viruses , 1989, Comput. Secur..

[19]  Stephen G. Kochan,et al.  Unix System Security , 1986 .

[20]  Fred Cohen,et al.  Computer viruses—theory and experiments , 1990 .

[21]  Matt Bishop A model of security monitoring , 1989, [1989 Proceedings] Fifth Annual Computer Security Applications Conference.

[22]  Eugene H. Spafford,et al.  The design and implementation of tripwire: a file system integrity checker , 1994, CCS '94.

[23]  Tom Duff,et al.  Experience with Viruses on UNIX Systems , 1989, Comput. Syst..

[24]  Rita C. Summers Secure Computing: Threats and Safeguards , 1996 .

[25]  A HughesLorine,et al.  Viruses, Worms, and Trojan Horses , 2007 .

[26]  Robert E. Tarjan,et al.  Self-adjusting binary search trees , 1985, JACM.

[27]  Lance J. Hoffman,et al.  Smoking out the bad actors: Risk analysis in the age of the microcomputer , 1989, Comput. Secur..

[28]  Eugene H. Spafford,et al.  The internet worm program: an analysis , 1989, CCRV.

[29]  Roy T. Fielding,et al.  Hypertext Transfer Protocol - HTTP/1.1 , 1997, RFC.

[30]  L. J. Hoffman Rogue programs: viruses, worms and Trojan horses , 1990 .

[31]  Jerome H. Saltzer,et al.  The protection of information in computer systems , 1975, Proc. IEEE.

[32]  Simson L. Garfinkel,et al.  Practical UNIX Security , 1991 .

[33]  Algirdas Avizienis,et al.  A fault tolerance approach to computer viruses , 1988, Proceedings. 1988 IEEE Symposium on Security and Privacy.

[34]  E. Tavares,et al.  On the security of the CAST encryption algorithm , 1994, 1994 Proceedings of Canadian Conference on Electrical and Computer Engineering.