A Software Based Approach for Trusted Agent Execution on Malicious Host

Absolute protection of mobile agents from attacks by malicious hosts is an open research problem. We propose a software based paradigm whereby an agent is protected from various static and dynamic attacks from a malicious host of an unknown hardware configuration, for a specific period of time. This time interval is computed by restricting the maximum resources that may be available to the adversary and the time complexity of the critical static and dynamic attacks that it may launch. We employ the technique of oblivious hashing (OH) using overlapped instructions [1], with multilevel pointer aliasing to thwart static analysis and instant code modifications. The host is required to obtain the aggregate OH value of the whole agent by executing it in an unobtrusive environment and to send it back to the agent originator within the specified time interval. To provide unobtrusive environment, we employ external timing analysis to detect major dynamic attack tools such as debuggers, virtual machines and emulators. Experimental results are presented that demonstrate the viability of the timing analysis mechanism in detecting dynamic attack tools on a range of Intel based machines.

[1]  Paul C. van Oorschot,et al.  A generic attack on checksumming-based software tamper resistance , 2005, 2005 IEEE Symposium on Security and Privacy (S&P'05).

[2]  Hyunsoo Yoon,et al.  Tamper Resistant Software by Integrity-Based Encryption , 2004, PDCAT.

[3]  Jack W. Davidson,et al.  Protection of software-based survivability mechanisms , 2001, 2001 International Conference on Dependable Systems and Networks.

[4]  Atsuko Miyaji,et al.  Software Obfuscation on a Theoretical Basis and Its Implementation , 2003, IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences.

[5]  G. Ramalingam,et al.  The undecidability of aliasing , 1994, TOPL.

[6]  David Aucsmith,et al.  Tamper Resistant Software: An Implementation , 1996, Information Hiding.

[7]  Ramarathnam Venkatesan,et al.  Towards integral binary execution: implementing oblivious hashing using overlapped instruction encodings , 2007, MM&Sec.

[8]  Amit Vasudevan,et al.  Stealth breakpoints , 2005, 21st Annual Computer Security Applications Conference (ACSAC'05).

[9]  William Landi,et al.  Undecidability of static analysis , 1992, LOPL.

[10]  John M. Brooke,et al.  Towards Time Limited Secure Agent Execution on Malicious Host: A Concept Paper , 2008, 2008 IFIP International Conference on Network and Parallel Computing.

[11]  Ramarathnam Venkatesan,et al.  Oblivious Hashing: A Stealthy Software Integrity Verification Primitive , 2002, Information Hiding.

[12]  Koen De Bosschere,et al.  Run-Time Randomization to Mitigate Tampering , 2007, IWSEC.

[13]  Barbara G. Ryder,et al.  Pointer-induced aliasing: a problem classification , 1991, POPL '91.