Black penguin: On the feasibility of detecting intrusion with homogeneous memory

Growing complexity in modern software is making signature-based intrusion detection an increasing challenge. Many recent intrusion detection systems rely on accurate recovery of application semantics from memory. In this paper, we approach the problem from a different angle. We observe that the user applications in corporate network often run in identical system environments due to standardized IT deployment procedure. The same applications share similar runtime statistics across different workstations through out the time, despite different uses by the end users. When an application is compromised on one workstation, its runtime profile would be different from the rest, similar to how a black penguin would look distinctly different from the rest of the colony. In this work, we present our preliminary study on Black Penguin, a compare-view based intrusion detection system leveraging homogeneity of application-level memory statistics in corporate environment. The detection system follows a three-step process that includes memory analysis, unsupervised learning and risk mitigation. To explore the feasibility of Black Penguin, we conduct two types of experiments using Internet Explorer and Firefox as target applications. First, we examine the statistical differences of the same application under different user usage. To this end, we collect and analyze memory statistics of browser when visiting the top 500 websites ranked by Moz. Second, we examine the difference when the application is under attack. Several browser attacks are used to generate the intrusion samples. Our preliminary evaluation demonstrates the feasibility of the approach. Lastly, we also provide discussions on the limitations of the proposed system as well as future directions.

[1]  Benjamin Livshits,et al.  NOZZLE: A Defense Against Heap-spraying Code Injection Attacks , 2009, USENIX Security Symposium.

[2]  Xuxian Jiang,et al.  Stealthy malware detection through vmm-based "out-of-the-box" semantic view reconstruction , 2007, CCS '07.

[3]  George Danezis,et al.  Proceedings of the 2012 ACM conference on Computer and communications security , 2012, CCS 2012.

[4]  Benjamin Livshits,et al.  ZOZZLE: Fast and Precise In-Browser JavaScript Malware Detection , 2011, USENIX Security Symposium.

[5]  Kang G. Shin,et al.  Detection of botnets using combined host- and network-level information , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[6]  Guofei Gu,et al.  BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection , 2008, USENIX Security Symposium.

[7]  Xuxian Jiang,et al.  SigGraph: Brute Force Scanning of Kernel Data Structure Instances Using Graph-based Signatures , 2011, NDSS.

[8]  Adam Barth,et al.  The Security Architecture of the Chromium Browser , 2009 .

[9]  Paul Movall,et al.  Linux Physical Memory Analysis , 2005, USENIX Annual Technical Conference, FREENIX Track.

[10]  Ross J. Anderson.  Security engineering - a guide to building dependable distributed systems (2. ed.) , 2001 .

[11]  Niels Provos,et al.  Preventing Privilege Escalation , 2003, USENIX Security Symposium.

[12]  Jeffrey Scott Vitter,et al.  Proceedings of the thirtieth annual ACM symposium on Theory of computing , 1998, STOC 1998.

[13]  Peng Ning,et al.  SEER: practical memory virus scanning as a service , 2014, ACSAC '14.

[14]  R. Sekar,et al.  A practical mimicry attack against powerful system-call monitors , 2008, ASIACCS '08.

[15]  Salvatore J. Stolfo,et al.  On the feasibility of online malware detection with performance counters , 2013, ISCA.

[16]  Leon J. Osterweil,et al.  Data Flow Analysis in Software Reliability , 1976, CSUR.

[17]  Piotr Indyk,et al.  Approximate nearest neighbors: towards removing the curse of dimensionality , 1998, STOC '98.

[18]  Christopher Krügel,et al.  Effective and Efficient Malware Detection at the End Host , 2009, USENIX Security Symposium.

[19]  Stephanie Forrest,et al.  A sense of self for Unix processes , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[20]  David A. Wagner,et al.  Mimicry attacks on host-based intrusion detection systems , 2002, CCS '02.

[21]  Christopher Krügel,et al.  Blacksheep: detecting compromised hosts in homogeneous crowds , 2012, CCS '12.

[22]  Zhongshu Gu,et al.  DSCRETE: Automatic Rendering of Forensic Information from Memory Images via Application Logic Reuse , 2014, USENIX Security Symposium.

[23]  Guofei Gu,et al.  EFFORT: A new host-network cooperated framework for efficient and effective bot malware detection , 2013, Comput. Networks.

[24]  Christopher Krügel,et al.  Accurate Buffer Overflow Detection via Abstract Payload Execution , 2002, RAID.

[25]  Christopher Krügel,et al.  Automating Mimicry Attacks Using Static Binary Analysis , 2005, USENIX Security Symposium.

[26]  Juan Caballero,et al.  FIRMA: Malware Clustering and Network Signature Generation with Mixed Network Behaviors , 2013, RAID.

[27]  Chao Wu,et al.  Discovering Semantic Data of Interest from Un-mappable Memory with Confidence , 2012, NDSS.

[28]  Deian Stefan,et al.  Data-Provenance Verification For Secure Hosts , 2012, IEEE Transactions on Dependable and Secure Computing.

[29]  Salvatore J. Stolfo,et al.  On the infeasibility of modeling polymorphic shellcode , 2009, Machine Learning.

[30]  Calton Pu,et al.  Buffer overflows: attacks and defenses for the vulnerability of the decade , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[31]  Evangelos Kranakis,et al.  DNS-based Detection of Scanning Worms in an Enterprise Network , 2005, NDSS.

[32]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.