Shellzer: A Tool for the Dynamic Analysis of Malicious Shellcode

Shellcode is malicious binary code whose execution is triggered after the exploitation of a vulnerability. The automated analysis of malicious shellcode is a challenging task, since encryption and evasion techniques are often used. This paper introduces Shellzer, a novel dynamic shellcode analyzer that generates a complete list of the API functions called by the shellcode, and, in addition, returns the binaries retrieved at run-time by the shellcode. The tool is able to modify on-the-fly the arguments and the return values of certain API functions in order to simulate specific execution contexts and the availability of the external resources needed by the shellcode. This tool has been tested with over 24,000 real-world samples, extracted from both web-based drive-by-download attacks and malicious PDF documents. The results of the analysis show that Shellzer is able to successfully analyze 98% of the shellcode samples.

[1]  R. Thayer,et al.  Activation states as assessed by verbal report and four psychophysiological variables. , 1970, Psychophysiology.

[2]  Robert E. Tarjan,et al.  Applications of a planar separator theorem , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[3]  Peter W. Shor,et al.  Algorithms for quantum computation: discrete logarithms and factoring , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[4]  Alexei Y. Kitaev,et al.  Quantum measurements and the Abelian Stabilizer Problem , 1995, Electron. Colloquium Comput. Complex..

[5]  Leonidas J. Guibas,et al.  Visibility-Based Pursuit-Evasion in a Polygonal Environment , 1997, WADS.

[6]  Mark Ettinger,et al.  On Quantum Algorithms for Noncommutative Hidden Subgroups , 2000, Adv. Appl. Math..

[7]  David S. Wise.  Ahnentafel Indexing into Morton-Ordered Arrays, or Matrix Locality for Free , 2000, Euro-Par.

[8]  Anders Lindgren,et al.  Probabilistic routing in intermittently connected networks , 2003, MOCO.

[9]  Ramakrishnan Srikant,et al.  Mining newsgroups using networks arising from social behavior , 2003, WWW '03.

[10]  C. Lomont.  THE HIDDEN SUBGROUP PROBLEM - REVIEW AND OPEN PROBLEMS , 2004, quant-ph/0411037.

[11]  Yoshifumi Inui,et al.  Efficient quantum algorithms for the hidden subgroup problem over semi-direct product groups , 2004, Quantum Inf. Comput..

[12]  Udo Payer,et al.  Hybrid Engine for Polymorphic Shellcode Detection , 2005, DIMVA.

[13]  Edward J. Caropreso,et al.  Effects of Personality on Small Group Communication and Task Engagement in an , 2005 .

[14]  Kevin C. Almeroth,et al.  Delay Tolerant Mobile Networks (DTMNs): Controlled Flooding in Sparse Mobile Networks , 2005, NETWORKING.

[15]  Sampath Kannan,et al.  Randomized pursuit-evasion in a polygonal environment , 2005, IEEE Transactions on Robotics.

[16]  Dave Bacon,et al.  From optimal measurement to efficient quantum algorithms for the hidden subgroup problem over semidirect product groups , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[17]  Evangelos P. Markatos,et al.  Network-level polymorphic shellcode detection using emulation , 2006, Journal in Computer Virology.

[18]  Bradley Taylor,et al.  Sender Reputation in a Large Webmail Service , 2006, CEAS.

[19]  Michael A. Nielsen,et al.  The Solovay-Kitaev algorithm , 2006, Quantum Inf. Comput..

[20]  Jesse D. Kornblum.  Exploiting the Rootkit Paradox with Windows Memory Analysis , 2006, Int. J. Digit. EVid..

[21]  Randy H. Katz,et al.  X-Trace: A Pervasive Network Tracing Framework , 2007, NSDI.

[22]  David Kotz,et al.  Evaluating opportunistic routing protocols with large realistic contact traces , 2007, CHANTS '07.

[23]  Giovanni Vigna,et al.  Multi-module vulnerability analysis of web-based applications , 2007, CCS '07.

[24]  Kevin Borders,et al.  Spector: Automatically Analyzing Shell Code , 2007, Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007).

[25]  Evangelos P. Markatos,et al.  Emulation-Based Detection of Non-self-contained Polymorphic Shellcode , 2007, RAID.

[26]  Giovanni Vigna,et al.  Swaddler: An Approach for the Anomaly-Based Detection of State Violations in Web Applications , 2007, RAID.

[27]  Pavel Laskov,et al.  Detection of Intrusions and Malware, and Vulnerability Assessment: 19th International Conference, DIMVA 2022, Cagliari, Italy, June 29 –July 1, 2022, Proceedings , 2022, International Conference on Detection of intrusions and malware, and vulnerability assessment.

[28]  Craig H. Martell,et al.  Topic Detection and Extraction in Chat , 2008, 2008 IEEE International Conference on Semantic Computing.

[29]  Chris Kanich,et al.  On the Spam Campaign Trail , 2008, LEET.

[30]  John R. Gilbert,et al.  Parallel sparse matrix-vector and matrix-transpose-vector multiplication using compressed sparse blocks , 2009, SPAA '09.

[31]  Morten Fjeld,et al.  Mixed Reality: A Survey , 2009, Human Machine Interaction.

[32]  Ivan Simecek.  Sparse Matrix Computations Using the Quadtree Storage Format , 2009, 2009 11th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[33]  Chen Lin,et al.  Simultaneously modeling semantics and structure of threaded discussions: a sparse coding approach and its applications , 2009, SIGIR.

[34]  Renato Portugal,et al.  Solutions to the Hidden Subgroup Problem on Some Metacyclic Groups , 2009, TCQ.

[35]  Richard Wolski,et al.  The Eucalyptus Open-Source Cloud-Computing System , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[36]  Abhishek Singh,et al.  Identifying Malicious Code Through Reverse Engineering , 2009, Advances in Information Security.

[37]  Ravi Kumar,et al.  Dynamics of conversations , 2010, KDD.

[38]  Dong Xuan,et al.  Malicious Shellcode Detection with Virtual Memory Snapshots , 2010, 2010 Proceedings IEEE INFOCOM.

[39]  Christopher Krügel,et al.  Toward Automated Detection of Logic Vulnerabilities in Web Applications , 2010, USENIX Security Symposium.

[40]  Christopher Krügel,et al.  Detection and analysis of drive-by-download attacks and malicious JavaScript code , 2010, WWW '10.

[41]  Michael S. Bernstein,et al.  Enhancing directed content sharing on the web , 2010, CHI.

[42]  Evangelos P. Markatos,et al.  Comprehensive shellcode detection using runtime heuristics , 2010, ACSAC '10.

[43]  Chris Kanich,et al.  Botnet Judo: Fighting Spam with Itself , 2010, NDSS.

[44]  Subhash Suri,et al.  Complete Information Pursuit Evasion in Polygonal Environments , 2011, AAAI.

[45]  Xiaowei Li,et al.  BLOCK: a black-box approach for detection of state violation attacks towards web applications , 2011, ACSAC '11.

[46]  Paramvir Bahl,et al.  Augmenting data center networks with multi-gigabit wireless links , 2011, SIGCOMM.

[47]  Christopher Krügel,et al.  Escape from Monkey Island: Evading High-Interaction Honeyclients , 2011, DIMVA.

[48]  Giovanni Vigna,et al.  Prophiler: a fast filter for the large-scale detection of malicious web pages , 2011, WWW.

[49]  Gianluca Stringhini,et al.  Hit 'em where it hurts: a live security exercise on cyber situational awareness , 2011, ACSAC '11.

[50]  Long Jiang,et al.  User-level sentiment analysis incorporating social networks , 2011, KDD.

[51]  Gianluca Stringhini,et al.  The Underground Economy of Spam: A Botmaster's Perspective of Coordinating Large-Scale Spam Campaigns , 2011, LEET.

[52]  Volkan Isler,et al.  Capturing an Evader in a Polygonal Environment with Obstacles , 2011, IJCAI.

[53]  John R. Gilbert,et al.  The Combinatorial BLAS: design, implementation, and applications , 2011, Int. J. High Perform. Comput. Appl..

[54]  Rachit Mathur,et al.  PREDICTING THE FUTURE OF STEALTH ATTACKS , 2011 .

[55]  Geoffrey A. Hollinger,et al.  Search and pursuit-evasion in mobile robotics , 2011, Auton. Robots.

[56]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[57]  Sushil Jajodia,et al.  NSDMiner: Automated discovery of Network Service Dependencies , 2012, 2012 Proceedings IEEE INFOCOM.

[58]  David L. Johnson,et al.  VillageCell: cost effective cellular connectivity in rural areas , 2012, ICTD.

[59]  Ralf Herbrich,et al.  De-Layering Social Networks by Shared Tastes of Friendships , 2012, ICWSM.

[60]  Tobias Höllerer,et al.  I’m feeling LoCo: A Location Based Context Aware Recommendation System , 2012 .

[61]  James Fogarty,et al.  Regroup: interactive machine learning for on-demand group creation in social networks , 2012, CHI.

[62]  Ben Y. Zhao,et al.  Mirror mirror on the ceiling: flexible wireless links for data centers , 2012, SIGCOMM '12.

[63]  John R. Gilbert,et al.  A Flexible Open-Source Toolbox for Scalable Complex Graph Analysis , 2012, SDM.

[64]  D. M. Clark.  Theory of Groups , 2012 .