Gamifying ICS Security Training and Research: Design, Implementation, and Results of S3

Our work considers the challenges related to education and research about the security of industrial control systems (ICS). We propose to address those challenges through gamified security competitions. Those competitions should target a broad range of security professionals (e. g., from academia and industry). Furthermore, they should involve both attack and defense components. This could include the development of new attack techniques and evaluation of novel countermeasures. Our gamification idea resulted in the design and implementation of the SWaT Security Showdown (S3). S3 is a Capture-The-Flag event specifically targeted at Industrial Control Systems security. We developed ICS-specific challenges involving both theoretical and applied ICS security concepts. The participants had access to a real water treatment facility and they interacted with simulated components and ICS honeypots. S3 includes international teams of attackers and defenders both from academia and industry. It was conducted in two phases. The online phase (a jeopardy-style capture the flag event) served as a training session and presented novel categories not found in traditional information security CTFs. The live phase (an attack-defense CTF) involved teams testing new attack and defense techniques on SWaT: our water treatment testbed. During the competition we acted as judges, and we assigned points to the attacker teams according to a scoring system that we developed internally. Our scoring system is based on multiple factors, including realistic ICS attacker models and effectiveness of the detection mechanisms of the defenders. For each phase of the S3 we present the results and relevant statistics derived from the data that we collected during the event.

[1]  John Viega,et al.  Defcon Capture the Flag: defending vulnerable code from intense attack , 2003, Proceedings DARPA Information Survivability Conference and Exposition.

[2]  Alvaro A. Cárdenas,et al.  Attacks against process control systems: risk assessment, detection, and response , 2011, ASIACCS '11.

[3]  Nils Ole Tippenhauer,et al.  On Attacker Models and Profiles for Cyber-Physical Systems , 2016, ESORICS.

[4]  Nick McKeown,et al.  A network in a laptop: rapid prototyping for software-defined networks , 2010, Hotnets-IX.

[5]  Nils Ole Tippenhauer,et al.  HAMIDS: Hierarchical Monitoring Intrusion Detection System for Industrial Control Systems , 2016, CPS-SPC '16.

[6]  Giovanni Vigna,et al.  Organizing Large Scale Hacking Competitions , 2010, DIMVA.

[7]  Nickolai Zeldovich,et al.  Experiences in Cyber Security Education: The MIT Lincoln Laboratory Capture-the-Flag Exercise , 2011, CSET.

[8]  Miguel Grinberg,et al.  Flask web development , 2014 .

[9]  Floris. A. Schoenmakers.  Contradicting paradigms of control systems security: how fundamental differences cause conflicts , 2013 .

[10]  Andrew Ruef,et al.  Build It, Break It, Fix It: Contesting Secure Development , 2016, CCS.

[11]  Martin Mink,et al.  Evaluation of the Offensive Approach in Information Security Education , 2010, SEC.

[12]  Sridhar Adepu,et al.  Distributed Detection of Single-Stage Multipoint Cyber Attacks in a Water Treatment Plant , 2016, AsiaCCS.

[13]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[14]  Eric A. M. Luiijf.  Cyber (In-)security of Industrial Control Systems: A Societal Challenge , 2015, SAFECOMP.

[15]  Derek Harp,et al.  The State of Security in Control Systems Today , 2015 .

[16]  Giovanni Vigna.  Teaching Network Security Through Live Exercises , 2003, World Conference on Information Security Education.

[17]  Bruno Sinopoli,et al.  Detecting integrity attacks on control systems using robust physical watermarking , 2014, 53rd IEEE Conference on Decision and Control.

[18]  Sridhar Adepu,et al.  Argus: An Orthogonal Defense Framework to Protect Public Infrastructure against Cyber-Physical Attacks , 2016, IEEE Internet Computing.

[19]  H.A.M. Luiijf,et al.  Cyber Security of Industrial Control Systems , 2015 .

[20]  Nils Ole Tippenhauer,et al.  MiniCPS: A Toolkit for Security Research on CPS Networks , 2015, CPS-SPC@CCS.

[21]  Dan Boneh,et al.  Webseclab Security Education Workbench , 2010, CSET.

[22]  Nils Ole Tippenhauer,et al.  Towards High-Interaction Virtual ICS Honeypots-in-a-Box , 2016, CPS-SPC '16.

[23]  John W. Rice,et al.  The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[24]  Nils Ole Tippenhauer,et al.  SWaT: a water treatment testbed for research and training on ICS security , 2016, 2016 International Workshop on Cyber-physical Systems for Smart Water Networks (CySWater).

[25]  Connie Malamed,et al.  Book Review: 'The Gamification of Learning and Instruction: Game-Based Methods and Strategies For Training And Education' by Karl Kapp , 2012, ELERN.

[26]  Jill Slay,et al.  Lessons Learned from the Maroochy Water Breach , 2007, Critical Infrastructure Protection.