Benefits and Pitfalls of Using Capture the Flag Games in University Courses

The concept of Capture the Flag (CTF) games for practicing cybersecurity skills is widespread in informal educational settings and leisure-time competitions. However, it is not much used in university courses. This paper summarizes our experience from using jeopardy CTF games as homework assignments in an introductory undergraduate course. Our analysis of data describing students' in-game actions and course performance revealed four aspects that should be addressed in the design of CTF tasks: scoring, scaffolding, plagiarism, and learning analytics capabilities of the used CTF platform. The paper addresses these aspects by sharing our recommendations. We believe that these recommendations are useful for cybersecurity instructors who consider using CTF games for assessment in university courses and developers of CTF game frameworks.

[1]  David Brumley,et al.  PicoCTF: A Game-Based Computer Security Competition for High School Students , 2014, 3GSE.

[2]  Manu Kapur,et al.  Designing for Productive Failure , 2012 .

[3]  David H. Tobey,et al.  An Argument for Game Balance: Improving Student Engagement by Matching Difficulty Level with Learner Readiness , 2014, 3GSE.

[4]  Tom Chothia,et al.  An Offline Capture The Flag-Style Virtual Machine and an Assessment of Its Value for Cybersecurity Education , 2015 .

[5]  John Aycock,et al.  Exercises for teaching reverse engineering , 2018, ITiCSE.

[6]  Nickolai Zeldovich,et al.  Experiences in Cyber Security Education: The MIT Lincoln Laboratory Capture-the-Flag Exercise , 2011, CSET.

[7]  Jens Mache,et al.  Finding the Balance Between Guidance and Independence in Cybersecurity Exercises , 2016, ASE @ USENIX Security Symposium.

[8]  Jelena Mirkovic,et al.  Class Capture-the-Flag Exercises , 2014, 3GSE.

[9]  Yanick Fratantonio,et al.  Ten Years of iCTF: The Good, The Bad, and The Ugly , 2014, 3GSE.

[10]  Diana L. Burley,et al.  Engaging learners in cybersecurity careers: lessons from the launch of the national cyber league , 2014, INROADS.

[11]  Giovanni Vigna,et al.  Shell We Play A Game? CTF-as-a-service for Security Education , 2017, ASE @ USENIX Security Symposium.

[12]  Pablo Arias,et al.  CTF: State-of-the-Art and Building the Next Generation , 2017, ASE @ USENIX Security Symposium.

[13]  Kevin Chung,et al.  Learning Obstacles in the Capture The Flag Model , 2014, 3GSE.

[14]  Chris Eagle.  Computer Security Competitions: Expanding Educational Outcomes , 2013, IEEE Security & Privacy.

[15]  Keith Kirkpatrick.  Coding as sport , 2016, Commun. ACM.

[16]  Richard E. Clark,et al.  Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching , 2006 .

[17]  Wu-chang Feng.  A Scaffolded, Metamorphic CTF for Reverse Engineering , 2015 .

[18]  Kevin Chung.  Live Lesson: Lowering the Barriers to Capture The Flag Administration and Participation , 2017, ASE @ USENIX Security Symposium.

[19]  Kees Leune,et al.  Using Capture-the-Flag to Enhance the Effectiveness of Cybersecurity Education , 2017, SIGITE.

[20]  Zachary N. J. Peterson,et al.  The Outcomes of Cybersecurity Competitions and Implications for Underrepresented Populations , 2016, IEEE Security & Privacy.

[21]  David Brumley,et al.  Automatic Problem Generation for Capture-the-Flag Competitions , 2015 .