Challenges in IT security preparedness exercises: A case study

Preparedness exercises need to be performed in several different ways.Only one goal should be defined for each exercise, and the design depends highly on this goal.All required competence needs to be included in an IT security preparedness exercise.Post-evaluation highly increases the learning effect from an exercise. The electric power industry is currently implementing major technological changes in order to achieve the goal of smart grids. However, these changes are expected to increase the susceptibility of the industry to IT security incidents. IT security preparedness exercises are not commonly performed in the electric power industry, even though this industry is considered part of society's critical infrastructure. Resolving an IT security incident requires inter-departmental collaborations between various categories of personnel, and to successfully achieve this, training is required. The process of preparing a response to incidents enhances the nature of collaboration, coordination, and communication within an organization. Our objective is to understand the challenges faced when performing IT security preparedness exercises, as challenges experienced during these exercises affect the response process during a real incident. By improving the exercises, the response capabilities would be strengthened accordingly. We have designed a multiple-case study with six teams in three organizations. We collected data by performing semi-structured interviews, participant observations, and from process artifacts. We identified six main challenges involving team composition and external expert involvement, goal definition, documentation, and time management. In summary, there are many ways of conducting preparedness exercises. Therefore, organizations need to both optimize current exercise practices and experiment with new ones in order to ensure continuous learning and improvement; hence, they can be adequately prepared to respond to IT security incidents.