In 2012, the European Network and Information Security Agency carried out research and analysis of national and international cyber-exercises. Udo Helmbrecht, executive director, summarises the findings, and outlines seven European-wide policy recommendations that firms in the banking industry and financial services would benefit from.
Cyber-exercises are an important tool to assess the preparedness of a community against cyber-crises, technology failures and critical information infrastructure incidents. They enable competent authorities to target specific weaknesses, increase cooperation across the critical information infrastructure sector, identify interdependencies, stimulate improvements in continuity planning and generate a culture of cooperative effort to boost resilience in the cyber-crisis cooperation area.
In 2009, EU agency the European Network and Information Security Agency (ENISA) issued a recommendation about the importance of cyber-exercises. Since then, the organisation has continued to support the stakeholders involved in cyber-exercises in Europe, and 2012's stocktaking report is one of its efforts to enhance this area, which aims to support the European and international cyber-exercises community with lessons and recommendations for the future.
ENISA examined 85 exercises from 2002 to 2012; 84 countries worldwide participated in the multinational exercises analysed in the report and 22 European countries conducted national cyber-exercises.
First European CIIP exercise
Supporting EU-wide cybersecurity preparedness exercises is one of the main items on the Digital Agenda for Europe, the new policy plan of the European Commission that emphasises the need for EU Member States to carry out large-scale attack simulations and test mitigation strategies in cooperation with the Commission. ENISA's newly proposed mandate highlights the significance of cybersecurity preparedness exercises in enhancing trust and confidence in online services across Europe, as well as the exchange of good practices in this area.
In this policy context, Cyber Europe 2010, the first pan-European exercise on Critical Information Infrastructure Protection (CIIP), was conducted. Built on a scenario concerning internet resilience, the exercise helped to increase trust and test the communication efficiency between the participating member states and the European Free Trade Association.
A Commission communication in 2011 on CIIP again stressed the importance of cyber-exercises for a coherent strategy for cyber-incident contingency planning and recovery at both national and European level. There is therefore an increasing appreciation of exercises as a means of validating CIIP resilience and improving stakeholder communication. As such, the cycle is now in full motion after the completion of the first joint EU-US CIIP exercise, Cyber Atlantic 2011, and of the second pan-European cyber-exercise Cyber Europe 2012.
ENISA research method
In this new research, ENISA focused on the way exercises are executed at the national or multinational level and in the private, public or combined sectors. Information was gathered from online sources and available relevant literature and a survey was then developed. ENISA invited stakeholders in the global cyber-exercise community to respond.
The final step was the organisation of the 1st International Conference on Cyber Crisis Cooperation: Cyber Exercises in June, which focused on objectives to:
- exchange good practices in the field of international cyber-crisis cooperation, specifically focusing on cyber-exercises
- bring together the stakeholders that organise and have experience in cyber-exercises in order to explore cooperation between them
- identify gaps and challenges in the field of international cyber-crisis cooperation and, in particular, cyber-exercises.
Figure 1 shows the number of cyber-exercises per year. The majority in this stocktaking, around 71%, were conducted in the past three years and reflect the seriousness with which governments and private organisations take cyber-threats. Based on the trend observed, the number of cyber-exercises is expected to increase.
Approximately two thirds of the exercises were national and a third multinational. This indicates a tendency toward cooperation at the international level, even though matters of national security are usually domestic concerns. The cross-border nature of cyber-threats gives rise to the need for international cooperation. Based on these results, the trend of a growing number of multinational exercises will continue.
Public and private participation
Another interesting aspect is the participation of the public and private sectors. More than half (57%) of the exercises (based on 88% of overall data gathered) combined the public and private sector, while 41% involved only the public sector. Only one exercise in this stocktaking took place with only the private sector involved, demonstrating that the private sector could be more proactive with testing security and contingency plans as they are the owners of the infrastructure and the experts.
Public-private cooperation occurs in more than half of the exercises, which is attributed to the fact that private stakeholders play a critical role in the area of cyber-crisis cooperation. As such, public-private cooperation in is likely to increase in the coming years.
Many kinds of exercises exist, each with different formats, benefits, challenges and costs. There is no international standard taxonomy of exercise types, although there are several commonly used terms and categories. The simplest forms of exercise are the 'desk check' and 'walk through' exercises that use a simple scenario to validate a plan or procedure to ensure that the participants are able to meet the requirements of the organisation. The most complex are 'full simulation exercises' where players experience the pressure of working in real time responding to an unfolding scenario.
Some 43% of the cyber-exercises (based on 61% of overall data gathered; the relevant data was not available for the remaining 39%) were executed as distributed tabletop exercises (for example, players remain in their usual place of work within their organisation/country), 19% were full simulation exercises and 5% took the form of a workshop.
Cyber-exercise objectives
Raising awareness and building trust are important objectives of cyber-exercises, as are procedures, plans, protocols, capabilities and players. Around half of the exercises (based on 25% of overall data gathered) made use of exercise management tools (such as tools and software to support preparation, execution and evaluation of an exercise); however, three quarters of all exercises gathered did not provide data about the exercise management tools. This could be either because they do not use tools or because the information about them could be made public.
Research also shows there's a need for structured evaluation in order to improve the learning of participants in exercises. Looking only at outcomes of an exercise tends to undermine its aims, is generally unfair to participants and encourages risk-avoiding behaviour. The focus should be on process characteristics that enhance the effectiveness of crisis management. Monitoring and evaluation tools help to structure feedback and formulate lessons learned.
The results of the ENISA stocktaking show that 31% of the exercises (based on 24% of overall data gathered) conducted real-time monitoring, 22% worked with status reports and 27% employed experts to monitor the exercise. Most of these approaches are not used exclusively and there are many exercises that employ a combination of the different methods.
