Cyber Exercises are a powerful and valuable tool but it is easy to confuse what we mean.
I was a member of the Scenario Design Group for the Bank of England’s Waking Shark 2 cyber exercise this year. It was a fascinating experience, seeing how the top cyber/technology risk people at the banks view a massive cyber attack, what really concerns them as well as seeing the regulators and other government agencies engaging with industry.
Waking Shark 2 garnered a lot of headlines but little of real meat made it to the public domain. I signed up to the participants non-disclosure agreement so I won’t be adding any details here. There will be a publicly published report from the Bank of England for that soon enough.
What I have found interesting is the parallel, unprecedented and increasing levels of demand for cyber exercises from clients in the financial services industry. Among the financial institutions I’ve spoken to there has been a serious focus on critical incident management. As a result good, experienced and exercised critical incident management teams exist. I suspect the focus had been due to regulator scrutiny following the RBS failure in 2012. However, it was also clear that few had run cyber exercises before and under increasing regulator scrutiny of their cyber arrangements they wish to not only find any issues but provide some evidence they have been looking.
It was also evident that some clients weren’t entirely clear what they were asking for. After discussions and explanatons on all sides it has tended to boil down to three different activities as follows:
- A Cyber War Game for the Board
- A Cyber Exercise for the Critical Incident Management Team
- A Cyber Simulation for the Network Defenders
The goals, the time available, the method of delivery and the content of all of these are different but they share enough of the subject to be confusing.
A War Game is literally a game. A shorter activity where the decidedly non-technical board members are assigned roles , the content is fairly generic, the game is a structured and guided exercise through a story and the purpose is awareness and education rather than truly testing the participants. It needs to be fun and engaging and relies heavily on strong facilitators who understand their audience (the Board)
A Cyber Exercise is a longer table top activity where a group of critical incident managers, likely consisting of mostly non-cyber specialists with some specialists to hand are presented with a series of events (or ‘injects’) that they respond to in their existing roles, using their existing processes. These are much more free form than a game but are still guided through an overall story. The outcome is a test of the processes and the people to identify weaknesses that can be addressed to better prepare for the real thing.
A Cyber Simulation is long running real world activity where an active adversary performs simulated attacks on the live systems and the fully technical specialist network defender attempts to detect, investigate and respond to those attacks. This isn’t penetration testing, I’ve discussed the weaknesses of penetration testing before, this requires simulating a real world attack and so cannot be as noisy or as fast as a penetration tends to be. There is still a role for onsite guidance as should the defenders fully fail to detect or investigate an event during the simulation they may need to be ‘prompted’ by an inject describing what they missed in order to advance the simulation into later stages. The goal of this is to get a sense of what would actually happen if the organisation came under attack and to get a feel for strengths and weaknesses of the operational security capabilities.
Confusing the needs of the different participants can involve mixing people together such as network defenders and board members who will take turns being bored and irrelevant as the exercise focuses on other areas of interest. Similarly not understanding the required outcomes can mean that the exercise achieves none of them.
It’s also worth bearing in mind that as the exercise type becomes more detailed, closer to the real world and more directly relevant to the participants (War Game -> Exercise -> Simulation) then the costs of development and facilitation rise.
In my experience Cyber Exercises are a very valuable and often an enjoyable addition to awareness and training activities.