Asset Management Measurement for Cyber

Some time ago I wrote about using the Goal-Question-Metric (GQM) method for identifying useful and organisationally relevant measurements in order to have a clear view of some aspect of security.

Often we think about metrics in terms of engaging security colleagues, executives and the board. However, occasionally in distributed organisations, we need to step away from delivery ourselves and provide clear guidance to other teams to deliver to our needs, we become the stakeholders of their projects.

This was the position I was in where I needed to document my requirements for a project to understand the current state of asset management and improve capabilities, where necessary, in the various engineering teams across an organisation in order to understand and improve the cyber posture of the organisation.

To do this I used the GQM methodology to state a clear goal, to identify the questions I would ask to see if those teams were delivering that goal and some indicative measurements or metrics that would answer those questions. Below is a worked example of using GQM for cyber measurement.

My clearly stated goal: “To know the environment accurately in order to accelerate the early stages of cyber incident response and to be able to make confident decisions sooner“.

In order to know we were delivering that goal I identified the following questions I would ask:

  1. Do we know what assets we have?
  2. Do we know enough about these assets in order to be able to analyse their role & potential impact in an incident?
  3. Do we know which of our assets are critical?
  4. Do we have an accurate map of our networks and how they connect to each other?
  5. Do we have an accurate map of gateways into and out of our networks?
  6. Do we have confidence that we still control our networks?
  7. Do we know where are critical assets are on our networks?
  8. Do we have appropriate network controls to limit the blast radius / contagion risk from an incident?
  9. Are we confident we can recover business services and assets following a destructive attack?

In order to know how to answer these questions here are the indicative metrics I identified (I was open to different metrics being proposed by engineering teams but this was a strawman to start that conservation):

Question 1 Do we know what assets we have?

QuestionMetricDefinition
11What % of network devices connected to the network are accurately described in the inventory?
12What % of servers connected to the network are described in the inventory?
13What % of end-user devices connected to the network are described in the inventory?

Question 2 Do we know enough about these assets in order to be able to analyse their role & potential impact in an incident?

QuestionMetricDefinition
21What % of server assets have business services mapped to them?
22What % of assets have MAC addresses recorded within the last week?
23What % of assets have IP Addresses recorded within the last week?
24What % of assets have Hostnames recorded within the last week?
25What % of assets have Owner recorded?
26What % of assets have Prod/PreProd/Dev status recorded?
27What % of assets have Operating System recorded?
28What % of assets have Operating System Version recorded?
29What % of assets have Deployed Software recorded?
210What % of assets have Deployed Software Versions recorded?
211What % of assets have which network services are provided by which software recorded?
212What % of assets have which TCP/UDP ports are in use by which network services recorded?
213What % of assets have encryption at rest configured recorded?
214What % of assets are identified as holding customer data?
215What % of assets are externally-facing?

Question 3 Do we know which of our assets are critical?

QuestionMetricDefinition
31How many critical applications are identified in the inventory?
32How many systems supporting critical application are identified in the inventory?
33What % of systems supporting critical applications are mapped to groups of assets in the inventory?
34What % of assets mapped to systems supporting critical applications have dependencies mapped?

Question 4 Do we have an accurate map of our networks and how they connect to each other?

QuestionMetricDefinition
41Do we have an automatically-generated network map/model?
42Do we have an automatically-generated list of devices connected to our network?

Question 5 Do we have an accurate map of gateways into and out of our networks?

QuestionMetricQuestion
51Do we have an accurate list of Internet gateways?
52Do we have an accurate list of 3rd party connections?
53Do we have an accurate list of remote access gateways?

Question 6 Do we have confidence that we still control our networks?

QuestionMetricDefinition
61How many named network administrator accounts do we have?
62How many generic/shared network administrator accounts do we have?
63How many remote access accounts do we have?
64Is transmitted data kept confidential?
65What controls limit access to our network environment?
66How are those controls configured?
67How are our Internet gateways configured?
68How recently were each of our Internet gateways assessed/tested?
69How many network-connected assets are monitored?
610Can external systems access internal systems directly?
611Do we authenticate access to our network environment?
612How recently were our identified assets assessed for vulnerabilities?
613How many of our assets are patched in line with BU policy/SLA?
614Do we model the security implications of changes to our network flow control rules?
615How do we manage privileged network administrator accounts?
616How many inactive network administrator accounts are there?
617Is there a change control procedure for network infrastructure changes?
618Is there egress logging collected at each Internet gateway?

Question 7 Do we know where are critical assets are on our networks?

QuestionMetricDefinition
71How close are our critical systems to our Internet gateways?
72Do we have appropriate  controls between our critical systems and our Internet gateways?
73Do we have appropriate controls between our critical systems and our less critical systems?
74What controls exist between remote access users and our critical systems?
75How many service accounts exist per host on each critical system?

Question 8 Do we have appropriate network controls to limit the blast radius / contagion risk from an incident?

QuestionMetricDefinition
81Have we implemented robust segmentation models in our network infrastructure?
82What exceptions to a robust network segmentation model have been implemented in our network infrastructure?

Question 9 Are we confident we can recover business services and assets following a destructive attack?

QuestionMetricDefinition
91How long since the last backup of the asset?
92Is there a hot/warm/cold DR capability available for the asset?
93What asset dependencies exist for the backup and recovery processes & systems?
94Are the backup and recovery systems running the same operating system as the systems they backup?

If we were able to answer all of these questions we would have an incredible level of situational awareness but we accepted that we would likely not be able to answer all questions fully or to an acceptable level of rigour but that by answering the questions as best we can we would smoke out systemic issues in our situational awareness, which proved to be true.