Cyber Risk Remediation Analysis
Definition: Cyber Risk Remediation Analysis (RRA) is a methodology for selecting countermeasures to reduce a cyber-asset's susceptibility to cyber-attack over a range of attack Tactics, Techniques, and Procedures (TTPs) associated with the Advanced Persistent Threat (APT). In the Mission Assurance Engineering (MAE) methodology, RRA is a follow-on to cyber Threat Susceptibility Analysis (TSA) and provides recommendations to sponsors seeking to reduce susceptibility to cyber-attack.
Keywords: advanced persistent threat, APT, CM, countermeasure, cyber-attack, MAE, mission assurance engineering, risk Remediation, RRA, threat susceptibility, TSA, TTP, recommendation, utility-cost ratio, U/C ratio
MITRE SE Roles & Expectations: MITRE systems engineers (SEs) are expected to understand the purpose and role of Mission Assurance Engineering (MAE) in the systems acquisition life cycle and its constituent activities, including cyber Risk Remediation Analysis (RRA). The MAE methodology has application throughout the system life cycle. MITRE SEs are expected to know the context(s) in which this methodology can be applied. They are also expected to help establish the scope of the assessment and set sponsor expectations regarding deliverables and schedules.
Introduction and Background
The MAE process framework is depicted in Figure 1. This framework provides an analytical approach to:
- Identify the cyber-assets most critical to mission accomplishment (the "crown jewels" of a Crown Jewels Analysis).
- Understand the threats and associated risks to those assets (accomplished via a subsequent cyber Threat Susceptibility Analysis (TSA)).
- Select mitigation measures to prevent and/or fight through attacks (cyber Risk Remediation Analysis (RRA) is used to identify recommended mitigation measures).
The MAE process framework provides a common repeatable risk management process that is part of building secure and resilient systems .
Cyber risk remediation analysis (RRA) is the final step in the MAE process framework. It is a methodology for selecting countermeasures (CMs) to reduce a cyber-asset's susceptibility to cyber-attack over a range of tactics, techniques, and procedures (TTPs) associated with the APT. A CM is defined as an action, device, procedure, or technique that opposes or counters a threat, a vulnerability, or an attack by eliminating or preventing it, by minimizing the harm it can cause, or by detecting and reporting it so that corrective action can be taken . The selection of CMs is governed by the system life cycle of the cyber-asset being evaluated. Recommended CMs are those judged to be effective at mitigating TTPs to which a cyber-asset may be susceptible. CMs cover a broad spectrum, including changes to requirements, system design, testing, deployment configuration, and/or operating procedures.
This article focuses on what we know today about cyber RRA, a fast-moving branch of systems engineering. The concepts and methods described are evolving and will continue to mature as more experience is gained in applying this discipline. Please revisit this article for additional insights as the community's collective knowledge builds.
The Advanced Persistent Threat (APT) refers to an adversary with sophisticated levels of expertise and significant resources that can apply multiple, different attack vectors to achieve its objectives, which include the establishment of footholds within the information technology infrastructure of an organization to continually exfiltrate information and/or undermine or impede critical aspects of a mission, program, or organization, or to place itself in a position to do so in the future . The APT pursues its objectives over an extended period of time, adapts to a defender's efforts to resist it, and maintains the level of interaction needed to execute its objectives.
Cyber RRA is a follow-on to a cyber Threat Susceptibility Analysis (TSA), which produces a Threat Susceptibility Matrix that ranks TTPs and maps them to cyber-assets. In a TSA assessment, a scoring model spreadsheet may be used to rank TTPs on a risk scale of [1..5], with 1 representing very low risk and 5 representing very high risk. Factors used in the TTP risk scoring spreadsheet can vary from one assessment to the next, but must be uniformly applied across all TTPs evaluated in an assessment to ensure consistent ranking. This scoring tool can be tailored, (e.g., add or remove criteria, modify weightings) or even replaced to meet a program's needs. The interested reader is referred to [4, 5] for details on TSA and the default TTP risk scoring model.
The first step in cyber RRA is to use the Threat Susceptibility Matrix to identify which TTPs to mitigate for each cyber-asset. There are several strategies for performing this selection. One strategy is to focus only on the highest risk TTPs of each cyber-asset. Another strategy is to focus on the cyber-asset(s) that have the highest aggregate susceptibility. Aggregate susceptibility is calculated for each cyber-asset and category of threat actor by summing the risk scores of the mapped TTPs. Note that these calculations use rounded risk scores and will be subject to rounding errors. A third strategy is for RRA to focus exclusively on crown jewel cyber-assets. A hybrid approach might select high-risk TTPs for the crown jewel cyber-assets with the highest aggregate susceptibility. Whatever strategy is used, the result will be a list of TTPs for each cyber-asset assessed.
Figure 2 provides a notional example of a Threat Susceptibility Matrix for two cyber-assets: cyber-asset #1 and cyber-asset #2. In this example, both assets are judged to be essentially equally susceptible to high-risk TTPs T000017 and T000030. Overall, cyber-asset #2 is more susceptible than cyber-asset #1 to a range of TTPs, as reflected by it higher aggregate susceptible scores. The color coding indicates the relative severity of the threat.
Because different cyber-assets are susceptible to different TTPs, cyber RRAs are conducted separately for each cyber-asset. The RRA uses a mapping table that associates TTPs with CMs. A sample TTP/CM mapping table is illustrated in Figure 3.
Each row in a TTP/CM mapping table corresponds to a countermeasure and each column corresponds to a TTP. A CM to TTP mapping is characterized by the mitigation effectiveness of the CM over a range of criteria: detect, neutralize, limit, and recover. Detect CMs serve to identify or uncover the action or presence of a TTP. Neutralize CMs stop or prevent execution of a TTP. Limit CMs serve to reduce or constrain the risk associated with a TTP, either by lessening the severity or likelihood. Recovery CMs facilitate recovery from attack. A given CM may be highly effective at detecting a certain TTP, moderately effective at neutralizing or limiting its impact, but provide no mitigation value in recovering from its effects. A 2-character notation is used to denote mitigation effectiveness within the mapping table, where the first character signifies the type of mitigation from the list: (N)eutralize, (D)etect, (L)imit and (R)ecover, and the second character represents the degree of effectiveness from the list: (L)ow, (M)edium, (H)igh, and (V)ery high. The value NH represents Neutralize-High mitigation effectiveness, while the value DM represents Detect-Medium mitigation effectiveness.
The RRA seeks to identify a list of CMs that mitigate a list of TTPs based on this mapping table. This list will be optimal if it identifies CMs that are highly effective at mitigating most or all of the listed TTPs at a minimal cost. One key assumption made with this approach is that CMs can be applied in combination to achieve greater mitigation effectiveness than they would individually, which is the basis for the onion model of security.
To identify an optimal list of CMs, it is first necessary to assess the relative merit of each CM. One approach, detailed below, is to calculate the utility/cost (U/C) ratio for each CM. A U/C ratio is a "bang-for-buck" valuation of a CM derived from its estimated utility and cost. With the default scoring model, CM utility is estimated by assigning a numeric score to each mitigation effectiveness value and multiplying by the number of mappings containing that mitigation effectiveness value across the list of TTPs being assessed. Once U/C ratios are calculated for each CM, the CM Ranking table is sorted by descending U/C ratios. This approach for calculating U/C rations is depicted in Figure 4.
The next step is to walk through the ranking table to identify sets of CMs that mitigate the list of TTPs, starting at the top and working down. Ordering CMs in the ranking table by descending U/C ratio facilitates early identification of optimal solutions. When a combination of CMs is identified that provides mitigation value over the range of TTPs, a solution effectiveness table is constructed to illustrate the coverage provided and to calculate the relative cost for that solution. These solution effectiveness tables are used to compare alternatives solutions. Figures 5 and 6 represent two alternative solutions that provide roughly equivalent mitigation over the same list of TTPs but at different costs.
In addition to providing a basis for comparing alternative solutions, solution effectiveness tables document the mapping between each TTP and the CMs that provide mitigation value. They can be used to identify cases where mitigation redundancies or coverage gaps exist. For example, additional countermeasures may be advisable for T000053 and T000127 in both solution alternatives above.
The final step is to translate the list of CMs reflected by the lowest cost solution into well-formed recommendations. A well-formed recommendation includes three pieces of information:
- The action, device, procedure, or technique recommended, i.e., which CM to apply
- The reason why the CM is required, i.e., the TTPs that it mitigates
- The implication or effect if the CM is not applied, i.e., the potential impact to mission capability resulting from compromise of the cyber-asset
A cyber RRA conducted as follow-on to a cyber TSA addresses the first two items above. To detail all three elements, however, a crown jewel analysis may be necessary in order to identify the range of mission impact(s) that result from compromise of a cyber-asset.
Best Practices and Lessons Learned
Adapt RRA to satisfy program needs. The objective of an RRA assessment may not be to identify an optimal solution but instead to understand the range of mitigation alternatives and/or areas where gaps exist. In this context, the RRA deliverable would consist of TTP/CM mapping table data for a specified set of TTPs.
Use TARA to evaluate cyber risks and mitigations. Early assessments demonstrated that a TSA conducted without a follow-on RRA provides limited value to sponsors who seek to reduce susceptibility to cyber-attack. The MAE portfolio now combines cyber TSA and RRA into a single engineering practice called Threat Assessment & Remediation Analysis (TARA) .
Consider alternative scoring approaches. A variety of more sophisticated scoring models and approaches can be considered prior to conducting an assessment. The RRA approach does not mandate use of either U/C ratios or the default RRA scoring model; any approach for estimating CM merit may be used provided all CMs are assessed consistently.
Automate to manage large amounts of data. Large catalogs of TTPs and CMs produce very large TTP/CM mapping tables, which require automation to be processed effectively.
Evaluate security measures for operational systems. For deployed and operational systems, one optional step not discussed above is the evaluation of existing security measures to determine whether effective TTP mitigations have already been applied. In cases where such security measures are judged to be highly effective, it may be expedient to remove the TTP from the list of TTPs being evaluated for that cyber-asset.
Reducing the CM search space. The process used to enumerate the solution set of CMs can benefit from the application of heuristics that reduce the search space. The heuristic outlined previously is to rank CMs by U/C ratio in order to facilitate early identification of optimal, i.e., lowest cost, solutions. Other heuristics may also apply.
Crown jewel analysis is essential. A well-formed recommendation details risk-to-mission impact, which is needed in order to make informed decisions. A crown jewel analysis or other form of mission impact analysis is essential for that determination.
References & Resources
- Goldman, H., Building Secure, Resilient Architectures for Cyber Mission Assurance, The MITRE Corporation, October 2010.
- NIST Special Publication 800-39, Integrated Enterprise-Wide Risk Management," March 2011.
- CNSS Instruction 4009, National Information Assurance (IA) Glossary, April 2010. Cyber Threat Susceptibility Analysis (TSA), Systems Engineering Guide.
- Wynn, J., and Montella, L., "Cyber Threat Susceptibility Analysis (TSA) Methodology," Version 2.0, MITRE Technical Report (MTR) 100379, October 2010.
- Wynn, J., et al., "Threat Assessment & Remediation Analysis (TARA)," Version 1.4, MITRE Technical Report 110176, The MITRE Corporation, October 2011.