Analyses of Alternatives
Definition: In defense communities, the Analysis of Alternatives (AoA) is "an analytical comparison of the operational effectiveness, suitability, risk, and life cycle cost (or total ownership cost, if applicable) of alternatives that satisfy validated capability needs" . AoAs document the rationale for identifying and recommending a preferred solution or solutions.
In civilian agencies, an AoA is an evaluation tool that eliminates "those ideas that are technically or financially unfeasible in order to permit the selection of alternatives for further feasibility testing based on the resulting cost estimates" . It helps stakeholders and agencies identify and select investments that will maximize benefit (i.e., achieve essential mission performance goals and objectives) within availability, affordability, cost, sustainability, and risk constraints.
Keywords: alternative assessment, analysis of alternatives, AoA, baseline alternative, cost analysis, criteria, evaluation, materiel solutions, operational effectiveness, risk analysis, suitability
MITRE SE Roles and Expectations: MITRE systems engineers (SEs) are expected to understand the purpose and role of AoAs and when it is appropriate to conduct an AoA (e.g., for government departments and agencies, where it occurs in the acquisition process). MITRE SEs may lead AoAs and will contribute as technical and analytic subject matter experts (SMEs) across the life cycle of the AoA—from initiating the AoA study plan, to developing and evaluating alternatives, to generating recommendations based on the analysis. SEs monitor and evaluate AoA progress and recommend changes when warranted. More broadly, SEs should understand similarities and differences across related analyses (e.g., cost/benefit analysis, business case analysis, economic analysis, and AoA) and are expected to advise the government on the type of analysis that is most appropriate for a given decision-making need. For information on other types of analyses, see the SEG topic Comparison of Investment Analyses.
AoAs provide a framework to consistently evaluate and compare the value of different solutions for providing a needed capability to specific end users. Typical steps in an AoA include:
- Plan: Define decision/objectives supported, identify stakeholders, define the schedule/funding/effort, establish the study team, prepare the study plan.
- Establish analysis foundation/framework: Define the analysis problem statement, context, scope, and framework for alternative comparison, including criteria to be used. Establish ground rules and assumptions that frame the analysis. Address data needs, collection, and sources prior to and during the study.
- Identify and define alternatives: Identify multiple alternatives (one of which could be to maintain the "status quo") that address the stated problem within the context and scope defined. The final set of alternatives evaluated should be the product of thorough research, vetting, and filtering.
- Assess alternatives: Evaluate each alternative against established criteria (e.g., cost, risk, effectiveness/benefit); conduct sensitivity analysis.
- Compare alternatives: Determine the relative merits of the alternatives as exposed by the analysis.
- Report results: Document results that support decision-maker/stakeholder needs.
For the Department of Defense (DoD), the Materiel Development Decision directs the execution of the AoA and authorizes the DoD component to conduct the Materiel Solution Analysis Phase .
For civilian agencies, the documented analysis of alternatives directly furnishes the required information  that justifies the selection of the chosen solution and indirectly contributes both to identifying the development and implementation milestones and activities and to establishing the performance goals for the preferred solution.
The Office of Management and Budget (OMB) provides instructions on performing an analysis of alternatives . The guidance is aimed at moving organizations from a single alternative to the comparison of multiple alternatives.
In addition, according to the Federal Government’s Cloud First Policy , agencies must evaluate secure cloud computing options before making new investments. Cloud computing alternatives evaluation specifies whether, as of the date of the submission, a cloud alternative was evaluated for the investment or components/systems within the investment, per the Cloud First policy.
Best Practices and Lessons Learned
Use AoAs as part of good project management. The AoA process should be seen as a good project management practice for different decisions on multiple types of projects. It should show that sound processes and techniques were followed in recommending a specific solution.
Create an appropriate AoA study plan. A major step leading to a successful AoA is the creation of a well-considered study plan. The study plan establishes a roadmap for how the analysis should proceed, who is responsible for doing what, and why it is being done. A sample AoA study plan template is provided in .
Ensure sufficient time and resources. A significant risk to success is the lack of time and other resources to adequately perform an AoA. Critical to implementing a robust AoA, advance planning should include maintaining awareness of key need dates and provisioning of funds, staff, data, and other resources. The Government Accountability Office (GAO) found that many AoAs are conducted under compressed time frames—six months or less—or concurrently with other key activities that are required for program initiation in order to meet a planned milestone decision or system fielding date. Consequently, AoAs may not have enough time to assess a broad range of alternatives and their risks, or they may be completed too late to inform pre-development trade discussions .
Know the baseline/status quo alternative. For many AoAs, a legacy capability exists and may either be near end of life or no longer satisfy current needs. In these cases, it is critical to understand the existing capability baseline so that the AoA alternatives include an upgrade path from the status quo. If information about the current capability is not already in hand, you will need to capture its baseline to compare against potential upgrades. Collecting baseline information can be a resource-intensive activity, which must be accounted for in developing schedule and staffing needs.
Know your stakeholders. Understand the decision makers and other stakeholders involved with or impacted by the AoA and how they will participate in the AoA and/or use the results. To inform the scope and execution of the analysis, assess the stakeholders' political, operational, economic, and technical motivations. Leverage community and user stakeholder knowledge to help determine objectives and criteria used to evaluate solutions. Develop recommendations for how the various stakeholders will participate in the AoA (e.g., Responsible, Accountable, Consulted, and Informed) and discuss them with the AoA principal customer.
Build an appropriate AoA team. Building an AoA team requires the early identification of individuals with the right skills/expertise (e.g., specific domains, analytic expertise), level of experience, organizational perspectives, and authorities. It also requires the specification of "rules of engagement" for interaction throughout the analysis, including gaining and maintaining "buy-in" of objectives and execution plans. SMEs should be recognized experts in their areas. They should not be selected for organizational affiliation or balance. Some civilian agency guidance  calls for the government to select an organization to conduct the AoA that is independent of both the organization that will acquire and use the solution and the organization sponsoring the acquisition.
No "box checking." Ensure that the analytic results inform the decision-making process rather than the validation of a preferred alternative so that the AoA functions as an unbiased assessment rather than a box-checking exercise.
Maintain broad context. Multiple perspectives should be brought to bear in the AoA, including operational/mission, technical/technological, programmatic, and any other areas that influence the value of solutions.
Use appropriate methods, tools, and data; understand links. Select methods, tools, and data to execute the AoA and its component analyses (e.g., cost, effectiveness/benefit, and risk) to produce products and recommendations that support the decision-making process. AoA participants must seek appropriate data sets that work together with selected methodologies and tools (e.g., simulation models) to avoid "garbage in/garbage out" issues. AoA participants must make appropriate linkages between the component analyses (e.g., schedule and cost).
Ensure the right number of alternatives. A 2009 GAO report on defense acquisitions attributes premature focus or convergence on a particular solution or range of solutions as a failing of AoAs . If stakeholders are already enamored of a particular solution, completing an unbiased AoA may be difficult. A narrow scope or attention paid to a particular solution renders the AoA ineffective for decision making and often foreshadows increased risk in the resulting program.
Identify multiple alternatives to address the problem within the AoA context and scope. Explore a broad range of alternatives to ensure the best value and technical match to the need. Brainstorming can be useful in identifying the superset of alternatives for consideration. The final set of alternatives should be the product of thorough research, vetting, and filtering and should be traceable and defensible. For each alternative, define a technical baseline as a foundation for its assessment.
Anticipate data problems. Analyzing a broad range of solutions involves collecting and developing a considerable amount of information on the representative solutions as well as on other contextual details/supporting information (e.g., specific guidance, regulatory environment). It is critical to develop credible data sets to support an AoA. Develop a data collection plan and create back-up plans in case data access problems arise, particularly when the AoA schedule is aggressive/compressed. Consider and leverage industry, government, contractor, and/or FFRDC data sources. Realistic assumptions going into an AoA are that not all information will be available and that workarounds will be needed. Be persistent!
Ensure appropriate risk analysis. The GAO reports that some AoAs do not examine risks at all and focus only on the operational effectiveness and costs of alternatives. Other AoAs have relatively limited risk assessments. For example, several AoAs did not include integration risks even though the potential solution set involved modified commercial systems that would require integration of subsystems or equipment. The GAO cited a Defense Science Board (DSB) report on buying commercially based defense systems. Programs that did not assess the systems engineering and programmatic risks of alternatives associated with militarizing commercial platforms or integrating various commercial components underestimated the true costs. Other AoAs did not examine the schedule risks of the various alternatives, despite accelerated schedules and fielding dates for the programs. Not surprisingly, the GAO found that programs that conducted a comprehensive assessment of risks tended to have better cost and schedule outcomes than those that did not . For more information on risk identification and management, see the Risk Management topic and articles in this SEG section.
Presentation of the final results. An AoA may not identify one definitive solution. In these situations, it is critical that decision makers have insights into the pros and cons of each alternative and are provided with all information that is relevant to them. The presentation of results should be thought about early in the AoA process, based on decision-maker and stakeholder preferences.
References and Resources
- Office of Aerospace Studies, Air Force Materiel Command (AFMC) OAS/A5, June 10, 2013, Analysis of Alternatives Handbook: A Practical Guide to Analyses of Alternatives, accessed August 22, 2017.
- OMB Circular A-11, Part 7 ("Planning, Budgeting, Acquisition, and Management of Capital Assets"), Exhibit 300, accessed August 30, 2017.
- Department of Defense, January 7, 2015, Instruction 5000.02, Operation of the Defense Acquisition System, accessed August 22, 2017.
- Kundra, V. (U.S. Chief Information Officer), February 8, 2011, Federal Cloud Computing Strategy, accessed August 22, 2017.
- Government Accountability Office (GAO), September 2009, Many Analyses of Alternatives Have Not Provided a Robust Assessment of Weapon System Options, GAO-09-665, accessed August 22, 2017.
- Homeland Security Studies and Analysis Institute, January 22, 2014, Analysis of Alternatives (AoA) Methodologies: Considerations for DHS Acquisition Analysis; Ver. 3.0, accessed August 30, 2017.
Additional References and Resources
Office of Information Technology (OIT), Division of Program Management and Budget, January 2015, Managing Capital Investments at the Indian Health Service: A “How-To” Guide for an Analysis of Alternatives, accessed August 30, 2017.