Comparison of Investment Analyses
Definition: Investment analyses take many different forms within the federal government and are applied to support a number of key decisions throughout the investment, acquisition, and program life cycles. Examples include Analysis of Alternatives, Business Case Analysis, Cost-Benefit Analysis, and Economic Analysis.
Keywords: analysis of alternatives, business case analysis, cost-benefit analysis, economic analysis, financial analysis, investment analysis
MITRE SE Roles & Expectations: MITRE systems engineers (SEs) are expected to understand key aspects of investment analyses to be performed at various stages of the life cycle, including decisions supported, primary objectives, and general approaches that can be applied. Engineers will frequently team with economic/cost analysts to perform investment analyses, and their familiarity with key aspects of prevalent analytic approaches can improve the overall quality of completed analyses and the efficiency of supporting activities performed.
Several different analysis approaches are designed to inform sponsor funding and expenditure decisions. These include Analysis of Alternatives, Business Case Analysis, Cost Benefit Analysis, and Economic Analysis. This article provides an overview of the commonalities, overlaps, and differences among prevalent business, investment, and economic analysis approaches and of which analyses are required by different government organizations.
Comparison of Key Investment Analysis Types
Analysis of Alternatives (AoA): An AoA is a technical assessment using distinct metrics and various decision criteria to objectively evaluate different potential courses of action (or alternatives). Typically the focus is on an analysis of alternative technical approaches for meeting a given set of functional requirements or mission needs. Alternatives can, in special cases, be distinguished by different acquisition-related approaches if a real range of feasible technical alternatives does not exist. The assessment also includes a life-cycle cost estimate (LCCE) and risk assessment for each alternative and a preliminary recommendation(s).
Business Case Analysis (BCA): A BCA is used to determine if a new approach and overall acquisition should be undertaken. A BCA provides justification for investment (either to invest or not) based on a comparison of life-cycle costs, benefits, and the results of applying financially oriented calculations such as return on investment (ROI), net present value (NPV), and discounted payback period (DPP) for each alternative. An AoA establishes the alternatives to be evaluated. Based on the results of the financial analysis, a BCA helps determine if a potential new acquisition is warranted and if the effort should proceed.
Cost Benefit Analysis (CBA): The primary objective of a CBA is to identify and obtain approval for the optimum way to solve a specific problem or capitalize on a specific improvement opportunity. A CBA documents the predicted effects (financial, operational, quantitative, and qualitative) of actions under consideration and describes the potential financial impacts and other business benefits to support a a decision regarding adopting a particular solution before significant funds are invested. Agency-specific policies may differ, and CBAs within some organization may need to include a detailed budget estimate and identification of funding sources for the preferred alternative. Many organizations consider a CBA a living document that should be updated as needed.
Economic Analysis (EA): An EA is a systematic approach to identifying, analyzing, and comparing costs and benefits of alternative courses of action. An EA should emphasize an identification of key variables that drive cost, benefit, risk, and uncertainty assessment results across competing alternatives. Depending on agency policy, an EA may be very similar to a CBA. In some government organizations, EAs will include post-investment management considerations, including the identification of key performance measures for monitoring and evaluating whether initiatives ultimately achieve expected or desired results.
AoAs and BCAs are typically conducted early in the acquisition process for a new Information Technology (IT) initiative. For example, the Department of Defense (DoD) Joint Capabilities Integration and Development System (JCIDS) process identifies that they should be conducted prior to Milestone A. For each subsequent year, they should be validated against the baseline as costs, schedule, etc. may change.
In the past, many civilian agencies were not necessarily required to prepare distinct AoAs or BCAs but would, instead, rely on the Alternatives Analysis that was prepared in support of an Office of Management and Budget (OMB) Exhibit 300 submission, as required by OMB Circular A-11, Part 7. Although the E-300 is a fairly comprehensive document, a typical AoA or BCA will be more detailed and rigorous.
The DoD requires distinct and more comprehensive AoAs and BCAs primarily because DoD initiatives are often larger dollar investments than civilian IT initiatives. Civilian agencies must periodically check progress against the baseline and, as necessary, revise the E-300 annually. Most recently, a number of civilian agencies (e.g., Department of Homeland Security) have produced guidance for preparing AoAs or BCAs to support investment decision making.
AoAs and BCAs are often well understood among IT acquisition professionals. The understanding of CBAs and EAs is often less clear. The terminology of these analyses is often used interchangeably depending on a given agency and individual's opinion. CBA and EA are basically similar analysis approaches.
Table 1 summarizes the general similarities and differences of the analytical approaches across various agencies/communities.
Table 1. Comparison of Analytical Approaches
Best Practices and Lessons Learned
Many identified best practices and lessons learned for particular investment analyses will also be relevant for other investment analysis types.
Analysis of Alternatives (AoA)
Develop viable alternatives. Alternative courses of action that could feasibly be pursued to achieve the mission should be developed. An analysis should not use "throwaway" alternatives to justify a preconceived course of action.
Examine three or more alternatives in addition to any Status Quo (SQ) baseline. For example, new alternatives for traditional software development initiatives might include commercial-off-the-shelf (COTS), custom software development, and COTS/software (SW) development hybrid. To meet certain requirements, it may be impractical to use all COTS or to solely embark on a custom software development. A well-developed AoA should include technically distinct, realistic alternatives. The SQ alternative should be viable; in particular, SQ does not necessarily mean "do nothing." For example, inclusion of technology refresh in the future may be necessary if it is essential to maintaining required functionality.
Alternatives need to be developed in collaboration with technical subject matter experts (SMEs). An AoA/BCA author should not invent alternatives for the purpose of checking a box and meeting a capital planning requirement. Technical SMEs should be consulted to think through the implications of how to best fulfill mission needs.
Alternatives can be differentiated by acquisition approaches. In certain cases, there may be few distinct technical approaches for fulfilling mission needs.
Establish evaluation criteria and scoring. Identify multiple, independent metrics and compare how each alternative impacts them. Metrics may be quantitative or qualitative. Every metric does not need to be equally important. Weightings can be applied to reflect priorities.
Identify technical and cost risks. Risks with each alternative should be identified, including ease of implementation, relative difficulty to meet certain challenging requirements, as well as an assessment of the probability that an alternative can meet cost and schedule goals.
Life-cycle cost estimating. An AoA should include a multi-year life-cycle estimate generally broken out by acquisition, operations and support (O&S), and SQ phase-out costs. A typical DoD approach is to analyze O&S costs after 10 years post-system full operational capability. This would include at least one tech refresh cycle and enable the analysis to calculate financial metrics and benefits for each alternative over a longer time period.
Recommendations that consider environmental changes. It is useful to recommend a specific alternative to pursue, and this can be done in conjunction with a subsequent BCA, as needed. In certain cases, it is useful to present a few recommendations that take into account likely changes to the surrounding environment. For example, Alternative 1 may be the preferred recommendation if the program receives adequate funding but not if its funding is cut. Recommendations for specific alternatives can be tied to potential future situations that may arise (which should also be documented in the ground rules and assumptions).
Government/sponsor buy-in. An AoA is sometimes viewed as a required check-the-box activity that is a necessary part of capital planning. For programs requiring large expenditures of funds, however, an AoA is a useful means for the program office to fully consider what must be done to move forward in an effective and efficient manner. The approach requires a degree of rigor that will help the program succeed in the long term.
Business Case Analysis (BCA)
Identify potential benefits for each alternative. In contrast to an AoA, a BCA identifies measurable benefits for each alternative. An example of a measurable, or quantitative, benefit includes an increase in productivity. Although qualitative benefits such as morale, better information sharing, and improved security cannot be readily measured, they may still be important to alternative selection.
Develop ROI statistics. Key financial metrics help gauge the ROI of alternative courses of action relative to the SQ alternative. No single calculation can fully describe ROI, and financial metrics typically calculated to support this description include net present value (NPV), rate of return (ROR), and discounted payback period (DPP). These three metrics take into consideration the "time value of money" (i.e., a dollar received today is worth more than a dollar received one year from now due to factors such as inflation). All calculations, except NPV, are typically performed relative to the SQ alternative. In other words, the calculations take into account how significantly an alternative course of action increases or decreases costs and benefits relative to the SQ. The NPV for an alternative describes current and future net expected benefits (i.e., expected benefits less expected costs) over the life cycle for that alternative. ROR, stated as a percentage, describes the discount rate at which the incremental current and future benefits of an alternative in comparison to the SQ equal the incremental costs of that alternative relative to the SQ baseline. DPP, stated in units of time, describes how much time it will take for the cumulative incremental benefits of an alternative course of action relative to the SQ alternative to exceed the cumulative incremental costs of an alternative course of action.
Clarify the acquisition/implementation timeline. A clear timeline is important to see how costs are phased in over an investment and O&S period. Programs typically have from two- to five-year investment periods depending on the size of the initiative and the type of acquisition approach.
Identify the risks and potential benefit impact. This pertains specifically to the risks that the LCCE will deviate from the baseline estimate. Specific major cost drivers that are risky should be called out. Software development labor is particularly prone to potential risk of under-estimation. There are others. The risks that certain benefits will not be realized should also be identified. This is similar in concept to identifying cost risks.
Cost Benefit Analysis (CBA)
Alternatives may differ from those in the AoA and BCA and be analyzed in more detail. Depending on how much detail was provided in the AoA and BCA, a CBA should break down the preferred alternative into more specificity. A CBA may focus on how best to implement the preferred alternative, which requirements may yield the biggest "bang for the buck" in higher ROI earlier in the life cycle, how different contracting approaches may reduce risk, etc.
Can analyze multiple alternatives. The standard rule of thumb for a BCA is three new alternatives plus the SQ (policies on the number of alternatives to evaluate may vary, and the number may be predicated on the problem being addressed). A CBA can offer many more, if necessary, with minor variations among each to help determine the best specific approach forward. It will still follow the basic format of a BCA, although the analysis for each alternative approach may not be as comprehensive as in the AoA/BCA.
Allows for incremental ROI results. A CBA is particularly useful for demonstrating the quantitative benefits that specific actions within a work breakdown structure (WBS) may yield. Therefore, a CBA is well suited to analyze if one alternative offers a greater ROI than a competing alternative. Incremental financial analysis helps decision makers move forward by considering which alternative approach at a given point along an integrated master schedule (IMS) will yield a greater "bang for the buck" to justify an investment decision at a particular acquisition milestone or increment.
Economic Analysis (EA)
Perform economic sensitivity analyses for key variables. EAs are often evaluated over the investment life cycle, and it is often unreasonable to assume perfect knowledge of what costs, benefits, and risks will be at all points in the future. For analysis variables that significantly drive analysis results and for which there is either (a) considerable uncertainty or (b) considerable impact if unlikely conditions actually occur, sensitivity analyses or uncertainty-weighted assessments should be performed to determine the potential impact of uncertainty on analysis results.
Evaluate all significant economic implications, but not necessarily in monetary terms. In the federal government, many investment costs, benefits, and risks are not readily translated into monetary terms. Regardless, if these implications are critically important to informing investment decisions, they should be evaluated. Analysts should consider whether other quantitative methods (e.g., comparison of numeric ratings for aspects of investment options) can be applied to assess the relative strengths and weaknesses of investment options.
References & Resources
- AFMAN 65-506, "Economic Analysis," August 29, 2011.
- Army Cost Benefit Guide, April 8, 2011.
- "Conducting a Business Case Analysis," K. Buck.
- "Economic Analysis Documentation," K. Buck.
- OMB Circular A-11, Part 7 ("Planning, Budgeting, Acquisition, and Management of Capital Assets"), Exhibit 300.
- "Performing Analyses of Alternatives," The MITRE Corporation, Systems Engineering Guide.