Follow Us:

Policy Analysis

Definition: Policy analysis is a disciplined process to help people make decisions in situations of multiple objectives and multiple perspectives.

Keywords: decision making, policy, policy analysis

MITRE SE Roles & Expectations: MITRE systems engineers are expected to understand the role and implication of policy in our customers' activities and how systems engineering relates to it. MITRE systems engineers are expected to know the basic characteristics of good policy analysis, so they can constructively collaborate with policy analysts on questions and issues arising at the boundary of systems engineering and government policy.

Government Interest and Use

Within the United States government, very few important decisions are made by a single individual. Congress and the Supreme Court make decisions by voting. While most Executive Branch decisions are made by an individual senior official, the decision is normally the result of a deliberative process in which numerous people with diverse expertise or diverse interests offer advice that it is imprudent to ignore. The output of a good policy analysis is a set of questions regarding priorities or a set of options among which to choose, along with the major arguments for each competing priority or the major pros and cons of each option. Either option will help organize the interactions that lead to choosing a course of action.

Systems engineering can be viewed as a process for arriving at a solution that represents an acceptable balance among multiple objectives. Traditionally, systems engineering typically presumed that the objectives and relevant operational constraints can be defined, and that the extent to which any given outcome meets a given objective can be quantified. When these conditions exist, systems engineering can usually arrive at a "best," "correct," or "optimal" design solution. Systems engineering also can derive the requirements for various subsystems on the basis of the overall system design, including the requirements that each subsystem must meet to interact properly with other subsystems. In contrast, policy analysis, when done well, leads to courses of action that may not be the "best" from any one perspective, but are "good enough" for enough players to win the necessary political support to move ahead. New forms of systems engineering are adopting this "good enough" solution perspective, particularly in large-scale enterprise settings. Indeed, a good policy analysis may be used by individuals who disagree with the government's objectives; in this case, one individual may conclude that the policy analysis shows option B is best while the other concludes that the same policy analysis shows option D is best, and they both agree that either option is acceptable.

Systems engineering and policy analysis must account for costs and affordability. An elegant engineering solution that the customer cannot afford is useless; so too is a policy option that would make many people happy, but at a prohibitive cost. Therefore, careful efforts to estimate the cost of a particular option and the risk that the actual cost may exceed the estimate are necessary for systems engineering and policy analysis. Engineers who design products for commercial sale are familiar with the concept of "price points," and a manufacturer may wish to produce several products with similar purposes, each of which is optimal for its own selling price. In the case of systems engineering for the government, it may be necessary to conduct a policy analysis to determine how much the government is willing to spend, before conducting a systems engineering analysis to arrive at the technically "best" solution at that cost level.

Policy Analysis Best Practices and Lessons Learned

Especially rigorous quality assurance. Policy analysis at MITRE poses a special concern. The missions of MITRE's Federally Funded Research and Development Centers (FFRDCs) are systems engineering and research, while policy analysis is the mission of other FFRDCs. At times, it is completely appropriate for MITRE to conduct policy analysis. MITRE has excellent policy analysts on its staff, but it falls outside the mainstream of our work. Thus, it important that all MITRE policy analysis delivered to the government is of high quality. If a MITRE policy analysis is substandard, we have few resources to fix the problem and are vulnerable to the accusation of taking on work that is outside our sphere of competency. Therefore, any MITRE policy analysis intended for delivery to the government typically requires a degree of quality assurance beyond our routine practices.

The technical-policy boundary—know and respect it. Some MITRE work requires policy analysis as a deliverable to our government sponsors (i.e., the sponsors ask us to provide analytical support for government policy-making). At times, MITRE conducts policy analysis for internal consumption only. This helps MITRE understand the multiple perspectives and objectives of our sponsors so that our technical work can be responsive to "real" needs that they may be precluded from expressing in official documents. Finally, MITRE is sometimes asked to support a government policy process by providing technical analysis that narrows the scope of the government's disagreements; the task of "taking the technical issues off the policy table" requires that MITRE staff sufficiently understand policy analysis to assure our technical analysis stops where true policy analysis begins.

Policy analysis basics for systems engineers. There are a few basics that characterize good policy analysis. MITRE systems engineers should be familiar with them, so they can constructively collaborate with policy analysts on questions and issues arising at the boundary of systems engineering and government policy. These are summarized below in the order in which they appear during the course of a policy analysis:

• Transform a situation into one or more issues. The analysis must identify the policy decisions that are most appropriate for the situation. Figuring out what questions to ask is the most critical, and often the most difficult, part of the analysis. Asking the right questions is what transforms a "messy situation" into an "issue" or a "set of issues." When policy analysis is being performed for an identifiable customer, it is of little use unless the analysis is framed in terms of decisions that the customer has the authority to make—or perhaps decisions that the customer's boss or boss's boss has the authority to make, provided that the customer has a charter to go to the boss and say, "I can't do my job until you make this decision."
• Create executable options. The analysis must identify options for each decision. This is where policy analysis can be genuinely creative, even while remaining rigorous. There are many options in a typical government policy dilemma; however, the number of options that the policy-makers can seriously consider is small. A senior government official looks for an option that will meet the most important objectives, can be implemented with the resources available, and will attract support from enough other perspectives to command a majority vote or support from a preponderance of advisers. A good set of options: (a) are responsive to the issues posed (see the previous bullet): (b) could be implemented, if chosen; and (c) none of the important players in the decision process will react by saying "none of the above."
• Options have advantages, disadvantages, and uncertainties. The analysis must identify the advantages, disadvantages, and uncertainties associated with each option. This is a straightforward process. However, if the analysis is to be credible, one must carefully state the pros and cons in ways that are recognized as accurate by those whose views they portray. For example, an analysis of an option for sharing extremely sensitive intelligence with an ally should state the pros in language that might be used by a proponent of this option, and the cons in language that might be used by an opponent. Otherwise, the product will be viewed as advocacy, not an analysis.
• Strategies for reducing uncertainty. Sometimes an analysis, having identified uncertainties that make it difficult to choose an option, may propose a strategy for reducing the uncertainties. Of course time reduces some uncertainties, and a serious effort to gather additional information will require time. Delaying a decision often permits a bad situation to become worse. Much of the art of the statesman is sensing the moment to make a difficult decision. When a policy analyst chooses to propose a strategy for reducing uncertainty, the analyst is helping the decision-maker understand how much time would be required to obtain additional information or understanding, and thus make a good judgment about when to decide.
• Identifying additional options, if needed. Sometimes if an analysis failed to identify an acceptable set of options, it may propose a strategy for identifying additional options. Such a strategy could be a targeted research program or consultation with other organizations that have not participated in the process.
• Decision-making strategies. Finally, the analysis may identify a strategy for arriving at a decision. In some circumstances, this is not necessary if the strategy is obvious; in other cases, some or all of the options may require concurrence of others or a process that is unusual in some way.

As is the case with many MITRE services and products, a policy analysis may contain extensive data and argumentation that the actual decision-maker will never read or hear. The executive summary of a paper and the first few slides of a briefing must clearly convey the information that the decision-maker should learn and understand, while the body of the paper and the extensive back-up slides in the briefing provide credibility to the message and a means for staff to check the validity of summary statements they find surprising. Therefore, it is highly desirable that the Executive Summary or the summary slides be well written. In contrast, the segments providing detail must be checked carefully for clarity and accuracy, but need not be models of graceful prose.

References & Resources

Allison, G. T. and P. Zelikow, 1999, Essence of Decision: Explaining the Cuban Missile Crisis, 2nd Edition, New York: Longman.

Conklin, J., Winter, 2009, "Building Shared Understanding of Wicked Problems," Rotman Magazine.

Rittel, H. and M. Webber, 1973, "Dilemmas in a General Theory of Planning," Policy Sciences, Vol. 4, Elsevier Scientific Publishing Company, Inc., Amsterdam, pp. 155-169.

The MITRE Corporation, "Tradeoff and Engineering Analysis." MITRE Project Leadership Handbook.

Wildavsky, A., 1979, Speaking Truth to Power: The Art and Craft of Policy Analysis.

Wildavsky, A., 1988, The New Politics of the Budgetary Process.