Follow Us:

Operational Needs Assessment

Definition: An operational needs assessment identifies and characterizes gaps in existing capabilities that are significant impediments to achieving the mission area objectives. It does so through the application of operational experience in a particular mission/business area, knowledge of related processes and elements involved in conduct of the mission, and knowledge of the mission's objectives and measures of success/effectiveness.

Keywords: acquisition development program, capability-based assessment, CBA, operational needs assessment, program management

MITRE SE Roles & Expectations: The operational or capability-based needs assessment is typically the responsibility of the operational requirements organization of the system's or capability's end user. MITRE systems engineers (SEs) are often requested to support such assessments, or even develop complete assessments for review and approval. Key roles for MITRE SEs in this process may include ensuring that operational needs statements are clear and complete; understanding and conveying areas of uncertainty or flexibility; ensuring analyses contain appropriate attributes and associated metrics supported by analytical or operational evidence, and are clearly tied to operational goals; modeling/prototyping/experimenting on needs/gaps for clarity, feasibility and integration; assessing technical readiness/risk/feasibility of technology-driven capability needs; and identifying risk/cost drivers in capability needs.

The MITRE SE should have a sound understanding of the principles of needs assessment, the needs assessment process of the supported government element, and the political, business, operational, and technical (enterprise) context of the capability area of interest as well as how the government customer/sponsor intends to continue evolution and sustainment of the product following product delivery.

Background

Operational needs assessments are frequently the initial step toward a new development or process improvement program. A needs assessment is conducted by the user community to determine the best capabilities that will help users accomplish their operational tasks. These assessments are accomplished through operational experiments, exercises, modeling and simulation of user tasks/operations, etc. From these, users formulate their needs and requirements. Assessment products are the basis for applicable technology assessments, solution alternatives analyses, operational requirements definitions, and ultimately the acquisition program (or programs). An operational needs assessment defines the business and mission need for providing systems, services, capabilities, or platforms to end users and other stakeholders, and develops a business case that justifies the return on investment in order to obtain funding for a system or multiple systems [1, 2].

New needs can arise for a number of reasons: new goals (e.g., manned mission to Mars), new conditions (new or improved threats, deterioration/discontinuation of existing capabilities, hardware/software end-of life), changing processes/regulations (new laws, new organizational responsibilities, new relationships), introduction of new technologies that enable previously unattainable capabilities or enable improved or more cost-effective solutions to existing capabilities, etc.

Why do we perform operational needs assessments? First, we are typically required to do so by Department of Defense (DoD), Federal Aviation Administration, Internal Revenue Service, and other formal organizationally run programs. But even if not required, an assessment of operational needs and lessons learned provides greater understanding by the user, acquisition, development, and support communities so that needs can be satisfied with capabilities, products, or improved processes valuable to mission success. The government has limited resources to obtain its goals. Operational needs must be described and quantified in the context of operational user needs and goals for decision makers to assess their validity, importance, and urgency in the context of competing needs, and determine the risk of not obtaining the capability. If new needs can be most cost-effectively met by changes to DOTLPF (Doctrine, Organization, Training, Logistics, Personnel, Facilities), then new materiel solutions may not be necessary. In any case, the operational needs must be defined and quantified in a manner that allows for assessment of the most cost-effective solution alternatives for the need.

Process

In the context of a systems engineering life cycle, an operational needs assessment forms the basis for defining requirements for a program and a system. It occurs as an initial step in the life cycle but also must be continuously addressed as operational environments, evolutionary strategies, priorities, and funding changes. As part of initial program planning, MITRE is frequently involved in the establishment of systems engineering processes, of which operational needs assessment is an important one. The process elements are:

1. Determine the specific requirements of the needs assessment process that apply.
2. Identify specific stakeholders in this particular needs assessment process, including their responsibility, goals, and roles/relationships.
3. Identify and obtain support from operational or capability domain experts.
4. Develop a needs assessment plan and schedule, which can include scenario-driven experiments, gap analysis, tradeoff studies, etc.
5. Identify and put in place any analytical tools necessary to define and quantify needs.
6. Implement and conduct needs assessment.

Operational Needs Considerations

As an example, the DoD Joint Capability Integration and Development System (JCIDS) process [3, 4] includes several steps leading to an operational requirements document (capability development document [CDD]) for acquisition of a system. Although other government departments and agencies may have different specifics, the basic approach has general applicability. It begins with a capabilities-based assessment that identifies the mission, the capabilities required and their associated operational characteristics and attributes, capability gaps, potential solutions (e.g., processes, systems, technologies), and associated operational risks. If a DOTLPF assessment determines that a new system (materiel) capability is required, an initial capability document (ICD) is developed. The ICD contains the definition of the capabilities needed along with their important attributes and associated metrics. This is used as the basis for an analysis of alternatives (AoA), which provides quantitative cost/effectiveness trades for alternative approaches to providing the capability. The results of this AoA are then used to develop the solution-approach specific CDD.

The ICD is the repository of the capability needs assessment results. The needs statements should have the following attributes:

Enterprise and Operational Context: It is important that needs be considered in an enterprise context. If related enterprise capabilities can address part of the need, define the unique characteristics of the new need in this context.

Complete (End-to-End) Need Defined: Ensure that the need is defined as completely as possible (e.g., detect, identify, and defeat incoming cruise missiles vs. detect incoming cruise missiles). Recognize where areas of uncertainty remain or areas of flexibility exist.

Conditions/Scenario: Define the conditions/scenario under which the capability/need will exist (e.g., indications and warning vs. major combat operations, jamming vs. clear, communications/power outage).

Attributes/Metrics: Consider quantifiable metrics for the capability that define how much, how well, how often, and how quickly the capability must perform. These metrics should be directly related to mission goals. Again, recognize where areas of uncertainty remain or flexibility exists.

Growth/Extensibility: If current needs are expected to increase or expand in the future, state those expectations so that expandability/extendibility of solution approaches can be properly taken into account and hooks are put in place to enable those extensions. Note that making design choices that favor enhanced adaptability is always prudent.

Independent of Solution Approach: If needs are stated as a particular solution approach, they can eliminate consideration of more effective approaches to meeting the actual need. Thus, needs should be articulated in terms that describe a successful operation or mission instead of a proposed solution.

Lessons Learned

• Beware solutions masquerading as needs: Operational or capability needs are often represented by users in terms of specific solution approaches. This can result from marketing or technology demonstrations, familiarity with a specific solution approach, or preference for a specific solution/approach due to unstated (or unrecognized) aspects of the need (political, economic, etc.). The challenge is to extract from the users the full definition of the underlying capability needed, and obtain stakeholder concurrence that the focus needs to be on those identified capabilities, not a solution-based approach. The best approach for understanding the needs is by observing and talking with the actual end users. It may be a challenge to get their time and access. If so, consider a user "surrogate," including MITRE employees with recent operational experience.
• State needs unambiguously: Key attributes and metrics are frequently missing, stated in ambiguous terms, or stated with no corroborating analysis or evidence basis. The challenge is to clarify needs in unambiguous terms, with attributes and metrics directly related to mission goals (measures of effectiveness), and supported by analysis or operational evidence.
• Get all relevant views: Operational needs can be driven by a subset of the key stakeholders (e.g., system operators vs. supported operational elements), and thereby miss key capability needs. The challenge is to ensure that all key stakeholders needs are taken into consideration.
• One size may (or may not) fit all: The union of a set of needs may lead to a solution that is too cumbersome to implement cost-effectively. Remember that multiple solutions to subsets of needs, or satisfying additional needs by iterative solution augmentations, may sometimes be the most practical approach, assuming operational needs can be adequately met. Methods such as modeling and simulation and prototyping/experimentation allow an examination of the needs-satisfaction approaches and evolution and help plan the augmentations that best satisfy operational needs and missions over time.
• The educated consumer is the best customer: Particularly in the case of new technology driven needs, operational requirements contributors can be unfamiliar with potential capabilities, the user's concept of operations or concept of use, organizational, and political implications. The challenge is to educate users on capabilities, limitations, cost drivers, and operational implications of the new technologies so that the capability delivered provides the best cost/performance balance for the customer. Prototyping and experimentation, particularly with heavy user involvement, can help educate not only the end user, but the systems engineers as well. For best practices and lessons learned, see the article, Competitive Prototyping, in the Acquisition Systems Engineering section.

References & Resources

1. The MITRE Institute, "Concept Development," Systems Engineering Competency Model, section 2.1, accessed February 22, 2010.
2. An example of a document that contains operational needs is US Citizenship and Immigration Service Concept of Operations, October 22, 2009, USCIS Transformation Program.
3. Joint Capabilities Integration and Development System, March 1, 2009, Chairman of the Joint Chiefs of Staff Instruction 3170.01.
4. Manual for the Operation of the Joint Capabilities Integration and Development System, July 31, 2009.