Home > News & Events > MITRE Publications > The Edge >

As technologists, our paradigm has been: build it, test it, and train them to use it. This paradigm fails when new technology brings about dramatic changes in organizational process. Introducing advanced decision-support technology into large, complex organizations via multiple, interdependent information systems demands an understanding of concurrent changes in technology requirements, organizational processes, and the learning required to improve organizational capability. When they are in training and learning to use complex systems that are themselves evolving, human decision-makers discover new ways to use those systems. At the same time they provide feedback on emerging requirements to the system developers. When these interdependent activities are not synchronized, organizations find that they have systems delivered and processes in place which solve last year s problems and fail to address current needs. How can organizations use training to discover technology requirements and invent process?

Challenges in Evolving C2 Decision Systems
The U. S. military faces tremendous challenges in effectively co-evolving advanced information technology, operational processes, and training as decision-support technologies permeate operational units. An effective approach to these challenges evolves system requirements through a series of advanced warfighting experiments integrating prototypes and existing systems. This iterative development approach is a discovery process that requires: (1) good systems engineering practices for analysis, design and evaluation; (2) innovative concepts of operation that focus technology application on decision makers needs; and (3) effective methods for analyzing and incorporating feedback from users.

The first two requirements are well known, though not often well executed. The third is problematic in developing C2 decision-support systems. Systems developers have difficulty incorporating cognitive and organizational models into requirements analysis, design, or evaluation. Successful synthesis of these factors is required to ensure that advanced information technology benefits operational users in a reasonable time frame without creating additional burdens for organizations.

Three Co-evolving Interdependent C2 Systems

Training designs with the new systems often portray the command and control organizational processes encompassing tactics, techniques, and procedures (TTP) as known and standard. In contrast, observations and interviews with commanders and other military leaders reveal invention and exploration as critical aspects of training exercises. Introducing new information technology results in discovery-based learning and experimentation on an individual level (how does this system perform and how can it help me?) and on an organizational level (how can this technology help us achieve missions?).

As active combat units use training activities to explore new TTP, two specific issues determine efficient use of training time. First, the training methods must assist the trainees in developing adequate conceptual models of system operation. Second, the organization needs to provide starter heuristics or baseline rules for an initial strawman process. Organizations must experiment with the systems (1) to discover their performance characteristics, (2) to discover how they affect organizational process, and (3) to use their experimental results to invent new TTP. Since decision-support systems interact with cognitive and decision-making processes, understanding their organizational effects is not easy and makes discovery and invention complicated. Knowledge discovered in training and educational exercises is invaluable to the iterative development of the systems, the evolution of the operational processes, and, ultimately, to the successful integration of new technology into the larger C2 decision system.

Capitalizing on Invention and Discovery in System Development
Qualitative evaluation of operational users discovery processes can provide feedback for systems engineers conceptual models of use, rather than gathering ad hoc critiques of system features. As an example, we recently observed battalions and a brigade field a new C2 decision-support system. Part of this system includes a situation picture, which displays blue forces (icons driven by near real-time GPS-based information) and red forces. Red force depictions are manual creations from raw intelligence, such as scout reports, local fused intelligence, and feeds from higher-echelon intelligence analysis. When command post staff viewed red icons, we often heard expressions such as, What does this mean? The staff understood the technical event and symbology, but the tactical meaning was not conveyed when the icon creator was not physically present to explain. This question has critical implications for command decision-making and systems engineering. It led to a host of discovery and invention activities to drive TTP and manual process characterized by decision makers as "red icon management."

This process involves shared beliefs, uncertainty and evidence management, and many practical issues such as filter setting, icon creation, monitoring, and tracking, and confidence in a synthetic presentation under time pressure.

During after action reviews (AAR) at the end of exercise segments, commanders created visual models to link phase of battle with system utility in order to discuss red icon management. AARs are often a missed opportunity for capturing feedback for use in formal system reviews and avoiding an endless cycle of change requests.

As forums for learning, AARs we observed focused on linking process with outcomes; learning occurs when commanders and staff establish how and why the battlefield operating systems and functions contributed to performance. Most of the conventional AAR support information remains limited to outcome measures of performance, such as targets destroyed, resources used, and attrition on both sides. New AAR tools provide capabilities to replay exercise events synchronized to the relevant C2 system displays. Additionally, operational combat units are severely time-constrained. While field exercises most closely resemble operational target environments, operational units do not have the luxury of investing significant time to explore the possibilities of new technologies; commanders and staff look for workable TTP and press on with other tasks.

Users concepts of system capabilities and quality affect not only their ability to exploit the system for operational objectives, but also their ability to imagine new ways to combine systems to achieve operational objectives. This is a reason that fidelity of the training environment is critical. For example, during recent experiments in the Fort Leavenworth CGSC/BCBL Digital Leader Reaction Course (see Simulation Trains Commanders), technical difficulties during pre-exercise training led some students to believe that the intelligence picture would significantly lag behind the main control system picture. As a result, they learned to get coordinates for processing fire missions from the main control system, but many of their fire missions found no targets. One group of students who did not receive this training learned during the exercise to use the intelligence system to focus the real-time search of the unmanned aerial vehicle (UAV), capture target coordinates and pass them to the Fire Support Officer to create fire missions.

This error did surface during an AAR, but none of the AAR support tools captured it. In order to uncover and document this issue, the officers conducting the AAR needed intermediate outcome information about the team processes that selected targets and created fire missions. Simulation message traces, system messaging, and observation can provide this information. Support for rapid analysis would map decision-making team members, information, and sources used to measures of fire mission effectiveness. In addition to benefits for decision makers, systems developers can use information from the exercises and AARs in refining conceptual models of use to improve their ability to predict and trace the ripple effects of design modifications.

Using Technology to Capture and Share Organizational Experience
Clearly operational units cannot participate in systematic research; they cannot trace every doctrinal and tactical implication of the new TTP they develop. It is the role of Army training and doctrine thinkers to develop scenarios, systematically explore the ramifications, and work these issues into exercises to complete the work begun at operational commands. Simulation-based training exercises furnish a low-risk, medium-fidelity environment for both user and organizational learning. We believe that these laboratory settings could support discovery and invention in the introduction and use of advanced information technology in command and control processes. Advances in simulation environments and AAR support present opportunities to provide robust feedback on technology-supported decision-making processes, as well as the combat effectiveness of those decisions. To realize these possibilities, research and development efforts must focus on innovative methods to support system design evolution through analysis that relates decision-making process and technology. Conceptual models for training and exercise support systems should include AAR process models that incorporate both instructional objectives and subsequent cognitive requirements.

Complex interactions between the human, machine, and communication components that define C2 decision systems require the synthesis of multiple model types. These models will help us discover requirements and invent better evaluation methods as we examine how simulations and conceptual models of C2 decision support systems may best support the multiple goals of system development, organizational process evolution, and combat team training.

Craig Doescher, Wesley Hamm, and Dan McConnell contributed to this research as part of the Army Experiment 5 team.

For more information, please contact Lee Ehrhart using the employee directory.

Solutions That Make a Difference.®
Copyright © 1997-2013, The MITRE Corporation. All rights reserved.
MITRE is a registered trademark of The MITRE Corporation.
Material on this site may be copied and distributed with permission only.

Privacy Policy | Contact Us