What happens when you've got five unmanned aerial vehicles (UAVs) circling their assigned targets and a new target is reported? How do you quickly assess the situation and select the right UAV to investigate the new target? If you're an Air Force collection manager for intelligence, surveillance, and reconnaissance (ISR) information, you'd probably want to perform a "what if" analysis to understand what assets you have available and how you can best use them. Getting answers faster becomes even more important as the number and variety of ISR assets grows and the decision process becomes increasingly complex.

ISR collection managers typically work in a Combined Air and Space Operations Center (CAOC) where teams plan, monitor, and direct battlefield coordination. Tasking ISR assets is a critical function of the CAOC. Although automated tools are available to perform that "what if" analysis, they weren't designed to return answers quickly. The current tools were designed for a long planning cycle of 24 to 72 hours. As a result, the collection managers don't use the tools because they aren't useful for ongoing analysis because the target list is constantly changing.

From the point of view of a MITRE research team, a key flaw with the existing system is the inability of humans to interact with the automation. The process doesn't allow collection managers to look at the pros and cons of different options in case a new target pops up. "Instead, the managers are likely to ignore the automatic planning tool and try to figure out the answer themselves," says MITRE's Erika Darling, a lead human factors engineer. "At the other extreme, where no automation is involved, the task of looking at all the data and making a decision is too much of a burden for a human.

"Current tools are either strictly visualizations that don't allow for what-if analyses, or they're very complex and require extensive training. Warfighters are therefore unable to easily explore alternative scenarios to see the impacts on a larger scale when retasking an individual asset."

To solve this dilemma of humans not being able to interact with the automation, Darling and Nikhil Kalghatgi, a human factors engineer, initiated an Air Force-supported research project (as part of our MITRE Technology Program) called START, for Spatio-Temporal Analysis for Rapid Tasking. "We're focusing on decision support to improve automation-human interaction," says Darling. The project, now in its second year, is currently being led by Craig Bonaceto and Alexander Enzmann.

Allocating Responsibility Is Context Dependent

"The START research is in an area of decision support where the allocation of responsibility between humans and automated systems is context-dependent," says Sandeep Mulgund, lead of MITRE's Decision Support Technology Area Team. "We want to be able to shift control back and forth between humans running a process or the process being run through automation. It's important that the automation be reliable and keep the human in the loop so the human decision maker can understand what's going on. It's a very complex challenge."

"This represents a shift in the way we've traditionally thought about the design of automation and artificial intelligence in the context of decision support systems," adds Bonaceto. "The model used to be that automation replaces a task that people originally performed because the technology can do it faster and better. But in complex and dynamic environments, the automation may have incomplete or uncertain input data. New situations may arise that were not considered when the automated algorithms were conceived and require special treatment.

"People are adaptive and flexible. They can fill in the missing pieces and decide how to best guide an automated system to an appropriate solution. We believe that the task can be performed better if the humans and automation engage in partnership with each other. We view automation and artificial intelligence as tools that can amplify and extend, rather than replace, human capabilities. The key is to provide visualizations that capture the structure of the decision-making problem and give the operator insight into the quality of potential solutions."

Before being applied to START, these concepts were developed and refined under a MITRE sponsored research project called Mental Models in Naturalistic Decision Making. That research applied the concepts to the design of a decision support system for weapon-target pairing in time sensitive targeting.

With START, the warfighter will get multiple, viable options about which ISR asset to use from an automatic optimization plan that identifies all the ISR assets, where they should go, and what targets they should collect information on. Darling and her team developed the START architecture, a research prototype of automation-human interaction techniques that include the warfighters as partners in the automated retasking. Their prototype is currently being extended to support new automation-human interaction techniques that enable operators to understand, constrain, and refine potential retasking solutions.

Two-Part Architecture

The START architecture has two parts—the Decision Support Client and the 3D Mission Rehearsal Client. The Decision Support Client helps the user understand the "what if" scheduling options for an asset and tradeoffs between the options. Katie Minardo, a designer for the START team, created an intuitive visualization of the comparative risks and values of each option. The design allows easy comparisons between the original plan and the replanning options. The "what if" options are then simulated on the 3D Mission Rehearsal Client, which is built on the World Wind software, developed by the National Aeronautics and Space Administration (NASA).

The START program could also enable UAV coverage to be more efficient by changing the "ownership model" of each vehicle from a "customer-based" operation to a "target-based" operation. Customer-based operations occur when a customer—say, an Army squad—controls which targets the UAV monitors. With target-based control, on the other hand, the collection managers in the CAOC can direct a UAV asset to any new target. Their focus is always on high-priority targets, so retasking occurs more frequently.

Getting Information Straight from the Source

Part of the START team's work involved interviewing collection managers who deal with ISR assets. "They thought our concepts were promising because they currently use standard office software for holding the replanning data and all the decision-making has to happen in their heads," says Darling. "They thought that our prototype could help—because they could understand the advantages and disadvantages of each retasking option. Currently, their software will give a single 'answer' and there's no way to work with it or trust it."

In discussing the current data collection process with collection managers, Darling and her team found a variety of problems, including:

• Priorities and requirements were often in conflict.
• Requests for ISR assets were not supported with actionable information, instead requesting analysts to "report any suspicious activity."
• When collected imagery is posted in a central location, it was difficult to find by the unit that requested it.

START was recently tested with users who participated in the Coalition Warrior Interoperability Demonstration (CWID) this June. CWID, which includes members of the U.S. combatant commands and their international partners, focuses mainly on new technologies that can be moved into operational use within six to 12 months after proving their usefulness.) The findings from the demonstration haven't been released yet, but Kalghatgi expects to get valuable suggestions for improving START.

Next Steps

The next step for the project is to conduct an experiment at the United States Military Academy, West Point, N.Y. Although START is tailored for the Air Force, the need to provide good decision support tools for operators applies to the Army as well. The START team will collaborate with Dr. Ericka Rovira, assistant professor of engineering psychology in the Department of Behavioral Sciences and Leadership at the Academy. Professor Rovira and her students, Cadets Austin Cross and Evan Leitch, will investigate varying the reliability of the automation and presenting those confidence scores along with the retasking options. Freshman cadets will participate in the study for research credits.

"The decision support system for ISR asset replanning needs to be aware of ad hoc targets not in the plans, targets that have been collected, and targets that no longer need intelligence collected," says Darling. "The cadets will provide an Army perspective as well as actual metrics on how the tool improves decision making."

Adds Bonaceto: "Armed with the results of the experiment, we'll have a clear idea of human-automation interaction concepts that work well and concepts that we'll need to improve and refine as we move forward."

—by David A. Van Cleave

Related Information

Articles and News

• The Edge: Unmanned Systems Issue
• UAVs Are Likely to Affect Our National Airspace System
• Selecting the Best Technologies for the Navy's BAMS UAV
• Outthinking the Enemy: Card Games Are War Games

Technical Papers and Presentations

• Spatio-Temporal Analysis for Rapid Tasking (START)
• Structure Mapping in Visual Displays for Decision Support [PDF, 150KB]

Websites

• MITRE's Command and Control Center (C2C)
• MITRE Research Program

Solutions That Make a Difference.®
Copyright © 1997-2013, The MITRE Corporation. All rights reserved.
MITRE is a registered trademark of The MITRE Corporation.
Material on this site may be copied and distributed with permission only.

Privacy Policy | Contact Us