Days of the Roundtable: Jousting with Big Challenges in Data AnalyticsNovember 2018
Topics: Data Analytics, Acquisition Management, Government Acquisition
There's a benefit to a roundtable: no one takes precedence over another. Everyone can contribute or take the lead equally.
That's one of the ideas behind the Analytic Technology Industry Roundtable, which is helping meet the needs of government customers. As host of the Roundtable, MITRE convenes representatives from both government agencies and commercial developers to hash out their biggest issues--together.
Although the Roundtable members meet monthly, and working groups meet throughout the month, the broader analytics community comes together at annual industry days. This year, Industry Day VII will reveal results from pilots featuring the Roundtable's signature achievement, the Analysis Exchange, or AE. The group will also talk about accomplishments in acquisition and procurement, notably, work with the Section 809 Panel.
The AE tackles a problem that's central to many areas of information technology: How do you exchange different kinds of data across different platforms and make sense of it? That's important to any IT-related endeavor. But when it comes to analytics for national security or for reducing tax-related fraud, that challenge takes on greater urgency.
It's a problem important enough that agencies from the U.S. Army and Air Force Intelligence took part in the first AE pilot programs. At the 2018 Industry Day, held on November 7, participants in AE pilots will discuss their findings.
"We ask hard questions, and we need industry to provide solutions," says MITRE's Angela O'Hanlon, the driving force behind the Roundtable's creation. "They huddle and come back with recommendations, and really sound and thoughtful plans. I'm hoping what we talk about at Industry Day—the pilots and work with the Section 809 Panel—will be examples of future work, higher impact and collaboration on hard problems."
The Analysis Exchange Broadens the Marketplace
At its most basic level, the Analysis Exchange acts as a hub that brokers analytic artifacts, knowledge, and results between analytics and analyst customers. Members of the Roundtable worked together to design an architecture that allows tools from different manufacturers to communicate. It's a big step forward in data interoperability.
MITRE encouraged the Roundtable members to swing for the fences to demonstrate the value of the unusual "collaboration-among-competitors" model. And with our unique vantage point working across government, we could serve as the neutral referee for the discussions. The goal: Come up with a technical solution that would make it easier for agencies to acquire systems—perhaps multiple systems—that could solve their analytics challenges.
"We wanted to collaborate on a big problem, so government agencies could see all the options," says Richard Games, a MITRE chief engineer and early champion of the Roundtable. "So, the first thing the Roundtable worked on was the Analysis Exchange. Developers can build technology that intermediates between the tools and demonstrate ways to solve a mission problem. And they can do it better together than separately."
Games notes that this is a natural role for MITRE—serving as a hub for collaboration. "The government isn't paying for this. We saw a need and filled it. If the companies can't develop products that agencies need, they can't stay in business, which in turn reduces innovation for the government. We are fortunate to have these kinds of trusted relationships that encourage innovation."
A Pilot to Benefit the Army's Training Capabilities
As chief information manager for the Army's TRADOC G2, Gary "Chip" Retzlaff offered to participate in an Analysis Exchange pilot. TRADOC, the U.S. Army's Training and Doctrine Command, is responsible for (among other things) assessing analytic tools for training, development, and operational environment products and services. (Retzlaff was already familiar with the precursor to the Roundtable, MITRE's Analysis Tool Shed Laboratory. He performed analysis-of-alternatives testing on products within the ATS Lab, which allows government agencies to evaluate more than 100 commercial analysis products to support their missions.)
"One of our biggest challenges is the accreditation of tools," Retzlaff says. "The core challenge was that the data and information from one vendor's tool doesn't flow easily to another's. This kept vendors from competing for contracts because they couldn't share information. And my agency couldn't get these tools."
This made acquisition of the best-possible solutions difficult, to say the least. When O'Hanlon asked to him to propose a use case to test the new Analysis Exchange system, Retzlaff proposed a real-life TRADOC G2 challenge—improve the recently launched Decisive Action Training Environment.
"They put two teams of industry partners to work against this use case," he says. "Team 1 had six industry partners. Team 2 has five. They work within their teams and share ideas. It's not really a competition. They strive to solve different parts of the problem, which can then be integrated."
He explains how the pilot worked. "We [TRADOC G2] provided the representative sets of data. The Decisive Action Training Environment is built for units that want to train—they can set up their 'variables.' Since that's now in a data format, it’s one of the data streams." Some of the other streams are open source, in yet another data format.
"We told the teams what data we go after and gave them sample products of output. In the past, we'd used a manual process, so we could show them what output should look like."
Closing a Gap in Analysts' Time
By this year's Industry Day, Retzlaff says his team will have worked on the pilot for about 10 months. He's happy with the results so far—and ready to take the AE further.
"Certainly, we've had savings on time, and the work has also provided us with a better product in the long run. In the next year, it will cut our analysts' time by about half, closing a significant gap. Analytic outflow from tools will be better, more efficient, and better for the training office across the Army."
Retzlaff notes that he was skeptical at first that such a process could work—with competitors collaborating—but he's been won over. He praises MITRE for making it happen.
"MITRE led the partners through the whole thing. They really served as the glue or middleware to show how this problem-solving can occur. For our use case, we needed a technical solution. But when you look back, the creation of the exchange wouldn't have happened without MITRE putting this process into place."
And he really appreciates that the AE was structured from the beginning for success beyond the pilot. "MITRE has released this as open source so anyone on the government or industry side can use it. It’s not a one-off development. Anyone can take advantage of it. My goal next year is to take it to a non-pilot implementation beyond TRADOC G2."
He encourages his representatives from other agencies to join in. "We certainly appreciate the Roundtable's work and want the rest of the sponsor community to know this is a great place to bring your challenges."
O'Hanlon agrees. "One of the things we learned early—if your competitors are at the table talking about something, it's better to be at the table than on the sidelines. When they discuss how to help government or sell to government, it's most productive to take a leadership role than sit back and observe."
—by Alison Stern-Dunyak