Locating the Right Geocoding Tools Helps Agencies Make Informed ChoicesJuly 2015
Topics: Geographic Information Systems
Nowadays, you can type an address into a search engine, and Google Maps will immediately transport you across cyber space to the site of a childhood haunt, favorite vacation spot, or ancestral landmark. With the technology literally at our fingertips—in our mobile phones and tablets--mapping the world seems easier than ever.
But it really isn't, says Carrie Muenks, a geospatial expert at MITRE. The current explosion of mapping tools ranges from unreliable and simplistic to highly sophisticated. Moreover, government agencies often need more than just an address or location on a map. They need the most precise information they can get. "That’s a huge challenge for any organization," she says.
To help the government make better mapping-technology choices, staff from MITRE's Homeland Security Systems Engineering and Development Institute (HSSEDI TM) developed an assessment framework to sort through the options, distractions, and challenges of the marketplace. (HSSEDI is the federally funded research and development center MITRE operates for the Department of Homeland Security.)
Geocoding Tools Identify Location, Location, Location
Why is this selection process so complex? For one thing, the growth of the geospatial-tool marketplace has fed the growing priority of organizations—both commercial and governmental—to locate things. "Things," of course, could be anything from potential terrorists' haunts to intended mail recipients and receptive advertising targets. The Internal Revenue Service might want to find taxpayers who still resist e-filing so it knows where to focus its efficiency campaigns. Insurers might want to identify homes located in floodplains to determine appropriate premiums.
There are numerous types of geospatial-related systems, some of which enable you to identify the precise latitude and longitude coordinates of a physical address. This is a process known as geocoding. Competition has led to an increase in the availability of these tools—referred to as geocoders—in the geospatial marketplace. This abundance of choices makes it difficult to assess what's the most appropriate software for a specific use.
As a result, without deeper investigation, there’s no guarantee that the geocoder you use to identify a physical address is pinpointing the right spot or if it’s off by a street—or a city. That's where the MITRE team stepped in.
The team originally set out to help DHS' various components and departments identify and evaluate the best geocoding systems for specific use cases. But the framework can also help other federal, local, and state entities understand how to choose the most appropriate geocoder.
"Government agencies want to harness big data to tackle a host of issues, including counter-proliferation, counter-terrorism, cybersecurity, fraud detection and prevention, and intellectual property protection," says MITRE's Chuck Lewis, an authority on big data. "Geospatial capabilities and the ability to accurately geocode addresses are an integral part of big data analytics. Conducting assessments like these allows the government to select the best systems to meet their requirements."
Assessing Options, Narrowing the Field
For the DHS assessment, Muenks led a team of geospatial and IT experts in evaluating a wide array of geocoders over about eight months in 2014. The tools included both commercial- and government-off-the-shelf products, as well as open source solutions. The MITRE team tested the tools' different capabilities against DHS requirements in several areas, such as degree of accuracy, ability to operate in a disconnected environment, and worldwide coverage.
Another key factor was a system's ability to handle "messy" data. This ranged from incorrect address structures (such as street name before box number) to incorrect state abbreviations and missing ZIP code digits. In addition, researchers considered nearly 100 non-functional aspects (those not directly related to technological capability). Some of these factors included how responsive the vendor's help desk was and how easily the system scaled up to handle more requests. Cost was another factor.
The MITRE team trimmed the initial list of nearly 40 contenders down to four. The team then tested these systems in MITRE's labs using real domestic and international data, as well as synthetic data.
Of the remaining four systems, no single geocoding solution out-performed the others; each had various flaws. However, the differences between the systems in terms of DHS requirements became evident. The framework allows organizations to make educated decisions based on their particular needs.
One Good Assessment Leads to Another
The MITRE team delivered a report to DHS outlining their findings and recommendations, as well as additional considerations for any similar studies in the future. The DHS Science and Technology Directorate (S&T) has since shared this independent assessment with the DHS Geospatial Management Office, DHS Components, and other federal agencies.
Based on MITRE’s assessment, the DHS S&T Homeland Security Advanced Research Projects Agency is now employing a geocoding solution in its Predictive Analytics Center. And because of the success of MITRE's methodology, DHS has requested we conduct additional independent assessments for other markets of analytical software.
"Organizations have typically had to rely on trial and error when it comes to geocoding," says Muenks. "But now we have given them a way to kick the tires and make fully informed choices."
—by Twig Mowatt