With HSI Microscopy, Every Pixel Tells a Story

April 2015
Topics: Meteorological Factors, Remote Sensing, Signal Processing, Image Processing, Sensor Technology
Airborne spectral imagery of objects on the ground can identify important details, such as their composition. But traditional imaging systems can leave information gaps. MITRE's Hyperspectral Imagery Microscopy project seeks to fill those gaps.
MITRE Principal Scientist Ron Resmini

Dirt. If someone asked you if you knew what it was, you'd say, "Of course." But MITRE Principal Scientist Ron Resmini would say, "I'm not sure. Tell me more." The need to know more—about pollutants in the dirt, about moisture and dust particles in the air, about enemy troops in a forest—is the driving motivation behind his Hyperspectral Imagery Microscopy research project.

Spectral imaging is used to identify the composition of an object by collecting and analyzing the light reflected by the object. MITRE's sponsors use spectral imaging sensors equipped on airplanes and satellites for surveillance or to conduct environmental, geological, or climatology studies. Researchers will sweep up spectral signature data, analyze it, label it, and store it in a database—called a spectral library.

The drawback to this process, says Resmini, is that spectral libraries carry too little information about each data entry. A scan of a soil sample in a Petri dish, for example, will too often consist of a single spectral signature under the label "SOIL."

Getting the Dirt on Dirt

"Dirt's dirt, right?" he says. "But of course it's not. There are differences in the organic and inorganic contents, moisture differences, particle size differences. So a guy like me will come along wanting to use that signature to do soil mapping and say, 'Tell me more about the dirt,' I need all the details."

Resmini's goal is to use hyperspectral imagery (HSI) microscopy to populate signature libraries with more information, more metadata. Currently, spectral libraries are built using laboratory sensors that measure a single spectral signature from a small sample of material. HSI microscopy will generate several thousands of spectra of the same sample as an image. This allows for a more detailed look at, and analysis of, the individual constituents of the soil. Ideally, by looking up a soil signature in a spectral library, researchers will be able to know the exact composition of that soil and all the ways it differs from other soil signatures in the database.

Smart Signature Storage

Won't vastly increasing the amount of information in a database library make it more difficult to find or match a signature? "If I scan a petri dish full of dirt using HSI microscopy, I'll end up with full-spectrum scans of tens of thousands of pixels, which is an enormous amount of data. Does every single pixel need to go into the library? No. We can take averages of the signatures and then build probability models of how the signatures vary. Then we use those models to tune our search algorithms.

"So you don't necessarily need to put every pixel's signature in the library to learn more about your material and to help search algorithms recognize the material."

Soiled Soil

Resmini had a chance to put HSI microscopy to the test when the U.S. Geological Survey came to him with a challenge. The USGS needed to determine if certain sites were contaminated with asbestos. However, asbestos wasn't present in enough quantity in the soil to show up on field-based spectral scans, forcing the USGS to bring the soil into the lab for testing. They wanted to know if HSI microscopy could better spot the asbestos.

"We conducted an experiment," he says. "We took some soil from the sites and sprinkled a little bit of asbestos on top. Then we measured it with a traditional point spectrometer and couldn't detect the asbestos. When we measured it with HSI microscopy, however, we could see the individual asbestos particles." The USGS is now a partner on the HSI microscopy project.

MITRE transitioned the HSI microscopy capability to the National Institute of Standards and Technology, its partner on the project for the last three years. NIST and the USGS will jointly study the effectiveness of HSI microscopy for other challenges such as detecting heavy metals in vegetation.

Eyes of HSI

Resmini and his colleagues have recently published two papers on their research. Now, the team is looking to tie its findings into current MITRE research into computer vision.

"We're currently working on a project to analyze data from multi- and hyperspectral video cameras. We're speeding up our hyperspectral imagery algorithms so we can apply them to video-rate spectral imagery."

Resmini believes that MITRE's hyperspectral algorithms will open up whole new vistas in imaging. "It all comes down to the question, 'What am I looking at?'" he says. "By filling in the gaps of information that exist with traditional imaging systems, MITRE is trying to better answer that question."

—by Christopher Lockheardt


Interested in MITRE's Work?

MITRE provides affordable, effective solutions that help the government meet its most complex challenges.
Explore Job Openings

Publication Search