Home > News & Events > MITRE Publications > The Edge >

The potential benefits of overhead imagery from both government and commercial sources are huge: early warning of agricultural problems, analysis of population changes, discovery of terrorist activity, etc. However, reaping these benefits assumes the availability of manpower to actually look at the images.

The reality is that we are able to review only a fraction of what is available, and with several new commercial satellites going up in the next few years, the problem will only get worse. We can use imagery, especially satellite imagery, to confirm what we suspect (e.g., training activities are underway in a remote area of the world) or to monitor activity at specific sites (e.g., production at a foreign weapons factory is increasing), but we simply don’t have the manpower to look through all this imagery to find something new.

Automated Analysis of Overhead Imagery
MITRE has addressed this problem of insufficient manpower by developing technology to detect “unusual” changes over time via automated analysis of imagery. Using data mining techniques, we compare “before” and “after” images of a site, and identify uncommon—and potentially interesting--changes between the images.

The first step is to factor out the expected, normal changes, such as seasonal differences or lighting intensity. We use data mining techniques to identify the normal changes over time between images. Given a pair of images, we train an Artificial Neural Network (ANN) to recognize what should be in one image given what is in the other image. That is, we feed the intensities of the various colors for a given point in one image as input to the neural network, and the intensities of that point in the other image as expected output. ANNs perform a weighted combination of inputs to generate outputs. Training automatically adjusts the weights until the resulting input-to-output function is correct for all (or most) of the training examples. Each pair of points in the images constitutes one training example.

We use all points in the image as training examples. This enables the trained ANN to recognize that, for example, the light brown of a dirt road should stay light brown, while the bright green color of newly sprouted wheat in the spring should become the brown of earth in a picture taken after harvest. The trained ANN is then used to get expected values for each point in the new image: for a given point, the intensity values from the old image are fed to the ANN, which produces expected values for the new image. The expected values are compared with the actual values in the new image; if they are substantially different, then that point is flagged as an unusual change. Defining what constitutes a “substantial” difference between the expected and actual values involves statistical analysis of the entire set of predictions as well as correlation of prediction between various colors. Note that something staying the same could also count as unusual. For example, a bright green spot that stays bright green would be identified as unusual, if the ANN would predict that it should turn brown.

Images taken near Portsmouth, New Hampshire several years apart, with circles indicating automatically detected changes.

The key to this process is that a new ANN is trained for each pair of images. The ANN doesn’t need to know the concepts of earth, wheat, or road. It learns what the common color changes are for that pair of images only. A different neural network is trained for the next set of images, and so on. However, the training for each ANN is done automatically from the images--no human effort or a prior knowledge is required.

Detecting changes between sets of grayscale images.

Significantly Reducing ANN Training Time
The main difficulty with the approach described above is that training ANNs is a computationally intensive process. With commercial data mining technology, however, the amount of ANN training time is significantly reduced. For example, MITRE tested a commercial data mining tool that uses neural networks and found that it trained in one percent of the time needed by the research neural network tool we had been using. This alone makes the process practical. We expect additional developments in data mining research to improve upon this even further.

---

For more information, please contact Chris Clifton using the employee directory.

---