Building Blocks of Intelligence for Threat Defense

January 2, 2015
Cyber Threat Intelligence: Post by Mike Goffin

In my first post, I introduced Collaborative Research Into Threats (CRITs), a MITRE-developed tool released to the open source community.

In this post, I'll talk about an approach for creating your own threat intelligence picture based on threat data – from both external and internal sources. The approach is tool-agnostic, but I will make use of terminology used in CRITs.

By understanding your own threat picture, you can fine-tune your defenses and better understand the capability and intent of the cyber adversaries attacking your organization.

Not all Artifacts are Equal

When beginning a threat intelligence program, there is a tendency to ingest data from every source available. Although this might help get you started, it usually ends up creating too much unnecessary data that could slow down the work of your threat intelligence analysts.

When selecting artifacts to ingest, consider the reputation and reliability of the source (e.g., security vendors posting APT exposés, internal logs generated by vendor tools and capabilities), and the relevancy of the data (e.g., email headers and content, forum postings).

The best advice is to start with small sets of data from your own logs and sensors and from external sources. Whether or not you’re using CRITs, annotating the data you ingest will help you determine its value and usefulness over time. By starting small, the collect/annotate-analyze/review- correlate/instrument cycle will be manageable and will help your threat intelligence analysts refine and calibrate their analytical judgment. This, in turn, will help make it possible to formulate intelligence about a particular incident or adversary.

Threat Sharing

Belonging to a threat sharing community can help fill out your own program. Rather than sharing raw data, most sharing organizations contribute data that has portions redacted due to sensitivities, such as employee names or customized attack links. This broader view of the threat landscape creates threat intelligence from which you can create signatures that could give you an edge in advanced detection. To reduce false positives that could affect your business operations, be sure to verify and validate the data before leveraging it for threat defense. Many advanced adversaries customize their attacks based on a specific target, such as customized malware for your environment, thus making your own raw data more valuable since it is specific to the threats faced by your own organization.


When considering your raw data sources, such as network traffic and email, it's important to have first determined what is "of interest." Perhaps a sensor triggered or an email was trapped by perimeter defenses. Or maybe you are doing a historical search based on new indicators and got a hit from one of your logs. Ingesting this raw data will help your analysts correlate related data to formulate a threat intelligence picture that becomes more comprehensive over time.

Network traffic

Placing taps on your network, such as at the perimeter or at key borders, is essential. Capturing network traffic (full packets (PCAP)), and keeping it for as long as possible forms a foundation that gets to the "ground truth" about activity on your network. This network data provides historical insights for your threat analysis: communications from a compromised system to its command and control channel; certificates that you may later learn are falsified; IPs and domains that originate or go to countries or sites determined to be suspect; and, potentially, data that might help you find an adversary's lateral movement within your network.


Consider your pre-scanning (e.g., perimeter AV) and post-scanning (e.g., server and endpoint AV) capabilities and how email traverses through these points on ingress and egress. Then ask yourself what happens if an email makes it past your detection mechanisms. Do you have threat intelligence from sharing partners that might provide timely detection of an attack, and do you have a process for removing the email from inboxes? What about a way for users to report suspicious emails? Because email is the most widely used attack delivery vehicle, it contains a wealth of potentially interesting data, such as the from address and actual sender address, subject lines (you'll begin to see patterns used by adversary groups), host names, IPs, and domains included in links and payloads, time zone information (adversaries from different regions), and x-mailer strings (which can tell you the email program used to send the email), and the "to" information, that is, who in your organization has been targeted. I won’t address targeting analysis at this time, but needless to say, this too can help you formulate a larger threat picture for your organization and industry.


Firewall, proxy, and citrix logs, to name a few, can help your analysts get a more complete picture of activity, and assist in determining what action to take, including keeping watch lists. This correlation activity brings into focus events that might be suspect. You will need to determine which logs, and the degree logging should be configured, to cater to your specific interests. Create a short-term (3-to-6 months) analysis cycle to try out a source and tune it, if necessary then assess its value to your program.


A variety of intrusion detection systems and dynamic threat detectors are on the market, from host-based capabilities to network and email-based sensors. Many of these products will take raw data inputs and groupings of atomic indicators to tune detection for your particular environment. Alerts, sent to your monitoring and response team, should also be considered artifacts for potential inclusion in your analysis tool.

Conclusion: Producing Threat Intelligence

Now that you have an idea about what to collect, you'll need a centralized tool, such as CRITs, to help store your internal and external artifacts and intelligence. By creating this warehouse of information, your analysts (e.g., malware, threat intelligence, and monitoring and response) can create an accurate threat landscape picture and then can respond to threats effectively. Regardless of which tool you select, it should be flexible, extensible, and programmable. While automatic analysis isn't possible, being able to automatically expand and correlate data and generate new artifacts leads to intelligence and to defining tradecraft used by cyber adversaries. And that can, and will, make a difference to your threat-based operations and defense strategy.