Performing network defense can be a daunting task. In an incident response scenario you must be able to answer questions such as, "What happened, and what data left our network?" Assuming you have packet capture from the incident, being able to answer these questions hinges on your ability to understand the protocols in question. If it's a common protocol that Wireshark has decoding capability for, then answering those questions is as easy as using Wireshark. But what do you do when it's a protocol Wireshark doesn't know how to decode? Well, you have to do it yourself.

There are a handful of available tools built around the need to decode network traffic. The most ubiquitous of them is tcpdump. While great at many things, its ability to do Layer 7 analysis is lacking and it is cumbersome to extend. Another commonly used tool is Wireshark. It has significant improvements to Layer 7 analysis over tcpdump but it's still cumbersome to extend. While the Lua interface eases this awkwardness to some extent, it's still an overbearing interface to use.

There has to be some middle ground when it comes to traffic analysis. We need some capability that lets us, as protocol analysis specialists, reassemble packets into their respective sessions and get at the payload with as little rework as possible.

Lucky for us there are open source projects which do just that. Specifically, I'm speaking about libnids. Its reassembly, while not perfect, is good enough for our needs. But who wants to write protocol analysis code in C? That's part of the reason why other protocol analysis tools are so cumbersome (and who wants to write in Lua? ;). Our language of choice for quick prototypes is Python, so it seemed an obvious choice to use pynids as our interface to libnids.

Using pynids is great because it gets us access to the reassembled data stream. That's a good enough start, but when it's 4:30 p.m. on a Friday and you want to get the decoder done, the last thing you want to do is write (or copy/paste) the necessary boilerplate pieces of code. I'm talking about the code that parses arguments to figure out how to get your pcap file as input, or how to open the file, or any of the other standard things a protocol analysis program does just to be able to get at the data that needs to be analyzed.

If you've ever done network traffic analysis with custom code, you know that the ratio of actual protocol decoder code to boilerplate code is not conducive to getting the job done quickly. This is why, time and time again, analysts will write a decoder once and then copy the program to a new decoder, immediately followed by deleting huge chunks of it and replacing it with the new decoder. While faster than writing it all from scratch, it's still not as fast as it can be and is rather error prone.

With all that said, I'd like to introduce chopshop. Chopshop is a MITRE-developed protocol decoder framework built around pynids. One of the goals of chopshop is to eliminate the mundane work that goes into writing protocol decoders, allowing decoder authors to quickly write the important parts. By abstracting away all the boilerplate code, chopshop only requires decoders to conform to a simple, easy-to-use interface.

Another goal of chopshop is to make it easier to share decoders. Because there is a standard interface for them, they can be easily shared. Users will not have to read the code to understand how to run the decoder. The logic for the decoder is logically isolated from the mundane pieces.

In a future post, I'll discuss chopshop in detail. I'll cover how to write decoders, what the API looks like, and what the data structures involved look like. I'll provide examples of different decoders and other ways to use chopshop for protocol metadata extraction.

If you want to pick up chopshop and explore it on your own, it's available at:
https://github.com/MITRECND/chopshop.