Introducing ChopShop 4

April 11, 2014
CND Tools: Post by Murad Khan

Since the time we first introduced ChopShop, we've received lots of feedback on how useful a tool it's been to CND. In our continuing effort to make it easier for developers to create decoders and other ChopShop modules, we recently released ChopShop Release 4.0, which brings in a number of new features, the most significant of which are adding arbitrary data or protocol types and module chaining.

The ChopShop Enhancement

Module chaining allows an analyst to focus on decoding malware without the burden of the underlying protocols used to transport the malware. Case in point: If you're analyzing a piece of malware that runs over HTTP, you shouldn’t have to worry about first turning TCP into HTTP.

ChopShop Release 3.x: the old way of decoding malware using HTTP as a transport

In ChopShop 3, you'd have to add code to first decode HTTP, and then decode the malware traffic. Malware that uses HTTP as a transport, without any modifications, adds an unnecessary burden to any new decoder you write.

Figure 1 illustrates the added burden on decoder developers utilizing ChopShop 3.

Figure 1 ChopShop 3 Module Design

Even if you used something like htpy to process HTTP, you'd still be adding possibly hundreds of lines of code, unnecessarily. Not to mention possibly propagating errors via copy/paste of the code into other modules.

ChopShop 4: the enhanced way of decoding malware using HTTP as a transport

ChopShop 4 alleviates the multi-step decoding within a new decoder by introducing arbitrary data types and adding the ability to chain modules. Module chaining separates out parsing underlying protocols from upper level protocols by forwarding them to modules designed for the specific data type.

So how would ChopShop 4 handle the case of malware delivered over HTTP? You could first create a module to parse HTTP that then feeds the parsed data into a decoder module. The decoder module, which expects to process 'http' typed data, then only has to parse the malware. To make things even easier, ChopShop 4 already provides an 'http' module!

Figure 2 illustrates the segregation of modules that ChopShop 4 provides.

Figure 2 ChopShop 4 Module Design

How Developing ChopShop Modules Just Got Easier

If you're already a ChopShop user, and wonder about the effects of upgrading to ChopShop 4, or how you can take advantage of the enhancements in ChopShop 4, then these questions and answers might help.

1. How does using ChopShop 4 change my existing modules?

Well, it shouldn't! Modules that use the ChopShop 3 format should still work without modification. However, there are two main points to consider. The first is that print function support in module_info has been entirely removed. It was deprecated in Release 3 and is no longer supported in Release 4. If you have a module that still uses it, when module_info runs, nothing will be printed. The second change is internal and has to do with how internal memory structures are handled. If you've stuck to the tenet of "Don't use globals," your module shouldn't be affected.

2. How would I run chained modules in ChopShop 4?

An entirely new command line parser has been built to handle the new grammar format that comes with Release 4.

The format in Release 3 was quite simple and didn't require much processing:

module [arguments] [ ; module [arguments]]

The new Release 4 format supports chaining modules (by using the pipe character [|]) and TEEs and reverse-TEEs (using parentheses and commas). Here are two examples of how to create complex chains:

http | (http_extractor, my_http_malware_decoder); dns | dns_extractor


(icmp, dns) | backdoor_detector | (logger, reporter)

There's more latitude in Release 4, which frees analysts to concentrate more on analyzing malware traffic.

3. How does a module author take advantage of ChopShop 4?

ChopShop 4 introduces a few changes to the structure of a module to support the new module format. Going over all the required elements would be quite lengthy, so I recommend taking a look at the docs folder in the distribution to see what's required or look at the http and http_extractor modules, since they're great examples of how chaining can be used.

Perpetual Beta

Work continues on ChopShop to make it even better for the CND community. If you have any comments, requests, or questions, check out the github repository or feel free to contact the developers.