How ChopShop Modules WorkFebruary 21, 2013
In the previous two posts I talked about the why of ChopShop and the what of ChopShop modules. In this post I'll talk about the how of a very simple module. I'll be using the terms and objects discussed in the previous post as I walk you through the
payloads module that ships with ChopShop. To provide further context, I'm including sample traffic in a pcap file.
Let's say that you've been called in to help analyze some traffic collected during an intrusion. The malware analyst has determined that the malware opens a socket, and when something connects to it, a command is sent that will cause the malware to spawn a shell, execute the command, and return the result to the caller. The malware analyst has also been able to determine that the data for both the command and the response is obfuscated by xor'ing each byte with
Your job is to determine, via the packet capture, exactly what was sent over that shell command channel.
First, let's examine how the
payloads module can help you answer that question quickly.
payloads module will print out the payload of each TCP packet in a stream (sample traffic pcap shown below). This is very useful for looking at plaintext shells. It is roughly equivalent to "follow TCP stream" in Wireshark.
In our case the data is not plaintext, but is in fact obfuscated with a single byte xor. The
payloads module can also handle this situation by using the
'-x' option. (Note: For more information on the usage of the payloads module please run
'chopshop -m payloads'.)
The first thing ChopShop is going to do is initialize the
payloads module, by calling the
This is about as simple as an
init function can be. It stores some module instance-specific variables in the module_data dictionary, sets up the return dictionary to specify that it is only interested in TCP streams, calls a
parse_args function and returns the dictionary. Although the
parse_args function is local to the
payloads module, it is not shown here for brevity. The
parse_args function is doing the usual argument parsing and storing any of the optional argument values in the module_data dictionary. It is also worth noting that the argument string that is parsed is passed in as the 'args' key in the module_data dictionary.
As ChopShop is reassembling the TCP stream from the pcap file, the first thing it will encounter is a completed TCP handshake. When a new handshake is completed, ChopShop will call the
taste function of each module that has specified an interest in TCP streams. The
taste function for
payloads is also simple.
tcp object that was discussed in the previous post is passed in to the function.
The purpose of any
taste function is twofold. The first is filtering unwanted traffic from your module. It is generally recommended to not do any filtering here but instead use a BPF filter, as it will be faster. If you have to do some kind of filtering at this point, remember that there is no payload data to look at. All you have available are the quad-tuple and the timestamp.
The second purpose of the
taste function is to allow your module to set up any stream-specific information in the stream_data dictionary. In our case the code is neither filtering traffic nor setting up any stream-specific information. It is, however, using this as an opportunity to print out a line so the user knows there is a new session starting. Because it is not filtering anything, the
taste function always returns
True. If it were filtering, it would selectively choose
False based upon some available criteria.
At this point the module has seen frames 1, 2 and 3 in the pcap file. Frame 4 is the first frame with a TCP payload. This is the next packet that ChopShop will reassemble, and will be the first payload passed to the
Directionality and Quad-Tuple Parsing
To make things easier, the
parse_addr function from
c2utils is used. This function takes a
tcp object and will return a correctly ordered quad-tuple based upon the direction of the packet. Without using this function the
dst (and their corresponding ports) would be swapped when it is a client packet. This is because the quad-tuple is fixed from the moment the first
syn packet is seen, and ChopShop uses this internally as a unique flow identifier, which means it is up to the modules to properly order the quad-tuple. (Note: For more information please read the
parse_addr function in
Now that the quad-tuple is properly ordered the module must determine if the packet is a server or client packet. It does this by checking if
tcp.server.count_new is greater than 0. If
tcp.server.count_new is not greater than zero, it is safe to assume this is a client packet. Remember that server packets are defined as the ones going to the TCP server.
Once the directionality is determined, some variables are set:
- data: A copy of the TCP payload portion for the current packet
- count: The number of new bytes being processed
- color: A string used later on to make things prettier when using the curses GUI
Data Manipulation and Printing
With the payload data collected, it is now time to process it. This particular module treats both client and server data identically, so the redundant code is outside of the directionality checks. The first thing to do is to (optionally) print out a new packet message in the appropriate
color. The module uses the
tsprettyprnt function from the
chop library. This function will prepend the current packet timestamp to the provided string, and print out the message in the given color (at the moment, colors are only viewable when using the curses GUI). There are multiple printing functions in the chop library for modules to use, but the two most common are tsprnt and prnt. (Note: For more information on the standard printing capabilities provided by the
chop library please see the ChopShop documentation).
Now that all the mundane stuff is done, the only thing left to do is to print out the payload data. The module provides a couple of options to the user when it comes to handling the data. If the user provided an xor key (the -o option), the
multibyte_xor function from
c2utils is used. This function will apply the given xor mask over every byte in the provided data and return the result. In our case, the data is the TCP payload. If the user wants to see the data in a hexdump format (the -x option), the data is converted to a hexdump string using the hexdump function from
c2utils. It's worth noting that at this point data will contain the data after the xor mask has been applied, if one was applied.
Finally, the data is printed out to the screen using the
prettyprnt function from the
chop library in the appropriate color. As this module tries to print payload data in a clean form, the timestamp is eliminated, which is why
prettyprnt is used instead of
Discarding Processed Data
Now that the entire payload has been processed and the data has been printed to the screen, the module tells the ChopShop core that it is done with the newly processed data. This is done by calling the
discard function from the
tcp object and passing in the total number of bytes that were just processed. It is possible to call the
discard function earlier in
handleStream with no negative side effects, as the bytes are actually discarded after the last modules
handleStream function has returned.
Teardown and Shutdown
This process is repeated for every reassembled packet in the session until the teardown state is reached. The rest of the code in
payloads is mostly optional and not essential to the data processing.
I hope that walking through the code with a pcap capture in front of you has helped you realize the ease with which you can write decoders in ChopShop.
Admittedly, this module is about as easy as it gets, so in the next post I will talk about processing more interesting protocols, like HTTP, by exploring how to use
htpy and ChopShop together.
By the way, I'll leave it to you to run this sample through the ChopShop module to discover what it contained.