To Check the Box or Not to Check the Box: Is that Really the Cybersecurity Question?

September 27, 2017
Cyber Policy: Post by Aaron Temin
Aaron Temin

Checklists are good for all sorts of things: keeping medical instruments from being sewn up unintentionally in patients' bodies; helping a pilot ensure an aircraft is airworthy before take-off. There’s even a popular book, The Checklist Manifesto, extolling their virtues. But apparently, checklists are bad for cybersecurity. Article after article complains about a "check box mentality"—meaning that someone checked off all the items on the cybersecurity defenses list and still was left with a vulnerable system. Despite this, checklists remain a very popular way to describe how to defend one’s systems.

Well, the problem may not be with the checklist itself but how it is used. Let’s see if we can rehabilitate the venerable checklist and improve our cybersecurity at the same time.

It is understandable why people want to rely on checklists. Cybersecurity is complicated. And a typical way to handle complexity is to divide and conquer—break the problem into a list of smaller problems, and handle each of those in turn. This works for some aspects of the cybersecurity of a system but not all. That is because some weaknesses arise as emergent properties when different IT components are combined. Each component by itself might be secure, but put them together and security is compromised.

For example, in one of our labs we have a smart light bulb and a smart door lock. Each can be controlled wirelessly from a mobile phone. Each is reasonably secure when used independently. But put them both on the same network and the door lock becomes susceptible to a replay attack that is enabled by a feature of the light bulb.

An important distinction to consider is between building and operating a system. Building a system is an activity that often takes considerable time and rarely requires quick response. There is generally time to ponder and deliberate. Operating a system, especially responding to an attack, can be time-critical. The medical and aviation analogies are both operational and support the conclusion that checklists can be most effective when a person is under stress and likely to forget something important.

Checklists belong in a security operations center to be used for the time-sensitive activities involved in incident response—when you want to know that the basic steps have all been covered, that all the high-level areas that need to be considered have been. However, there are many articles proposing various checklists for designing and implementing cybersecurity. Some of these are caveated as being "good starting points," though the tendency is probably to view them as providing complete guidance.

Checklists should not be seen as a substitute for expertise. They can help provide some consistency to a cybersecurity approach in an enterprise with multiple experts designing and monitoring cybersecurity. They can also help remind an expert of areas where protection is needed (or needs to be examined), or serve as an aid when organic memory might fail.

And just as they won't teach you how to fly a plane, checklists cannot be used to train a novice in how to properly secure an IT system. Judgement and adaptation are critical skills that are needed, and no checklist will be able to meaningfully inform someone who does not yet have those skills.

The ultimate solution is to improve the process by which they are used. We don’t want to accept a bunch of check marks as a substitute for good cybersecurity. But we can use a solid checklist that serves as an aid, not a hindrance, to help us remember to defend all the aspects that are both applicable to the system and to the level of risk that is acceptable to the owning organization. They can be used as partial memory aids, as long as higher-level principles with broader reach take precedence.

A good approach to cybersecurity will consider what you want to protect and defend (with a justification as to why) and how much you want to spend on the protection. You can use an expectation of the threats to your system to further prioritize what defenses to use and where to place them. At some point, a checklist might be helpful to get to the next level of detail in your design (though there is a good chance that you have enough experience and expertise that you won’t need it.)