Hurdling the bug bar
A look at Microsoft's Secure Development Lifecycle.
After spending some time critiquing qualitative vulnerability assessment descriptions as well as the ostensibly quantitative CVSS, I will now turn to the Microsoft Security Development Lifecycle (SDL). The SDL is a relatively well-known software engineering framework with many aspects to it, but I am going to focus on one in particular: the “bug bar.” For those of you with experience with it, you are probably getting ready to say “but this system uses primarily qualitative terminology!” This is true, and Microsoft’s categorization is by no means perfect. Furthermore, as Microsoft itself states, what they provide is merely a suggestion which can (and should) be customized further. For organizations that are just beginning to develop a vulnerability management program, however, and have to choose something already in common usage, I would recommend picking this tool over the CVSS.
The reasons for this recommendation are that it:
is a more effective risk management tool than the CVSS (though still imperfect);
provides easily understandable guidelines for classifying vulnerabilities; and
is internally consistent and results in far fewer odd outcomes than the CVSS.
The Microsoft SDL bug bar does not provide much information with respect to how quickly to resolve security vulnerabilities, but it does provide a starting point for relative prioritization. For startups that need to move quickly, and which might not even have a single dedicated information security employee, making a good, rapid decision is more important than making a perfect but delayed one.
As a quick aside, I would like to share a quotation from Jeff Bezos, who said that “being wrong might hurt you a bit, but being slow will kill you.” As others have written on this note, moving quickly is critical in business. It is always possible to delay decision-making (and more dangerously, action) to gather more information or input. This isn’t always the best way to proceed, though, as a failure to decide can sometimes be more catastrophic than a sub-optimal decision that you are able to iterate on quickly.
The need for rapid decision-making is especially relevant to the world of software security, where you should generally make decisions as soon as you have 70% certainty (this standard of certainty originated in the military and has migrated into the business world). As I have mentioned, people are generally not good at assessing opportunity costs and are often willing to expend immense energy when faced with making a tough call. Unfortunately, I have seen such indecision in the case of relatively trivial issues, which can cost many teams incredible amounts of time that would be better focused on more pressing items.
In a perfect world, achieving the highest possible fidelity decision immediately would obviously be optimal. Improving your decision-making tool kit, which is the purpose of Deploy Securely, will help get you closer to this ideal. You will always fall short of this goal, though, so you should embrace a degree of uncertainty and ambiguity when making decisions. Sometimes just picking a framework and moving ahead is the best option. You can always revisit the decision later and optimize.
With that detour completed, I’ll turn back to the SDL bug bar.
Broadly, it is built on top of the classic STRIDE framework. I won’t go into depth as to what this acronym means since there is so much available material on it. If you aren’t familiar, check out the previous hyperlink or do some Google sleuthing.
Using STRIDE as the foundation, the Microsoft model further divides vulnerabilities into server- and client-side ones, separating issues primarily by whether or not they require user interaction to exploit. This can be an important distinction but isn’t always a perfect tool for delineating attacks. For example, in the case of the Internet of Things (IoT), an agent running on an edge device that communicates back with a centralized platform is generally considered a “client” in the relationship, where the latter is a “server.” In the context of the SDL, though, evaluating vulnerabilities in such an edge agent would be more appropriate through the rubric of a server, as it sees minimal or no human interaction.
In terms of additional critiques, the SDL bug bar does not make a clear distinction between the probability of a vulnerability being exploited and the consequences of such an exploitation. Thus, like most qualitative rating scales, it does not allow for an easy distinction between high-likelihood and high-impact events. Additionally, while it makes distinctions between vulnerabilities based on the ease of exploitation, it does not incorporate much in terms of evaluating the threat landscape (e.g. are there hackers actively using this attack vector?). Finally, on the impact side, the system refers to “high value assets” but only provides examples of them - such as certificate server or domain controllers - without defining them in monetary or operational terms.
Despite these shortcomings, however, the SDL bug bar generally avoids any of the nonsensical outcomes that can result from using CVSS, such as obviously higher risk vulnerabilities having similar or lower scores than clearly trivial issues. Furthermore, it is relatively easy to understand for technical folks like software engineers, who are likely the people triaging security issues identified during your development process.
By when do we need to fix this?
If you decide to use the SDL for vulnerability management, a critical subsequent step is determining how quickly you will remediate issues based on their risk as defined by the framework. Please see my post on vulnerability management policies for general suggestions as to how to structure these types of decisions.
The right way to do this when using the SDL bug bar is not immediately apparent, though, and I have not been able to find an example where this is done well. Also, as I mentioned, Microsoft’s bug bar is an imperfect risk management tool, as it critically excludes factoring in anything related to the existence of an exploit or other factors related to the threat environment.
With that said, developing a remediation policy based on the bug bar is possible, but will need to be narrowly tailored to your organization’s business and security requirements. Generally speaking, I would recommend that the required timelines for resolving issues expand geometrically rather than linearly.
For example, a “critical” server-side vulnerability under the SDL bug bar means that a self-propagating network worm could “own” the target completely without user intervention. This type of situation - even barring the identification of exploit code in the wild - could reasonably lead everyone in an organization to drop what they are doing and fix the vulnerability as soon as possible. Thus, a 24-hour remediation timeline might be appropriate.
Conversely, an “important” priority issue for a server-side deployment could include an unauthenticated denial-of-service vulnerability, the exploitation of which would impact something other than a “high-value asset.” A hacker conducting an attack through this vector would cause extreme annoyance to the target organization and greatly disrupt productivity. But absent evidence of active or imminent exploitation, it is conceivable that a business might wait 15-30 days to resolve this type of problem.
Finally, a “low” priority vulnerability, such as the leak of random heap memory, might be reasonably fixed on an annual cycle. Deferring it indefinitely might also be appropriate if the engineering resources required to resolve it are substantial.
Conclusion
Compared to the CVSS, I think the Microsoft SDL bug bar is a better vulnerability scoring framework to use, as it is an internally consistent system that generally does not drive perverse outcomes. With that said, it is a qualitative framework that does not take into account things such as exactly how “high value” a given system is. Furthermore, it generally omits any parameters related to the threat environment, focusing primarily on the ease of exploitation. If you are just getting started with vulnerability management and need to implement a program immediately, it’s not a bad place to begin. With that said, I think there are better ways to value cyber risk effectively, so will continue with my analysis.
Next up: the DREAD system.