How to track AI vulnerabilities?

Not by logging CVEs.

Check out this issue on YouTube, Spotify, or Apple Podcasts.

Vulnerability management is already a thorny problem.

Throwing AI into the mix makes vulnerability management even tougher:

• How do you identify a discrete vulnerability in an AI model, which might only be reproducible using the same exact context and user input, or potentially, is not reproducible at all?

• Given this situation, how would I describe the likelihood of exploitation of a given vulnerability, even if I could describe it unambiguously?

• How would I know when the vulnerability is “exploited?” With the Exploit Prediction Scoring System (EPSS), for example, it is possible to discretely measure whether an attacker has attempted to exploit a given CVE. This is because all of the sensors connected to it can detect either a) a specific signature for malware designed to exploit it or b) a series of actions taken by a user or agent which suggest an attempt to do so.

I have been thinking about this problem for some time but a recent blog post by the Cybersecurity and Infrastructure Security Agency (CISA) spurred me to write this post.

The government was vague, typical of its discussions of cyber risk management. But there was one exception, which stuck out to me:

The AI engineering community must institute vulnerability identifiers like Common Vulnerabilities and Exposures (CVE) IDs.

Bottom line: if we are going to develop a vulnerability identification system for artificial intelligence systems, it most should be not be like the existing CVE regime (at least in its current form)!

The existing CVE system is fraught with problems

• The relationship between a piece of software having a CVE logged against it and that vulnerability actually being exploitable is already highly non-deterministic. 90% or more of CVEs are not exploitable in a given context, meaning that merely identifying one gives you very little idea as to your level of risk.

• The National Vulnerability Database (NVD), which lists all CVEs, is full of “science projects.” These are issues exploited by researchers in extremely limited or controlled situations, and which do not represent realistic vulnerabilities for most organizations. I’m sure they mean well, but these findings do little to advance real-world security and do much to obstruct it.

• The component naming problem makes it very difficult to even be sure whether a given CVE actually impacts a specific piece of software at all. This is on top of the question of whether the CVE can be exploited in context. This problem is already hugely impactful to non-as-a-Service (-aaS) products, and gets an order of magnitude more complex when you incorporate them into the mix.

All of the problems with CVEs get worse with AI

• If detecting CVEs already tells you so little and generally causes a lot of “noise” because of their non-deterministic relationship with risk in a given system, how are we going to deal with this problem with under even less deterministic circumstances? How will we triage the hundreds of thousands - potentially millions - of AI vulnerabilities that get logged if we follow CISA’s advice?

• What is to stop enterprising AI security researchers from logging even more “science project” vulnerabilities using extremely complex and unrealistic attack methods against AI models?

• How are we going to describe the already exploding array of different AI models in a standardized manner? What about fine-tuned versions of them? I hope not by attempting to reproduce the existing Common Platform Enumeration (CPE) system.

The existing CVE system is not a good place to start.