A risk analysis of Meta's Code Llama

Data retention and unintended training with self-hosted AI models.

Check out the YouTube, Spotify, Apple podcast versions.

Self-hosting AI models seems to quickly have become the de facto choice for those concerned about data security.

But if you read Deploy Securely regularly, you’ll know this isn’t necessarily the right answer.

On that note, someone recently asked me about the unintended training and data retention risk associated with using Meta’s Code Llama. This person was comparing the model against GitHub Copilot and was concerned about if and how Meta might process data provided to Code Llama.

My bottom line response: these two specific risks are the same for Code Llama as for every other model you host and operate on your own.

And, all other things being equal, the risk for these two issues is lower than that of anything operating -as-a-Service (-aaS) like Copilot or ChatGPT’s Code Interpreter.

LLM-based code analysis

Because Code Llama is not offered -aaS and thus Meta does not host it in production, the only conceivable way in which the company could process and train on your data is if there is some sort of callback.

If there were any, that would certainly be odd (although not the first-time open source software is piping data back to another party without the clear awareness of the operator!), but it looks clean on a surface-level scrub using ChatGPT.¹

It seems highly unlikely that Meta would try to implement such a “phone home” capability due to the almost certain outcry that would result, especially considering the intense scrutiny they already face when it comes to data privacy.

And if you are extremely paranoid, you could physically or logically air-gap the model or follow ChatGPT’s advice and monitor/block outgoing request during runtime.

Comparative risk

Your unintended training risk is not zero here, though.

Consider what would happen if you fine-tune it on some super secret code that only a select team is supposed to see. If employees on other teams (or the public at large, for some reason) end up having access to this model instance, then you have some exposure risk for your intellectual property.

That’s why having a neutral security policy for your AI applications is vitally important.

On the data retention side, your internal architecture will be critical here. If you hoover up every prompt and response and store it in an Infrastructure-as-a-Service (IaaS) database without any built-in data destruction processes, your retention risk can be higher than if you are using the OpenAI (default: 30 day retention) or Anthropic Enterprise (default: 28 day retention) APIs.

Bottom line

Unintended training is definitely a real risk.

But there is no “right answer” for how to deal with it.

So don't index too heavily on it. There are many other things to worry about, like:

• known “traditional” vulnerabilities

• misconfigurations

• human error

And if you’re ready to start managing the risk from AI related to:

• Cybersecurity

• Compliance

• Privacy

1. The code is available without authentication on GitHub, but Meta makes you do a click-through agreement to get access to the model weights. Some have thus disputed Meta’s characterization of Code Llama as being “open source” as a result, but the most important thing for this analysis is that we can review the model’s code.