Discover more from Deploy Securely
Manage 4th party AI processing and retention risk
Map artificial intelligence tools throughout your supply chain.
And AI makes the problem even more challenging.
That’s why having a policy that extends throughout your digital supply chain is vital. Otherwise, you are just playing whack-a-mole.
For example, suppose you ban processing confidential information with third-party generative AI tools. This is a fairly restrictive approach, but if that’s what your risk appetite dictates, do what you need to.
Asana Intelligence - Example 1
Unfortunately, it’s quite possible your already-approved tools (which are not themselves dedicated generative AI platforms) are integrating generative AI capabilities into them (which can be disabled in this case). And these can introduces fourth-party AI risk, like in the case of Asana:
You can implement all the:
you want, but if you don't address the fact your already-approved tools are passing confidential information to:
it’s pretty much a waste of time and effort.
Need help mapping your AI-related risk?
And in any case, you need to understand how data is being processed and retained throughout your entire supply chain. If a 4th party is retaining your confidential information indefinitely, that greatly increases your risk surface due to the necessarily higher risk of a breach over an infinite timeline.
And at least Asana is clear about exactly what they are doing. For an example of something more vague, look at Databricks Assistant, an AI copilot.
Databricks Assistant - Example 2
While the company has a good FAQ about how things work, there is a gaping whole obscured by some clever wording.
They talk a whole lot about Azure OpenAI and all the security measures in place, but that is only ONE of the options used for processing! The others are unnamed.
While it’s good to know they aren’t training on user inputs, it would be good to know which other tools Databricks is leveraging and what their retention policies are. Anthropic, for example, has a default 28 day retention period for enterprise users while Azure OpenAI (in this case) doesn’t retain any prompts or outputs.
The fact that they don’t identify the other third-party services and index so heavily on Azure OpenAI is somewhat concerning because it suggests they are trying to bury something customers might not be happy about.
Based on public information, there is no way to be sure. But this is why transparency is the best policy.
Zoom AI Companion - Example 3
This is the most complex example of 4th party AI processing risk that I have encountered. According to to their AI Companion data handling article:
developing its own Large Language Model (LLM),
operating its own instance of Llama 2 (probably in their own Infrastructure-as-a-Service (IaaS) environment), and
using Software-as-a-Service (SaaS) models from
This is a hugely complex web of 4th parties. And their retention policies are a little bit vague, so it’s not entirely clear who is keeping what data and for how long.
This setup is not necessarily wrong, and they do make clear that these models aren't training on your data. But having so many different repositories for your information, with unclear retention periods, can potentially increase your cyber and compliance risk.
Don’t confuse this with unintended training
The above examples represent a separate issue from unintended training, whereby you accidentally train an AI model on data you wish you hadn’t. I’m just talking about processing and retaining sensitive data here, which basically every organizations already lets third parties do.1
Obviously you should be concerned about whether fourth parties are training on your data as well, but that doesn’t appear to be happening in either of these examples.
AI governance is not optional
This is just one example of why you need a comprehensive approach to AI governance that lets you manage:
risk without slowing down the business.
With the right framework in place, you can leverage AI for huge productivity gains without undue risk.
Let us do an risk assessment for you:
Training does represent a form of processing, but is more worrisome from a security perspective because of the potential for the underlying data (or derivations of it) being exposed unintentionally to other parties using the same AI model.