AI inventory, governance, and testing for SEC-regulated firms
I read 239 pages of proposed SEC rules on AI so you don't have to.
[Update 16 July 2024] The SEC announced its intent to “re-propose” this rule by October of 2024. The analysis below applies to the original.
The SEC is coming for your AI.
In addition to ordering a recent “sweep” of regulated companies to identify AI use, the Securities and Exchange Commission (SEC) also proposed levying an enormous set of new AI-related requirement in mid-2023.
The rules [from which I’ll cite page numbers like this] are still only in draft form. And two Senators have proposed killing them before they take force. But the financial services industry is unlikely to escape unregulated when it comes to AI.
The stated reason for the proposal is to identify - and eliminate - conflicts of interest stemming from the use of what the SEC calls predictive data analytics (PDA). And the requirements specifically target registered broker-dealers and investment advisers using certain technologies during investor interactions [p. 42]. Since Deploy Securely isn’t a blog about finance, I won’t get into the weeds on the conflict of interest side of things.
What I will do, however, is provide some actionable recommendations for cybersecurity and compliance leaders in the impacted financial services firms.
An accurate, continuously-updated AI asset inventory will be a hard requirement
Before you can begin to understand potential conflicts of interest related to PDA, you need to know exactly what types of PDA you use. The draft rule requires exactly such an inventory [p. 10].
The scope of the proposed rule is vast
The definition of “covered technology” would include any:
model
function
algorithm
correlation matrix
or similar method or process that:
optimizes for
forecasts
predicts
guides
directs
investment-related behaviors or outcomes [p. 42].
The SEC wants you to know how expansive this definition is by saying that
if a firm utilizes a spreadsheet that implements financial modeling tools or calculations, such as correlation matrices, algorithms, or other computational functions, to reflect historical correlations between economic business cycles and the market returns of certain asset classes in order to optimize asset allocation recommendations to investors, the model contained in that spreadsheet would be a covered technology [p. 44].
This scope goes way beyond even broadest common definitions of AI.
So I hope you have a solid spreadsheet inventory in place!
Because the scope is limited to PDA involved in investor interaction, the definition does exclude tools used solely for:
anti-money laundering purposes [p. 54]
“basic” customer support [p. 45]
back-office processes [p. 51]
But in any case, just compiling a list of everything meeting the definition will be an enormous effort. That doesn’t even include keeping it updated.
The SEC makes what I consider to be a laughable estimate of 100 man-hours initially (and 50 hours each subsequent year) to inventory the use of covered technology and determine whether it creates a conflict of interest [p. 184].
Note that this estimate is for a “complex” firm! The agency assesses a “simple” firm would require only 10 hours to do both tasks, which is a ludicrously small number. For a company (StackAware) with a single employee (me) and limited regulatory oversight, just inventorying all of our AI tools itself took 10 hours. And I run an AI governance company!
Unless you have the right combination of tools and expertise, I think the inventory alone could easily be a full-time equivalent (2,000 man-hours per year) job.
This doesn’t even include the conflict identification and elimination tasks.
Your software supply chain is also included
The regulation would also apply to any covered technology built or maintained by third parties. And basically every company is or soon will be using PDA meeting these conditions.
The SEC also specifically cites a lack of access to the underlying source code or architecture as not constituting an exception to the rule [p. 32, 64, 66]. Since the third parties maintaining these systems won’t necessarily themselves be subject to the regulation, contractual enforcement - throughout the entire supply chain - will be the only way to achieve any sort of assurance that the end customer will be compliant [p. 167]
Because shadow IT generally - and shadow AI in particular - are big problems already, staying on top of them will become increasingly important for regulated companies. The risk of unmanaged ChatGPT or similar tool use will increase - and substantially - if this rule comes to pass.
Tracking data lineage will become key
The SEC also specifically calls out corrupted data as a potential failure mode for covered technologies [p. 29]. Due to the potentially huge payoffs of well-crafted data poisoning attacks, this is another risk impacted firms will need to address. So you’ll need to be able to clearly monitor what data your AI was trained on and uses to make decisions, regulation or not.
How the heck am I going to keep track of all of this?
Given the extreme granularity of the proposed inventory requirements, the only way I see this working is by using an extensible and flexible Software Bill of Material (SBOM) format. I am a big fan of the CycloneDX standard, and StackAware already describes all known AI training processes in our SBOM.
Covered firms in the the financial services sector will also need:
a way to continuously update the SBOM as it changes
fields to record conflict of interest countermeasures
methods for tracking data lineage
CycloneDX has all of these things, so would definitely start looking there. StackAware is building our AI governance platform using it.
Compliance with the new rules will be a big part of your AI governance program and infrastructure
An inventory of covered technologies will be just the starting point.
Whether the proposed rules impact you or not, at the core of your AI governance program should be a policy that lays out:
organizational goals when it comes to AI
risk appetite and tolerance
roles and responsibilities
If the SEC gets its way, though, regulated firms will need to include some additional things. My recommendation for these companies is to incorporate into their policy an additional requirement for building a procedure (or updating an existing one) to document:
use (and changes in use) of covered technology in investor interaction [p. 139-140]
specific features of the technology used in these interactions
disclosures of where they are used to investors [p. 139]
any associated potential conflicts of interest [p. 137]
how these are eliminated/neutralized [p. 138]
yearly reviews of the above [p. 41, 138- 139]
Attempting to do any of the above in any sort of manual or spreadsheet-driven way seems doomed to failure. The good news is that StackAware is build out a policy-as-(no-)code platform for AI governance that will help you track all of these things.
Check out this demo for a glimpse.
Testing and evaluation will be key to compliance
The SEC takes a novel approach in how it wants regulated firms to deal with identified conflicts of interest:
Due to the inherent complexity and opacity of these technologies as well as their potential for scaling, we are proposing that such conflicts of interest should be eliminated or their effects should be neutralized, rather than handled by other methods of addressing the conflicts, such as through disclosure and consent [p. 26].
The neutralization/elimination standard appears to be a completely new one, which one critic has described as being basically unattainable. If a firm is going to even have a chance of proving its compliance with this standard, though, it will need comprehensive testing and documentation.
To address these the SEC proposes things like:
having someone with sufficient knowledge of both the applicable programming language and the firm’s regulatory obligations (these people are likely to be in high demand!) review the source code of the technology [p. 63]
building “explainability” features to give covered technologies the capacity to explain why it reached a particular outcome [p. 63]
A/B testing [p. 26]
How feasible any of these will be in reality remains to be seen. But whatever a firm does, it will need to:
test each covered technology prior to implementation or material modification, and periodically thereafter [p. 73]
document any research or third-party outreach related to any testing of a covered technology [p. 136-137]
track and justify whenever it believes pulling a covered technology out of service due to a conflict of interest would be a greater risk to investors than the conflict itself [p. 101]
eliminate identified conflicts “promptly” [p. 101]
The only way this can ever be done accurately will be to use a single, machine-readable source of truth. And StackAware’s AI governance platform will eventually have the capability to track all of these things in your asset inventory, using the CycloneDX SBOM format.
Whether the regulation comes into force or not, SEC regulated firms need to govern AI effectively
Even if the proposed SEC rule is itself eliminated through congressional action, that doesn’t mean AI governance can go on the back-burner. One of the proposed alternative approaches in the regulation is a detailed disclosure [p. 200]. Even this watered-down requirement would demand a huge investment in terms of simply inventorying and documenting PDA in use.
And the SEC requirements will likely overlap with other emerging frameworks like:
The EU AI Act
Existing security and privacy regulations
So if you need help wrapping your head around the complex web of AI-related requirements and risks, StackAware can help.
Related LinkedIn posts