How to comply with New York City Local Law 144 on Automated Employment Decision Tools
An actionable procedure for companies using AI for hiring in the Big Apple.
The web of AI regulation is thick, and growing.
New York City led the pack by passing its own Local Law 144 in 2021, which regulates the use of Automated Employment Decision Tools (AEDT).
I have reviewed several AI-powered recruiting tools that are almost certainly being used as AEDTs, and only one of them fully complies with this law…
…by entirely disabling its AI functionality for candidates in New York City.
Business demands will eventually make this approach infeasible, though, so I put together a simplified procedure for compliance that take into account the law itself and the enforcing regulation that greatly expands on some of the definitions.
This isn’t legal advice, so check with your attorneys when integrating this into your AI governance program.
But it gives you an actionable starting point.
New York City Local Law 144 Compliance Procedure
Scope
Artificial Intelligence systems1 used for Employment Decisions within the 5 boroughs of New York City.
Requirements
Data owners must ensure:
The system is not an Automated Employment Decision Tool (AEDT) because it does not provide Simplified Output related to Employment Decisions (and affirmative documentation of this fact);
The system is not an AEDT because, for all Employment Decisions, the system’s Simplified Output is not weighted more than any other criteria2 (and documented support of this assessment for each Employment Decision); or
If the system is an AEDT:
The following are available on COMPANY_NAME web site:
Hyperlink to a summary of results of a Bias Audit of the AEDT completed in the last year, including at least:
Date conducted.
Source and explanation of data used.
Number of applicants or candidates.
Selection or scoring rates.
Impact ratios for all categories.
Number of individuals the system assessed that fall within an unknown category (not identified by sex and race/ethnicity).
Type(s) and source(s) of data collected by the system.
Retention policy for such data.
Date of first use of system.
Candidates or employees are notified via email 15 or more days3 prior to assessing them with an AEDT:
That COMPANY_NAME will use an AEDT to do so.
About which job qualifications the AEDT assesses.
How to opt-out of assessment by the AEDT and request an alternative selection process or accommodation.
Definitions
“Artificial Intelligence (AI) system,” means an engineered system that generates outputs such as content, forecasts, recommendations[,] or decisions for a given set of human-defined objectives. (per ISO/IEC 22989:2022).
“Automated Employment Decision Tool” means any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons. The term "automated employment decision tool" does not include a tool that does not automate, support, substantially assist or replace discretionary decision-making processes and that does not materially impact natural persons, including, but not limited to, a junk email filter, firewall, antivirus software, calculator, spreadsheet, database, data set, or other compilation of data.
“Bias Audit” means an impartial evaluation by an independent auditor. Such a bias audit shall include but not be limited to the testing of an automated employment decision tool to assess the tool’s disparate impact on persons of any component 1 category required to be reported by employers pursuant to subsection (c) of section 2000e-8 of title 42 of the United States Code as specified in part 1602.7 of title 29 of the Code of Federal Regulations.
New York City provides granular requirements for a Bias Audit in the enforcing regulation of Local Law 144.
“Distribution Date” means the date the employer or employment agency began using a specific AEDT.
“Employment Decision” means to screen candidates for employment or employees for promotion within the city.
“Impact Ratio” means either (1) the selection rate for a category divided by the selection rate of the most selected category or (2) the scoring rate for a category divided by the scoring rate for the highest scoring category.
“Machine learning, statistical modeling, data analytics, or artificial intelligence” means a group of mathematical, computer-based techniques:
i. that generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate's fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; and
ii. for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models in order to improve the accuracy of the prediction or classification.
“Screen” means to make a determination about whether a candidate for employment or employee being considered for promotion should be selected or advanced in the hiring or promotion process.
“Simplified Output” means a prediction or classification as specified in the definition for "machine learning, statistical modelling, data analytics, or artificial intelligence." A simplified output may take the form of a score (e.g., rating a candidate's estimated technical skills), tag or categorization (e.g., categorizing a candidate's resume based on key words, assigning a skill or trait to a candidate), recommendation (e.g., whether a candidate should be given an interview), or ranking (e.g., arranging a list of candidates based on how well their cover letters match the job description). It does not refer to the output from analytical tools that translate or transcribe existing text, e.g., convert a resume from a PDF or transcribe a video or audio interview.
The definition for Artificial Intelligence Systems is much broader than that for AEDT, to ensure all potential AEDTs are properly handled.
The enforcing regulation for NYC LL 144 provides 3 different definitions for “to substantially assist or replace discretionary decision making”:
i. to rely solely on a simplified output (score, tag, classification, ranking, etc.), with no other factors considered; or
ii. to use a simplified output as one of a set of criteria where the simplified output is weighted more than any other criterion in the set; or
iii. to use a simplified output to overrule conclusions derived from other factors including human decision-making
I view these as duplicative and overlapping. They all come back to whether or not the AEDT is weighted more than any other factor, hence the simplified language in the procedure.
The law says “ten business days” without defining what a business day is. I hate the term because it is vague, and thus have used 15 calendar days to account for a situation where there is a 2-day weekend followed by a 3-day weekend.