Does your pre-hire assessment tool require a bias audit?

Introduction

If you’re a recruiter leveraging artificial intelligence (AI), a resume screening tool, or other automated methods in your pre-hire assessment, new regulations such as NYC Law 144, may impact your practices. 

NYC Law 144 is the pioneering regulation that prohibits the use of Automated Employee Decision Tools (AEDT) to screen candidates or employees based in New York City for an employment decision, unless the tool has been subject to a bias audit conducted within the prior year. 

In this article, we’ll guide you in determining whether your tool qualifies as an AEDT, provide a checklist for assessing the need for a bias audit, and offer suggestions for fostering a more equitable, diverse and inclusive pre-hire process.

What is an Automated Employee Decision Tool (AEDT), and Why Are They Being Scrutinized?

NYC Local Law 144 centers on AEDTs, tools that significantly augment or even replace human decision-making. 

AEDTs are designed to simplify complex decisions by using input data and algorithms to predict future success.They are often used to determine outcomes like which candidates are selected for interviews or the identification of individuals up for promotion. It’s life-changing stuff that can reshape careers. 

The NYC Local Law 144 is the first of what is likely to be many more regulations all over the world, aimed at protecting candidates and employees from algorithmic bias. 

The law necessitates audits of AEDTs to help ensure they do not exhibit bias based on factors like ethnicity, gender, or age. This is crucial in light of past cases involving biased algorithmic decisions, such as sexism, racist code, and age discrimination.

Breach of this regulation can result in substantial fines, adding up to $500 the initial violation or $1500 for a second violation.

3 Steps to Determining Whether you Need a Bias Audit

1. 1. Is your tool actually an AEDT? 

First, evaluate whether your tool fits the AEDT definition. 

Automated Employment Decision Tool means “Automated employment decision tool” as defined by 20-870 of the Code where the phrase to “Substantially assist or replace discretionary decision making” means: 

i. to rely solely on a simplified output (score, tag, classification, ranking, etc), with no other factors considers; or

ii . to use a simplified output as one of a set of criteria where the simplified output is weighted more than any other criterion in the set; or

Iii. to use a simplified output to overrule conclusions derived from other factors including human decision making. 

If we break down this definition, we can simplify it as a tool that is:

(1) A computational process derived from machine learning, statistical modelling, data analytics, or artificial intelligence.

AND

(2) Issues simplified output, including a score, classification, or recommendation. Alll pre-hire assessments do this. 

AND

(3) Is used to substantially assist or replace discretionary decision-making for making employment decisions. All pre-hire assessments do this. 

All pre-hire assessment tools meet the final two criteria, so the part of the definition that matters is (1) above. They clarified this.

Importantly, your AEDT must “at least in part identify the inputs”. It’s this part of the definition that targets the “black box” element found in AI and machine learning.

The “black box” element that’s found in AI and machine learning is defined as: 

“Machine learning, statistical modeling, data analytics, or artificial intelligence means a group of mathematical, computer-based techniques: 

i: That generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate’s fit or likelihood of success, or that generate a classification meaning an assignment of an observation to a group, such as categorizations based on skill sets of aptitude; and 

ii. For which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and if applicable, other parameters for the models in order to improve the accuracy of the prediction or classification.” 

To confirm whether your tool is an AEDT, understand how your AEDT obtains its input data. 

Some pre-hire assessment systems draw on personality data, academic achievements, CV skills, or demographic information from common attributes of successful employees to predict future performance. However, if this data isn’t representative of the entire population, bias may occur, and that’s what NYC Law 144 is trying to avoid.

If the inputs are decided by humans, like a job analysis survey conducted by current employees, your pre-hire assessment doesn’t rely on ‘black box’ inputs described above, and so isn’t an AEDT. 

2. 2. Do you employ staff in New York City? 

If you hire employees residing in New York City, you must comply with NYC Law 144. 

However, implementing best practices to mitigate bias is advisable, as similar regulations may emerge in other U.S. territories, and the European AI Act also emphasizes assessing the impact of automated decisions.

3. 3. My vendor says they’ve been audited, do I still need one? 

If your tool qualifies as an AEDT and you operate in New York City, your vendor’s audit may not suffice. 

Vendors are not liable; the responsibility falls on the employer. 

Many audits are general and may not address your specific context, and audit costs can exceed $5000 per audit, per job, per year.

Tips and Resources for Fairer Hiring and Avoiding Bias

If you want to avoid the costs, headaches, and risks of non-compliance with existing and new AEDT laws, you may be looking for alternatives.

Here at ThriveMap, we believe that the best way to reduce bias in the recruitment process is to look at the skills required to do a job, and assessing candidates on those skills using a Realistic Job Assessment. Using Realistic Job Assessments has other benefits, such as reduced attrition, improved on the job performance, and stronger business growth.

Next Steps

Complying with NYC Law 144 and addressing bias in pre-hire assessments necessitates a thorough understanding of AEDTs, careful consideration of your candidate pool, and a readiness to scrutinize your entire recruitment process for fairness and inclusivity.

Here are more resources to support your journey to ensuring a fairer hiring process:

• Read our Fair Hiring Guide: With over 40 actionable tips you can implement today
• ThriveMap’s recruitment process: Achieving excellence with pre-hire assessments
• An overview of AI bias