Who Actually Performs AI Hiring Bias Audits
7 min read · Last reviewed 6 Jul 2026
General information, not legal advice. Laws in this area change; verify against the official sources at the end of this guide and confirm specifics with employment counsel.
New York City's Local Law 144 requires an annual bias audit by an independent auditor, and then tells you nothing about where to find one. There is no city-approved list, no register, and no licence: the law defines independence by exclusion and leaves the market to sort itself out. So employers reach the same question from every direction, which is simply who does this.
This page answers that. It names the firms that visibly do this work, says what each is known for, and gives you the questions that separate a real audit from a report. We do not perform bias audits and we take no fee from anyone listed here, which is exactly why we can write it. Nothing below is an endorsement or legal advice; verify independence and scope with counsel before you sign anything.
First, check whether you need to buy one at all
Before shopping, establish which audit you actually need, because for most small employers the answer is none of your own.
The audit attaches to the tool, not to you. If your screening software's vendor commissions an annual independent audit of their product, often on historical data pooled across their customers, your obligation shrinks to verifying that audit exists, is current, was genuinely independent, and covers the tool as you use it, then publishing the summary on your careers page with the distribution date. That is a morning of work, not a five-figure engagement.
So the first call is to your vendor, not to an auditor. Ask who performed their most recent audit, on what date, and for a copy of the published summary. If they cannot answer, you have learned something important about the vendor, and you now have a reason to price your own audit or change tools.
You need to commission your own when you built the tool yourself, when you configured or trained it enough that the vendor's audit no longer describes how you use it, or when your vendor has none.
Who actually files these audits
A 2025 study presented at the ACM Conference on Fairness, Accountability and Transparency reviewed the LL144 audit reports employers have actually published and found the work concentrated in a small number of firms, with Holistic AI appearing in roughly a fifth of published reports and BABL AI in roughly a sixth. That concentration is the most useful public signal available, since there is no register.
Holistic AI runs a dedicated NYC bias audit line and is the most frequently named auditor in published summaries. Worth knowing: they also sell AI governance software, so if you use their platform as well, ask directly how they wall off the audit, because independence is defined by financial and operational separation rather than by good intentions.
BABL AI is the other firm with a clear public track record of completed LL144 reports, and it audits without requiring you to integrate a platform, which matters if you want the audit to be a discrete engagement rather than a subscription.
Warden AI provides independent third-party audits aimed at HR technology, and publishes a good deal of practical LL144 material, which is a reasonable proxy for how well a firm understands the rule.
FairNow, now part of AuditBoard, is the one to look at when you do not have enough historical applicant data to compute selection rates, because its synthetic-data approach was built for exactly that blocking problem: a tool that has not yet processed enough real candidates cannot be audited conventionally.
The Big Four also sell this work, and Deloitte publishes an LL144 practice. They are the expensive end and the right end if your audit needs to survive scrutiny alongside other regulated assurance work; they are usually overkill for a 20-person company.
Beyond these, university statistics departments and boutique industrial-organizational psychology practices perform the same analysis competently. The maths is not exotic. What you are buying is independence, defensible methodology and a report someone else will accept.
Six questions that separate an audit from a PDF
Ask every candidate firm the same six things and compare the answers, not the brochures.
One: what exactly makes you independent of both us and our vendor, in writing? Independence under the rule excludes anyone who helped build, sell or configure the tool, anyone with an employment relationship on either side, and anyone with a financial interest beyond the audit fee.
Two: will you compute scoring rates as well as selection rates? Most modern screening software outputs scores rather than accept and reject decisions, and the rules use an above-median scoring-rate variant for those. A firm that only knows the binary method has not audited software like yours.
Three: how do you handle intersectional categories and small groups? The audit must report sex, race and ethnicity, and the intersections, and must handle categories too small to be meaningful without producing nonsense ratios.
Four: what data do you need from us, in what shape, and who prepares it? Data preparation, not statistics, is the real cost driver in every engagement.
Five: what does the published summary look like, and do we publish it or do you? You are required to publish, so see the artifact before you buy it.
Six: what is not covered? An LL144 audit is a statistical snapshot of outcomes by group. It is not a code review, not a validation that your criteria predict job performance, and not a certificate of fairness. A firm that implies otherwise is selling you comfort.
What it costs, honestly
The market is young and pricing is not standardised, so treat any number, including ours, as orientation rather than a quote.
A single-tool audit on clean, well-structured data has been quoted in the low-to-mid four figures by boutique firms. Engagements covering several tools, messy data or bespoke methodology run well into five figures. Big Four engagements start higher still.
The variable that moves the bill is almost never the analysis. It is whether you can hand over a joinable per-candidate table of tool outputs, outcomes and voluntarily self-identified demographics, or whether someone has to reconstruct it from an applicant tracking system, a spreadsheet and somebody's memory. Employers who build that export once and keep it current pay the low end every year afterwards.
And for the many small employers covered by a vendor's pooled audit, the real cost is an hour of verification and a page on your website.
Before the audit: run the maths yourself
Whoever you hire will compute impact ratios: each group's selection or above-median scoring rate divided by the highest group's rate, flagged when it falls below four-fifths. There is no reason to meet those numbers for the first time in an auditor's report.
Our free bias audit self-check at /tools/bias-audit-check runs that arithmetic in your browser, flags groups under the four-fifths line, and uploads nothing, so the demographic data never leaves your machine. It is explicitly not the official audit, because it is not independent of you, and it does not satisfy LL144 on its own. It is the thing that stops the official audit from surprising you, and in California and Illinois, where testing is evidence rather than mandate, the record it produces is the point.
If you want the underlying per-candidate data in an auditor-ready shape, SiftFirst exports every screening as a clean table of rubric criteria, per-criterion scores with quoted evidence, and outcomes, with no demographics in it, designed to be joined against separately held self-ID data.
Key takeaways
- ✓There is no official list of approved LL144 auditors, and no licence: independence is defined by exclusion, so due diligence is on you.
- ✓Call your screening vendor before you call an auditor. A vendor-commissioned pooled audit of the tool usually discharges a small employer's obligation, leaving you to verify and publish.
- ✓Published LL144 reports concentrate in a few firms, with Holistic AI in roughly a fifth and BABL AI in roughly a sixth per a 2025 ACM FAccT study; Warden AI and FairNow are the other names that appear in the HR-tech segment, and FairNow's synthetic-data method exists for tools without enough historical data.
- ✓Ask six questions: independence in writing, scoring rates as well as selection rates, intersectional and small-group handling, who prepares the data, what the published summary looks like, and what is explicitly out of scope.
- ✓Costs run from low four figures to five-plus, and data preparation drives the bill, not the statistics. Build the export once.
Screening built for these rules
SiftFirst scores candidates against criteria you set, quotes the resume line behind every score, never auto-rejects, and exports the records these laws expect. The candidate notice generator and bias audit self-check are free.
FAQ
Is there an official list of approved bias auditors in NYC?
No. Local Law 144 requires the auditor to be independent and impartial but sets up no licensing, registration or approval scheme, and the city does not publish a roster. That is why employers keep asking who does this: the law creates a market without naming anyone in it. Your protection is documented independence and a methodology you can defend, not a credential.
Can the company that sells us our screening software also audit it?
Not as your independent auditor. What a vendor legitimately does is commission an independent third party to audit their tool and hand customers the summary and distribution date to publish. Be careful with firms that sell both governance software and audits: using their platform does not automatically disqualify them, but you should ask in writing how the financial and operational separation works, because independence is judged on ties and interests rather than intent.
We have a brand new tool with almost no applicant data. Can it be audited?
Conventionally, no: selection and scoring rates need enough candidates per demographic category to mean anything. This is the specific gap that synthetic-data methods were built for, and FairNow's approach is the best documented of them, generating representative synthetic applicant data to evaluate the tool before it has processed real volume. Discuss with counsel how a synthetic-data audit sits against your publication duty.
How much of this applies outside New York City?
The audit mandate itself is NYC-only. Elsewhere the same arithmetic still matters: California's FEHA regulations make anti-bias testing relevant evidence, Illinois attaches liability to discriminatory effect regardless of intent, and any federal disparate-impact claim runs on selection-rate statistics. Colorado repealed its AI Act in May 2026 before it took effect and its replacement, effective January 2027, asks for notice, explanations and records rather than audits.