The Question
Your HR technology vendor has added an AI-powered resume screening feature to the applicant tracking system your company has used for years. It was not a separate procurement decision — it shipped as a product update. Your talent acquisition team loves it. Processing time dropped by 60 percent. Your legal counsel just sent you a copy of NYC Local Law 144 and a question: has this system been audited?
This scenario is playing out across large employers in New York City and increasingly across regulated industries nationally. AI-assisted hiring decisions are subject to mandatory bias audit requirements. AI-assisted credit decisions are subject to adverse action notice requirements that apply regardless of whether the decision was made by a human or an algorithm. AI used in insurance underwriting is under active scrutiny from state regulators. The tools are deployed; the governance infrastructure is not.
The broader challenge is that enterprise AI bias governance requires fluency in two separate domains — legal and technical — that rarely communicate well. The legal standard is about protected class disparate impact. The technical standard involves competing mathematical definitions of fairness that cannot simultaneously be satisfied. Most enterprises are trying to comply with the former without understanding the latter.
AI bias governance is not about making your AI perfectly fair — it is about demonstrating to regulators that you measured, monitored, and addressed disparate impact in the AI systems making material decisions about people.
Why This Matters Now
In November 2023, the EEOC issued updated guidance on employer use of algorithmic decision-making tools in hiring, explicitly stating that the same disparate impact framework that applies to human employment decisions applies to automated and AI-assisted decisions. An AI screening tool that disproportionately screens out candidates in a protected class — even without any discriminatory intent in its design — creates Title VII exposure for the employer.
NYC Local Law 144, which took effect July 2023 and began enforcement in April 2023, requires that any automated employment decision tool used in hiring or promotion decisions for New York City-based roles undergo an annual bias audit by an independent auditor, and that the audit results be published and disclosed to candidates before use. The law defines bias audit as "an impartial evaluation by an independent auditor" that includes impact ratio analysis across sex, race, and ethnicity categories.
The CFPB, in guidance issued throughout 2024, confirmed that lenders using AI or algorithmic models for credit decisions must provide specific, accurate adverse action reasons — and that "the system gave you a low score" does not satisfy that requirement. The model must be capable of producing the specific factors that drove the decision for the specific individual.
By early 2026, at least seven additional US states had introduced legislation modeled on NYC LL 144 or the EU AI Act's provisions for high-risk AI in employment and credit contexts. The EU AI Act classifies AI used in employment, education, access to credit, and essential services as high-risk under Annex III, triggering conformity assessments, technical documentation requirements, and fundamental rights impact assessments before deployment.
The regulatory trajectory is not speculative. For enterprises operating in financial services, healthcare, HR technology, and insurance, AI fairness compliance requirements are active, enforceable, and growing in geographic scope.
What the CURVE™ Data Shows
The 2026 Stackcurve AI Governance CURVE™ Report evaluated the vendor landscape for AI bias monitoring, fairness auditing, and regulatory compliance workflows. The evaluation specifically assessed coverage of NYC LL 144, EU AI Act high-risk AI requirements, EEOC guidance, and CFPB adverse action notice requirements.
Credo AI ranked as a Leader for structured AI governance workflows that map technical bias metrics to specific regulatory frameworks, with pre-built compliance templates for NYC LL 144, EU AI Act, and EEOC guidance. Arthur AI scored highly for production bias monitoring — ongoing monitoring of deployed models for demographic drift in outcomes over time, not just point-in-time audits. Holistic AI and Fairly AI both specialize in independent bias audit services for employment and financial services contexts, with the audit documentation workflows that NYC LL 144 requires. Weights & Biases and MLflow provide the model evaluation infrastructure that internal teams use to run fairness metrics during development, though neither provides the regulatory workflow layer that compliance teams need.
For the legal technology side, Veritone and HireVue have both invested in bias testing for their own AI tools — worth noting as a procurement criterion when evaluating AI-powered HR technology vendors.
The full vendor rankings are in the 2026 Stackcurve AI Governance CURVE™ Report — free to download.
The Gap Most Buyers Miss
Legal fairness and technical fairness are not the same thing — and they can conflict
Legal fairness analysis centers on disparate impact: does the AI system's outputs (decisions, rankings, scores) fall disproportionately on members of a protected class, compared to a majority group? This is measured as an impact ratio — the selection rate for one group divided by the selection rate for another. A ratio below 0.80 (the "four-fifths rule") is a standard indicator of adverse impact under EEOC guidance.
Technical fairness research has produced multiple competing mathematical definitions: demographic parity (equal selection rates across groups), equalized odds (equal true positive and false positive rates across groups), calibration (equal predictive accuracy across groups), and others. A 2016 ProPublica analysis of the COMPAS recidivism algorithm showed that the tool could be simultaneously calibrated (equally accurate) and inequitable (higher false positive rates for Black defendants) — and that these two properties cannot both be satisfied when base rates differ across groups.
This is not a solved problem. Governance programs need to understand which fairness definition is legally required in their context, which technical metrics approximate it, and what the tradeoffs are when multiple fairness criteria conflict.
The measurement problem: demographic data and proxy variables
Bias audits require demographic data to measure. This creates a series of compounding problems. Demographic data is often not available in training sets because it was never collected. It is illegal to collect race, sex, or national origin data in some employment contexts. When it is not directly available, researchers use proxy variables (zip code, name patterns, educational institution) that introduce their own biases and methodological disputes.
Enterprises conducting bias audits for existing deployed models frequently discover that they cannot run the required analysis because they do not have the demographic data needed. Governance programs should address this prospectively: what demographic data should be collected (where legally permissible), how it should be stored, and what the methodology will be when direct measurement is not available.
Point-in-time auditing is not sufficient
NYC LL 144 requires annual bias audits. The EU AI Act requires ongoing monitoring for high-risk AI. A model that was fair at deployment can drift — if the underlying population changes, if the model is updated, or if usage patterns shift. Production bias monitoring, distinct from point-in-time auditing, is a governance requirement for any AI system making ongoing material decisions.
Questions Your Buying Team Should Be Asking
1. For AI systems used in employment, credit, insurance, or healthcare decisions — what bias audit has been conducted, by whom, and when?
Vendor claims of "bias testing" are not equivalent to an independent audit that meets NYC LL 144 or EU AI Act standards. Ask for the actual audit report, the methodology used, the auditor's independence credentials, and the date. A bias test conducted by the vendor's own data science team during development is not an independent audit.
2. Which specific fairness definitions does the vendor test for, and can they explain the relationship between those metrics and the disparate impact standard used in your regulatory context?
Most AI vendors can tell you what fairness metrics they measure. Fewer can explain how those metrics map to the four-fifths rule or the specific disparate impact framework that applies to your use case. If the vendor cannot bridge technical metrics to legal standards, your compliance team will have to — which is a governance risk.
3. What demographic data does the model require to run bias analysis, and what is the vendor's methodology when that data is not available?
This question surfaces the measurement problem directly. If the vendor has no answer, or if the answer relies on proxy variables without disclosing the methodology, that is a significant audit and compliance risk.
4. How does the vendor notify us of model updates that could affect bias performance, and what re-auditing is required?
A model that passes a bias audit can fail the same audit after an update. Governance requires a contractual commitment to notification of model changes and a process for re-evaluation before updated models are used in regulated decision contexts.
5. What adverse action explanation can the system produce for individual decisions, and does that meet CFPB or applicable state requirements?
For credit and financial services AI specifically, "the model assessed you as high risk" is not a legally sufficient adverse action explanation. The system must be capable of producing the specific factors, with weights, that drove the decision for a specific individual. Ask the vendor to demonstrate this for a sample adverse decision.
The Stackcurve Take
The enterprises that are handling AI bias governance well share one characteristic: they have mapped their AI inventory to the regulatory frameworks that apply by use case, geography, and decision type — and they have built that mapping into their procurement process rather than treating it as a post-deployment compliance exercise.
Bias audits commissioned after a regulatory inquiry are expensive, disruptive, and may document a problem you are now legally obligated to remediate. Bias governance built into AI procurement and deployment — vendor selection criteria, contractual audit rights, production monitoring requirements — converts a reactive liability into a manageable operational program.
The window for treating AI bias as a research topic rather than a compliance requirement has closed. The regulations are in force. The enforcement actions are happening. The question for enterprise governance programs is no longer whether to build fairness governance infrastructure but whether to build it before or after the first enforcement event.
The 2026 Stackcurve AI Governance CURVE™ Report covers AI bias monitoring and fairness audit platforms, including evaluation criteria mapped to NYC LL 144, EU AI Act Annex III, EEOC, and CFPB requirements. Download it free →
Stackcurve Advisory Briefs are independent research. No vendor pays for placement, tier assignment, or editorial influence. The CURVE™ methodology is disclosed in full at stackcurve.net/research/methodology.