CASRAI Dictionary

Tag: AI compliance

  • EU AI Act: Counting Down to High-Risk Obligations

    The most demanding tier of the European Union’s Artificial Intelligence Act concerns high-risk AI systems. Their core obligations — risk management, data governance, documentation, human oversight, conformity assessment and CE marking — were tied to a 2 August 2026 milestone, with further phased dates running to 2027 and beyond. This explainer sets out the framework and the moving timeline as it stood. It is news analysis, not legal advice.

    What counts as high-risk

    The Act treats two broad groups as high-risk. The first covers AI systems that are safety components of products already regulated under EU harmonisation law (Annex I), such as machinery or medical devices. The second covers stand-alone systems used in defined areas (Annex III), including biometrics, critical infrastructure, education, employment, access to essential services, law enforcement, migration and the administration of justice. Our pillar page on the EU AI Act sets out the full risk taxonomy.

    What providers must do

    For high-risk systems, providers face a structured set of obligations, including:

    • A continuous risk-management system across the lifecycle.
    • Data governance covering training, validation and testing datasets.
    • Detailed technical documentation and automatic logging.
    • Transparency to deployers and appropriate human oversight.
    • Appropriate accuracy, robustness and cybersecurity.

    Deployers carry their own duties under Article 26, such as using systems in line with instructions and ensuring human oversight in practice.

    Conformity assessment and CE marking

    Before a high-risk system is placed on the market, it must undergo a conformity assessment to demonstrate it meets the requirements. Depending on the system, this may be a self-assessment by the provider or, in some cases, involve a designated notified body. Systems that pass are registered in an EU database and carry the CE marking, the same conformity mark already familiar from EU product law. This product-safety mechanism is what distinguishes the high-risk tier from the lighter transparency duties elsewhere in the Act. Many organisations preparing for it align their internal controls with frameworks such as the NIST AI RMF and ISO/IEC 42001, though these are not a substitute for the Act’s legal conformity route.

    The phased timeline — and a proposed deferral

    The original calendar tied the main body of high-risk obligations to 2 August 2026, with product-embedded (Annex I) systems following on a longer transitional schedule into 2027. However, the timeline became the subject of legislative change. A “Digital Omnibus” proposal advanced through the EU institutions sought to defer key high-risk obligations — including those for Annex III stand-alone systems and the fundamental-rights impact assessment — to later dates, with reporting indicating a move toward December 2027 for Annex III and August 2028 for certain Annex I systems.

    An important caveat applies: until any amending instrument is published in the Official Journal, the original dates remain the binding law. Readers should therefore verify the current position against the authoritative European Commission timeline rather than relying on any single snapshot.

    Why the dates are layered

    The phasing reflects the Act’s dependence on supporting infrastructure: harmonised standards, notified-body capacity and Commission guidance all need to mature before conformity assessment can operate at scale. Spreading the high-risk obligations across multiple dates was intended to give that ecosystem time to develop, and the proposed deferral was framed around readiness concerns of a similar kind.

    The role of harmonised standards

    A defining feature of the high-risk tier is its reliance on technical standards. The Act anticipates that European harmonised standards will translate its broad legal requirements — such as “appropriate” data governance or “adequate” robustness — into specific, testable technical specifications. A system that conforms to a relevant harmonised standard benefits from a presumption of conformity with the corresponding legal requirement, which is what makes self-assessment viable for many systems. The development of these standards through the European standardisation bodies is therefore on the critical path, and delays in finalising them were among the readiness concerns cited in discussions about the timeline.

    Registration and the EU database

    Beyond conformity assessment and CE marking, providers of many high-risk systems must register them in an EU database before placing them on the market. The database is intended to give regulators and, in part, the public visibility into which high-risk systems are in circulation and who is responsible for them. Combined with logging and documentation duties, registration reflects the Act’s traceability ambition: the ability to reconstruct, after the fact, how a high-risk system was built, tested and used.

    What this means in practice

    For organisations building or deploying systems that may be high-risk, the practical task is classification first: determining whether a system falls within Annex I or Annex III, and in what role. A single product can involve multiple parties — a model provider, a system provider and one or more deployers — each with distinct duties, so mapping responsibilities along the value chain is as important as the technical build. Terminology such as provider, deployer, notified body and conformity assessment is defined in the Regulation; our dictionary offers plain-language entries for readers approaching these concepts for the first time.

    In summary

    The high-risk tier is the operational heart of the EU AI Act, turning principles such as oversight and robustness into documented, assessable requirements backed by CE marking. Its precise effective dates were in flux as deferral proposals moved through the legislative process, so the exact calendar should always be checked against the official record. This article describes the framework and the state of the timeline; it does not provide legal advice.

  • NYC Local Law 144: Bias Audits for Hiring AI Tools

    New York City’s Local Law 144 of 2021 requires employers and employment agencies that use certain hiring algorithms to commission an annual bias audit. Enforcement began on 5 July 2023, making it one of the earliest US measures to place concrete, testable obligations on AI used in employment. This article explains what the law covers and what the audit involves. It is informational and not legal advice.

    What the law covers

    Local Law 144 applies to automated employment decision tools (AEDTs) — broadly, computational tools that substantially assist or replace discretionary decision-making in hiring or promotion. Where an AEDT is used to screen candidates or employees for a position in New York City, the law imposes audit, notice and publication duties. The official guidance is published by the city’s Department of Consumer and Worker Protection (DCWP).

    What the bias audit requires

    At the centre of the law is the audit itself. Key features, as described in the city’s rules, include:

    • The audit must be independent and impartial, conducted by an auditor that is not involved in using or developing the tool.
    • It must be carried out no more than one year before the tool is used.
    • It tests the AEDT for disparate impact — assessing how selection or scoring outcomes differ across categories such as sex, and race or ethnicity, and intersections of those categories.
    • It typically reports metrics such as selection rates and impact ratios across groups, drawing on the tool’s historical data or test data.

    The audit is descriptive: it surfaces and quantifies differences in outcomes rather than certifying a tool as fair or unfair. The result is a defined set of figures that must then be disclosed.

    Notice and publication duties

    Beyond the audit, the law imposes transparency obligations:

    • Employers must publish a summary of the most recent bias-audit results, and the tool’s distribution date, in a clear and conspicuous place on their website.
    • Candidates and employees who live in New York City must be notified at least ten business days before an AEDT is used, including notice of the job qualifications and characteristics the tool will assess.

    These notice duties connect Local Law 144 to broader debates about disclosing the use of automated systems, a theme we track under generative-AI disclosure even though AEDTs are not necessarily generative.

    Enforcement and penalties

    The law is enforced by the DCWP. Reported penalty structures include a civil penalty for a first violation and escalating penalties for subsequent violations, with each day of non-compliant use potentially treated as a separate violation. The enforcement posture has been the subject of public scrutiny, including review of how actively the requirements are being enforced.

    How it fits the wider landscape

    Local Law 144 is narrow by design: it targets a specific use of AI — employment screening in a single city — rather than AI broadly. That makes it a useful contrast with comprehensive frameworks. Where the EU AI Act classifies employment AI as high-risk within a sweeping regime, Local Law 144 takes a single, audit-and-disclose mechanism and applies it precisely. Organisations operationalising bias testing often reference voluntary tools such as the NIST AI RMF and management standards like ISO/IEC 42001 to structure the surrounding governance, although neither defines the city’s specific audit requirements.

    What “independent” and “impartial” mean here

    The independence requirement is central to the law’s credibility, and the city’s rules address what disqualifies an auditor. Broadly, an auditor should not have been involved in using, developing or distributing the tool, and should not have a financial interest that would compromise objectivity. The practical effect is that the bias audit cannot simply be a vendor’s self-assessment; it must be carried out by a party with sufficient distance from the tool. This independence is part of what distinguishes a Local Law 144 audit from internal fairness testing that organisations may already perform.

    The role of test data and small samples

    An operational challenge the rules confront is what to do when an employer lacks sufficient historical data about a particular tool’s outcomes. The rules permit the use of test data in defined circumstances, and they address how to handle categories with too few data points to produce a meaningful figure. These provisions matter because the headline metrics — selection rates and impact ratios across groups — depend on having enough data to compute them reliably. The framework therefore acknowledges that an audit’s quality is bounded by the data available, a recurring theme in algorithmic-fairness measurement.

    Why it matters

    As one of the first laws to require a concrete, recurring, publicly disclosed algorithmic audit, Local Law 144 became a reference point in discussions about how to make AI accountability measurable. Its emphasis on independent testing, quantified disparate-impact metrics and candidate notice illustrates a disclosure-and-audit model distinct from outright prohibition. It also prompted debate about the limits of the model: critics asked whether disclosure of impact ratios alone changes employer behaviour, while supporters pointed to the value of forcing measurement and transparency where previously there was none. Readers new to terms such as disparate impact, impact ratio or selection rate may find plain-language explanations in our dictionary.

    In summary

    NYC Local Law 144 requires annual independent bias audits of automated employment decision tools, public disclosure of summary results, and advance notice to candidates. It is a targeted, audit-based approach to AI accountability in hiring. This article describes the requirements as published by the city; it is not legal advice, and employers should consult qualified advisers and the official DCWP guidance.