Written and maintained by CASRAI Editorial Board
Last updated
Read the frontier-AI safety literature end to end — the lab frameworks, the California and New York statutes, the EU AI Act’s general-purpose AI chapter, the safety-institute testing agreements — and one of the world’s largest standards bodies is simply not there. IEEE has published a full international standard on the transparency of autonomous systems, and runs a certification mark that assesses AI products against four ethics criteria. Neither is cited by any of the frontier lab frameworks or frontier statutes covered in this cluster. That absence is not an oversight anyone needs to correct; it is structural, and understanding why is the fastest way to learn what a conformity standard can and cannot do for you. (In CASRAI’s own NIKOLAI dictionary, the distinction lands on the evaluation run element — more on that below.)
This guide covers what IEEE 7001-2021 actually specifies, how its graded transparency levels work, what IEEE CertifAIEd certifies and who performs the assessment, why neither appears in frontier governance, and the two settings — procurement and deployed-system transparency claims — where both are genuinely the right instrument. University procurement and IT-governance offices, in particular, are a far better-fit buyer for a certification mark than a frontier lab ever was.
What IEEE 7001-2021 Actually Is
IEEE 7001-2021 is the IEEE Standard for Transparency of Autonomous Systems. IEEE’s standards catalogue records it as published on 4 March 2022 and currently active. Its stated scope is unusual enough to quote directly: the standard describes “measurable, testable levels of transparency, so that autonomous systems can be objectively assessed, and levels of compliance determined.”
That sentence is the whole reason the standard is worth knowing about. Almost every other transparency instrument in AI governance — model cards, system cards, the EU AI Act’s Annex XI documentation duties, the transparency reporting in a frontier framework — specifies what must be disclosed. IEEE 7001 specifies how much transparency a system has, on a graded scale, per audience. It is a measurement instrument rather than a disclosure checklist.
Its origins explain some of its shape. The standard was developed by the P7001 working group under a sponsorship split between the IEEE Vehicular Technology Society’s Intelligent Transportation Systems committee and the Robotics and Automation Society Standards Committee — robotics and autonomous vehicles, not machine-learning research. The worked examples the drafters had in mind were physical autonomous systems that can injure someone and then be investigated, which is why two of its five stakeholder groups are accident investigators and lawyers.
The Graded Transparency Levels, and the Five Stakeholder Groups
The design that makes IEEE 7001 distinctive is that transparency is not one number. Writing in Frontiers in Robotics and AI while the standard was still in draft as P7001, members of the drafting group set out its architecture: the standard defines transparency requirements separately for five stakeholder groups, because each needs something different from the same system.
- End users — need to understand what the system is doing and why, well enough to use it appropriately.
- The general public and bystanders — people affected by a system they did not choose to interact with.
- Safety certification agencies — need evidence sufficient to support a certification or assurance judgement.
- Incident and accident investigators — need recorded system data detailed enough to reconstruct what happened.
- Lawyers and expert witnesses — need findings from that investigation in an interpretable, admissible form.
Each group gets its own scale, running from level 0 to level 5. Level 0 means “none” — no transparency provision at all for that group. Higher levels move from basic documentation, through interactive explanation tools, toward richer explainability, and the drafters were explicit that the levels are not uniformly cumulative across groups: what counts as level 3 for an accident investigator is a different kind of artefact from level 3 for a bystander.
Two consequences follow, and both are frequently misread.
The compliance floor is very low. As the drafting group put it: “A system would be compliant with P7001 if it meets at least level 1 transparency for at least one stakeholder group.” A bare claim of “IEEE 7001 compliant” therefore carries almost no information on its own. It is consistent with level 1 for end users and level 0 — nothing — for investigators, regulators and the public.
The correct output is a profile, not a grade. Assessment runs through what the standard calls a System Transparency Assessment (STA), and the drafters are direct about how it should be reported: “The correct way to describe P7001 compliance is through the multi-element description of the STA,” rather than a simple pass or fail. If a vendor hands you a one-line 7001 claim, the useful reply is to ask for the STA itself, broken out by stakeholder group.
This graded, per-audience framing is a genuinely different lens from the one most explainable-AI methods supply. XAI techniques ask what can be extracted from a model; IEEE 7001 asks what a named audience actually receives, and scores it.
What IEEE CertifAIEd Certifies
IEEE CertifAIEd is a separate programme and a different kind of object. It is not a standard: it is a conformity assessment scheme that issues a certification mark. In IEEE’s own words, “the IEEE CertifAIEd mark recognizes that a product, service, or system has been verified to meet relevant ethical criteria.” The unit certified is the autonomous or intelligent system and the AI-enabled product, not the organisation that built it.
The programme assesses four sets of criteria, defined by IEEE as follows:
- Transparency — criteria that “relate to values embedded in a system design, and the openness and disclosure of choices made.”
- Accountability — criteria that “recognize that the system/service autonomy and learning capacities are the results of algorithms and computational processes designed by humans.”
- Algorithmic bias — criteria that “relate to the prevention of systematic errors and repeatable undesirable behaviors.”
- Privacy — criteria “aimed at respecting the private sphere of life and public identity of an individual, group, or community.”
The assessment is performed by IEEE Authorized Assessors, and the product pathway runs through pre-assessment alignment, an independent ethical evaluation, issuance of the certification and mark, and continuing improvement guidance. A parallel professional credential exists for individuals, who complete training, pass an exam, and hold a three-year certification that must then be renewed — note that this three-year term applies to the individual credential. IEEE’s public pages do not state a validity period for the product mark itself, and we did not find one; treat any vendor claim about how long a product mark lasts as something to verify against the certificate.
The most-cited adopter is a government rather than a technology company. IEEE announced on 15 November 2021 that “the City of Vienna has become the first city worldwide to earn the IEEE CertifiAIEd AI Ethics (AIE) Certification Mark,” in support of Vienna’s Digital Humanism strategy. That is a revealing first customer: a public body that needed to demonstrate to its residents that a deployed municipal system had been independently reviewed against stated ethics criteria.
Why Frontier AI Governance Skipped Both
Neither instrument shows up in frontier-AI safety frameworks or statutes, and the reason is not that IEEE was late or unknown. Both predate almost every document in this cluster. The mismatch is structural, and it comes down to three properties.
A certification mark attests to a point in time; frontier risk is conditioned on capability. A conformity assessment establishes that a specified system, in a specified configuration, met specified criteria when it was examined. Frontier safety frameworks are built on the opposite premise: that the relevant question changes when the model changes. A capability threshold is a trigger that re-fires on every new model and every meaningful capability increase, which is why frontier frameworks describe evaluations that re-run per model rather than certificates that are issued once.
The unit of assessment is different. CertifAIEd certifies a product or deployed system. A frontier framework governs a model — a set of weights that will be deployed across many products, by many parties, some of them not yet known at assessment time. A mark on one deployment says nothing about the next one, and a general-purpose model has an open-ended deployment surface by design.
The risk vocabulary does not overlap. CertifAIEd’s four criteria — transparency, accountability, algorithmic bias, privacy — are the canonical responsible-AI set, and they are real risks. They are not the risks frontier frameworks are written against: chemical, biological, radiological and nuclear uplift, offensive cyber capability, autonomous AI research and development, loss of control. An assessment scheme can be rigorous and still be pointed at a different threat model entirely.
The same structural argument applies, with less force, to IEEE 7001. Its transparency levels are defined around a system whose behaviour can be observed and explained to a bystander or reconstructed by an investigator. That framing transfers reasonably well to a deployed AI product and poorly to the question of whether a frontier model can be elicited into producing dangerous capability under adversarial conditions.
What Each Is Genuinely Useful For
Dismissing both because they do not solve frontier risk would be the wrong conclusion. Each is well-matched to problems frontier instruments handle badly or not at all.
IEEE 7001: making a transparency claim falsifiable
The practical value of 7001 is that it converts “our system is transparent” from marketing into something with a testable structure. If you are writing a requirement, specifying a minimum level per named stakeholder group is enormously more useful than asking for “appropriate transparency.” If you are receiving a claim, the per-group STA profile tells you immediately whether the transparency was built for users, for regulators, or for nobody in particular. It is also the only widely available instrument that treats incident investigators and legal proceedings as first-class transparency audiences — a gap that matters once a deployed system is involved in a dispute.
IEEE CertifAIEd: procurement and public accountability
A certification mark’s real function is to let a buyer rely on someone else’s assessment. That is exactly the position a procurement office is in: it cannot audit every AI-enabled product it buys, and it needs a defensible basis for saying a product was independently reviewed. Vienna’s use of the mark is the clearest illustration — a public body demonstrating to its residents that a deployed system had been assessed against published ethics criteria by an external assessor.
In that role it sits alongside, not against, the management-system route. ISO/IEC 42001 certification attests that an organisation runs a documented, auditable AI management system; CertifAIEd attests that a specific product was assessed against ethics criteria. They answer different questions, and a mature vendor file may contain both. Neither is a substitute for third-party AI auditing where you need an examination scoped to your own use case.
What Neither Can Substitute For
- Capability-conditional evaluation. No certificate tells you whether a model crosses a dangerous-capability threshold, because the assessment was not designed to look for one and was not re-run on the model you are actually using.
- Model-level assurance from a product mark. A mark on a product built on a general-purpose model says nothing about the model, and nothing about anyone else’s product built on the same weights.
- Currency. A point-in-time assessment ages. If the underlying model was updated after the assessment, the certificate describes a system that no longer exists in the form examined.
- Statutory compliance. Neither instrument is a harmonised standard conferring a presumption of conformity under the EU AI Act, and neither is named in the US state frontier statutes. Holding the mark does not discharge a legal obligation.
- Scope you did not check. A bare “IEEE 7001 compliant” claim is satisfiable at level 1 for a single stakeholder group. Always ask which groups, and at what level.
Why This Matters for Research Administration
This is one of the few AI-governance topics where the research-administration relevance is stronger than the frontier-lab relevance, not weaker.
University procurement offices, research computing groups and IT-governance committees buy AI-enabled products constantly — proctoring and plagiarism tools, admissions and advising analytics, clinical decision support in an academic medical centre, lab instrumentation with embedded models, research administration software with AI features. None of these purchases is a frontier-model deployment, and a frontier safety framework is the wrong lens for all of them. A conformity mark and a graded transparency profile are, in principle, exactly the right lens: they are artefacts a purchasing officer can put in a file.
Three practical uses follow. First, in vendor questionnaires: institutions already run structured vendor reviews such as the HECVAT vendor security assessment, and an IEEE 7001 stakeholder-level profile is a cleaner ask than an open-ended “describe your transparency measures.” Second, in specifications: naming a minimum transparency level per stakeholder group in an RFP gives evaluators something to score. Third, in institutional risk registers, where a certification mark belongs recorded with its scope, its assessor and its date — alongside the rest of your third-party AI vendor risk assessment, not as a replacement for it.
The honest caveat for a procurement office is the one above: a mark tells you a specified system passed a specified review on a specified date. If the vendor’s underlying model has been swapped or updated since, you are holding a document about a previous system. Record the assessment date, and re-ask at renewal.
Where NIKOLAI Fits
NIKOLAI is CASRAI’s own independent frontier-AI-safety dictionary. It is unendorsed: no organisation named in a crosswalk has adopted it, and every crosswalk row is a shadow mapping — CASRAI’s reading of how an external document uses a term — unless that organisation has filed a Mapping Declaration confirming it. IEEE has not filed one, so nothing here should be read as IEEE’s own position.
Two NIKOLAI elements name the structural gap this guide describes.
Evaluation run (track N5, evidence and evaluations) is defined as “a single execution, or a declared batch of executions, of an evaluation against a specified model checkpoint and configuration, recorded with attempt count, scoring rule, and date.” The phrase “specified model checkpoint and configuration” is precisely what a product certification mark does not bind to. A certificate identifies a product; an evaluation run identifies the exact artefact evaluated, which is what makes the result re-checkable when the model changes.
Coverage date (track N1, actors, models and scope) is “the date as of which a report’s risk statements are asserted to hold, distinct from the report’s publication date.” That element exists because a safety claim has a shelf life. A certification mark has the same property and usually states it less clearly: the date the assessment was performed, not the date the certificate was printed, is the date the claim is about.
Frequently Asked Questions
Is IEEE 7001-2021 a certification you can hold?
No. IEEE 7001 is a published standard describing how to assess transparency; it is not itself a certification scheme. A system undergoes a System Transparency Assessment producing a per-stakeholder-group level profile. IEEE CertifAIEd is the separate programme that issues a mark, and it assesses four ethics criteria rather than 7001’s transparency levels.
Does “IEEE 7001 compliant” mean a system is fully transparent?
No, and this is the most important thing to know about the standard. The drafting group stated that a system is compliant if it meets at least level 1 transparency for at least one of the five stakeholder groups. The claim is therefore satisfiable with minimal transparency for a single audience and none for the rest. Ask for the full STA profile.
Do any frontier AI safety frameworks require IEEE 7001 or CertifAIEd?
None of the frontier lab frameworks or frontier statutes covered in this cluster cites either instrument. The mismatch is structural: both assess a product or system at a point in time, while frontier frameworks are built around capability thresholds that trigger fresh evaluation whenever a model changes.
How does IEEE CertifAIEd compare with ISO/IEC 42001?
CertifAIEd certifies a product, service or system against four ethics criteria, assessed by IEEE Authorized Assessors. ISO/IEC 42001 certifies an organisation’s AI management system — its processes for governing AI — not any particular product. They are complements, and a vendor holding one has not answered the question the other asks.
Who performs an IEEE CertifAIEd assessment?
IEEE Authorized Assessors conduct the independent ethical evaluation. The product pathway runs from pre-assessment alignment through that evaluation to issuance of the certification and mark, with continuing improvement guidance afterwards. A separate professional credential exists for individual practitioners, carrying a renewable three-year term.
Is IEEE 7001 relevant to large language models?
Partially. It was developed under robotics and intelligent-transportation sponsorship, and its stakeholder model — including accident investigators and expert witnesses — reflects physical autonomous systems. The per-audience graded structure transfers usefully to a deployed AI product; it was not designed for, and does not address, adversarial capability elicitation on a general-purpose model.
Sources
- IEEE Standards Association, IEEE 7001-2021: IEEE Standard for Transparency of Autonomous Systems — standards.ieee.org/ieee/7001/6929/
- IEEE Standards Association, IEEE CertifAIEd (Industry Connections programme page) — standards.ieee.org/industry-connections/ieee-certifaied/
- IEEE Standards Association, IEEE CertifAIEd product and professional certification pages — standards.ieee.org/products-programs/icap/ieee-certifaied/
- Winfield et al., “IEEE P7001: A Proposed Standard on Transparency,” Frontiers in Robotics and AI, 2021 — describes the standard’s stakeholder groups, level scale and System Transparency Assessment while it was in draft.
