Algorithmic Disgorgement vs Machine Unlearning

This page is not legal advice. It is a reference account of an enforcement remedy and of the research literature on the technique people assume can satisfy it. Nothing here establishes a legal standard, and the enforcement record described is US-specific. In NIKOLAI, CASRAI’s own independent frontier-AI-safety dictionary, the element that bears most directly on this subject is Model identifier on track N1 — a persistent identifier for a specific model version or checkpoint, together with a rule for which modifications keep the identifier stable and which require a new one. That second half is exactly the clause a disgorgement order needs and almost never has. NIKOLAI is unendorsed: it is a proposal, and its element statuses and crosswalk rows are CASRAI’s own reading of published sources, not anybody’s standard.

Last verified against primary sources: 25 September 2026.

Two Different Things With One Name Between Them

“Delete the model” describes a legal remedy and an engineering task, and the two have almost nothing in common except the verb.

Algorithmic disgorgement — also called model destruction or model deletion — is an equitable remedy. A regulator finds that data was obtained or used unlawfully, and orders the respondent to destroy not only the data but the derived artefacts: the models, the algorithms, the embeddings. It is a legal instrument, enforced by an order, and its measure of success is a compliance certification.

Machine unlearning is a research programme. Given a trained model and a subset of its training data, produce a model that behaves as though that subset had never been in the training set — without paying the cost of retraining from scratch. It is a technical instrument, and its measure of success is an evaluation metric.

The temptation is to treat the second as the cheap way to satisfy the first. That is where things go wrong, because a disgorgement order asks for a guarantee about the artefact and unlearning research mostly delivers a claim about observed behaviour. Those are not the same claim, and the gap between them has been measured.

What the FTC Has Actually Ordered

The remedy is not hypothetical and it is not new. Since 2019 the Commission has used it at least six times.

Cambridge Analytica, final order 6 December 2019

The Commission’s final opinion and order issued on 6 December 2019, following an administrative complaint filed in July that year. The order reached past the collected personal information to the “algorithms or equations” that originated, in whole or in part, from it. This is the origin point for the remedy, and it is worth noticing what it is not: it is not a statute, not a rule, and not a general policy. It is a term in a single litigated order.

Everalbum, announced 11 January 2021

The Everalbum settlement is the one that gave the remedy its vocabulary. The FTC alleged that Everalbum had applied facial recognition to users’ photos in its Ever app by default, without the consent it had told users it would obtain. The order required deletion of the photos and videos of deactivated users, deletion of all face embeddings derived from photos of users who had not given express consent, and — the novel part — deletion of what the order defines as Affected Work Product: any models or algorithms developed in whole or in part using the biometric information collected from users of the Ever app. The deletion deadline is ninety days.

Everalbum, by then trading as Paravision, confirmed it had deleted the Affected Work Product. The defined term has since been reused, close to verbatim, in later orders.

Kurbo and WW International, 3 March 2022

A COPPA action over a weight-management app marketed to children as young as eight. The settlement carried a $1.5 million civil penalty, required destruction of personal information collected from children under 13 without verifiable parental consent, and required destruction of any models or algorithms developed using it.

Ring, 31 May 2023

The Ring order is the bluntest statement of the principle. Ring was required to delete “data products such as data, models, and algorithms derived from videos it unlawfully reviewed.” The phrase data products is doing a great deal of work: it is a category defined by provenance rather than by form.

Edmodo, May 2023

Another COPPA matter, against an ed-tech platform. The stipulated order reused the Everalbum construction: delete or destroy any Affected Work Product, meaning any models or algorithms developed in whole or in part using personal information collected from children through the platform without verifiable parental consent or school authorisation.

Rite Aid, announced 19 December 2023

The Rite Aid order banned the company from using facial recognition technology for five years and required it to delete, and to direct third parties to delete, any images or photos collected because of its facial recognition system, together with any algorithms or other products developed using those images and photos. The complaint concerned the deployment of facial recognition in hundreds of stores between 2012 and 2020.

Two features of that order matter more than the headline ban. The first is the third-party direction: the obligation follows the data downstream into other organisations’ systems. The second is that this was the Commission’s first use of its Section 5 unfairness authority against an allegedly discriminatory deployment of AI, which means model deletion is not tied exclusively to a consent or privacy defect.

The Clause Every One of These Orders Needs

Read the six together and one phrase recurs: developed in whole or in part using. Everything turns on it.

In a 2019 recommender system, tracing which artefacts were “developed in part using” a given dataset was tractable. In a 2026 model estate it is a research problem. A foundation model is pre-trained once; then fine-tuned; then distilled into a smaller model; then quantized for serving; then used to generate synthetic data that trains a further model. Embeddings computed from the tainted data sit in a vector index that nothing in the order describes. If the order names one artefact and the organisation ships six descendants of it, the compliance question is not “did you delete it” but “delete what.”

This is the practical reason NIKOLAI’s Model identifier element is written the way it is. Its definition is not just an identifier; it is an identifier plus a stated rule for which modifications preserve it. The element is at Proposed status in nikolai-v0.1, and the published frameworks it surveys do not agree with each other: one treats checkpoints and versions as the same model where base capabilities stem primarily from the same foundational training run, another applies an inheritance rule under which non-material modifications carry the same risk threshold forward. Those are different answers, and an organisation under a disgorgement order is the party that has to pick one and defend it.

Why the Technical Side Cannot Yet Carry the Legal Side

Retraining from scratch without the offending data is the only method that indisputably satisfies a deletion requirement, and for a frontier-scale model it may cost millions of dollars. Machine unlearning exists to avoid that bill. The question is what it buys.

The first large-scale competition was about evaluation, not methods

The NeurIPS 2023 unlearning competition drew close to 1,200 teams. Its published findings report genuine progress — leading entries outperformed prior algorithms under the competition’s framework — but the framework itself was the contribution: an evaluation that measures forgetting quality against a formal notion of unlearning while accounting for retained model utility. The organisers’ conclusions point at trade-offs between forgetting and utility, and at how poorly methods generalise to new datasets. The competition made it much clearer how hard the measurement problem is.

A model can pass a deletion audit and fail it after routine deployment work

The sharpest single result is “Catastrophic Failure of LLM Unlearning via Quantization” (Zhang et al., ICLR 2025). For unlearning methods that operate under a utility constraint — that is, methods designed not to wreck the model while forgetting — the unlearned model retained on average 21% of the intended forgotten knowledge in full precision, rising to 83% after 4-bit quantization.

The mechanism is mundane, which is what makes it serious. To forget without destroying utility, these methods use a small learning rate and regularise against the retained set, so the unlearned weights stay very close to the originals. Quantization then maps both sets of weights onto the same low-precision values, and the difference that constituted the forgetting disappears.

Read that as a compliance timeline. A model is unlearned. It is audited at BF16 and shows near-zero accuracy on the forget set. A certification of compliance is filed. The model is then quantized to INT4 for cheaper serving — an ordinary engineering decision that no one thinks to route past counsel — and the forgotten material comes back. Nothing in the certification was false when it was signed.

The gap is a stated finding in the literature, not an inference

Cooper and colleagues make the point directly in “Machine Unlearning Doesn’t Do What You Think: Lessons for Generative AI Policy and Research” (December 2024, revised October 2025). Their argument is that policy discussion treats unlearning as a general-purpose mechanism for removing problematic content — personal data, copyrighted work, unsafe capability — and that it is not one. They separate two goals that regulators tend to merge: removing information from the model’s parameters, and preventing the model from producing particular outputs. A method that achieves the second is not evidence of the first.

Related work has shown the same thing from the attack side: supposedly forgotten knowledge can be recovered by light fine-tuning on unrelated data, by low-bit compression, or by adversarial prompting. If the remedy is deletion and the test is behavioural, the test is measuring the wrong object.

Europe Has Three Answers and They Do Not Agree

Anyone operating across jurisdictions should understand that the question “is the model itself in scope for erasure” currently has no settled answer.

The Hamburg position. On 15 July 2024 the Hamburg Commissioner for Data Protection and Freedom of Information published a discussion paper on large language models and personal data. Its thesis is that an LLM does not store personal data in the relevant sense — it stores numerical correlations between tokens — and therefore the model itself is not the object of data subject rights. Inputs and outputs of an LLM-supported system remain fully in scope; the weights, on this reading, are not. It is a discussion paper from one German supervisory authority, not a binding position.

The EDPB position. The European Data Protection Board adopted Opinion 28/2024 in December 2024. It declines to say that AI models are anonymous as a class. Anonymity is to be assessed case by case, and a model qualifies only where it is very unlikely both that individuals in the training data can be identified from it and that their personal data can be extracted from it by query. On unlawfully processed training data, the Board’s conclusion is that the unlawfulness may affect the lawfulness of the model’s subsequent deployment unless the model has been duly anonymised.

The FTC position. Destroy the model.

These are not three phrasings of one rule. They are three different objects of regulation: Hamburg regulates the system’s behaviour, the EDPB regulates the model conditionally on an extraction test, and the FTC regulates the artefact. An organisation cannot design a single deletion control that satisfies all three without deciding which one it is actually building for.

The Research-Administration Case Is Genuinely Different

Universities and academic medical centres are in an odd position here, because their own human-subjects rules point the other way, and this is not widely appreciated inside research computing.

OHRP’s 2010 guidance on withdrawal of subjects from research, announced in the Federal Register on 21 September 2010, states that investigators may retain and continue to analyse data already collected about a subject who withdraws. OHRP reads 45 CFR part 46 as permitting that retention without regard to the subject’s consent, provided the analysis falls within the scope described in the IRB-approved protocol — and it says so even where the data include identifiable private information. Withdrawal stops prospective collection. It does not, as a regulatory matter, reach backwards.

So a US-regulated study is, by default, permitted to keep the training set. That is a defensible position for a fixed statistical analysis. It becomes considerably less comfortable when the “analysis described in the protocol” is the training of a model that will be released as a research artefact, licensed to a spin-out, or deposited in a repository. The protocol-scope test was written for a planned analysis with a defined endpoint, and a model that continues to serve predictions after the study closes is not obviously that.

Data-sharing obligations pull differently again. Controlled-access datasets under the NIH Genomic Data Sharing framework are closed out through dbGaP’s Authorized Access System, and project close-out requires confirmation that the data have been destroyed. What a close-out certification means for model weights fitted to those data before destruction is a question that institutional practice has largely not had to answer yet.

Four practical consequences for a research institution:

Your IRB approval is not a defence against a consumer-protection theory. The FTC orders above rest on deception, unfairness and COPPA, not on the Common Rule. An institution with a commercial deployment surface can be exposed on a theory its IRB never evaluated, and CASRAI’s page on OHRP’s unactioned AI recommendations sets out how little federal guidance IRBs have been given on AI specifically.
Consent language should not promise what unlearning cannot deliver. A consent form that tells a participant their data can be removed from a trained model at any time is making a technical claim. On the evidence above, it is not currently a claim anyone can substantiate for a large model.
Tech transfer is where the exposure concentrates. A licensed or spun-out model inherits its training-data provenance. Diligence that examines the code and the licence but not the lineage of the weights is not examining the thing that a disgorgement order operates on.
Vendor terms should name the artefact class. If your institution’s data are used to train or fine-tune a supplier’s model, an agreement that requires deletion of “institutional data” on termination does not reach the weights. Our guide to assessing third-party AI vendor risk covers the wider diligence set.

What an Organisation Can Actually Do Now

Nobody can promise verified unlearning today. The controls that are available are provenance controls, and they work by making the “in whole or in part” question answerable before it is asked under an order.

Keep a derivation record for every model artefact you ship. Base checkpoint, fine-tune, distillation, quantized serving build, and every dataset that entered each. Without it, the scope of any deletion obligation is unbounded, and a regulator’s reading of it will not be the generous one.
Write an identity rule and publish it internally. State which modifications preserve a model identifier and which mint a new one. The rule matters less than having a consistent one you applied before the dispute.
Index the embeddings and the synthetic data too. Face embeddings were named explicitly in the Everalbum order. Vector indexes and model-generated training corpora are derived artefacts and should be recorded as such.
Separate the two claims in any deletion certification. “The model no longer reproduces this material under our test suite” and “this material is not recoverable from the weights” are different statements. Sign only the one you can support, and say which one it is.
Treat quantization, distillation and further fine-tuning as re-triggering the test. This is the single most actionable lesson from the ICLR 2025 result. A deletion certification is valid for the artefact tested, not for its descendants.
Retraining remains the only unambiguous remedy. Price it. An organisation that cannot afford to retrain a model has, in effect, no deletion capability for it, and should know that before it signs something that assumes otherwise.

What This Page Does Not Claim

It does not say machine unlearning is useless. It says the published evidence does not currently support using it as proof that data has been removed from a model’s parameters.
It does not say the FTC will order disgorgement in any particular case. The remedy has appeared in a small number of orders, all of them settlements or litigated administrative orders, and its outer limits have not been tested in court.
It does not state any search-volume or traffic figure for this topic, because CASRAI could not verify one at the time of writing.
It does not present NIKOLAI as a standard. The Model identifier element is at Proposed status and its framework comparisons are CASRAI’s shadow mappings — readings of published documents, not endorsements by the organisations named. A mapping becomes an organisation’s own position only where that organisation has filed a Mapping Declaration.