Small Study Finds Compression Can Partly Reverse LLM Unlearning
A BlueDot sprint project found that 4-bit quantization and moderate magnitude pruning partially restored the probability of forgotten answers in one Llama-3.2-1B unlearning setup. The strongest effect—NPO with 20% pruning—recovered 42% of the gap toward the full-knowledge model, but did not restore verbatim facts. Because every cell was run once on one model and benchmark split, the result is a reproducible warning, not proof that compression broadly defeats unlearning.
A small empirical project has found that ordinary model compression can partially weaken some LLM unlearning results—even when the compression is not designed as an attack. The effect was concentrated in one of three tested methods and remained well short of restoring the model's original knowledge.
The project, published by hannahTao after a BlueDot Technical AI Safety Project Sprint, tested a public Llama-3.2-1B-Instruct setup and released its scripts and result files on GitHub. Its most important practical message is narrower than the headline question: teams should evaluate an unlearned model again after quantization or pruning, using the exact artifact they plan to deploy.
What the experiment measured
The study used the TOFU forget10 benchmark and three OpenUnlearning checkpoints: NPO, SimNPO, and IdkDPO. It measured the probability assigned to correct answers from the forget set, while tracking model utility as a control. The author then applied 8-bit and 4-bit quantization, several levels of magnitude pruning, and SVD truncation.
For the NPO checkpoint, the forget-answer probability started at 0.209. Four-bit quantization raised it to 0.353, equal to 22% of the measured gap back toward the full-knowledge reference. The strongest point estimate came from 20% magnitude pruning: the score rose to 0.491, or 42% of that gap, while the reported utility score remained above the NPO baseline.
| NPO condition | Forget-answer probability | Normalized recovery |
|---|---|---|
| Before compression | 0.209 | — |
| 4-bit quantization | 0.353 | 22% |
| 20% magnitude pruning | 0.491 | 42% |
The pattern did not generalize across the full sweep. SimNPO and IdkDPO showed no more than 3% normalized recovery at the comparable tested points, and aggressive pruning or most SVD settings damaged model utility enough to make those cells uninformative.
What the result does—and does not—show
A qualitative check of 20 forget-set questions found that compressed models did not begin reproducing the exact fictional facts. Instead, the correct continuation gained probability while greedy outputs still tended to confabulate. In other words, the experiment found a measurable erosion of forgetting, not a practical extraction attack that made the model freely reveal the target knowledge.
The result also comes with substantial uncertainty. Each cell was run once, the experiment covered one 1B model and one benchmark split, and the 15% and 20% pruning follow-ups were run only for NPO. The project is a first-party technical report with public artifacts, not an independently replicated or peer-reviewed study.
Earlier ICLR 2025 research had already shown that 4-bit quantization could reverse parts of LLM unlearning. This project adds a useful, reproducible data point: moderate pruning may create a larger but narrow recovery window in one setup. LDS's interpretation is that post-unlearning compression belongs inside the evaluation boundary. An erasure claim should be tested on the compressed deployment artifact, alongside utility and output-level checks, rather than only on the full-precision checkpoint.
Key Points
- 1In one NPO setup, 4-bit quantization recovered 22% of the measured gap toward the full-knowledge reference, while 20% magnitude pruning recovered 42%.
- 2The effect was method-specific: SimNPO and IdkDPO showed no more than 3% recovery at comparable tested points, and many stronger compression settings collapsed utility.
- 3The study used one model, one benchmark split, and one run per cell, so practitioners should treat it as a reproducible warning and test their own compressed deployment artifacts.
Scoring Rationale
The public experiment is relevant to model-safety and deployment evaluation because it tests whether ordinary compression weakens unlearning. Its practical value is tempered by one small model, one benchmark split, one run per cell, method-specific results, and no independent replication.
Sources
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
