Synthetic data is not one training policy
Studies of recursive training reach different outcomes because “uses synthetic data” hides the decisive choice: whether generated examples replace the original corpus or accumulate beside it. Specify retention and sampling before judging the risk.
A training proposal describes a new corpus as forty per cent synthetic. The review follows a familiar route. Somebody asks which model generated it, whether the examples were filtered and how the new model performs on the holdout set. The percentage becomes the risk label.
That label leaves out what happens between one training run and the next. The generated examples might supplement a versioned source corpus. They might replace it. They might be stored beside it while a fixed training budget quietly stops sampling most of the originals. Those are different training policies, even when a data inventory calls all three “synthetic augmentation”.
The risk in recursive training depends on retention and sampling, not only on whether generated data appears in the corpus. A dataset owner should approve the sequence by which examples survive and enter each training run, rather than approve a synthetic percentage in isolation.
Replacement compounds what the generator missed
A generative model is an estimate of the distribution it learned from. Samples from that model carry its approximation error. When those samples become the whole corpus for the next model, another round of estimation starts from that imperfect projection. Repeating the cycle can amplify small errors and remove low-probability parts of the original distribution.
Shumailov and colleagues call this model collapse. Their experiments across language models, variational autoencoders and Gaussian mixture models found that indiscriminate recursive use of model-generated content caused models to forget parts of the original distribution, with the tails disappearing first. Their central warning is also a data-management condition: access to the original data distribution matters through the sequence, and provenance is needed to distinguish generated material from other sources. Nature, 2024
That result is sometimes compressed into a rule that synthetic data poisons a model. The compression removes the experimental policy. Gerstgrasser and colleagues compared replacing each corpus with the latest generated data against accumulating successive generated batches alongside the original data. In their controlled experiments, replacement tended towards collapse while accumulation kept error bounded across the model families and settings tested. Their linear model analysis reaches the same distinction, although it does not turn the experimental finding into evidence about every production system. Gerstgrasser and colleagues, 2024
| What the inventory says | What the training sequence does |
|---|---|
| Synthetic data replaces the source corpus | Each generation learns only from the previous model’s samples |
| Synthetic data accumulates beside the source corpus | Original and generated examples remain available to later runs |
| All generations are retained under a fixed training budget | Availability grows, but the sampler decides what the model actually sees |
| Synthetic share is reported as one percentage | Lineage, generation and sampling weight are hidden inside the total |
The distinction explains why two studies can report different directions without one of them making synthetic data universally safe or unsafe. “Contains generated examples” describes material. It does not describe the process that made one generation evidence for the next.
Retention without sampling is only an archive
Keeping the original corpus is not enough if the training process stops drawing from it. Kazdan and colleagues tested three workflows across Gaussian estimation, kernel density estimation and language-model fine-tuning. Replacing the source data produced collapse in the settings studied. Training on all accumulated source and generated data remained stable. When data still accumulated but each generation trained on a fixed-size subset, performance degraded slowly rather than failing explosively. Kazdan and colleagues, 2024
That third workflow is the enterprise case worth noticing. Storage is cheap enough that a team can truthfully say the originals were retained. Compute is bounded, so a sampler selects a subset for every run. As generated batches grow, the probability of drawing an older source example can fall unless the pipeline sets weights by origin and generation. The archive preserves the evidence. The sampler determines whether the model learns from it.
Verification adds another decision. Feng and colleagues studied selection of generated examples with human or machine verifiers. In two practical tasks and several theoretical settings, they found that even imperfect verification could prevent the collapse observed when generated data was accepted without that gate. The experiments do not establish a universal verifier threshold. They do show why “filtered” needs a measurable acceptance rule and a checked relationship to downstream performance. Feng and colleagues, 2024
Approve a corpus policy, not a percentage
The reviewable object is a manifest for every training run. It should say which source-corpus version was eligible, which generated batches were eligible, how many examples the sampler drew from each stratum and what verification each generated batch passed. An aggregate percentage can be derived from that manifest. It cannot substitute for it.
The holdout has a different job from the retained source corpus. Retained source examples influence training and can interrupt a recursive feedback loop. A holdout stays outside it and measures what happened. Combining the two makes the reported result optimistic and removes the independent evidence needed to test the policy.
The lineage fields matter even when the synthetic examples are good. Without them, a later team cannot reconstruct which model trained on which ancestor, cannot compare policies and cannot remove a defective generated batch without discarding unrelated material. This extends the earlier case that training-data provenance belongs in a register. That article concerns rights and acquisition. Here provenance records a statistical dependency across model generations.
What this does not tell you
These studies use controlled recursive-training settings. They do not establish that an enterprise fine-tune containing generated examples will collapse, or that accumulation will protect every task, model architecture and sampling scheme. One of the accumulation experiments itself reports deterioration in a fixed-size sampling workflow. The operational proposal is to make that workflow visible and testable, not to infer a safe recipe from one paper.
Nor is source data automatically correct, representative or lawful to retain. Human-created records carry selection effects, label errors and historical policy choices. Contractual, copyright and data-protection duties can require material to be removed. When that happens, the manifest should record the change and the evaluation should test its consequence. “Original” identifies a lineage position. It is not a quality certificate.
Synthetic data can be useful for coverage, simulation, privacy engineering and tasks where verified answers are cheaper to check than to produce. The decision is therefore not whether to permit it as a category. The data owner deciding on the next training run should require the retention, sampling, verification and evaluation policy that gives the category meaning. If the proposal provides only a percentage, it has described the contents of a bucket and left the training system unspecified.