Synthetic data
Generated rather than collected examples: when that holds, where it breaks, and why a model that only learns from models gets worse.
The idea
Instead of collecting examples you produce them: rendered images of a component from a thousand angles, simulated sensor traces, generated sample sentences for a rare category.
When it pays
| Situation | Suitability |
|---|---|
| Rare defect images in manufacturing | very good, with CAD and rendering |
| Edge cases with known ground truth | very good |
| Balancing rare classes | good, with care |
| Replacing personal data | with limitations |
| Replacing expertise you do not have | unsuitable |
| Replacing a test set | unsuitable |
The routes
| Route | Principle | Typical for |
|---|---|---|
| Augmentation | Vary real data | Images, audio, text |
| Simulation | Render a physical model | Manufacturing, robotics, vehicles |
| Generative model | Learn from real data and generate | Text, images |
| Rule-based | Generate from templates | Forms, structured texts |
| Domain randomisation | Deliberately unrealistic variety | Transfer to reality |
The last row is counter-intuitive and well supported: when lighting, textures and backgrounds vary strongly and partly unrealistically in simulation, the model learns to ignore those factors and transfers better to real images.
- Always mix with real data; rarely train on synthetic alone.
- Treat the synthetic share as a hyperparameter and measure it.
- Mark synthetic data and version it separately.
- The test set stays real, always.
Model collapse
The practical consequence is worse than the formula suggests: the loss hits the tails of the distribution first. A model trained over several rounds on generated text first loses rare phrasings, technical terms and unusual cases, precisely what it is later needed for.
Remedy: mix real data into every round and never let its share fall below about a third.
Anonymity is not automatic
A generative model trained on real records can reproduce individual ones almost verbatim. Two checks are the minimum:
- Nearest-neighbour distance. For every generated record, measure the distance to the most similar real one. A cluster near zero shows memorisation.
- Membership test. Check whether the generator reveals whether a particular real record was in training.
The claim only becomes defensible with differential privacy in the generator's training:
Only with such a guarantee and a documented ε can you argue to a supervisory
authority that the generated data no longer carries a personal reference. An ε
above about 10 counts as weak.