Generating images
Diffusion models from the inside: how noise becomes an image, what step count and guidance scale do, and which legal questions remain open.
The idea
Take a photo and pour noise over it step by step until only static remains. A model is trained to undo each of those steps: turning a slightly noisy image into a slightly less noisy one.
To generate, you start at pure noise and let the model denoise fifty times. What remains is an image that never existed.
The dials
| Dial | Effect |
|---|---|
| Steps | More steps, cleaner detail, longer compute |
| Guidance scale | How closely the prompt is followed |
| Seed | The same seed reproduces the same image |
| Negative prompt | What must explicitly not appear |
Steering guidance
The key trick is classifier-free guidance: the model computes every step twice, once with and once without the prompt, and amplifies the difference.
The formula explains both extremes: at w = 1 only the conditioned prediction
remains; at very large w the difference is amplified until colours clip and
content turns stereotyped.
Controlled generation
| Method | Purpose |
|---|---|
| Image to image | An existing image as a starting point, strength sets the departure |
| Inpainting | Regenerate a masked region only |
| ControlNet | Edges, depth map or pose as an additional constraint |
| LoRA | A style or object learned from a handful of images |
For business use ControlNet is usually the decisive building block: it turns a random generator into a tool that respects a given geometry.
The forward process
That property is what makes training affordable at all: for every training step a
random t is drawn, x_t computed in one line, and the model trained to predict
the ε that was mixed in.
Why latent diffusion
Diffusion directly on pixels at 1024 by 1024 means over a million dimensions per
step. Latent diffusion compresses the image first with an autoencoder by a factor
of 8 per side, down to 128 × 128 × 4 = 65,536 dimensions: a factor of 48 less
compute per step at barely visible quality cost. That is why image generation runs
on a single 8 GB card.
The legal position
Two questions must be kept apart:
- Training. In the EU, text and data mining rests on Art. 4 of the DSM Directive, implemented as § 44b UrhG in Germany and § 42h UrhG in Austria. The rightsholder's reservation must be expressed machine-readably. Whether and how effectively that reservation bites is the subject of ongoing litigation.
- Output. An image produced without human creative shaping is generally not a work in the EU and therefore unprotected. Anyone wanting exclusive use needs means other than copyright.
- Labelling. Under Art. 50 AI Act, providers must mark outputs machine-readably as artificially generated, and deployers must disclose deepfakes.
See Copyright and AI.
Related courses and sources
Denoising Diffusion Probabilistic Models
Removing noise step by step instead of generating an image in one pass. The basis of every current image model.
For understanding every current image model, thought through from noise.
Generative Adversarial Networks
Two networks learning against each other. The first approach that produced convincing images, now largely superseded.
Historically important; largely superseded today but useful for understanding how things developed.
High-Resolution Image Synthesis
Diffusion in a smaller latent space rather than in pixels. The step that made image generation possible on ordinary hardware.
For anyone running image generation themselves; explains why it works on ordinary hardware.