Why a prompt fails
The seven causes prompts actually fail on, how to tell them apart, and what to change for each.
The seven causes
| Cause | How you recognise it |
|---|---|
| The task is unclear | The result is generic and arbitrary |
| The context is missing | The result does not fit the situation |
| The format was not stated | Content is right, form is unusable |
| Instructions contradict each other | Part is followed, part is not |
| The information is not in the material | It gets invented instead of refused |
| The task is too large | The start is right, the end falls apart |
| It is the wrong kind of task | Arithmetic, counting, current facts |
What to change per cause
| Cause | The change |
|---|---|
| Task unclear | Describe the result, not the activity: "a table with four columns" rather than "summarise" |
| Context missing | Who reads it, what for, what happened before |
| Format not stated | Give the format, and an example of the wanted output if needed |
| Contradiction | Cut the prompt until only one instruction per aspect remains |
| Information missing | Add the rule: "If it is not in the text, write: not present" |
| Task too large | Break it into steps, one call per step |
| Wrong kind of task | Change tools: let something that computes do the computing |
Diagnosis in three steps
- 01
Same instruction, different material
If it succeeds there, it is the material and not the prompt.
- 02
Shorter prompt, same material
If it improves, the prompt was overloaded or contradictory.
- 03
One example of the wanted output
If that fixes it, it was a format problem and not a comprehension problem.
Why the middle gets lost
Models attend to the beginning and the end of a long input more reliably than to the middle. In long prompts with many instructions, the criterion sitting in the middle is precisely the one that drops out.
- What matters most at the start, the format requirement at the end.
- Instructions as a short list rather than as prose.
- Separate material and instruction visibly, with a marker that does not occur in the text.
- Beyond five instructions, check whether two calls beat one.
The contradiction you cannot see yourself
"Summarise briefly" and "name every point" are incompatible. So are "factual" and "promotional". The model then follows one of the two, and which one changes between calls.
- Exactly one instruction per aspect: one for length, one for tone, one for format.
- Where there is a genuine conflict, resolve it explicitly: "If length and completeness collide, completeness wins."
- Examples beat adjectives. One example of the wanted tone works better than three descriptive words.
When it is the model
Only once task, context, format and material have been checked is the model the suspect. Then what helps is:
| Case | Measure |
|---|---|
| Multi-step reasoning fails | A model with reasoning steps, see Reasoning models |
| Numbers and units wrong | Hand the arithmetic to a tool |
| Domain terms missed | A glossary in the prompt, or retrieval over your own sources |
| Output varies widely | Lower the temperature, see Sampling and temperature |
| Format not respected | Enforce a schema rather than asking for one |
The diagnosis sheet
FREE ACCOUNT
Diagnosis sheet for a failed prompt
A sheet that narrows the cause down in four steps before you change the wrong end.
Worksheet2 items