Piloting
What a pilot that carries a decision looks like: task, metrics, duration, stopping criteria.
The task
- Occurs at least weekly, ideally daily.
- Has a result that can be checked unambiguously.
- Contains no or little personal data.
- Has a named responsible person.
- Has a baseline from today, at least roughly collected.
If one of those five is missing, it is not a good first task. The most common failure is the first: an interesting but rare task yields twelve cases after eight weeks and therefore no basis to decide on.
The two metrics
One for time or volume, one for quality. Both collected before the start.
| Example | Time metric | Quality metric |
|---|---|---|
| Document capture | Minutes per document | Share of mandatory fields captured correctly |
| Ticket triage | Minutes to assignment | Share of tickets assigned correctly |
| Quotation draft | Minutes to ready-to-send | Share of drafts sent unchanged |
| File summary | Minutes per file | Share of paragraphs changed |
The sequence
- 01
Week 0: collect the baseline
Measure twenty tasks under today's process. Without those figures there is no comparison later.
- 02
Weeks 1 to 2: set up and accompany
Everything gets checked. That phase produces the corrections that make quality measurable.
- 03
Weeks 3 to 8: operation with sampling
Lower the review rate by the error rate. Record errors and corrections continuously.
- 04
Week 9: evaluate and decide
Extend, continue with conditions, or stop. With reasons, documented.
Stopping criteria, fixed in advance
- The quality metric stays below today's process.
- The review effort exceeds the time saved.
- An error occurs that would not have been noticed and was expensive.
- The people involved stop using it after four weeks.
The last row is the most informative and is rarely measured. A tool that is not used voluntarily has not hit the task.
What gets documented at the end
| Item | Why |
|---|---|
| Task and boundary | What exactly was trialled |
| Baselines with collection date | The comparison basis |
| Results with case counts | Without a count there is no statement |
| Errors that occurred, with examples | The most valuable finding |
| Effort for setup and review | For the economics |
| Decision and reasoning | Including where it reads "stop" |
The fourth point is missing from most pilot reports and is the only one from which anything can be learned for the next task.
The pilot record
FREE ACCOUNT
Eight-week pilot record
A record with baselines, weekly collection and the evaluation sheet the decision rests on.
Worksheet3 items