Preparing for audit
What an inspection actually asks for, which eight documents suffice, and where organisations routinely fail.
The eight documents
| Document | Content |
|---|---|
| Inventory | Every system with role, class, purpose, ownership |
| Classification reasoning | Per system, with date and categories assessed |
| Purpose description and boundary | For what, and expressly not for what |
| Data description | Provenance, categories, representativeness, bias assessment |
| Evaluation results | Fixed evaluation set, quality per category, with date |
| Oversight arrangement | Who, with what authority, on what basis |
| Logs | Per transaction, with model version and review note |
| Competence evidence | Who, for which system, when |
How an inspection runs
- 01
Is it known what is deployed?
The inventory is held against reality, often via network logs or invoices.
- 02
Is the classification traceable?
Not whether it is right but whether it is reasoned and dated.
- 03
Is operation under control?
Measurement, oversight, logs, reporting routes. This is where it is decided.
- 04
Do documents and reality agree?
Samples from operation against the documentation. The most common finding sits here.
Annex IV, for the provider role
- General description: purpose, version, interaction with other software, form of deployment.
- Detailed description: development steps, prior decisions, architecture, computational resources.
- Monitoring, functioning and control: accuracy, limits, foreseeable misuse.
- Risk management: risks identified and measures taken.
- Changes across the lifecycle.
- Standards and specifications applied.
- Declaration of conformity.
- Post-market monitoring plan.
What actually shortens preparation
- Dated documents. Without a date a document cannot be examined.
- A fixed evaluation set. It makes quality comparable across model changes and is the only number that carries.
- Trigger-based updating. Model change, purpose extension, new data category. Not an annual rhythm.
- A register of where things are. Searching for documents costs more time in practice than producing them.
- Named people. A department cannot be questioned; a person can.
The question that decides everything
An inspector rarely asks about the model. They ask: how would you have noticed it getting worse? Anyone who can name a number, a threshold and a reporting route passes. Anyone who answers that nobody complained does not.
The complete evidence list
FREE ACCOUNT
Evidence list for an audit
Four blocks covering the documents an audit actually asks for, each with a location and an owner.
Checklist4 items
Related courses and sources
Fraunhofer IAIS on artificial intelligence
German-language guidance on auditing, certifying and operating AI systems, from applied research.
For German-language audit and certification questions where English sources do not help.
Interpretable Machine Learning
What explainability methods deliver and where they get over-interpreted. The most sober treatment of the topic, freely available.
For anyone who has to promise explainability and should know what the methods actually deliver.
Model Cards for Model Reporting
The proposal to document purpose, limits and tested groups for every model. Today effectively a precondition for any audit.
For anyone preparing an audit; model cards have effectively become a precondition.
NIST AI Risk Management Framework
A structured frame for your own risk assessment, independent of the AI Act. Useful as an outline when none exists internally yet.
For building your own risk assessment, independent of the AI Act.
NIST AI RMF Playbook
The practical build-out of the risk framework: concrete suggestions per function on what to do and what to document.
For anyone building an internal risk assessment who wants an outline that has already been thought through.