Research and development
Reviewing literature, evaluating experiments, searching patents: where AI helps in R&D and why every citation must be checked.
What this is about
| Task | Value | Condition |
|---|---|---|
| Reviewing and summarising literature | high | only with a real database |
| Structuring experiment records | high | none |
| Screening measurements for anomalies | high | your own data |
| Comparing patent documents | medium | database and expert review |
| Proposing hypotheses | medium | as a prompt, not a finding |
| Interpreting results | low | stays with the expert |
Introduction
- 01
Search only with grounding
A tool that searches a real database and cites its hits is usable. A model answering from memory is not.
- 02
Start with your own material
Experiment records, reports and measurement series are in house, structurable and raise no legal questions.
- 03
Do not replace statistics
A pattern found is a hypothesis. It is tested on new data, not confirmed on the same set.
- 04
Record everything
Model, version, parameters, data state. Without them the analysis is not reproducible and therefore not publishable.
- Check every reference against the primary source, including the plausible ones.
- Mind multiple testing: check twenty hypotheses and one will look significant by chance.
- Hold back a confirmation dataset, touched exactly once.
- On publication, disclose where AI was used. Most publishers now require it.
Text and data mining, legally
Articles 3 and 4 of the DSM Directive distinguish two cases, implemented nationally in § 60d and § 44b UrhG in Germany and § 42h UrhG in Austria:
| Case | Precondition | Reservation possible |
|---|---|---|
| Scientific research | Research organisation, lawful access | no |
| Other purposes | Lawful access | yes, machine-readable |
The first is broadly drawn but requires non-commercial purpose and secure storage of the copies. The second is the normal case for companies and is subject to the rightsholder's reservation.
Reproducibility
- Model name and exact version, not only the family. Providers update behind the same label.
- All parameters: temperature, system prompt, context length, seed where available.
- Data state with a checksum.
- The number of variants tried before the reported result emerged.
- Keep the raw outputs, not only the tidied version.
The fourth point is the one most often missing from publications and the one that decides defensibility. See Statistics for AI and Reading benchmarks.
Evaluating measurement series
A model that finds patterns in measurements also finds patterns in noise. Three safeguards are the minimum:
- Form the hypothesis before looking at the data, or label the work explicitly as exploratory.
- Correct for multiple testing, for instance by Benjamini-Hochberg.
- Confirm on an independent dataset, not on a different analysis of the same one.
Choosing a task in research and development
FREE ACCOUNT
Selection sheet and templates: research and development
A sheet that picks the first task out of the ones research and development actually has, with two tested templates and the review steps for them.
Worksheet3 items