Learn · Research integrity

How to document an AI-assisted analysis (so you can defend it later)

A reviewer asks why the exclusion happened. The decision lives in a chat prompt you can no longer find. The analysis itself runs fine — but which code the model wrote, why that variable was transformed, and whether Table 1 still comes from the current script all live in memory. Here is what to record while you work, a plain-text way to do it today, and the checks to run before submission.

Rerun is not reconstruct

Reproducibility guidance asks one question of your analysis: can someone rerun it? AI-assisted work adds a second question that rerunning cannot answer: can someone reconstruct it? A rerun shows the code produces the numbers. It does not show which code was model-written, which analytic decisions were the model's suggestions, or why an exclusion that is not in the protocol appeared in the final script.

This is not carelessness. Nothing in an ordinary workflow records the path — prompts sit in one tool, code in another, decisions in nobody's file. So reconstruction falls to memory, months later, under a deadline. A bench scientist would recognise the fix immediately: it is a lab notebook. Contemporaneous, dated, and never rewritten. The analysis side of research mostly skipped that habit, and AI assistance is what makes the gap visible.

The four things to record

Record	What it answers	What to write down
Provenance	Who or what wrote this code?	File, declared origin (human-written / AI-generated / AI-modified), model and version, the prompt, what files the model saw.
Decisions	Why is the analysis shaped this way?	What was chosen (variable, model, exclusion, transformation, sensitivity analysis), the reason, and — if it started as an AI suggestion — whether you accepted, edited, or rejected it.
Runs	What actually executed?	Which script version, on which data, with which package versions and seeds, producing which outputs.
Claims	Does the manuscript still match?	Each table and figure, tied to the run and file it came from. Plus any deviation from the analysis plan, with the reason.

The decision record earns its keep most. Reviewers rarely ask "show me your terminal history"; they ask why — why this denominator, why drop those three patients, why a sensitivity analysis appeared in revision. A one-line rationale written at the moment of the decision answers in seconds what memory cannot answer in a week. And when the AI proposed the decision, saying so plainly — accepted, edited, or rejected — is exactly the part a careful reader wants to see.

The plain-text version — start today, no tools

A dated log file next to the analysis is enough to begin. One entry per event, appended at the moment it happens:

## 2026-07-11 14:05 · decision (exclusion)
Dropped ECOG 4 stratum. Why: n=3, prespecified minimum is 10.
Source: AI-suggested; edited (model proposed dropping ECOG 3-4).

## 2026-07-11 14:20 · provenance
survival_analysis.py rewritten by claude-sonnet-5 from prompt
"primary survival analysis per SAP v2". Model saw: sap_v2.md, codebook.csv.

## 2026-07-11 14:31 · run
python survival_analysis.py on data_2026-07-04.csv, seed 42.
Outputs: table1.csv, fig2.png.

Three rules keep it defensible:

1. Record at the moment, not at submission. A record reconstructed later is a guess with dates on it.
2. Append, never rewrite. If an entry was wrong, add a correcting entry. A log whose past changes under its own history has stopped being a record.
3. Write the why, not just the what. "Dropped ECOG 4" is a fact; "n=3, prespecified minimum is 10" is a defence.

This costs perhaps a minute per event, against an evening of archaeology per reviewer question. If you adopt nothing else from this guide, adopt the file.

Making the record tamper-evident

A plain log has one honest weakness: nothing shows whether it was quietly edited afterwards. The standard fix is a hash chain — each entry carries a cryptographic fingerprint of the entry before it, so anything edited, deleted, or reordered later breaks the chain visibly from that point on. Tamper-evident, not tamper-proof: it cannot stop a rewrite, but it cannot be rewritten silently.

AuditLite, our open-source tool, keeps exactly this record as one command per event — and adds what a text file cannot do by itself: it fingerprints every run (script hash, data hash, package versions, seeds, outputs) and then checks the record against your project:

auditlite decision --what exclusion --choice "drop ECOG 4" \
    --why "n=3 in stratum, prespecified minimum is 10" \
    --source ai_suggested --disposition edited

auditlite run --input data.csv --output table1.csv -- python survival_analysis.py

auditlite check   # before submission

auditlite check answers seven questions, each three ways (ok, flag, or not checkable): is the chain intact, were scripts edited after their last recorded run, does the data on disk still match what the runs consumed, were outputs modified outside a recorded run, do manuscript tables still match their files, did any script run without a declared origin, does every decision carry a reason. A hand-edited results table — the classic quiet disaster — comes back as a flag, not a surprise in review. It is free, Apache-2.0, works from Python and R on the same trail, and everything stays on your machine.

At submission, the disclosure writes itself

Journals increasingly require an AI-use disclosure statement, and the statements that get authors in trouble are the vague ones written from memory. If you kept the record above, the statement is a summary, not a reconstruction: which tools, which versions, what they were used for, what you reviewed and take responsibility for — all of it is already in the trail with dates attached. How to phrase the statement itself, and where journal policies disagree, is its own guide: how to write an AI-use disclosure statement.

An honest caveat

A record shows support for reconstruction, not correctness. An intact chain shows the trail was not rewritten — not that the recorded events are true, and not that the analysis is right. Nothing here detects AI-written code either: provenance is declared by you, and the useful property is that missing declarations become visible rather than invisible. The science stays your judgement. The record just makes that judgement defensible.

Related guides

At submission: write the AI-use disclosure statement. For the code itself: make your analysis code citable with a Zenodo DOI. More local-first research guides are on the Learn page.