Learn · Research integrity

Document AI-assisted analysis before memory rewrites it

The analysis reruns. Unfortunately, your memory does not. When a reviewer asks why an exclusion happened, the answer should not depend on finding a six-month-old chat. Record provenance, decisions, runs, and claims while the work is happening.

By Heidi Andersén, MD, PhD. Updated 16 July 2026.7 minute read

I will remember why.Future you has opened seventeen tabs.

Rerun is not reconstruct

Reproducibility guidance asks one question of your analysis: can someone rerun it? AI-assisted work adds a second question that rerunning cannot answer: can someone reconstruct it? A rerun shows the code produces the numbers. It does not show which code was model-written, which analytic decisions were the model's suggestions, or why an exclusion that is not in the protocol appeared in the final script.

Ordinary workflows scatter the path: prompts in one tool, code in another, decisions in nobody’s file. Months later, under a deadline, reconstruction falls to memory. A bench scientist would recognise the fix immediately: a contemporaneous, dated lab notebook that is never rewritten. The analysis side of research mostly skipped that habit, and AI assistance has made the gap visible.

The four things to record

The decision record earns its keep most. Reviewers rarely ask "show me your terminal history"; they ask why — why this denominator, why drop those three patients, why a sensitivity analysis appeared in revision. A one-line rationale written at the moment of the decision answers in seconds what memory cannot answer in a week. And when the AI proposed the decision, saying so plainly (accepted, edited, or rejected) is exactly the part a careful reader wants to see.

The plain-text version

A dated log file next to the analysis is enough to begin. One entry per event, appended at the moment it happens:

## 2026-07-11 14:05 · decision (exclusion)
Dropped ECOG 4 stratum. Why: n=3, prespecified minimum is 10.
Source: AI-suggested; edited (model proposed dropping ECOG 3-4).

## 2026-07-11 14:20 · provenance
survival_analysis.py rewritten by claude-sonnet-5 from prompt
"primary survival analysis per SAP v2". Model saw: sap_v2.md, codebook.csv.

## 2026-07-11 14:31 · run
python survival_analysis.py on data_2026-07-04.csv, seed 42.
Outputs: table1.csv, fig2.png.

Three rules keep it defensible:

1. Record at the moment, not at submission. A record reconstructed later is a guess with dates on it.
2. Append, never rewrite. If an entry was wrong, add a correcting entry. A log whose past changes under its own history has stopped being a record.
3. Write the why, not just the what. "Dropped ECOG 4" is a fact; "n=3, prespecified minimum is 10" is a defence.

This costs perhaps a minute per event, against an evening of archaeology per reviewer question. If you adopt nothing else from this guide, adopt the file.

Making the record tamper-evident

A plain log has one weakness: nothing shows whether it was quietly edited afterwards. The standard fix is a hash chain — each entry carries a cryptographic fingerprint of the entry before it, so anything edited, deleted, or reordered later breaks the chain visibly from that point on. Tamper-evident, not tamper-proof: it cannot stop a rewrite, but it cannot be rewritten silently.

AuditLite, our open-source tool, keeps exactly this record as one command per event, and adds what a text file cannot do by itself: it fingerprints every run (script hash, data hash, package versions, seeds, outputs) and then checks the record against your project:

auditlite decision --what exclusion --choice "drop ECOG 4" \
    --why "n=3 in stratum, prespecified minimum is 10" \
    --source ai_suggested --disposition edited

auditlite run --input data.csv --output table1.csv -- python survival_analysis.py

auditlite check   # before submission

auditlite check answers seven questions, each three ways (ok, flag, or not checkable): is the chain intact, were scripts edited after their last recorded run, does the data on disk still match what the runs consumed, were outputs modified outside a recorded run, do manuscript tables still match their files, did any script run without a declared origin, does every decision carry a reason. A hand-edited results table, the classic quiet disaster, comes back as a flag, not a surprise in review. It is free, Apache-2.0, works from Python and R on the same trail, and everything stays on your machine.

At submission, the disclosure is already written

Journals increasingly require an AI-use disclosure statement, and the statements that get authors in trouble are the vague ones written from memory. If you kept the record above, the statement is a summary, not a reconstruction: which tools, which versions, what they were used for, what you reviewed and take responsibility for — all of it is already in the trail with dates attached. How to phrase the statement itself, and where journal policies disagree, is its own guide: how to write an AI-use disclosure statement.

What the record cannot prove

A record shows support for reconstruction, not correctness — the trail can be intact while the analysis is wrong. And provenance is declared by you, not detected: the useful property is that a missing declaration becomes visible rather than invisible. The science stays your judgement. The record makes that judgement defensible.

Related guides

At submission: write the AI-use disclosure statement. For the code itself: make your analysis code citable with a Zenodo DOI. More local-first research guides are on the Learn page.