What this is
What. When two or more people screen the same papers for a review, they won't agree on every one. ReviewVahti compares them record by record and reports how much they agreed overall, in your browser — so you can report the statistic and find the papers to reconcile.
Why. A systematic review is only as trustworthy as its screening. PRISMA and most journals expect a reliability statistic, and disagreement you can't quantify is a reviewer red flag. ReviewVahti gives you the number, and the list of conflicts to resolve.
How. Each reviewer screens the same papers in their own MatchVahti and exports a file. Load every reviewer's files here together; ReviewVahti checks they used the same protocol, compares them record by record, then reports overall agreement — raw % and Cohen’s κ (with PABAK and AC1) for a pair, Krippendorff’s α for three or more. These statistics summarise the whole set of decisions, not a single record.
New to this? Use Load a worked example below to see a filled-in result with no files.
Which number do I report?
For two reviewers, report raw % agreement with Cohen’s κ, and read PABAK and AC1 alongside, because κ alone understates agreement when almost every paper is excluded (the “kappa paradox”). For three or more, report Krippendorff’s α (≥ 0.80 reliable, 0.667–0.80 tentative).
How many reviewers can I load?
Two to seven. Two gives κ, PABAK, AC1 and a bootstrap CI; three or more gives Krippendorff’s α, Fleiss’ κ (when everyone rated every paper), and a pairwise matrix showing which pair diverged most.
What do I load, and where does it come from?
The protocol and ballot files (or the .zip) each reviewer exports from MatchVahti after screening the same papers under the same protocol. ReviewVahti matches them by a fingerprint (protocol_hash) and flags any ballot that doesn’t match, rather than scoring it silently.
Does my data leave the browser?
No. ReviewVahti makes no network calls at all — it reads only the files you load, computes everything on your device, and uploads nothing.
Does the AI count toward agreement?
No. Reliability is measured over human reviewers only. If an AI ballot is present, it is scored separately against the records where the humans agree, and never enters κ or α.
What is the “kappa paradox”?
When almost every paper is excluded, chance agreement is high, so κ can read low even when reviewers agree on nearly every paper. That is why ReviewVahti always shows κ next to raw %, PABAK, and AC1 — never alone.
1 · Load reviewers' files
or click to choose — protocol-*.json, ballot-*.json, or the reviewvahti-*.zip MatchVahti exports
New here? Load a worked example to see a populated result without any files. In real use, each reviewer screens the same papers in their own MatchVahti and exports a zip; load every reviewer's files together. The shared protocol is matched by its protocol_hash; a ballot whose hash doesn't match is flagged, never silently scored.