Systematic reviews
“This review is drowning me. When do I stop?”
A systematic review for beginners, and a defensible stopping rule.
Two things send people here: not being sure what a systematic review actually is, and — once started — not knowing when the searching ends. Every new search finds a few more papers, and letting any go feels like a mistake. Here is the shape of the whole thing, and the rule that lets you stop on purpose.
“Any literature research is like an asymptote — getting closer and closer, but never hitting zero. You can quickly feel like you’re drowning.” — a reviewer describing the screening pile
First, plainlyWhat a systematic review actually is
A systematic review answers one focused question by finding all the relevant studies, using a method you decide and write down before you start, so that someone else could repeat your search and reach the same set. That last part is the whole point: “systematic” means the process is explicit and reproducible, not that you read everything ever written. In order, it is: a focused question (often framed as PICO), a written protocol (ideally registered on PROSPERO), a documented search, two-person screening against pre-set criteria, extraction, and synthesis. The rules exist so the endpoint is a decision, not exhaustion.
Why it feels endlessThe asymptote is real — and that’s fine.
You feel like you’re drowning because the returns really are diminishing and never quite reach zero: there is always one more database, one more citation chain, one more grey-literature source. Without a rule written in advance, every one of those feels obligatory, and letting a paper go feels like negligence. The fix is not to search harder; it is to define “enough” before you begin, so stopping is a planned step rather than an act of guilt.
Common mistakesFour ways the search runs away
- No protocol. Without pre-set criteria, every borderline paper becomes a fresh judgement call, and the pile never resolves.
- Searching before defining “enough”. If the stopping rule is decided at the end, it looks like you stopped when you were tired.
- Chasing every citation forever. Backward and forward citation searching needs a boundary, or it recurses without end.
- Screening the whole pile in one sitting. One giant, undocumented pass burns you out and leaves no trail. Slices, recorded, beat a marathon.
The stopping ruleDecide “enough” before you start
- Write the protocol and criteria first.
Fix your question, databases, search terms, and inclusion/exclusion criteria in advance. Register it on PROSPERO if it is eligible. Now “in or out” is a rule, not a mood.
- Define your sources up front.
Name the databases you will search and the citation-searching you will do (e.g. one round of backward + forward on included studies). Searching a source not on the list is a documented amendment, not an open door.
- Stop when new searches stop changing the set.
Your practical stopping signal: when an additional planned source returns no new includable studies, the search is saturated for your question. Record that it happened.
- Screen in slices, and log each one.
Handle a fixed batch (say 100 titles) per session, record the count in and out, and stop for the day. The trail turns an overwhelming pile into a series of finished tasks.
- Record everything for PRISMA.
Keep the running counts — identified, screened, excluded (with reasons), included — as you go. The PRISMA flow diagram is just those numbers, and filling it is trivial if you logged them and painful if you didn’t.
Worked exampleA stopping decision, recorded
A review that ends on purpose
Protocol. Question fixed in PICO; databases MEDLINE, Embase, CINAHL; one round of citation searching on included studies; criteria written and registered on PROSPERO.
Screening. 1,240 records after de-duplication, screened in slices of ~100. Title/abstract screening by two people against the criteria; disagreements resolved by discussion.
Stop. The planned citation-search round added two records and zero new includable studies; grey-literature check added none. Search declared complete.
PRISMA line: “1,240 records screened, 1,180 excluded at title/abstract, 60 full-text assessed, 18 included. The final citation-search round yielded no new eligible studies.”
Checklist & templateYour stopping rule + PRISMA counts
Downloadable · stopping-rule worksheet
The stopping-rule & PRISMA-count worksheet
- Question fixed (PICO) and written down.
- Databases and citation-searching plan named in advance.
- Inclusion/exclusion criteria set before screening; protocol registered if eligible.
- Stopping signal defined: a planned source returns no new includable studies.
- Screening done in fixed slices, each logged.
- Running PRISMA counts kept (identified, screened, excluded + reasons, included).
- The stopping decision recorded with a date and reason.
The honest limit. A stopping rule makes your search defensible, not exhaustive — no review captures every study that exists. Its benefit is that a reader can see exactly where you drew the line and why, which is what “systematic” actually asks for.
Related guidesRead next
Recommended tool · the next step
MatchVahti-Lite
Screen a manageable slice at a time and keep the trail — MatchVahti-Lite tracks what you included and why, and keeps your running counts ready for the PRISMA flow. It runs on your own machine; the decisions stay yours.
Explore MatchVahti-Lite →Further readingSources worth your time
- Page et al. (2021), The PRISMA 2020 statement — BMJ.
- PROSPERO, international register of systematic reviews — crd.york.ac.uk/prospero.