Systematic reviews
“How do I know when my systematic search is finished?”
A defensible stopping rule for your systematic search.
Once you have started, every new search finds a few more papers, and letting any go feels like a mistake, so the pile never resolves and you feel like you are drowning. The way out is a stopping rule you set before you begin.
“Any literature research is like an asymptote, getting closer and closer, but never hitting zero. You can quickly feel like you’re drowning.” — a reviewer describing the screening pile
In one line“Systematic” means the search has a rule.
A systematic review answers one focused question by finding the relevant studies through a method you write down before you start, so someone else could repeat it. “Systematic” means the process is explicit and reproducible, not that you read everything ever written, which is also why the search has a defined endpoint rather than running until you are exhausted.
Why it feels endlessThe asymptote is real, and that’s fine.
You feel like you’re drowning because the returns are diminishing and never quite reach zero: there is always one more database, one more citation chain, one more grey-literature source. Without a rule written in advance, every one of those feels obligatory, and letting a paper go feels like negligence. Rather than search harder, define “enough” before you begin, so stopping becomes a planned step, not an act of guilt.
Common mistakesFour ways the search runs away
- No protocol. Without pre-set criteria, every borderline paper becomes a fresh judgement call, and the pile never resolves.
- Searching before defining “enough”. If the stopping rule is decided at the end, it looks like you stopped when you were tired.
- Chasing every citation forever. Backward and forward citation searching needs a boundary, or it recurses without end.
- Screening the whole pile in one sitting. One giant, undocumented pass burns you out and leaves no trail. Slices, recorded, beat a marathon.
The stopping ruleDecide “enough” before you start
- Write the protocol and criteria first.
Fix your question, databases, search terms, and inclusion and exclusion criteria in advance. Register it on PROSPERO if it is eligible. Now “in or out” is a rule, not a mood.
- Define your sources up front.
Name the databases you will search and the citation-searching you will do (say one round of backward and forward on included studies). Searching a source not on the list is a documented amendment, not an open door.
- Stop when new searches stop changing the set.
Your practical stopping signal: when an additional planned source returns no new includable studies, the search is saturated for your question. Record that it happened.
- Screen in slices, and log each one.
Handle a fixed batch, say 100 titles, per session, record the count in and out, and stop for the day. The trail turns an overwhelming pile into a series of finished tasks.
- Record everything for PRISMA.
Keep the running counts (identified, screened, excluded with reasons, included) as you go. The PRISMA flow diagram is just those numbers, and it is trivial to fill if you logged them and painful if you didn’t.
Worked exampleA stopping decision, recorded
A review that ends on purpose
Protocol registered on PROSPERO, three databases, one planned round of citation searching. That leaves 1,240 records, screened in slices of 100 by two people against the criteria.
The citation round then adds zero new includable studies and the grey-literature check adds none, so the search is complete. The PRISMA line: “1,240 screened, 1,180 excluded at title and abstract, 60 assessed in full text, 18 included. The final citation round yielded no new eligible studies.”
ChecklistYour stopping rule and counts
Set before you search
- Question fixed in PICO, with databases, criteria, and citation-searching named in advance (and registered if eligible).
- Stopping signal defined: a planned source returns no new includable studies.
- Screening done in fixed slices, each logged as you go.
- Running PRISMA counts kept, and the stopping decision recorded with a date and reason.
Related guidesRead next
Further readingOld sources worth your time
- Page et al. (2021), The PRISMA 2020 statement. BMJ (open access).
- PROSPERO, international register of systematic reviews. crd.york.ac.uk/prospero.
Recommended tool · the next step
MatchVahti-Lite
MatchVahti-Lite helps with one part of this: screening a manageable slice at a time and keeping the trail, with running counts ready for the PRISMA flow. It does not run the whole review, but it takes the weight off the screening pile, and it runs on your own machine, with the decisions staying yours.
Explore MatchVahti-Lite →