Vahtian

Systematic reviews

“This review is drowning me. When do I stop?”

A systematic review for beginners, and a defensible stopping rule.

Two things send people here: not being sure what a systematic review actually is, and — once started — not knowing when the searching ends. Every new search finds a few more papers, and letting any go feels like a mistake. Here is the shape of the whole thing, and the rule that lets you stop on purpose.

“Any literature research is like an asymptote — getting closer and closer, but never hitting zero. You can quickly feel like you’re drowning.” — a reviewer describing the screening pile

First, plainlyWhat a systematic review actually is

A systematic review answers one focused question by finding all the relevant studies, using a method you decide and write down before you start, so that someone else could repeat your search and reach the same set. That last part is the whole point: “systematic” means the process is explicit and reproducible, not that you read everything ever written. In order, it is: a focused question (often framed as PICO), a written protocol (ideally registered on PROSPERO), a documented search, two-person screening against pre-set criteria, extraction, and synthesis. The rules exist so the endpoint is a decision, not exhaustion.

Why it feels endlessThe asymptote is real — and that’s fine.

You feel like you’re drowning because the returns really are diminishing and never quite reach zero: there is always one more database, one more citation chain, one more grey-literature source. Without a rule written in advance, every one of those feels obligatory, and letting a paper go feels like negligence. The fix is not to search harder; it is to define “enough” before you begin, so stopping is a planned step rather than an act of guilt.

Common mistakesFour ways the search runs away

The stopping ruleDecide “enough” before you start

  1. Write the protocol and criteria first.

    Fix your question, databases, search terms, and inclusion/exclusion criteria in advance. Register it on PROSPERO if it is eligible. Now “in or out” is a rule, not a mood.

  2. Define your sources up front.

    Name the databases you will search and the citation-searching you will do (e.g. one round of backward + forward on included studies). Searching a source not on the list is a documented amendment, not an open door.

  3. Stop when new searches stop changing the set.

    Your practical stopping signal: when an additional planned source returns no new includable studies, the search is saturated for your question. Record that it happened.

  4. Screen in slices, and log each one.

    Handle a fixed batch (say 100 titles) per session, record the count in and out, and stop for the day. The trail turns an overwhelming pile into a series of finished tasks.

  5. Record everything for PRISMA.

    Keep the running counts — identified, screened, excluded (with reasons), included — as you go. The PRISMA flow diagram is just those numbers, and filling it is trivial if you logged them and painful if you didn’t.

Worked exampleA stopping decision, recorded

A review that ends on purpose

Protocol. Question fixed in PICO; databases MEDLINE, Embase, CINAHL; one round of citation searching on included studies; criteria written and registered on PROSPERO.

Screening. 1,240 records after de-duplication, screened in slices of ~100. Title/abstract screening by two people against the criteria; disagreements resolved by discussion.

Stop. The planned citation-search round added two records and zero new includable studies; grey-literature check added none. Search declared complete.

PRISMA line: “1,240 records screened, 1,180 excluded at title/abstract, 60 full-text assessed, 18 included. The final citation-search round yielded no new eligible studies.”

Checklist & templateYour stopping rule + PRISMA counts

Downloadable · stopping-rule worksheet

The stopping-rule & PRISMA-count worksheet

  • Question fixed (PICO) and written down.
  • Databases and citation-searching plan named in advance.
  • Inclusion/exclusion criteria set before screening; protocol registered if eligible.
  • Stopping signal defined: a planned source returns no new includable studies.
  • Screening done in fixed slices, each logged.
  • Running PRISMA counts kept (identified, screened, excluded + reasons, included).
  • The stopping decision recorded with a date and reason.
Systematic review — stopping rule + counts Question (PICO): __________________________________ Databases: _______________________________________ Citation searching planned: _______________________ Criteria written / registered (PROSPERO?): ________ Stopping signal: Stop when [named source] returns 0 new includable studies. Reached on: ____________________________ Screening (log each slice): Slice size: ____ Screened: ____ In: ____ Out: __ PRISMA counts: Identified: ____ After de-dup: ____ Title/abstract screened: ____ Excluded: ____ Full-text assessed: ____ Excluded (reasons): ____ Included: ____ Stop decision + date + reason: ____________________
Download the worksheet (Markdown)

The honest limit. A stopping rule makes your search defensible, not exhaustive — no review captures every study that exists. Its benefit is that a reader can see exactly where you drew the line and why, which is what “systematic” actually asks for.

Related guidesRead next

Recommended tool · the next step

MatchVahti-Lite

Screen a manageable slice at a time and keep the trail — MatchVahti-Lite tracks what you included and why, and keeps your running counts ready for the PRISMA flow. It runs on your own machine; the decisions stay yours.

Explore MatchVahti-Lite →

Further readingSources worth your time