Round summary
Modified-Delphi / RAND–UCLA appropriateness, two rounds. Topic: surveillance of incidental pulmonary nodules (illustrative).
Chance-corrected ordinal agreement is interval Krippendorff’s α across the statement set (0.71 — substantial). One statement did not reach consensus and is carried to a further round rather than forced to a recommendation.
Per-statement result
Median, IQR, the share of votes in the appropriate region, the RAND/UCLA classification, and the resulting GRADE recommendation.
S4. Acquire nodule CT at slice thickness ≤ 1.5 mm. Consensus · for
S1. Use volumetry (not diameter) as the primary growth metric for solid nodules 6–15 mm. Consensus · for
S5. Apply a validated baseline malignancy-risk model (e.g. Brock). Consensus · for
S2. Report incidental micro-nodules < 4 mm without measuring them. Consensus · for
S3. Discharge a stable solid nodule after 12 months of follow-up. No consensus · re-vote
Implementation priority
Importance × feasibility (median, cut at 7). The split statement (S3) is omitted until it resolves.
Final statement table
| # | Statement | Median (IQR) | % 7–9 | Classification | GRADE |
|---|---|---|---|---|---|
| S4 | Slice thickness ≤ 1.5 mm | 9 (0.5) | 100% | Consensus · for | Recommend (strong, high) |
| S1 | Volumetry as primary growth metric | 8 (1) | 92% | Consensus · for | Recommend (strong, moderate) |
| S5 | Validated baseline risk model | 8 (1.5) | 83% | Consensus · for | Suggest (conditional, low) |
| S2 | Report micro-nodules without measuring | 7 (2) | 75% | Consensus · for | Suggest (conditional, low) |
| S3 | Discharge stable nodule at 12 months | 5 (6) | 50% | No consensus | No recommendation — re-vote |
Methods appendix (draft text)
Panel consensus was established using a modified-Delphi / RAND–UCLA appropriateness method. Twelve panelists independently rated each statement from 1 (strongly inappropriate) to 9 (strongly appropriate) across two rounds. Between rounds, each panelist received their own prior rating and the anonymized group distribution; no panelist was identified at any stage.
A statement was classified as consensus (for) when the group median fell in 7–9 with at least 75% of ratings in that region, consensus (against) for the symmetric 1–3 case, and no consensus otherwise, with an extreme-disagreement rule for bimodal distributions. Ordinal agreement across the statement set was summarised with interval Krippendorff’s α (α = 0.71). Recommendations were graded by combining the panel’s consensus direction with the certainty of evidence using GRADE / Evidence-to-Decision, yielding strong (“recommend”) or conditional (“suggest”) recommendations; statements without consensus received no recommendation and were carried to a further round.
Analyses were computed in GuidelineVahti, in the browser, on aggregate ratings only, with a versioned data file retained as the source record. The panel and its chairs are responsible for the recommendations; the tool records what the panel decided and does not certify that a recommendation is correct.
Run a pilot on your own statements
This is a synthetic example. To produce a record like this from your panel, start with a scoped pilot — from €1,500.
See the pilot tiers →