EpiNet — the Epistemic Network toolkit

Honest evaluation is the default, not an afterthought

Where most modelling code optimizes a headline number, EpiNet is built to resist fooling itself. A label-permutation null, calibration, bootstrap intervals, and community-aware splitting run alongside every headline metric, so a good score reflects real signal rather than leakage or chance. Contestability makes each call inspectable; an optional federated layer shares findings across sites without pooling records.

EpiNet's contribution is methodological reproducibility, not a new predictor: the model is a standard random forest, and the value is the conservative, auditable evaluation workflow wrapped around it — so the same checks travel with every analysis.

Scope: research and education only. EpiNet is not clinical or public-health decision support. Any model it produces must be validated on independent, outcome-linked data before it means anything clinically. It does not infer causality, outbreak dynamics, clinical risk, or intervention effects — use the outputs as exploratory evidence, not as decisions.

What it does

You load entities and relationships from CSVs; EpiNet computes graph features, honestly evaluates an outcome model, finds shortest paths, clusters nodes in feature space, and scores how contestable each call is — with publication-quality figures and a model card.

• Graph features — degree, weighted degree, clustering, component size, isolate flag, and optional betweenness / closeness / PageRank.
• Honest outcome model — a random forest over graph features + node attributes, reported with discrimination (AUROC, AUPRC), calibration (Brier; slope/intercept for binary outcomes), bootstrap CIs, permutation importance, a label-permutation null, community-aware splitting, small-cohort warnings, a provenance block, and a TRIPOD+AI-flavoured model card.
• Contestability — the closed-form smallest feature-space move that flips a node's nearest-centroid class, plus a per-feature value-of-information ranking.
• Shortest paths & feature-space clustering — source-to-target paths with per-target coverage; k-means centroids with per-node distance to each outcome-class centroid.
• Baselines & external validation — compare graph features against a node-embedding baseline and a no-information floor under the same harness, and validate a model on an independent cohort.
• Federated pipeline (optional) — reconstruct the scaler, centroids, and contestability from per-site aggregates only, behind a fail-closed governance gate.
• LLMvahti (experimental) — the same contestability lens pointed at LLM-as-judge verdicts: a blinded second-rater audit where the human rates first, with bootstrapped inter-rater agreement and judge-confidence calibration.

Install

pip install vahtian-epinet   # the package + the `epinet` command

From a source checkout, pip install -e ".[dev]" adds pytest, ruff and hypothesis; ".[lidc]" and ".[excel]" add the example extras.

Quick start

epinet \
  --nodes synthetic_nodes.csv \
  --edges synthetic_edges.csv \
  --outcome-column Outcome \
  --target-outcome 1 \
  --output-dir epinet_outputs

This runs graph-feature generation, an honestly-evaluated outcome model, and shortest-path summaries side by side. Outputs include model_metrics.json, a human-readable model_card.md, node_features.csv, provenance.json, and calibration / learning-curve / network plots. Full data format and methods are in the docs.

EpiNet Workbench Local CSV-to-report — plan → run → optional UI

The Workbench is a local interface that writes a reviewable analysis.yaml and runs the same EpiNet engine as the CLI, so every interactive run is reproducible without the interface. It is not AutoML — it never searches models or chases a metric. The config, not hidden UI state, is the source of truth.

epinet-workbench plan --nodes nodes.csv --edges edges.csv --outcome Outcome --output analysis.yaml
epinet-workbench run  --config analysis.yaml
epinet-workbench ui   # optional local Streamlit workbench

Every run distils its diagnostics into plain-language claim gates — permutation null, split sensitivity, baseline floor (paired per split with a Nadeau–Bengio-corrected interval: beats floor / at floor / not resolvable at this n), and external validation — written into the model card, claims_check.json, and a self-contained offline HTML report. A standing "do not claim clinical utility unless…" caveat is generated into every report and cannot be removed by theming.

Scope and caveats

The model is intentionally simple. Network features can be useful descriptors, but they can also encode sampling bias, measurement bias, and structural confounding. Before using EpiNet for health, education, welfare, employment, or public-sector decisions, add: domain-specific data validation; directed / temporal assumptions; uncertainty and sensitivity checks; external validation on independent outcome-linked data; privacy and governance review; and human review of any operational recommendation. For clinical prediction, align reporting with TRIPOD+AI.

Questions