Baseline risk is not a neutral anchor for disconnected evidence

Disconnected networks & bridges
High priority
Overstated
Arm-based baseline-synthesis, reference-prediction, and aggregate-level matching models link disconnected subnetworks through a baseline-outcome structure, allowing disconnected RCTs and single-arm studies to be included. An observed control-arm risk depends on prognosis, endpoint definition, follow-up, ascertainment, care context, and calendar time, so it is not intrinsically exchangeable across designs or components. Thom et al. 2022 measured what this costs: disconnected-treatment bias ran from -0.16 to 0.392 for aggregate-level matching and from -0.102 to 0.40 for reference prediction, with coverage of 0.30 to 0.82 and 0.64 to 0.94 respectively, and they recommend reference prediction on coverage grounds. Beliveau et al. 2017 is not the cautionary result the source made it: across the two disconnected datasets it examined it found the use of random baseline treatment effects appropriate, while warning that the normality and exchangeability assumptions could be inappropriate in datasets it did not observe.

DIS-03 Overstated High priority Promising

ImportantA study has been run against this problem

A baseline-risk bridge transports whatever makes the subnetworks’ baselines differ. When a design nuisance (such as an endpoint definition) coincided with the subnetwork it passed one for one into the cross-gap contrast (bias 0.2 to 0.7) and no adjustment was possible. When a quarter of trials used the other definition, adjusting for it removed the nuisance but not prognostic drift between subnetworks (bias 0.19 to 0.20). Read the study below.

Statement

Arm-based baseline-synthesis, reference-prediction, and aggregate-level matching models link disconnected subnetworks through a baseline-outcome structure, allowing disconnected RCTs and single-arm studies to be included. An observed control-arm risk depends on prognosis, endpoint definition, follow-up, ascertainment, care context, and calendar time, so it is not intrinsically exchangeable across designs or components. Thom et al. 2022 measured what this costs: disconnected-treatment bias ran from -0.16 to 0.392 for aggregate-level matching and from -0.102 to 0.40 for reference prediction, with coverage of 0.30 to 0.82 and 0.64 to 0.94 respectively, and they recommend reference prediction on coverage grounds. Beliveau et al. 2017 is not the cautionary result the source made it: across the two disconnected datasets it examined it found the use of random baseline treatment effects appropriate, while warning that the normality and exchangeability assumptions could be inappropriate in datasets it did not observe.

Why it is open

Where a single baseline parameter both carries the cross-gap bridge and absorbs study-level nuisance variation, nothing inside a disconnected graph separates the two roles, so a difference in endpoint definition can be absorbed as a difference in prognosis and propagated into the relative effect. Observed baseline heterogeneity and design associations can be examined, but no such check reaches the cross-gap counterfactual, which is exactly what the bridge supplies, and regression to the mean and measurement error in the baseline term add distortion invisible in the fitted output. Thom et al. 2022 show the residual bias and undercoverage are material rather than theoretical, and better relative coverage for reference prediction in one simulation program is a conditional preference under stated assumptions, not evidence that disconnected networks are solved.

What has been tried

Work What it contributes
Chandler and Ishak 2026, Anchors Away (arXiv:2606.20341v1) states that in fully disconnected single-arm evidence treatment effects are not separately identifiable from study, population and design effects without additional structural assumptions, so unanchored comparisons typically require a common conditional baseline risk across study populations; it calls that assumption highly restrictive and unlikely to be plausible in most applications, and recommends fixed study effects with hierarchical pooling as a sensitivity analysis
Beliveau et al. 2017 asks how dangerous random baseline treatment effects are in disconnected networks and finds their use appropriate for the two datasets considered, with an explicit caveat about datasets it did not observe
Thom et al. 2022 extends baseline-effect and aggregate-level matching to disconnected RCTs and single-arm studies, and quantifies the residual bias and undercoverage of both by simulation

Probable solution or research direction

Use separate baseline distributions for connected, disconnected, and single-arm evidence, and use commensurate rather than fully exchangeable baseline priors so the bridge is down-weighted when source and target baselines conflict. Condition reference prediction on design covariates where the relevant covariates are measured, overlap, and can be modeled correctly, since the improvement depends on those conditions rather than being automatic, and calibrate with negative controls that detect design-driven baseline differences. Report an explicit posterior decomposition of how much the baseline bridge moves each treatment contrast, and treat reference prediction as a conditional method rather than a solution to disconnection.

Our study

A baseline-risk bridge across a disconnected network

The question put to it. When disconnected subnetworks are linked through exchangeable baseline risks, does study-level design nuisance pass into the cross-gap contrast, and can design covariates separate it from prognosis?

What it found. A baseline-risk bridge transports whatever makes the subnetworks’ baselines differ. When a design nuisance (such as an endpoint definition) coincided with the subnetwork it passed one for one into the cross-gap contrast (bias 0.2 to 0.7) and no adjustment was possible. When a quarter of trials used the other definition, adjusting for it removed the nuisance but not prognostic drift between subnetworks (bias 0.19 to 0.20).

Results.

  • Refuting sentence fails: without crossover, adjustment possible in 0% of analyses and bridge bias 0.207 to 0.712.
  • With 25% crossover and no drift, adjusted bias within 0.003.
  • With prognostic drift 0.2, adjusted bias 0.191 to 0.204.

Design.

Estimand
Cross-gap contrast C versus A on the log odds scale
Methods compared
exchangeable-baseline bridge, bridge adjusted for a design covariate
Data-generating mechanism
Two subnetworks with prognostic drift, a design nuisance on the baseline and a design covariate overlapping the gap or not; exchangeable-baseline bridge with and without design adjustment. 24 cells, 2000 replicates each.
Replicates
2000 per scenario, 48,000 in total

What this does not answer. Study-level summaries, no Bayesian arm-based model or commensurate prior, one design covariate. Peer review has not been done.

Peer review. Not yet done. The result stands on the study alone until it is.

Read it: PDF · Markdown · OpenDocument · Code and data: studies/DIS-03-one-parameter-doing-two-jobs · Protocol: registered before the run

Verification

Verdict. Overstated. Auditors judged the problem open, and the source attributed it to the wrong work. Chandler and Ishak, Anchors Away (arXiv:2606.20341v1, appendix A.7), supplies the formal support the closure said was absent: in disconnected single-arm evidence, study-level effects are not separately identifiable from treatment effects, so a common conditional baseline risk is an identifying constraint and not a neutral anchor. The same paper calls that assumption highly restrictive and recommends fixed study effects with hierarchical pooling, which is this entry’s own proposed direction, so the problem is real but its strong form is overstated.

The source said. “Arm-based baseline-synthesis, reference-prediction, and aggregate-level matching models link disconnected subnetworks through a baseline-outcome structure, allowing disconnected RCTs and single-arm studies to be included. Baseline risk is not a universal bridge: it mixes prognosis with outcome definition, ascertainment, follow-up, supportive care, and calendar time, and simulations show cross-disconnected estimates can still be materially biased.”

What was wrong with it. The source’s gloss on Beliveau et al. 2017, its severity line, and the associated claim all inverted the paper’s finding; the literature auditor quoted doi:10.1002/jrsm.1256 saying that for the two datasets considered the use of random baseline treatment effects in disconnected networks was appropriate. GPT-5.6 Sol added that not every implementation forces one baseline parameter to carry both nuisance variation and the whole bridge, and that observed baseline heterogeneity and its association with design features can be partly assessed, so the untestability narrows to the missing cross-gap counterfactual.

Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).

verdict-corrected
not-supported -> overstated The adjudication rule let a misattributed citation set the published verdict to not-supported, though every auditor on this entry voted the problem open. A misattribution is a fault in the provenance, not in the claim. The rule is corrected in build/adjudicate.mjs and the verdict re-derived from this entry’s own audit record. Auditors judged the problem open, and the source attributed it to the wrong work. Chandler and Ishak, Anchors Away (arXiv:2606.20341v1, appendix A.7), supplies the formal support the closure said was absent: in disconnected single-arm evidence, study-level effects are not separately identifiable from treatment effects, so a common conditional baseline risk is an identifying constraint and not a neutral anchor. The same paper calls that assumption highly restrictive and recommends fixed study effects with hierarchical pooling, which is this entry’s own proposed direction, so the problem is real but its strong form is overstated.
Corrected
severity: removed a line that inverted the cited finding The correction_note already recorded that the source’s gloss on Beliveau et al. 2017, its severity line and its associated claim all inverted the paper’s finding, and that the auditor quoted doi:10.1002/jrsm.1256 saying the use of random baseline treatment effects in disconnected networks was appropriate for the two datasets considered. The correction had reached the statement but not this field. claims_to_check keeps the inverted wording deliberately, since it records the claim as the source made it, which is what was checked. Previously: “random baseline treatment effects can be dangerous in disconnected networks”
Flagged for reopening
14 findings from the reading assert this problem is open while the registry carries overstated Flagged, not applied: reversing an earlier audit’s closure needs the closure’s own evidence re-examined alongside this, which the reading did not do. See documentation/audit/reading/PROPOSED.md.
Literature and prior-art check
open / misattributed. I read the Beliveau et al. 2017 abstract in full. The title is a question and the paper’s answer is largely reassuring, not cautionary: ‘For the 2 datasets considered, we found that the use of random baseline treatment effects in disconnected networks was appropriate.’ The entry’s prior_work gloss (‘shows random baseline treatment effects can be dangerous’), the severity line, and claim 1 all invert the paper’s finding. The authors do add a hedge about other datasets, which is the defensible version. This is the kind of error the people who wrote the paper will notice immediately. Everything attributed to Thom et al. 2022 checks out exactly: disconnected-treatment bias -0.16 to 0.392 (ALM) and -0.102 to 0.40 (reference prediction), coverage 0.30-0.82 versus 0.64-0.94, and ‘Reference prediction has greater coverage and may be recommended overall.’ The substantive problem (baseline risk mixes prognosis with ascertainment and design era) remains open; only the attribution needs fixing.
GPT-5.6 Sol, technical and source-code lens
open / supported-with-caveat. The technical core of C2-C9 and C13 is sound. An observed control-arm risk depends on prognosis, endpoint definition, follow-up, ascertainment, care context, and calendar time, so it is not intrinsically exchangeable across designs or components. A baseline bridge requires an appropriate arm-level outcome model plus explicit exchangeability or regression assumptions, and aggregate relationships do not identify individual-level transportability. The report nevertheless overgeneralizes. Not every implementation makes one baseline parameter carry both nuisance variation and the entire bridge; separate distributions, design covariates, measurement-error models, and partial pooling are possible. Observed baseline heterogeneity and design associations can be assessed, although such checks cannot validate the missing cross-gap counterfactual. Conditioning on design covariates is helpful only with measured relevant covariates, overlap, and a correctly specified model. C14-C15 are universal current-method absence claims, while C1 and C10-C12 are paper-attribution claims; none is established by the permitted technical and source-code lens. The issue is high priority only when baseline structure materially carries the decision contrast.
Decision path
R5:default-open → R6:counter-quote-uncontested

How the evidence falls in time

14 papers in the reviewed corpus bear on this problem, published 2015 to 2026. Every paper found here asserts the problem is open; none reports progress.

This problem is recurrent: three or more papers spanning at least eight years assert it independently. Sustained restatement by authors who mostly do not cite each other is the strongest evidence this reading can offer that a gap is real rather than one group’s framing.

Year Says Paper
2015 confirms it is open Network meta-analysis combining individual patient and aggregate data from a mixture of study designs with an application to pulmonary arterial hypertension
2017 confirms it is open An investigation of the impact of using different methods for network meta-analysis: a protocol for an empirical evaluation
2017 confirms it is open Performing Arm-Based Network Meta-Analysis in R with the pcnetmeta Package
2018 confirms it is open Estimating relative efficacy in acute postoperative pain: network meta-analysis is consistent with indirect comparison to placebo alone
2019 confirms it is open A comparison of arm-based and contrast-based models for network meta-analysis
2019 confirms it is open Incorporating adjustments for variability in control group response rates in network meta-analysis: a case study of biologics for rheumatoid arthritis
2021 confirms it is open Joining the Dots: Linking Disconnected Networks of Evidence Using Dose-Response Model-Based Network Meta-Analysis
2022 confirms it is open Meta-Research: Methods and Protocols (Evangelou E, Veroniki AA, eds), Methods in Molecular Biology vol. 2345
2022 confirms it is open Bayesian network meta-analysis methods for combining individual participant data and aggregate data from single arm trials and randomised controlled trials
2022 confirms it is open Network Meta-analysis on Disconnected Evidence Networks When Only Aggregate Data Are Available: Modified Methods to Include Disconnected Trials and Single-Arm Studies while Minimizing Bias
2022 confirms it is open Medical Therapy, Radiofrequency Ablation, or Cryoballoon Ablation as First-Line Treatment for Paroxysmal Atrial Fibrillation: Interpreting Efficacy Through the Shiny Method
2025 confirms it is open An investigation of the impact of using contrast- and arm-synthesis models for network meta-analysis
2025 confirms it is open Lord’s Paradox and two network meta-analysis models
2026 confirms it is open Anchors Away: Navigating Unanchored Indirect Comparisons with Multilevel Unanchored Meta-Regression (ML-UMR) (Chandler & Ishak; arXiv:2606.20341v1). The batch file points at media/media/Table1.pdf; the full LaTeX source mlumr.tex sits in the same arXiv package and was read.

Source

Derived from C:L187-188, C:L197, C:L214-215, C:L237, C:L1090-1100, C:L1102-1104 of the reviewed corpus.