No criterion for discarding a present but non-exchangeable anchor
Statement
Guidance makes the connectedness of the randomized evidence decide whether an unanchored comparison is admissible, but an analyst can declare a connected network effectively disconnected by judging that the nominally common comparator is not the same condition across trials, for example because the placebo response depends on the mode of administration or because rescue and subsequent therapies differ. Nothing governs that declaration. There is no test or threshold for how much comparator drift is enough, no requirement to report the anchored analysis beside the unanchored one, no sensitivity analysis over the disconnection decision itself, and no way to price the exchange being made: a partially checkable constancy assumption on relative effects is traded for an untestable constancy assumption on absolute outcomes. The two assumptions are not on a common scale, so the trade has no natural currency, and the resulting bias is unsigned in both directions, since comparator drift biases the anchored estimate by an amount nobody computes and disconnection introduces residual confounding by an amount nobody computes either. The election is consequential: in a migraine example a fixed-effects network meta-analysis and a network meta-regression found few differences between anti-CGRP antibodies, while an unanchored simulated treatment comparison run on the premise that mode of administration creates a distinct contextual effect came out strongly in favor of one product, and the same analysis found no evidence of a non-zero placebo-response coefficient because too few studies contributed to each comparison (Fawsitt et al. 2023, doi:10.57264/cer-2023-0021). The discretion has been named without being closed: population adjustment methods were criticized for giving very considerable leeway to investigators to choose anchored or unanchored approaches (Phillippo et al. 2018, doi:10.1177/0272989X17725740), and no criterion has followed.
Why it is open
Comparator drift is real and sometimes large, so a blanket rule that a connected network must always be analyzed anchored would be wrong; the remedy therefore has to be a criterion, and criteria are harder to write than prohibitions. The evidence needed to check the premise is largely the evidence the disconnection discards, because whether the comparator arms differ beyond chance is a question the anchored model would answer, and it is underpowered in exactly the small networks where the question arises. There is no scale on which to express the exchange, since the anchored bias is a function of unobserved comparator drift and the unanchored bias is a function of unobserved prognostic imbalance, and neither quantity is identified from the data at hand. The declaration is also made by the party that assembles the evidence and that benefits from the outcome, and because the anchored analysis is often not reported at all a reviewer cannot see what the decision cost. Reporting conventions have not caught up: nothing obliges a submission to present both analyses, so the discretion leaves no audit trail.
What has been tried
| Work | What it contributes |
|---|---|
| Fawsitt, Thom, Regnier, Lee, Kymes and Vase 2023, Comparison of indirect treatment methods in migraine prevention to address differences in mode of administration, Journal of Comparative Effectiveness Research 12 | Runs network meta-analysis, network meta-regression and unanchored simulated treatment comparison on the same connected evidence base, reaches materially different conclusions, and justifies treating the network as disconnected by a contextual effect of administration route that its own regression could not support because too few studies informed each comparison. |
| Phillippo, Ades, Dias, Palmer, Abrams and Welton 2018, Methods for Population-Adjusted Indirect Comparisons in Health Technology Appraisal, Medical Decision Making 38:200-211 | States that unanchored comparisons make much stronger assumptions widely regarded as infeasible, and identifies the specific discretion at issue here, that MAIC and STC give very considerable leeway to investigators to choose anchored or unanchored approaches, without proposing a criterion for the choice. |
| NICE DSU Technical Support Document 18: Methods for Population-Adjusted Indirect Comparisons in Submissions to NICE | Makes connectedness of the randomized evidence the admissibility rule for unanchored analysis; it does not say what qualifies a nominally shared comparator as non-exchangeable, which is where the rule is circumvented. |
| Bucher, Guyatt, Griffith and Walter 1997, The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials, Journal of Clinical Epidemiology 50:683-691 | The anchored indirect comparison whose validity rests on the common comparator being the same condition in both trials; the requirement is stated qualitatively and has never been operationalized as a checkable threshold. |
| Phillippo, Dias, Ades and Didelez 2018, Sensitivity of Treatment Recommendations to Bias in Network Meta-Analysis, JRSS-A 181:843-867 | Threshold analysis: computes how much bias in a piece of evidence would have to be present before the recommendation changes; this is the existing machinery for pricing the anchored-versus-unanchored election and has not been applied to it. |
| Chandler and Ishak 2026, Anchors Away: Navigating Unanchored Indirect Comparisons with Multilevel Unanchored Meta-Regression (ML-UMR), arXiv:2606.20341 | Extends unanchored comparison beyond the pairwise case and makes its assumptions explicit so they can be tested in sensitivity analysis; it improves what happens after the election is made rather than constraining the election. |
Probable solution or research direction
Write the decision as a comparison of two bias budgets rather than as a judgment: express the anchored estimator’s bias as a function of comparator drift, express the unanchored estimator’s bias as a function of residual prognostic imbalance, and require the analysis to report the drift magnitude at which the two are equal, which is a threshold analysis of a kind already built for network meta-analysis. Require both analyses to be presented, so an unanchored analysis of a connected network appears alongside the anchored one it replaced with the difference attributed, and a reviewer sees the cost of the election rather than only its result. Test the premise wherever the data allow: a comparator-arm contrast, a network meta-regression on the proposed contextual variable, or a design decomposition restricted to the comparator arms each give an observable implication of the claim that the anchor is not shared, and reporting the result of that check should be a precondition rather than an optional extra. Prespecify the criterion, since the choice is currently made after both answers are visible. Where the premise cannot be checked at all, the disconnection should be presented as a sensitivity analysis rather than as the base case.
Verification
Verdict. Confirmed open. One reader proposed this from a single applied comparison, and the reviewer accepted it as a real design-election problem not settled by the ordinary rule that randomized connectedness permits an anchored analysis, while declining the broader empirical claim that unanchored analyses generally give larger and more precise effects. The evidence for the gap is one worked case plus the long-standing observation that the choice is left to investigator discretion; no survey has measured how often connected networks are analyzed unanchored or what it changes.
Source
Derived from reading:core-02 of the reviewed corpus.