No criterion says when a network supports surrogate-based prediction
Statement
The strength of a surrogate relationship is contrast-specific and class-specific rather than a property of a whole network, so validating a surrogate across all trials does not establish it for the contrast a decision rests on; in metastatic renal cell cancer the pooled analysis of all trials failed to produce a surrogate threshold effect at all, while the targeted-therapy subset gave a threshold of 3.9 months with R-squared 0.44. The estimator built for this, bivariate network meta-analysis, separates study-level surrogacy within each treatment contrast from treatment-level surrogacy across treatments, and its stated data requirement is a relatively large number of trials per contrast for the first and a relatively large number of treatments for the second. Real networks do not supply that: its own motivating advanced colorectal cancer network drew 15, 24, 4 and 4 trials on its four best-represented contrasts and a single trial on each of the remaining three, and model-comparison statistics could be reported only for contrasts with at least four trials. Where the treatment-level model is used for the case that motivates surrogacy, predicting the effect of a new treatment on a final outcome it has never been measured on, the predictions carry large uncertainty and were found in larger networks to be comparable to or wider than those from an ordinary bivariate meta-analysis that ignores the network entirely, with the properties and assumptions of those models left to future work. Applied networks meanwhile consume surrogate predictions as if they were data: a network meta-analysis of 49 studies from 45 randomized trials and 12 regimens in metastatic colorectal cancer predicted the overall-survival effect for 22 comparisons from tumor response and progression-free survival where survival was unreported, thereby adding three direct comparisons with no survival trial behind them, then ran node-splitting and ranked regimens by SUCRA on the augmented network, from a surrogate relationship whose posterior R-squared was 0.54 with 95 percent credible interval 0.25 to 0.76. What does not exist is a criterion with known operating characteristics: no rule states how many trials per contrast, how many treatments, or what between-study correlation makes contrast-level or treatment-level surrogacy estimable well enough to license prediction; no contrast-specific analog of the surrogate threshold effect has been defined for a network; and no convention requires a network to label which of its contrasts are surrogate-derived or to say whether its inconsistency assessment was computed on measured edges only.
Why it is open
The quantity a criterion would govern is a between-trial slope and correlation for a single contrast, estimated in practice from one or two trials on that contrast, so within the contrast there is nothing to estimate it from; the only routes are to pool contrasts, which assumes away the contrast-specificity that motivated the model, or to borrow across treatment classes, which imports the same assumption in softer form. Borrowing has been built, with full and partial exchangeability of surrogate relationships across classes, but the amount borrowed is governed by a hierarchical variance the data barely inform and by an analyst’s choice of which classes are exchangeable, so the answer to whether the surrogate holds for this contrast is largely supplied by that choice. Validation by held-out prediction is available only where the final outcome was measured, which is exactly where prediction is not needed; the case that matters, a contrast or treatment with no final-outcome trial, admits no in-sample check, and this is the same structural obstacle that makes cross-outcome borrowing unfalsifiable. The empirical record shows the relationship is not stable across mechanisms of action, so a criterion calibrated on pooled data would not transfer to the contrast it is applied to. Finally the reporting machinery that has been built for surrogate endpoints, the CONSORT extension and its definitional framework, addresses trials, so nothing obliges an evidence synthesis to disclose which of its contrasts were predicted rather than measured, and the incentive to solve the problem is correspondingly weak.
What has been tried
| Work | What it contributes |
|---|---|
| Bujkiewicz, Jackson, Thompson, Turner, Stadler, Abrams and White 2019, Statistics in Medicine 38:3322-3341 | Introduces bivariate network meta-analysis for surrogate endpoint evaluation, separating study-level surrogacy within a treatment contrast from treatment-level surrogacy across treatments, and states in its discussion that estimating the former needs a relatively large number of trials per contrast and the latter a relatively large number of treatments; reports that new-treatment predictions from the treatment-level models carry large uncertainty and in larger networks are comparable to or more uncertain than those from bivariate meta-analysis, and calls for research on the properties and assumptions of those models. |
| Papanikos, Thompson, Abrams, Stadler, Ciani, Taylor and Bujkiewicz 2020, Statistics in Medicine 39:1103-1124 | Attacks the sparsity directly with two Bayesian hierarchical models that borrow information about the surrogate relationship across treatment classes, one assuming full exchangeability of surrogate relationships and one partial exchangeability to limit borrowing from distinctly different classes, assessed in nine simulated data scenarios against class-by-class subgroup analysis; supplies precision but not a criterion for when the borrowed relationship licenses prediction for a given contrast. |
| Wheaton, Papanikos, Thomas and Bujkiewicz 2023, Medical Decision Making 43:539-552 | Addresses the same sparsity from the data side by supplementing 7 randomized trials with 4 comparative and 2 matched single-arm real-world studies when evaluating progression-free survival as a surrogate for overall survival in metastatic colorectal cancer, reducing uncertainty around the surrogacy parameters and improving predicted treatment effects, at the cost of a selection-of-evidence judgment flagged in the report itself as a bias risk. |
| Hoang and Kim 2020, Cancers 12:2663 | Shows the practice the missing criterion would govern: predicts overall-survival hazard ratios for 22 comparisons from tumor response and progression-free survival using multivariate random-effects surrogacy models, inserts them into a 12-regimen network where they create three new direct comparisons, and then reports node-splitting consistency tests, heterogeneity and SUCRA rankings on the augmented network, from a progression-free survival to overall survival relationship with posterior R-squared 0.54 (95 percent credible interval 0.25 to 0.76). |
| Poad, Khan, Wheaton, Thomas, Sweeting and Bujkiewicz 2022, Cancers 14:5391 | Measures how far surrogacy varies by mechanism of action, evaluating progression-free survival and tumor response as surrogates across 66 randomized trials in metastatic colorectal cancer and finding the tumor response relationships weak overall but stronger within the epidermal growth factor receptor inhibitor plus chemotherapy class, with little evidence that KRAS status matters. |
| Johnson, Liauw and Lassere 2015, Annals of Oncology 26:485-496 | Computes the surrogate threshold effect by drug class in metastatic renal cell cancer across 30 qualifying trials, finding that the all-trials and immunotherapy analyses failed to demonstrate a threshold at all while the targeted-therapy trials gave 3.9 months with R-squared 0.44, and concluding that surrogacy is not generalizable across drug classes. |
| Rotolo, Paoletti and Michiels 2018, Computer Methods and Programs in Biomedicine 155:189-198 | Implements the classical two-step copula method and one-step auxiliary Poisson models for failure-time surrogacy in the surrosurv package, including the option to adjust the second-step regression for measurement error in the estimated treatment effects, so the measurement-error correction at the pairwise level is available software rather than a missing method; supplies no network structure. |
| Bujkiewicz, Singh, Wheaton, Jenkins, Martina, Hyrich and Abrams 2022, Journal of Clinical Epidemiology 150:171-178 | Reuses the bivariate network machinery as an identification device rather than a validation device, bridging disconnected first-line and second-line rheumatoid arthritis networks from 21 trials and 6 emulated target trials so that second-line effects can be predicted where no second-line trial exists; the same construction as surrogate-based prediction, and the same absence of a rule for when the borrowed relationship is trustworthy. |
| Manyara et al. 2024, BMJ 386:e078524 | Gives the CONSORT-Surrogate reporting extension for randomized trials whose primary outcome is a surrogate, nine modified items and two new ones; it is the reporting standard the synthesis setting lacks, since nothing analogous requires a network meta-analysis to report which contrasts were surrogate-predicted or on what surrogacy evidence. |
| Wheaton and Bujkiewicz 2025, International Journal of Technology Assessment in Health Care 41:e11 | Measures the decision exposure: of 47 NICE oncology technology appraisals published between February 2022 and May 2023, 18 used surrogate endpoints and 37 separate surrogate endpoints were discussed, of which 11 were supported by randomized trial evidence, 7 by observational studies, 12 by clinical opinion and 7 by no evidence at all. |
Probable solution or research direction
Turn the stated data requirement into a checkable precondition: derive, for the bivariate network model, the replication and correlation levels at which the contrast-level and treatment-level slopes are estimated with usable precision, and run it before any prediction is issued, in the way contrast estimability is checked before a contrast is reported. Establish operating characteristics by simulating over network topologies rather than single scenarios, varying trials per contrast, number of treatments, and the strength and heterogeneity of the surrogate relationship, and scoring bias, coverage and interval width of predicted final-outcome effects against both the ordinary bivariate model and against issuing no prediction, since the existing evidence is that the network model can be the worse of the two for a new treatment. Define a contrast-specific surrogate threshold effect, the effect on the surrogate needed on that contrast to predict a nonnull effect on the final outcome, and report it per contrast rather than for the pooled network. Validate by deletion where the final outcome was measured: mask final-outcome data one contrast at a time, refit, and report prediction error and coverage indexed by trials per contrast and by whether the masked contrast shares a class with observed ones. Separately, require any network carrying surrogate-predicted contrasts to label them, to report the network with and without them, and to state whether inconsistency was assessed on measured edges only, since an edge that is a deterministic function of other edges cannot supply an independent consistency check.
Verification
Verdict. Partially addressed. This rests on weaker provenance than its neighbors. It was not proposed by a reader arguing from a paper; it was reported by 2 of 12 gap labellers as something they looked for and did not find, and the three papers behind it carry unequal weight. One is a pairwise individual-patient-data surrogacy analysis in esophageal cancer whose noted absence, copula estimates unadjusted for measurement error, is an option the package it used already offers, so that strand supports nothing. One is an applied colorectal cancer network whose quoted sentence asks for further individual-patient-data validation, which is a call for more studies; what supports the entry there is what that analysis did rather than what it said, namely feeding 22 predicted survival effects and three manufactured direct comparisons into a ranked network. Only the bivariate network meta-analysis paper states a methodological limit, and it states it in two sentences of its discussion together with a call for research on the properties of its own treatment-level models. The registry was searched for surrogate, surrogacy, bivariate, multivariate, correlated-outcome, trial-level and intermediate-endpoint content and contains no entry on surrogate endpoints; the nearest neighbors, DIS-17 on cross-outcome borrowing as a bridge, OUT-19 on multivariate population adjustment and HET-09 on multivariate inconsistency covariance, concern borrowing between simultaneously measured outcomes rather than the predictive validity of a surrogate for a final outcome, and DEC-08 concerns provenance labeling for connected and disconnected networks rather than surrogate validity. It is filed as partially-addressed because hierarchical borrowing across treatment classes and augmentation with real-world evidence both attack the sparsity that makes contrast-level surrogacy unestimable; neither supplies the criterion with operating characteristics, the contrast-specific threshold, or the labeling requirement, and no work located since 2019 extends the bivariate network model for surrogacy or reports its operating characteristics.
Source
Derived from reading:uncovered of the reviewed corpus.