No multivariate population-adjusted model for correlated outcomes
Statement
Multivariate network meta-analysis of correlated efficacy and safety outcomes is established rather than promising methodology, covering borrowing of information across outcomes (PMC4142066), joint synthesis of multiple correlated outcomes in networks of interventions (PMC4481542), multivariate and network meta-analysis of multiple outcomes and treatments (PMC5596393) and applied Bayesian multivariate network meta-analysis (doi:10.1002/sim.8668). What has not been developed is the population-adjusted counterpart: no multivariate ML-NMR and no copula outcome model composed with transport. Running endpoints separately discards covariance only where the endpoint-specific fits are treated as independent, since a joint bootstrap, paired study-level estimates or shared posterior draws can retain it, and incoherence is not automatic either, because a decision based on expected net monetary benefit is coherent from marginal means while correlation affects its variance, the probability of cost-effectiveness, nonlinear utilities and patient-level joint quantities. The observation that different endpoints may have different effect modifiers, so separate population-adjusted analyses can standardize to subtly different populations, appears to be original to this entry.
Why it is open
The generic joint-synthesis problem is addressed while the population-adjusted version is not, so independent endpoint-specific PAICs can estimate target marginal means but do not identify within-patient outcome dependence and can misstate joint decision uncertainty wherever covariance is discarded. Within-patient correlation is not recoverable from separately reported marginal summaries, which is all aggregate publications provide, although between-study dependence may be estimable when the same studies report paired endpoints. Population adjustment compounds the problem because each outcome may have a different effect-modifier set, so separate analyses can be standardized to subtly different target populations without that being visible in the output. In cpaic, the unreleased development package on GitHub that is not installable from CRAN, one outcome family is fitted per cmlnmr() call and its Gaussian copula models the target covariate distribution rather than cross-outcome dependence.
What has been tried
| Work | What it contributes |
|---|---|
| Achana et al., Network meta-analysis of multiple outcome measures accounting for borrowing of information across outcomes (PMC4142066) | network meta-analysis across multiple outcome measures that borrows information between outcomes, part of the established multivariate synthesis literature the entry treated as merely promising |
| Efthimiou et al. 2015, Joint synthesis of multiple correlated outcomes in networks of interventions, Biostatistics (PMC4481542) | establishes multivariate network meta-analysis for jointly synthesizing multiple correlated outcomes across networks of treatments, including estimation of between-outcome correlation |
| Riley et al., Multivariate and network meta-analysis of multiple outcomes and multiple treatments, BMJ (PMC5596393) | practical account of multivariate and network meta-analysis across multiple outcomes and treatments, showing the approach is standard rather than emerging |
| Waddingham et al. 2020, Exploiting relationships between outcomes in Bayesian multivariate network meta-analysis with an application to relapsing-remitting multiple sclerosis, Statistics in Medicine | implemented Bayesian multivariate network meta-analysis exploiting cross-outcome relationships, resolving the generic claim that efficacy and harms must be synthesized independently; it supplies no population adjustment and no multivariate ML-NMR |
Probable solution or research direction
Build the population-adjusted counterpart of multivariate synthesis, namely target-standardized multivariate ML-NMR, rather than multivariate network meta-analysis itself, and state where the dependence inputs come from, since marginal aggregate data do not identify them. Where a full joint model is unavailable, preserve covariance by carrying joint posterior draws or a joint bootstrap through to the decision quantity instead of recombining independent marginal summaries. Make the effect-modifier set for each endpoint explicit so that separate analyses cannot silently standardize to different populations.
Verification
Verdict. Overstated. 2 independent auditors judged the claim as stated too strong. Multivariate network meta-analysis of correlated efficacy and safety outcomes is established methodology with a scoping review of its applied use; what has not been developed is its population-adjusted counterpart, so PAICs in practice are still run one endpoint at a time and can standardise different endpoints to subtly different populations when the effect-modifier sets differ.
The source titled this. “Efficacy and harms synthesized separately, ignoring correlation” The title says separate synthesis ignores correlation, but the audit found separate marginal models can retain covariance and that multivariate network meta-analysis is established.
The source said. “Reimbursement decisions depend on correlated efficacy, safety, quality of life, and cost outcomes, but population-adjusted comparisons are estimated one endpoint at a time in separate network meta-analyses, discarding the correlation structure and permitting internally incoherent decisions.”
What was wrong with it. The source rated multivariate network meta-analysis as promising and treated separate analyses as necessarily discarding correlation and producing incoherent decisions; the literature auditor documented a mature multivariate NMA literature (PMC4142066, PMC4481542, PMC5596393, doi:10.1002/sim.8668) plus a scoping review of its applied use, and the code auditor showed that separate marginal models can retain covariance and that expected net monetary benefit is linear in the marginal means. The same auditor narrowed the copula proposal, since copula association parameters are not identified from marginal aggregate data and require individual patient data, covariance summaries, assumptions or sensitivity analysis.
Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).
- closure-upheld
- 11 findings from the reading assert this problem is open; the closure was re-examined and stands Put to an independent reviewer with the closing auditors’ reasoning, the work they relied on, and the new findings in view together. None of the findings contradicts the mature multivariate-NMA literature on which closure relied. Several findings document non-adoption, unavailable within-study correlations, or practical limitations of aggregate-data multivariate models, all of which the closure already concedes. The new 2025 paper is aggregate-data multivariate synthesis without covariates or transport, so it reinforces rather than fills the multivariate population-adjustment gap. No finding supplies multivariate ML-NMR, a transport-composed outcome-dependence model, or evidence that the cited methods were mischaracterized. (reviewer confidence: high)
- Flagged for reopening
-
12 findings from the reading assert this problem is open while the registry carries
overstatedFlagged, not applied: reversing an earlier audit’s closure needs the closure’s own evidence re-examined alongside this, which the reading did not do. See documentation/audit/reading/PROPOSED.md. - Literature and prior-art check
- partially-solved / overstated. The claim that multivariate NMA is ‘promising’ for jointly modelling correlated decision outcomes understates a mature literature. I retrieved Achana et al., ‘Network meta-analysis of multiple outcome measures accounting for borrowing of information across outcomes’ (PMC4142066), Efthimiou et al., ‘Joint synthesis of multiple correlated outcomes in networks of interventions’ (Biostatistics, PMC4481542), Riley et al., ‘Multivariate and network meta-analysis of multiple outcomes and multiple treatments’ (BMJ, PMC5596393), Waddingham et al. (Stat Med 2020, doi 10.1002/sim.8668) applying Bayesian multivariate NMA to relapsing-remitting MS, and a 2023 J Clin Epidemiol scoping review of the conduct and reporting of multivariate NMAs, which by its existence establishes that these are routinely conducted. Multivariate NMA handling continuous, binary, time-to-event or mixed outcomes with or without within-study correlations is established methodology. What is genuinely open is the population-adjusted version: I found no multivariate ML-NMR and no copula outcome model composed with transport, and the point that different endpoints may have different effect modifiers so separate analyses standardise to subtly different populations appears to be original to this entry. Cites https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4481542/: Establishes multivariate NMA for jointly synthesising multiple correlated outcomes across networks of treatments, including estimation of between-outcome correlation.. Cites doi:10.1002/sim.8668: Applied Bayesian multivariate NMA exploiting cross-outcome relationships, showing the approach is in routine methodological use rather than merely promising..
- GPT-5.6 Sol, technical and source-code lens
- partially-solved / overstated. The generic joint-synthesis problem is partially solved by multivariate NMA; the population-adjusted multivariate extension remains open. C2 is true only when endpoint-specific fits are treated as independent. Separate marginal models can retain covariance through a joint bootstrap, paired study-level estimates, or shared posterior draws. C3 is therefore not automatic: a decision based on expected net monetary benefit can be coherent from marginal means because E(lambda Q minus C) equals lambda E(Q) minus E(C). Correlation affects its variance, probability of cost-effectiveness, nonlinear utilities, and patient-level joint quantities. C4 is correct for within-patient dependence from marginal summaries alone, but too broad for between-study dependence, which may be estimated when the same studies report paired endpoints. Copulas are a possible dependence model, but their association parameters are not identified from marginal AgD and require IPD, covariance summaries, assumptions, or sensitivity analysis. In cpaic@9d150e9, one outcome family is fitted per cmlnmr() call, and its Gaussian copula models the target covariate distribution, not cross-outcome dependence. Cites doi:10.1002/sim.8668: Demonstrates an implemented Bayesian multivariate NMA that jointly exploits relationships between outcomes, resolving the generic claim that efficacy and harms must be synthesized independently. It does not supply population adjustment or multivariate ML-NMR..
- Decision path
-
R3:partially-solved(n=2) → R8:overstated-majority(n=2)
How the evidence falls in time
11 papers in the reviewed corpus bear on this problem, published 2014 to 2025. Partial progress is on the record, and later work still calls this open. Both belong in view.
This problem is recurrent: three or more papers spanning at least eight years assert it independently. Sustained restatement by authors who mostly do not cite each other is the strongest evidence this reading can offer that a gap is real rather than one group’s framing.
| Year | Says | Paper |
|---|---|---|
| 2014 | partly addresses it | Joint synthesis of multiple correlated outcomes in networks of interventions |
| 2014 | confirms it is open | Evidence-based prescribing: combining network meta-analysis with multicriteria decision analysis to choose among multiple drugs |
| 2014 | confirms it is open | Joint synthesis of multiple correlated outcomes in networks of interventions |
| 2016 | confirms it is open | Living cumulative network meta-analysis to reduce waste in research: A paradigmatic shift for systematic reviews? |
| 2016 | confirms it is open | The rank-heat plot is a novel way to present the results from a network meta-analysis including multiple outcomes |
| 2017 | confirms it is open | A matrix-based method of moments for fitting multivariate network meta-analysis models with multiple outcomes and random inconsistency effects |
| 2018 | confirms it is open | Different Methods for Modelling Severe Hypoglycaemic Events: Implications for Effectiveness, Costs and Health Utilities |
| 2018 | confirms it is open | Generating the evidence for risk reduction: a contribution to the future of food-based dietary guidelines |
| 2019 | confirms it is open | Combining Multiple Treatment Comparisons with Personalized Patient Preferences: A Randomized Trial of an Interactive Platform for Statin Treatment Selection |
| 2019 | confirms it is open | Joint Estimation of Remission and Response for Methotrexate-Based DMARD Options in Rheumatoid Arthritis: A Bivariate Network Meta-Analysis |
| 2020 | confirms it is open | Uptake of methodological advances for synthesis of continuous and time-to-event outcomes would maximize use of the evidence base |
| 2025 | confirms it is open | Exploiting Multivariate Network Meta-Analysis: A Calibrated Bayesian Composite Likelihood Inference |
Source
Derived from C:L214-215, C:L247, C:L1620-1628 of the reviewed corpus.