Target population is unnamed and set by data availability
Statement
Applied ITC and PAIC practice still defaults the target population to whichever aggregate-data trial reported enough baseline covariates, and reported effects are rarely labeled with the population they refer to. The normative half of the source’s Direction A is not new: NICE DSU TSD18 (Phillippo et al. 2018, doi:10.1177/0272989X17725740) has required since 2016 that the target population be stated explicitly and that population-adjusted effects be generated for it, and it names a cohort or registry population relevant to the decision as the appropriate external source. The machinery is not missing either: multilevel network meta-regression estimates every contrast in a network in one externally supplied target population (Phillippo et al. 2020, doi:10.1111/rssa.12579; doi:10.1002/sim.8759), and multinma implements it through relative_effects(), predict() and marginal_effects(), each of which accepts an arbitrary target via newdata. What survives is the gap between that requirement and practice, together with the substantive judgment of which target to name, which no statistical criterion settles.
Why it is open
The requirement exists but nothing enforces it at the point where the target is actually chosen, so the estimator’s target is still fixed by which trial reported enough baseline variables, and no diagnostic run on the source analysis can flag a target that was never declared. Transporting a whole network to one external population needs IPD for at least one study and a covariate law for the target, so the capability that exists in ML-NMR is unavailable in exactly the aggregate-only networks where the default is most damaging. Choosing among candidate target sources is a policy judgment no statistical criterion settles, and falling back to an overlap population restores comparability only if the whole network shares one declared overlap estimand rather than a different overlap per pair. Because a single unlabeled symbol denotes both the estimated and the required quantity, the substitution leaves no trace for a reviewer to challenge.
What has been tried
| Work | What it contributes |
|---|---|
| Phillippo, Ades, Dias, Palmer, Abrams and Welton 2018, Methods for Population-Adjusted Indirect Comparisons in Health Technology Appraisal, Medical Decision Making (NICE DSU TSD18) | Requires that the target population for any treatment comparison be stated explicitly and that population-adjusted relative effects be generated for that target, and names a cohort or registry population relevant to the clinical decision as the appropriate source, which is unlikely to match the aggregate-data trial population. It supplies the normative half of Direction A a decade before this agenda; it does not make the declaration a field anyone checks outside NICE submissions. |
| Phillippo, Dias, Elsada, Ades and Welton 2020, Multilevel network meta-regression for population-adjusted treatment comparisons, JRSS-A | Defines the one-stage individual-level regression that fits aggregate data by integration and produces population-adjusted contrasts for a specified target population across a whole network, which is the machinery the entry described as nonexistent. It requires IPD from at least one study. |
| Phillippo, Dias, Ades, Belger, Brnabic, Schacht, Saure, Kadziola and Welton 2020, Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study, Statistics in Medicine | Simulation evidence that ML-NMR extends to larger treatment networks and produces estimates in any target population, unlike pairwise MAIC and STC. |
| multinma: Bayesian Network Meta-Analysis of Individual and Aggregate Data (R package) | Implements ML-NMR, with relative_effects(), predict() and marginal_effects() all accepting newdata describing an arbitrary target population’s covariate distribution, so estimating every network contrast in one declared target is available software rather than a proposal. |
Probable solution or research direction
Do not rebuild the declaration rule or the network estimator, since both exist. The work is to make the declared target binding: carry the TSD18 requirement into journal and submission templates, label every reported contrast with its population, and treat target-specific prediction from a network model as the default output rather than an option. Where the full decision target lacks covariate support, change the estimand to one common overlap population for the whole network, and use several targets as sensitivity analysis rather than in place of a primary decision target. The measurable question is whether an enforced declaration changes what appears in submissions, not whether the method exists.
Verification
Verdict. Overstated. 2 independent auditors judged the claim as stated too strong. Although NICE DSU TSD18 has required since 2016 that the target population be explicitly stated, and although ML-NMR/multinma can estimate every network contrast in one externally supplied target population for anchored comparisons, which is not yet a general solution for unanchored comparisons or for all outcome types, applied ITC and PAIC practice still defaults the target to whichever aggregate-data trial reported enough covariates, and reported effects are rarely labelled with the population they refer to.
The source said. “Population-adjusted analyses adjust to whichever aggregate-data trial population happens to be reportable, while the decision concerns a different population, and the reported effect carries no label identifying the population it refers to. The report’s Direction A instead requires every relevant contrast to be estimated in one externally defined target population P*, chosen from a named source such as a jurisdictional registry, the eligible treatment population, a pooled trial-eligible population, a standardized reference population, or an overlap population where the full target is unsupported.”
What was wrong with it. The source treated both explicit target declaration and one-target network estimation as absent; the literature auditor found both already in place, in Recommendation 6 of NICE DSU TSD18 (doi:10.1177/0272989X17725740) and in ML-NMR as implemented in multinma (https://cran.r-project.org/package=multinma). GPT-5.6 Sol added that the target need not come from an external data source, that an overlap target is acceptable only as one common network estimand rather than pair by pair, and that reporting several targets is sensitivity analysis rather than a substitute for naming a primary decision target.
Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).
- scope-narrowed
- verdict_rationale: the closure stands, but its boundary was drawn in the wrong place Unanchored-method papers from 2024 and 2025 state that ML-NMR was then applicable to anchored comparisons only, and a 2025 quantitative-bias-analysis paper lists estimation in an arbitrary target population as a remaining need for unanchored PAICs. The closure stands; its counterexample was stated more broadly than the machinery reaches. Replacement wording composed from the reviewer’s description; reviewer confidence high. Previously: “2 independent auditors judged the claim as stated too strong. Although NICE DSU TSD18 has required since 2016 that the target population be explicitly stated, and although ML-NMR/multinma can estimate every network contrast in one externally supplied target population, applied ITC and PAIC practice still defaults the target to whichever aggregate-data trial reported enough covariates, and reported effects are rarely labelled with the population they refer to.”
- Flagged for reopening
-
19 findings from the reading assert this problem is open while the registry carries
overstatedFlagged, not applied: reversing an earlier audit’s closure needs the closure’s own evidence re-examined alongside this, which the reading did not do. See documentation/audit/reading/PROPOSED.md. - Literature and prior-art check
- partially-solved / overstated. I read the full text of Phillippo et al. 2018 Med Decis Making (the peer-reviewed NICE DSU TSD18). Its Recommendation 6 reads: ‘The target population for any treatment comparison must be explicitly stated, and population-adjusted estimates of the relative treatment effects must be generated for this target population.’ It also already names the external source the entry asks for: ‘the target population should be a cohort or registry study population relevant to the clinical decision, which is unlikely to match the population of the [AgD] trial.’ The practice half of the entry survives (Phillippo 2019 NICE review; Serret-Larmande 2023 found 3/133 PAIC articles reported all key methodological aspects). What does not survive is ‘there is no standard machinery for … transporting every contrast in a network to it at once’: ML-NMR does exactly that, and multinma implements relative_effects(), predict() and marginal_effects() with a newdata target population. The claim that the report ‘names exactly five candidate sources’ is self-referential and not externally checkable. Cites doi:10.1177/0272989X17725740: Already mandates explicit target-population declaration for NICE submissions and names registry/cohort populations as the appropriate external source, i.e. the normative half of Direction A predates this agenda by a decade.. Cites doi:10.1002/sim.8759: Establishes ML-NMR as machinery that transports all contrasts of a network into a single declared target population.. Cites cran:multinma: Software implementation: relative_effects(), predict() and marginal_effects() all accept newdata describing an arbitrary target population’s covariate distribution..
- GPT-5.6 Sol, technical and source-code lens
- partially-solved / overstated. C1 describes conventional pairwise anchored MAIC/STC, not ITC or PAIC generally; C2 and C6 are prevalence or adoption claims that this technical audit cannot establish. C3 is a second transport only when a population-indexed effect is reused in another population; a genuinely invariant conditional parameter is different. A coherent multi-treatment decision normally needs a common decision target, but C4 is too absolute: the target need not come from an external data source, and some decisions legitimately require several populations. C5’s count of five is textually true but methodologically vacuous. C7 is acceptable only as an explicit change to a common network overlap estimand; pair-specific overlap targets destroy contrast comparability. C8 is useful sensitivity analysis, not a substitute for choosing a primary decision target. Target declaration and common-target network standardization already have methodological and software machinery, so the technical problem is partly solved; data support and disciplined use remain. Cites doi:10.1177/0272989X17725740: Already requires an explicit target population and effects generated for that target, resolving the claim that no target-declaration machinery exists.. Cites doi:10.1111/rssa.12579: Provides one-stage network machinery for population-adjusted contrasts in a specified target population.. Cites cran:multinma: Implements target-specific prediction and marginal-effect calculations for a network, so the capability is not absent..
- Decision path
-
R3:partially-solved(n=2) → R8:overstated-majority(n=2)
How the evidence falls in time
19 papers in the reviewed corpus bear on this problem, published 2014 to 2026. Partial progress is on the record, and later work still calls this open. Both belong in view.
This problem is recurrent: three or more papers spanning at least eight years assert it independently. Sustained restatement by authors who mostly do not cite each other is the strongest evidence this reading can offer that a gap is real rather than one group’s framing.
It is also being closed in pieces: partial results come from more than one paper in more than one year, each covering a different part.
| Year | Says | Paper |
|---|---|---|
| 2014 | confirms it is open | The importance of baseline viral load when assessing relative efficacy in treatment-naive HBeAg-positive chronic hepatitis B: a systematic review and network meta-analysis |
| 2015 | partly addresses it | Accounting for Heterogeneity in Relative Treatment Effects for Use in Cost-Effectiveness Models and Value-of-Information Analyses |
| 2017 | confirms it is open | Methods for Population-Adjusted Indirect Comparisons in Health Technology Appraisal |
| 2018 | confirms it is open | Comparing pharmacological treatments for cocaine dependence: Incorporation of methods for enhancing generalizability in meta-analytic studies |
| 2019 | confirms it is open | Population Adjustment Methods for Indirect Comparisons: A Review of National Institute for Health and Care Excellence Technology Appraisals |
| 2020 | partly addresses it | Extending inferences from a randomized trial to a new target population (Dahabreh, Robertson, Steingrimsson, Stuart, Hernan; tutorial, arXiv:1805.00550v3) |
| 2022 | partly addresses it | Validating the Assumptions of Population Adjustment: Application of Multilevel Network Meta-regression to a Network of Treatments for Plaque Psoriasis |
| 2022 | confirms it is open | Parametric G-computation for compatible indirect treatment comparisons with limited individual patient data |
| 2022 | confirms it is open | Calibrating a network meta-analysis of diabetes trials of sodium glucose cotransporter 2 inhibitors, glucagon-like peptide-1 receptor analogues and dipeptidyl peptidase-4 inhibitors to a representative routine population: a systematic review protocol |
| 2023 | confirms it is open | A Critical Appraisal of Matching-Adjusted Indirect Comparisons in Spinal Muscular Atrophy |
| 2023 | confirms it is open | The Acceptance of Indirect Treatment Comparison Methods in Oncology by Health Technology Assessment Agencies in England, France, Germany, Italy, and Spain |
| 2024 | confirms it is open | Advancing unanchored simulated treatment comparisons: A novel implementation and simulation study |
| 2024 | confirms it is open | Methods for Indirect Treatment Comparison: Results from a Systematic Literature Review |
| 2025 | confirms it is open | Quantitative bias analysis for unmeasured confounding in unanchored population-adjusted indirect comparisons |
| 2025 | confirms it is open | Unanchored simulated treatment comparison on survival outcomes using parametric and Royston-Parmar models with application to lenvatinib plus pembrolizumab in renal cell carcinoma |
| 2025 | confirms it is open | A critical assessment of matching-adjusted indirect comparisons in relation to target populations |
| 2026 | confirms it is open | BEhavioural Weight Management: COMponents of Effectiveness (BE:COME) Synopsis |
| 2026 | confirms it is open | Evidence Synthesis via Indirect Treatment Comparisons in the European Framework of Joint Clinical Assessment |
| 2026 | confirms it is open | Arbitrated Indirect Treatment Comparisons |
Source
Derived from C:L214-215, C:L219, C:L312-324 of the reviewed corpus.