Non-collapsible marginal effects are not portable across populations
Statement
Odds ratios and hazard ratios can vary across populations even without confounding, so the shared effect modifier assumption alone does not make a marginal effect portable to an arbitrary population. The result is not new and does not belong to the citation the source gave it: Remiro-Azocar (doi:10.1002/sim.10111) established the stronger result that under non-collapsibility even purely prognostic variables shift the marginal effect, Phillippo et al. 2021 (doi:10.1002/sim.8965) had already noted that marginal effects are less transportable than population-average conditional ones, and Chandler and Ishak (arxiv:2602.17041) restate it inside an explicit two-step transportability framing for PAIC. Collapsibility is also not the same as transportability: a risk difference or an RMST difference marginalizes cleanly but still varies across populations when effects are heterogeneous, and a standardized risk is a target-specific absolute quantity rather than a treatment-effect scale.
Why it is open
Non-collapsibility is a property of the effect measure rather than a bias, so it cannot be removed by better adjustment, better models or larger samples. An assumption about effect modification of the conditional contrast does not license moving the marginal contrast, because the marginal effect depends on the covariate distribution it is averaged over, and under non-collapsibility even purely prognostic variables move it. What remains open is propagation rather than theory: the formal results are published and target-specific standardization is implemented, but reporting conventions and HTA submission scales still carry marginal odds ratios and hazard ratios without a population index. Because the same number is valid in its own population and invalid in the target, no diagnostic run on the source analysis reveals the error.
What has been tried
| Work | What it contributes |
|---|---|
| Remiro-Azocar 2024, Transportability of model-based estimands in evidence synthesis, Statistics in Medicine 43(22):4217-4249 | Establishes that for non-collapsible measures purely prognostic variables that are not determinants of treatment response can modify marginal effects even under individual-level treatment effect homogeneity, which is why an assumption about effect modifiers cannot suffice for portability. This is the work that establishes the entry’s central result. |
| Phillippo, Dias, Ades and Welton 2021, Target estimands for efficient decision making: Response to comments on Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study, Statistics in Medicine | States that marginal population-average treatment effects depend more strongly on the population of interest than population-average conditional effects and are less generalizable as a result, foreshadowing the formal result by three years. |
| Chandler and Ishak 2026, Reframing Population-Adjusted Indirect Comparisons as a Transportability Problem | Restates that the shared effect modifier assumption alone does not guarantee direct transportability of marginal effects, inside an explicit two-step transportability framing for PAIC. It restates rather than establishes the result. |
| Jiang et al. 2025, A critical assessment of matching-adjusted indirect comparisons in relation to target populations, Research Synthesis Methods | Documents the population dependence of MAIC results in relation to target populations. |
| multinma: Bayesian Network Meta-Analysis of Individual and Aggregate Data (R package) | Implements target-population marginal-effect calculation, with marginal_effects() defaulting to a risk difference, so the standardization step is available and the collapsible default is already in software. |
| cpaic, unreleased development package on GitHub (not on CRAN), commit 9d150e9 | Distinguishes a conditional profile contrast from a marginal population-standardized effect in its documentation, stating that the returned value is the conditional contrast at the covariate profile rather than the marginal effect standardized over a population. The labeling is correct; the marginal path is absent. |
Probable solution or research direction
Index every marginal estimand by the population it is defined in and standardize explicitly to a named target covariate law before any decision use, rather than assuming scale invariance. Directly collapsible contrasts make that standardization easier, so prefer them where the decision permits, but do not present them as empirically constant across populations, since the target still has to be declared. The concrete work is in outputs rather than derivations: get the population index into reporting templates and into packages that do not yet separate a conditional profile contrast from a population-standardized marginal effect, as in cpaic, an unreleased development package on GitHub that is not installable from CRAN, where at commit 9d150e9 the cML-NMR path returns only the conditional contrast at a covariate profile.
Verification
Verdict. Partially addressed. Part of this problem is addressed by existing work and the entry names which part. The source additionally attributed the claim to the wrong work, which is corrected in the entry.
The source said. “Odds ratios and hazard ratios can vary across populations even in the absence of confounding, so a marginal effect estimated in one population is not automatically transportable to another. The shared-effect-modifier assumption, routinely invoked to justify population adjustment, does not by itself make non-collapsible marginal effects portable to an arbitrary population.”
What was wrong with it. The source credited Chandler and Ishak (arxiv:2602.17041) with establishing that shared effect modification does not make non-collapsible marginal effects portable; the literature auditor found the result established two years earlier by Remiro-Azocar (doi:10.1002/sim.10111) and foreshadowed by Phillippo et al. 2021 (doi:10.1002/sim.8965), and found the claim that it has not reached software defaults false, since multinma’s marginal_effects() defaults to a risk difference. GPT-5.6 Sol separately struck the equation of collapsibility with portability and the treatment of a standardized risk as an effect scale.
Dissent. literature dissented (open): Chandler & Ishak 2026 does say what is attributed (‘SEMA alone does not guarantee direct transportability of marginal treatment effects’; non-collapsible marginal effects ‘are not directly transportable, even when SEMA holds’), so claims 1-3, 5 and 6 hold. Claim 4 does not: Chandler & Ishak are not the work that establishes this. Remiro-Azocar, Statistics in Medicine 43(22):4217-4249 (2024) establishes a strictly stronger result two years earlier - for non-collapsible measures even purely prognostic variables that are not determinants of treatment response modify marginal effects - which is exactly why an assumption about effect modifiers cannot suffice. Earlier still, Phillippo, Dias, Ades & Welton (Stat Med 2021) wrote that marginal effects ‘depend more strongly on the population of interest than the population-average conditional treatment effects, and are less generalizable/transportable as a result’. The why_open assertion that the result ‘has not propagated into guidance, software defaults’ is also wrong at the software level: multinma’s marginal_effects() defaults to mtype = ‘difference’, a collapsible absolute scale. Status stays open because these are formalizations of an unfixable property. codex dissented (overstated): C1-C3 are mathematically correct: a marginal odds ratio or hazard-ratio projection can change with baseline risk, prognostic covariate distribution, or follow-up composition despite no confounding and a stable conditional treatment parameter. C4 is a priority-attribution claim outside this assigned technical lens; the result is an established consequence of non-collapsibility, not something that can be credited from technical truth alone. C5 is sloppy: risk differences and RMST differences are collapsible linear contrasts, but collapsibility neither guarantees transportability under effect modification nor makes them universally preferable; a standardized risk is target-specific and is not itself a treatment-effect scale. C6 should require a named target population, and numerical standardization needs a target covariate or outcome law, but the abstract marginal estimand need not encode one particular covariate representation. Target-specific standardization is established and implemented, so the management problem is partly solved; no population-free marginal OR or HR exists.
Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).
- closure-upheld
- 1 findings from the reading assert this problem is open; the closure was re-examined and stands Put to an independent reviewer with the closing auditors’ reasoning, the work they relied on, and the new findings in view together. The new paper restates the closure’s accepted residual: marginal estimates are population-specific under non-collapsibility, including through baseline-risk distributions not captured by matching covariates. Its qualification that a shared effect-modifier assumption suffices only in restricted linear continuous-outcome settings is compatible with the closure’s distinction between stable conditional parameters and nonportable marginal projections. It does not show that target-specific standardization is unavailable or that a population-free marginal odds ratio or hazard ratio can be recovered. Thus it supports the established theory, not reversal of the overstated closure. (reviewer confidence: high)
- verdict-corrected
-
not-supported -> partially-addressed The adjudication rule let a misattributed citation set the published verdict to
not-supported, though every auditor on this entry voted the problempartially-solved. A misattribution is a fault in the provenance, not in the claim. The rule is corrected in build/adjudicate.mjs and the verdict re-derived from this entry’s own audit record. Part of this problem is addressed by existing work and the entry names which part. The source additionally attributed the claim to the wrong work, which is corrected in the entry. - Flagged for reopening
-
3 findings from the reading assert this problem is open while the registry carries
partially-addressedFlagged, not applied: reversing an earlier audit’s closure needs the closure’s own evidence re-examined alongside this, which the reading did not do. See documentation/audit/reading/PROPOSED.md. - Literature and prior-art check
- open / misattributed. Chandler & Ishak 2026 does say what is attributed (‘SEMA alone does not guarantee direct transportability of marginal treatment effects’; non-collapsible marginal effects ‘are not directly transportable, even when SEMA holds’), so claims 1-3, 5 and 6 hold. Claim 4 does not: Chandler & Ishak are not the work that establishes this. Remiro-Azocar, Statistics in Medicine 43(22):4217-4249 (2024) establishes a strictly stronger result two years earlier - for non-collapsible measures even purely prognostic variables that are not determinants of treatment response modify marginal effects - which is exactly why an assumption about effect modifiers cannot suffice. Earlier still, Phillippo, Dias, Ades & Welton (Stat Med 2021) wrote that marginal effects ‘depend more strongly on the population of interest than the population-average conditional treatment effects, and are less generalizable/transportable as a result’. The why_open assertion that the result ‘has not propagated into guidance, software defaults’ is also wrong at the software level: multinma’s marginal_effects() defaults to mtype = ‘difference’, a collapsible absolute scale. Status stays open because these are formalizations of an unfixable property.
- GPT-5.6 Sol, technical and source-code lens
- partially-solved / overstated. C1-C3 are mathematically correct: a marginal odds ratio or hazard-ratio projection can change with baseline risk, prognostic covariate distribution, or follow-up composition despite no confounding and a stable conditional treatment parameter. C4 is a priority-attribution claim outside this assigned technical lens; the result is an established consequence of non-collapsibility, not something that can be credited from technical truth alone. C5 is sloppy: risk differences and RMST differences are collapsible linear contrasts, but collapsibility neither guarantees transportability under effect modification nor makes them universally preferable; a standardized risk is target-specific and is not itself a treatment-effect scale. C6 should require a named target population, and numerical standardization needs a target covariate or outcome law, but the abstract marginal estimand need not encode one particular covariate representation. Target-specific standardization is established and implemented, so the management problem is partly solved; no population-free marginal OR or HR exists. Cites doi:10.1002/sim.10111: Formalizes the population dependence of model-based marginal estimands and the role of prognostic variables under non-collapsibility.. Cites cran:multinma: Implements target-population marginal-effect calculations, resolving the computational standardization step while not creating a portable population-free effect.. Cites choxos/cpaic@9d150e9cad88f2e6fbcd94dbb92c3dee3b7c2dca: Explicitly distinguishes a conditional profile contrast from a marginal population-standardized effect, resolving the software-labeling part in cpaic..
- Decision path
-
R3:partially-solved(n=1) → R6:counter-quote-uncontested
How the evidence falls in time
3 papers in the reviewed corpus bear on this problem, published 2025 to 2026. The most recent paper to touch this reports progress on it, and every paper calling it open is older.
Does the later work answer the earlier work? Put to an independent reviewer with the older and newer claims side by side, it judged that the later work answers a component of it, leaving a named part (medium confidence). Chandler and Ishak give the requested general transportability classification for the principal non-collapsible PAIC measures, and ML-UMR quantifies the consequence even without effect modification. Neither makes marginal odds ratios or hazard ratios portable across populations, so the practical problem survives despite the theoretical question being substantially clarified. What remains: A method for producing decision-relevant portable effects, or for handling the population dependence in practice, remains; the later work mainly characterizes and quantifies the limitation.
| Year | Says | Paper |
|---|---|---|
| 2025 | confirms it is open | Effect modification and non-collapsibility together may lead to conflicting treatment decisions: A review of marginal and conditional estimands and recommendations for decision-making |
| 2026 | partly addresses it | Reframing Population-Adjusted Indirect Comparisons as a Transportability Problem: An Estimand-Based Perspective and Implications for Health Technology Assessment (Chandler & Ishak; arXiv:2602.17041v3). The batch file points at media/media/Table_1_v2.pdf; the full LaTeX source Transportability_PAIC.tex sits in the same arXiv package and was read. |
| 2026 | partly addresses it | Anchors Away: Navigating Unanchored Indirect Comparisons with Multilevel Unanchored Meta-Regression (ML-UMR) (Chandler & Ishak; arXiv:2606.20341v1). The batch file points at media/media/Table1.pdf; the full LaTeX source mlumr.tex sits in the same arXiv package and was read. |
Source
Derived from C:L214-215, C:L221, C:L298, C:L46 of the reviewed corpus.