Decision QBA: net benefit, tipping surfaces and extrapolation
No. The probability of negative net benefit was 0.38 under point propagation of the reported hazard ratio and 0.67 when the bias (log hazard ratio within 0.15) and the treatment-effect extrapolation family were varied jointly. Almost all of the difference came from the extrapolation: a symmetric bias region alone moved the probability to 0.42, while a waning or cure extrapolation reversed the decision at the reported estimate. The worst joint combination (net benefit -9,618) was worse than any one-at-a-time scenario (-6,657). Read the study below.
Statement
For an HTA governed by cost-effectiveness the decision-relevant QBA output is the economic model evaluated over the plausible bias region, reported as incremental net benefit surfaces, decision-reversal proportions and expected regret against clinical and cost-effectiveness thresholds, and because HTA needs lifetime survival the bias region must be carried through the extrapolation model as well. A p = 0.05 tipping boundary is a valid answer to an inferential question and merely insufficient for the decision, though the entry’s exemplar analysis does read its result off exactly that boundary. The claim that nothing links the ITC sensitivity region to the economic model is wrong: Ren et al. 2025 (doi:10.1017/rsm.2025.13) explicitly recommend incorporating sensitivity results into economic scenario analyses and judging robustness by whether all scenarios yield ICERs below the threshold; what does not exist is a link to the model’s probabilistic sensitivity analysis through a shared parameter space, or any framework indexing the survival extrapolation model jointly with the bias parameters.
Why it is open
The map from a bias parameter to a decision runs through survival extrapolation, costs and QALYs and is strongly nonlinear, so a modest change in a hazard ratio can move the decision quantity far more than it moves the effect estimate, and propagating a single corrected point estimate cannot reproduce that. Economic models are usually built by a different team in different software from the ITC, so evaluating the model at every point of the bias surface is organizationally and computationally expensive, and nothing shares draws or parameters between the ITC sensitivity region and the model’s probabilistic sensitivity analysis even though nesting or conditioning over bias scenarios is statistically possible. Bias parameters are naturally specified for the observed follow-up while the decision depends mostly on a period in which no data exist, and no framework indexes the extrapolation model choice by the bias parameters, so the two are explored separately and their interaction is never displayed. With several bias parameters the tipping point is a surface with no accepted summary, and judging whether the tipping region is attainable still requires the elicitation the method was often adopted to avoid.
What has been tried
| Work | What it contributes |
|---|---|
| Ren et al. 2025, Research Synthesis Methods | Recommends that sensitivity analysis results be incorporated into scenario analyses within economic models, explicitly assessing their impact on the ICER, with robustness judged by whether all scenarios fall below the threshold. Counter-evidence to the claim that no link to the economic model exists; it does not link to the model’s probabilistic sensitivity analysis or share a parameter space. Its case study also reads a significance-crossing point off a deterministic sensitivity curve. |
Probable solution or research direction
Move from the scenario analysis published guidance already recommends to a shared parameter space: nest or condition the economic model’s probabilistic sensitivity analysis over bias scenarios using shared draws, and report incremental net benefit surfaces, decision-reversal proportions and expected regret against noninferiority margins and cost-effectiveness thresholds alongside the feasibility and support map. Index the extrapolation model jointly with the bias parameters so the sensitivity surface reports lifetime survival and net benefit rather than in-trial effects only. Build interfaces so a PAIC QBA object can be consumed directly by the health-economic model, and pass a single effect parameter only where it demonstrably carries the whole modeled bias.
Our study
Carrying a hazard-ratio bias into net benefit: the extrapolation family moved the decision more than the bias did
The question put to it. Does propagating a bias-corrected hazard ratio and its interval into an economic model reproduce the decision uncertainty, or does the bias need to share a parameter space with the extrapolation model?
What it found. No. The probability of negative net benefit was 0.38 under point propagation of the reported hazard ratio and 0.67 when the bias (log hazard ratio within 0.15) and the treatment-effect extrapolation family were varied jointly. Almost all of the difference came from the extrapolation: a symmetric bias region alone moved the probability to 0.42, while a waning or cure extrapolation reversed the decision at the reported estimate. The worst joint combination (net benefit -9,618) was worse than any one-at-a-time scenario (-6,657).
Results.
- Reversal probability by family, joint over the bias: persisting 0.42, waning 0.82, cure 0.75.
- Net benefit moved roughly linearly and symmetrically in the bias within each family, so a symmetric bias region barely changes the reversal probability.
- Scenario analysis as currently recommended missed the joint worst case.
Design.
- Estimand
- Probability of negative incremental net benefit (decision reversal)
- Methods compared
- point propagation, scenario analysis, joint probabilistic analysis over bias and extrapolation family
- Data-generating mechanism
- Decision model with Weibull or cure control survival, a log hazard ratio bias elicited on [-0.15, 0.15], three treatment-effect extrapolation families; point propagation, scenario analysis and a joint probabilistic reference.
- Replicates
- 4000 probabilistic draws; 31 x 3 deterministic surface
What this does not answer. Deterministic survival curves instead of fitted extrapolations; three families, no splines; one symmetric bias parameter; costs and utilities fixed; one price. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/QBA-25-a-bias-region-that-has-to-survive-extrapolation · Protocol: registered before the run
Verification
Verdict. Partially addressed. Auditors agreed the problem is open and no counterevidence surfaced. Reopened by the full-text reading: 2 findings from 2 papers survived both the reader and an independent second reviewer, so the verdict moves from confirmed-open to partially-addressed.
The source said. “The decision-relevant QBA quantity is incremental net benefit as a function of the bias parameters, together with the probability of recommendation reversal and expected regret, not whether a 95% interval for a log hazard ratio excludes zero. Tipping points should be surfaces computed against clinical and cost-effectiveness thresholds rather than a p = 0.05 boundary, and because HTA needs lifetime survival the bias region has to be carried through the extrapolation model as well.”
What was wrong with it. The literature auditor found the claim that nothing links the ITC sensitivity region to the economic model contradicted by Ren et al. 2025 (doi:10.1017/rsm.2025.13), which recommends precisely that link as scenario analysis against the ICER threshold, leaving only the probabilistic-sensitivity-analysis link and the shared parameter space missing. GPT-5.6 Sol softened two absolutes: a p = 0.05 tipping boundary is legitimate when the prespecified conclusion is statistical significance, and passing one effect parameter into the economic model is adequate when that parameter is a sufficient conduit for the modeled biases and its full scenario or uncertainty distribution is propagated, rather than a misuse in every case.
Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).
- Verdict changed
- confirmed-open -> partially-addressed 2 surviving findings from: Accounting for Heterogeneity in Relative Treatment Effects f; Quantitative bias analysis for unmeasured confounding in una. New citations: 10.1017/rsm.2025.13, 10.1177/0272989X15570113
- Literature and prior-art check
- open / supported-with-caveat. Claim 2 is corroborated by the very method the agenda holds up as the state of the art: Ren et al. 2025’s case study reads its sensitivity result off a significance boundary, ‘If the proportion is greater than 28%, then the treatment effect becomes statistically significant’, which is exactly the p=0.05 anchoring claim 2 calls a misuse. The caveat is claim 9. Ren et al. 2025 do explicitly link the sensitivity region to the economic model, recommending that ‘These sensitivity analysis results can be incorporated into scenario analyses within economic models, explicitly assessing their impact on the incremental cost-effectiveness ratio (ICER)’, with decision rules based on whether all scenarios fall below or above the threshold. So a link to the economic model exists in published guidance; what does not exist is a link to the model’s probabilistic sensitivity analysis or a shared parameter space, and no published work indexes the survival extrapolation model jointly with the bias parameters (claims 7, 8). Claims 1, 3, 4, 5, 6 are normative and correct in substance. Tipping-point machinery for evidence synthesis exists but only for arm-based network meta-analysis correlation parameters, not for unanchored PAIC bias surfaces.
- GPT-5.6 Sol, technical and source-code lens
- open / overstated. C1 is correct only after defining the reference point, feasible set, direction, or metric: with several parameters the tipping set is a boundary or hypersurface, but ‘smallest departure’ is otherwise undefined. C2 is too categorical. A p=0.05 tipping boundary is valid when the prespecified conclusion is statistical significance; it is merely insufficient for an HTA decision. C3 is appropriate for a conventional cost-effectiveness decision rule, but incremental net benefit is not the only possible HTA decision quantity and cannot replace clinical, budget, equity, or other constraints. C4, C5, C7, and C8 are valid possibility statements. C6 is false as a universal rule: if bias acts only through one treatment-effect parameter and the full scenario or uncertainty distribution for that parameter is passed to the economic model, it can be sufficient; a single corrected point estimate is inadequate only for a question about the whole bias region. C9 is an absolute practice claim outside a technical audit and is not a statistical impossibility: probabilistic sensitivity analysis can be nested or conditioned over bias scenarios using shared draws. The remaining open work is integration, extrapolation specification, and software workflow, not a new identification result, so the asserted very-high severity is inflated.
- Decision path
-
R5:default-open → R9:substantiated-single-overstatement:codex
How the evidence falls in time
8 papers in the reviewed corpus bear on this problem, published 2015 to 2026. Partial progress is on the record, and later work still calls this open. Both belong in view.
This problem is recurrent: three or more papers spanning at least eight years assert it independently. Sustained restatement by authors who mostly do not cite each other is the strongest evidence this reading can offer that a gap is real rather than one group’s framing.
It is also being closed in pieces: partial results come from more than one paper in more than one year, each covering a different part.
| Year | Says | Paper |
|---|---|---|
| 2015 | partly addresses it | Accounting for Heterogeneity in Relative Treatment Effects for Use in Cost-Effectiveness Models and Value-of-Information Analyses |
| 2018 | confirms it is open | Different Methods for Modelling Severe Hypoglycaemic Events: Implications for Effectiveness, Costs and Health Utilities |
| 2019 | confirms it is open | Threshold Analysis as an Alternative to GRADE for Assessing Confidence in Guideline Recommendations Based on Network Meta-analyses |
| 2022 | confirms it is open | Using threshold analysis to assess the robustness of public health intervention recommendations from network meta-analyses: application to accident prevention in households with children under five |
| 2023 | confirms it is open | Bayesian hierarchical model-based network meta-analysis to overcome survival extrapolation challenges caused by data immaturity |
| 2024 | confirms it is open | Twenty years of network meta-analysis: Continuing controversies and recent developments |
| 2025 | partly addresses it | Quantitative bias analysis for unmeasured confounding in unanchored population-adjusted indirect comparisons |
| 2026 | confirms it is open | Anchors Away: Navigating Unanchored Indirect Comparisons with Multilevel Unanchored Meta-Regression (ML-UMR) (Chandler & Ishak; arXiv:2606.20341v1). The batch file points at media/media/Table1.pdf; the full LaTeX source mlumr.tex sits in the same arXiv package and was read. |
Source
Derived from C:L709-713, C:L709-725, C:L725, C:L770-780 of the reviewed corpus.