Simulated-covariate QBA is an STC route, not a general solution
The MAIC extension of the simulated-covariate bias analysis was feasible in at least 99.8% of replicates across assumed target prevalences from 0.1 to 0.9, lost effective sample size to 85 to 109 of 300 only at the extreme, and had a reported SE 3% to 14% above STC’s. Both routes were unbiased. The restriction to STC was conservative rather than necessary. Both routes’ multiply imputed intervals widened along the sweep more than the estimate’s real spread (SE up to 0.28 against about 0.13), so coverage exceeded nominal. Read the study below.
Statement
Ren et al. 2025 (doi:10.1017/rsm.2025.13) extend the NORTA-based marginal unanchored STC of Ren et al. 2024 (doi:10.1002/jrsm.1718) by simulating an assumed synthetic joint distribution for covariates measured in the IPD but unreported in the comparator publication, averaging individual predictions on the natural outcome scale before transforming to the log odds scale, and presenting deterministic sensitivity curves with bootstrap confidence intervals; they do not present tipping-point plots, and the case study instead reads a significance-crossing value off the curve. This is the beginning of formal QBA for limited-aggregate-data PAIC rather than a completed solution: the workflow is implemented only for STC, and the authors restrict it there because a MAIC extension would suffer extreme effective-sample-size loss and might fail to produce feasible weights. Their statement that no additional assumptions are required cannot be read literally, since target marginals, dependence, source-to-target outcome associations, copula form and regression form are all assumed.
Why it is open
The workflow is embedded in an STC and g-computation route, so it does not transfer to weighting, one-stage or network estimators, and the authors themselves rule out the direct MAIC extension on effective-sample-size and feasibility grounds. Its inputs, the outcome association of the omitted variable and its dependence with measured covariates, are estimated in the source study and assumed to hold in the target, which cannot be checked against the published comparator data. It also inherits the requirements of the underlying estimator, a sufficiently correct outcome model and adequate overlap, neither of which a covariate-omission sensitivity analysis can repair.
What has been tried
| Work | What it contributes |
|---|---|
| Ren et al. 2025, Research Synthesis Methods | Simulated-covariate QBA extending unanchored STC for covariates unreported in the comparator study, with deterministic sensitivity curves and bootstrap confidence intervals. Restricted to STC because a MAIC extension would suffer extreme effective-sample-size loss and may fail to produce feasible weights. |
| Ren et al. 2024, Research Synthesis Methods (formerly Journal of Research Synthesis Methods) | NORTA (Gaussian copula) based marginal unanchored STC implementation evaluated by simulation, the estimator the 2025 QBA extends. |
Probable solution or research direction
Treat the simulated-covariate route as one family within a broader QBA framework, extend it to multiple correlated omitted variables and treatment-specific associations, and state its assumptions about target distributions, correlations, outcome associations, copula form and model form explicitly rather than describing it as assumption-free. Develop parallel routes for weighting and one-stage estimators so the same sensitivity question can be asked of any PAIC estimator, addressing the feasibility and effective-sample-size loss that led the authors to restrict the published version to STC. Report the sensitivity curve against a clinical or decision threshold rather than reading it off a significance boundary.
Our study
A simulated-covariate bias analysis by STC and by MAIC
The question put to it. Is a MAIC version of the simulated-covariate bias analysis infeasible or too imprecise, as its authors assumed when restricting it to STC?
What it found. The MAIC extension of the simulated-covariate bias analysis was feasible in at least 99.8% of replicates across assumed target prevalences from 0.1 to 0.9, lost effective sample size to 85 to 109 of 300 only at the extreme, and had a reported SE 3% to 14% above STC’s. Both routes were unbiased. The restriction to STC was conservative rather than necessary. Both routes’ multiply imputed intervals widened along the sweep more than the estimate’s real spread (SE up to 0.28 against about 0.13), so coverage exceeded nominal.
Results.
- Refuting sentence holds: feasibility 0.998 to 1.000; MAIC SE / STC SE 1.03 to 1.14.
- Effective sample size 85 to 231 of 300 along the sweep.
- Bias at most 0.009 for both routes; coverage 0.956 to 1.000 (conservative toward the extremes).
Design.
- Estimand
- A’s marginal log odds in a target with the assumed prevalence of the unmeasured covariate
- Methods compared
- simulated-covariate STC, simulated-covariate MAIC
- Data-generating mechanism
- Unanchored transport with an unmeasured binary covariate imputed from its assumed relations; STC and MAIC sensitivity curves over the assumed target prevalence, with feasibility and effective sample size. 2 cells, 500 replicates each.
- Replicates
- 500 per scenario, 1,000 in total, each over a 5-point sweep with 10 imputations
What this does not answer. One binary unmeasured covariate, correct assumed relations, unanchored A side only, no decision reading. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/QBA-13-testing-the-reason-maic-was-excluded · Protocol: registered before the run
Verification
Verdict. Confirmed open. Auditors agreed the problem is open and no counterevidence surfaced.
The source said. “Ren et al. (2025) extend a NORTA-based marginal unanchored STC by simulating covariates measured in the IPD but unreported in the comparator publication, then marginalizing predictions and presenting tipping plots. The report judges this the beginning of formal QBA for limited-AgD PAIC rather than a completed solution.”
What was wrong with it. Both auditors found the entry’s description of tipping plots inaccurate: Ren et al. 2025 (doi:10.1017/rsm.2025.13) present deterministic sensitivity curves with bootstrap confidence intervals and read a significance-crossing point off them, and the phrase tipping point appears only in background prose about QBA modes. GPT-5.6 Sol also corrected the claim that the method simulates the comparator population’s full covariate distribution; it simulates an assumed synthetic joint distribution.
- Literature and prior-art check
- open / supported-with-caveat. Read the Ren et al. 2025 full text. Claims 1 to 6 and 8 verified verbatim. NORTA/Gaussian copula: ‘Ren et al. (2024) developed a novel way to implement unanchored STC by incorporating marginalisation and the NORmal To Anything (NORTA) algorithm (also is known as a Gaussian copula method)’. STC-only: ‘MAIC approach would suffer an extreme reduction of the effect sample size and may even fail to produce feasible weights. Hence, we will only present the QBA for the STC approach.’ Marginal estimand preserved: prediction is done ‘based on the natural scale … and then transformed to the log odds scale for the indirect comparison step’. Overlap and model requirements are stated verbatim in the discussion. Claim 7’s target text is real: ‘No additional assumptions are required for our approach.’ Caveat on the entry’s statement (not a numbered claim): the paper presents no tipping plots. ‘Tipping point’ appears once, in background prose describing QBA modes generally; the case study presents deterministic sensitivity curves with bootstrap CIs, from which the authors read off a significance-crossing point (‘If the proportion is greater than 28%, then the treatment effect becomes statistically significant’), which is the p=0.05-anchored reading QBA-25 criticises.
- GPT-5.6 Sol, technical and source-code lens
- open / supported-with-caveat. The locally available full text supports C1-C8. Ren et al. follow the 2024 NORTA or Gaussian-copula STC, average individual predictions on the natural outcome scale before transforming, restrict the implemented QBA to STC, require correct outcome-model specification and overlap, rely on important omitted covariates being measured in the IPD source, and warn of severe MAIC effective-sample-size loss. The report is also correct that the paper’s statement that no additional assumptions are required cannot be literal: target marginals, dependence, source-to-target outcome associations, copula form, and regression form are assumed. Two caveats remain. C2 should say the method simulates an assumed synthetic joint distribution, not the comparator population’s known full distribution. The entry’s statement that the paper presents tipping plots is inaccurate; it presents deterministic sensitivity curves and significance-crossing interpretations rather than formal tipping plots. The broader estimator-general QBA problem remains open.
- Decision path
-
R5:default-open → R11:caveated(n=2)
How the evidence falls in time
2 papers in the reviewed corpus bear on this problem, published 2025 to 2026. Every paper found here asserts the problem is open; none reports progress.
| Year | Says | Paper |
|---|---|---|
| 2025 | confirms it is open | Quantitative bias analysis for unmeasured confounding in unanchored population-adjusted indirect comparisons |
| 2026 | confirms it is open | Can We Trust PAICs in Rare Diseases? Methodological Challenges and Limitations |
Source
Derived from C:L494-521 of the reviewed corpus.