No criterion chooses pooled or arm-separate matching in anchored MAIC
The product claim is wrong. Given an aggregate-trial imbalance kappa, pooled matching is biased by kappa (beta/2 + gamma_X + gamma_U rho_T) and arm-separate matching by kappa gamma_U (rho_T - rho_S); simulation matched both (slopes 1.00 and 0.96). Arm-separate matching reproduces the aggregate imbalance in the individual-data contrast so it cancels, and is biased only when an unmatched prognostic covariate relates to the matched one differently in the two trials. It had smaller bias than pooled matching in every scenario with an imposed imbalance and 10% to 27% lower RMSE under chance imbalance. Read the study below.
Statement
In an anchored matching-adjusted indirect comparison the analyst may estimate one weight vector matching the pooled individual patient data trial to the pooled aggregate trial, or two weight vectors matching the active arm to the aggregate active arm and the control arm to the aggregate control arm separately. The two choices identify different quantities and fail in opposite directions. Arm-separate matching reproduces whatever arm imbalance the aggregate trial had and was more precise and less biased in the simulation that introduced it; it also reweights the two arms of a randomized trial by different functions, so the within-trial balance that randomization supplied on covariates outside the matching set is no longer guaranteed. The evaluation that introduced it recommends it while stating that the approach might break the randomization, that covariates might be imbalanced after weighting, and that further investigation under a more diverse array of situations is needed. No diagnostic reports whether randomization-induced balance on unmatched covariates survived a particular arm-separate weighting, and no rule maps the observed arm imbalance in the aggregate trial and the prognostic strength of unmatched covariates onto a choice. Because arm-level aggregate summaries are routinely published the choice arises in most anchored analyses, it is exposed as an ungoverned user option in the tooling built around the method, and no reporting requirement asks which was used.
Why it is open
The two options trade one bias against another and neither bias is estimable from the data at hand: matching each arm separately protects the comparison against arm-level covariate differences that a pooled match would average away, while differential reweighting of randomized arms can introduce confounding through covariates that were never in the matching set. A diagnostic for the second failure would have to demonstrate balance on covariates that are by construction not reported for the comparator trial, so the check needs precisely the information the design lacks. The two published simulation-based positions were generated under different data-generating mechanisms, so reading them against each other cannot settle which is right; a resolution requires a study whose factors include the size of the aggregate trial’s arm imbalance and the prognostic strength of unmatched covariates, which is the combination nobody has crossed. Practice leaves no trail either, since submissions need not state which weighting was used, so the field cannot learn retrospectively which choice moved which result.
What has been tried
| Work | What it contributes |
|---|---|
| Petto, Kadziola, Brnabic, Saure et al. 2019, Value in Health 22(1):85-91 | Introduces and evaluates weighting each arm of the individual-data trial to the corresponding arm of the aggregate trial via the common comparator, reports better precision and lower bias than pooled matching in simulation, and states that the approach might break the randomization, that covariates might be imbalanced after weighting, and that further investigation across a more diverse array of situations is needed. |
| Jiang, Cappelleri, Gamalo, Chen, Thomas & Chu 2024, Research Synthesis Methods 15(4):671-686 | Comprehensive review of matching-adjusted indirect comparison, its derivation, assumptions and connection to calibration estimation in survey sampling, accompanied by the Shiny-MAIC application that offers anchored and unanchored analyses and alternative variance estimators as user settings. Reproduces the weighting options without adjudicating between them. |
| Remiro-Azocar 2022, BMC Medical Research Methodology 22:217 | Two-stage matching-adjusted indirect comparison, the closest formal treatment of arm-differential weighting: a treatment-assignment model inside the individual-data trial produces inverse probability weights that are combined with trial-assignment odds weights, so arms are weighted differently in order to correct chance imbalance in the randomized trial rather than to reproduce the aggregate trial’s imbalance. |
| Signorovitch, Wu, Yu, Gerrits et al. 2010, PharmacoEconomics 28(10):935-945 | The original matching-adjusted indirect comparison, which matches on pooled trial-level summaries and so sets the pooled option as the default from which the arm-separate variant departs. |
| Phillippo, Ades, Dias, Palmer, Abrams & Welton 2018, Medical Decision Making 38(2):200-211 | The appraisal-facing framework for population-adjusted indirect comparison, which sets out what an anchored analysis is assumed to preserve; it does not address how the weighting problem should be partitioned across randomized arms. |
Probable solution or research direction
Run one simulation study whose factors are the ones that separate the two options: the size and direction of arm imbalance in the aggregate trial, the prognostic strength and the number of covariates outside the matching set, the effective sample size after weighting, and the true effect modification. Score bias, coverage and precision for pooled and arm-separate weighting together with balance on the unmatched covariates, so the randomization-breaking failure appears as a measured quantity rather than a warning. Derive and report the estimand each option targets, since a comparison that reproduces the aggregate trial’s arm imbalance and one that removes it are answers to different questions, and state which corresponds to the decision population. Where the choice cannot be made on evidence, make it disclosable: require that the weighting partition, the resulting per-arm effective sample sizes and the balance achieved on all covariates be reported, and offer both fits as a standard sensitivity pair.
Our study
Arm-separate matching cancels the aggregate trial’s chance imbalance; pooled matching carries it
The question put to it. Arm-separate matching weights each individual-data arm to the corresponding aggregate arm; pooled matching uses one weight function. When does each give a biased anchored comparison, and is the bias of arm-separate matching the product of unmatched prognostic strength and aggregate arm imbalance, as the design claims?
What it found. The product claim is wrong. Given an aggregate-trial imbalance kappa, pooled matching is biased by kappa (beta/2 + gamma_X + gamma_U rho_T) and arm-separate matching by kappa gamma_U (rho_T - rho_S); simulation matched both (slopes 1.00 and 0.96). Arm-separate matching reproduces the aggregate imbalance in the individual-data contrast so it cancels, and is biased only when an unmatched prognostic covariate relates to the matched one differently in the two trials. It had smaller bias than pooled matching in every scenario with an imposed imbalance and 10% to 27% lower RMSE under chance imbalance.
Results.
- With the matched-unmatched correlation shared between trials, arm-separate bias was within Monte Carlo error of zero in all 16 scenarios with an unmatched prognostic covariate and an imposed imbalance (largest 0.011).
- Slope of simulated on predicted bias: pooled 1.004 (SE 0.011); arm-separate 0.958 (SE 0.035).
- Under chance imbalance, arm-separate RMSE over pooled 0.73 to 0.90; its sandwich interval overcovered (0.965 to 0.991) because the two trials’ imbalances are positively correlated.
- Two-stage MAIC behaved like pooled matching, correcting the individual trial’s imbalance but not the aggregate trial’s.
- The design’s candidate diagnostic, the weighted difference in the unmatched covariate between arms, estimates rho_S kappa and cannot see rho_T; its correlation with arm-separate bias was -0.41.
Design.
- Estimand
- Anchored B-versus-A mean difference in the target
- Methods compared
- unadjusted Bucher, pooled MAIC, arm-separate MAIC, two-stage MAIC
- Data-generating mechanism
- Anchored MAIC with a continuous outcome, 300 per arm; 96 scenarios crossing an imposed aggregate-arm imbalance in the matched covariate (0, 0.1, 0.2 SD or chance), the unmatched covariate’s prognostic strength, the matched-unmatched correlation in each trial and shared effect modification; unadjusted, pooled, arm-separate and two-stage MAIC.
- Replicates
- 1000 per scenario, 96,000 in total
What this does not answer. Continuous outcome, linear model, one matched and one unmatched covariate, shared effect modification; the non-collapsible case, where arm-separate matching compares arms in different populations, is not studied, and the arm-separate interval’s conservatism is not repaired. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/MOD-16-pooled-versus-arm-separate-matching · Protocol: registered before the run
Verification
Verdict. Confirmed open. One reader proposed this and the reviewer accepted it as a real and consequential design choice inside anchored matching, narrower than covariate selection or trimming and with competing implications for target-population alignment, precision and preservation of randomized balance. It rests on a single primary paper, whose own discussion calls for the investigation that has not been done; the conflicting recommendation elsewhere in the literature is reproduced in review rather than resolved.
Source
Derived from reading:core-05 of the reviewed corpus.