# Protocol: what an anchored comparison must balance on a non-collapsible scale

**Target problem.** COV-03 *Design-dependent roles of prognostic covariates.*
Reporting standard ADEMP (Morris, White and Crowther 2019). Committed before any
replicate of the registered grid; the commit is the timestamp. Design rationale:
[`DESIGN.md`](DESIGN.md). Probe numbers: [`results/probes.md`](results/probes.md),
generated by `R/02-probes.R`.

## 1. Claim

On the logit scale a covariate that is purely prognostic conditionally still
changes the marginal odds ratio (Remiro-Azócar 2024,
[doi:10.1002/sim.10111](https://doi.org/10.1002/sim.10111)). So an anchored MAIC
that balances only effect modifiers, or only covariate means, can leave bias that
current collapsible-scale guidance says is absent. **Refuting sentence:** the
contribution is second order and small at realistic prognostic strength, so the
guidance holds in practice.

## 2. Mechanism, corrected

With $u = \gamma^\top x$ normal, no effect modification, and a population law $F$,

$$\Delta(F) = \operatorname{logit}\int \operatorname{expit}(\alpha + u + \delta)\,dF - \operatorname{logit}\int \operatorname{expit}(\alpha + u)\,dF,$$

which depends on the mean and variance of $u$. To second order,
$\Delta(F) \approx \delta - \mathrm{Var}_F(u)\{\operatorname{expit}(\alpha + \bar u + \delta) - \operatorname{expit}(\alpha + \bar u)\}$.
**DESIGN.md section 2 said $\Delta$ depends on $F$ only through $\mathrm{Var}(u)$.
That is wrong whenever $\mathrm{Var}(u) > 0$**: the mean enters through the risk
difference at the mean index. P2 measures both parts exactly: with equal index
variances a mean shift alone gives bias 0.004 to 0.018; a variance ratio of 0.5 or 2
gives 0.017 to 0.114. P2 also shows the second-order formula is accurate only at
$\mathrm{Var}(u) = 0.25$ and overstates the bias up to fivefold at $\mathrm{Var}(u) = 4$,
so the mechanism check below uses the exact population bias, not the expansion.

## 3. Data-generating mechanism

Source: A versus C trial, individual data, $x \sim N(0, r I_3)$. Target: B versus C
trial, reporting arm event counts and pooled covariate sample means and SDs,
$x \sim N(s\mathbf 1, I_3)$. Outcome $\operatorname{logit} P(y=1) = \alpha + g\sum_j x_j + \delta_t + \beta x_1\mathbb 1[A]$,
$\alpha = \operatorname{logit}(0.3)$, $\delta_A = -0.8$, $\delta_B = -0.5$, 500 per
arm in each trial, baseline shared across trials.

| factor | levels |
|---|---|
| design | anchored, unanchored |
| $\mathrm{Var}_T(u) = 3g^2$ | 0.25, 1, 4 |
| variance ratio $r = \mathrm{Var}_S(u)/\mathrm{Var}_T(u)$ | 0.5, 1, 2 |
| target mean shift $s$ (covariate SDs) | 0, 0.3, 0.6 |
| effect modification $\beta$ on $x_1$ for A | 0, 0.4 |

108 cells, **1000 replicates each** (coverage MCSE 0.0069 at nominal; P4 prices the
grid at 7.6 core-hours). The conditional model is correct; covariates are
independent normal, so matching all means and variances also matches
$\mathrm{Var}(u)$.

## 4. Estimand

Marginal log odds ratio of B versus A in the target population, by 120-point
Gauss-Hermite quadrature (P1 checks it against 2,000,000 Monte Carlo draws).

## 5. Methods

| method | what is balanced or modeled |
|---|---|
| unadjusted | nothing (Bucher anchored; naive unanchored) |
| MAIC means | means of $x_1, x_2, x_3$ |
| MAIC means and variances | means and second moments of all three |
| MAIC index | means of all three and the second moment of $\hat u = \hat\gamma^\top x$, $\hat\gamma$ from a logistic model in the source |
| G-computation | logistic model in the source (with the $A \times x_1$ term when modification is present), marginalized over $N(\text{published means}, \text{published SDs})$ |

MAIC intervals use the sandwich variance of weighted proportions with weights
treated as fixed; G-computation uses the delta method. Unanchored versions weight
or model the A arm alone. The comparator that can win on RMSE without the index
construction is G-computation.

## 6. Performance, controls, decision

Bias, empirical SE, model SE, coverage, RMSE, mean ESS, each with MCSE; common
random numbers across methods.

**Mechanism check.** Across anchored cells without modification, regress the
unadjusted method's cell bias on the exact population bias
$\Delta_A(T) - \Delta_A(S)$. Slope 1 expected.

**Controls.** *Null:* $r = 1$, $s = 0$ (identical populations); every method's bias
within 3 MCSE in every such cell. *Positive:* anchored, no modification,
$\mathrm{Var}_T(u) = 4$, $r \in \{0.5, 2\}$; unadjusted bias beyond 3 MCSE.
*Design's claim:* anchored, no modification, $r = 1$, $s > 0$; the unadjusted bias
is compared with its exact nonzero prediction, which DESIGN.md said would be zero.

**Primary cells.** Anchored, no modification, $r \in \{0.5, 2\}$,
$\mathrm{Var}_T(u) \in \{1, 4\}$, all shifts: 12 cells.

- **Confirmed:** MAIC means has a 95% Monte Carlo interval for bias entirely
  beyond $\pm 0.05$ in at least one primary cell, and in each such cell MAIC means
  and variances or MAIC index has $\lvert \text{bias} \rvert < 0.05$.
- **Refuted:** MAIC means has $\lvert \text{bias} \rvert < 0.05$ in every primary
  cell.
- **Mechanism not supported:** MAIC means is biased and neither variance-balancing
  arm brings bias under 0.05.

RMSE and ESS of every method in the primary cells are reported in every branch,
since the trade-off decides what to recommend.

## 7. Departures from DESIGN.md, before the run

1. Mechanism corrected (section 2).
2. The risk-difference null control is dropped. A logistic model without
   modification on the logit scale has modification on the risk-difference scale,
   so no method is unbiased there by construction and the control could not hold.
   The null control is identical populations instead.
3. Overlap is not a separate factor: it is produced by the variance ratio and
   shift, and ESS is reported. The log hazard ratio arm and the skewed-covariate arm
   are not run.
4. The augmented weighting estimator is not run; G-computation is the
   outcome-model comparator.
5. The index-variance arm's interval ignores estimation of $\hat\gamma$ and of the
   weights. P3's pilot shows MAIC sandwich intervals undercover when weights are
   extreme (empirical SD 0.40 against model SE 0.35 at $r = 0.5$), so coverage of all
   MAIC arms is reported as measured, not as validated.
6. $n_{sim} = 1000$ rather than 2000.