Unreleased cpaic shares one tau and one IPD-row residual variance
This entry describes a limitation of a specific implementation rather than of the field. It is catalogued because the underlying methodological problem is general, and because a reference implementation is where such problems become concrete and checkable.
Largely yes for a contrast that sums both subnetworks. With one subnetwork five times as heterogeneous, shared, stratified and shrunk heterogeneity models covered within 0.016 of each other (shared 0.906 to 0.932 and stratified 0.917 to 0.943 across 3 to 12 studies per subnetwork), because a shared tau overstates one part’s variance and understates the other’s. The undercoverage came from ignoring uncertainty in tau: Hartung-Knapp intervals covered 0.97 to 0.997 at three studies, at about 1.5 times the width. Read the study below.
Statement
cpaic, an unreleased development R package on GitHub (choxos/cpaic@9d150e9) that a reader cannot install from CRAN, declares at most one between-study heterogeneity parameter tau when random effects are requested and applies it to all study-arm deviations, with no component-, class-, or subnetwork-indexed heterogeneity. Its Gaussian component ML-NMR likelihood uses one residual sigma across all IPD rows, although aggregate arms use their own supplied standard errors and the two-stage routes fit study-specific regressions, so the shared residual scale is narrower than stated. A common tau is also the near-universal default in netmeta and multinma, so this is a field-level convention rather than a package idiosyncrasy.
Why it is open
A single tau borrows strength across parts of a network that may not be exchangeable, so a heterogeneous subnetwork can inflate uncertainty in a homogeneous one and vice versa, and in a disconnected network that directly changes how much support the bridge appears to have. Forcing one residual scale across IPD studies that differ in instruments, populations, and follow-up transfers weight between studies and makes model-based intervals wrong even where the mean model is right. Allowing structured heterogeneity or study-specific scales immediately runs into identifiability, because sparse disconnected networks rarely have enough studies per stratum to estimate several variance components, so shrinkage is needed and prior dependence returns.
What has been tried
| Work | What it contributes |
|---|---|
| cpaic 0.1.0 (development version, choxos/cpaic@9d150e9) | unreleased development R package on GitHub, not on CRAN; every Stan family declares at most one tau applied to all study-arm deviations, and the Gaussian likelihood uses one residual sigma across IPD rows while aggregate arms use supplied standard errors |
Probable solution or research direction
Allow heterogeneity structured by component, component class, or subnetwork with shrinkage across strata, and add identifiability checks that flag a variance component driven entirely by its prior, since that is the binding constraint in sparse networks. Support study-specific or hierarchically shrunk residual scales for the IPD likelihood and robust residual distributions so heavy tails and unequal precision do not distort weighting. Report the diagnostic separately from the correction, so an analyst can see whether stratum-specific heterogeneity is supported before fitting it.
Our study
Sharing one heterogeneity parameter across a bridged network cost little; the interval method mattered more
The question put to it. When two subnetworks of a bridged network have different between-study heterogeneity, does a shared tau give the bridged contrast valid intervals, and do stratified or shrunk alternatives do better at realistic study counts?
What it found. Largely yes for a contrast that sums both subnetworks. With one subnetwork five times as heterogeneous, shared, stratified and shrunk heterogeneity models covered within 0.016 of each other (shared 0.906 to 0.932 and stratified 0.917 to 0.943 across 3 to 12 studies per subnetwork), because a shared tau overstates one part’s variance and understates the other’s. The undercoverage came from ignoring uncertainty in tau: Hartung-Knapp intervals covered 0.97 to 0.997 at three studies, at about 1.5 times the width.
Results.
- Registered rule: mixed; shared coverage never below 0.90 in the registered cells, lowest 0.897 elsewhere.
- Every Wald interval covered about 0.90 to 0.92 at a 5:1 ratio with three studies per subnetwork.
- The shared tau estimate sat between the two true values.
Design.
- Estimand
- Bridged contrast formed as the sum of two subnetworks’ pooled effects
- Methods compared
- shared tau, stratified tau, shrunk stratified tau, Hartung-Knapp versions
- Data-generating mechanism
- A bridged contrast summing two subnetworks’ pooled effects; heterogeneity ratio 1, 2 or 5 in the bridging or the other subnetwork; 3, 6 or 12 studies each; shared, stratified and shrunk REML heterogeneity with Wald and Hartung-Knapp intervals. 15 cells, 2000 replicates each.
- Replicates
- 2000 per cell, 15 cells
What this does not answer. A contrast drawing on one subnetwork only would carry the shared tau’s error without the offset; frequentist REML without priors; no residual-variance arm or IPD. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/CMP-16-one-tau-across-a-network-that-is-not-exchangeable · Protocol: registered before the run
Verification
Verdict. Unverifiable. Auditors disagreed and the disagreement was evidence-bearing on both sides.
The source titled this. “cpaic assumes a single heterogeneity and residual variance” The title implies an installable package and a single residual variance across all studies, but cpaic is a GitHub development package and the shared sigma covers IPD rows only.
The source said. “cpaic’s models assume one common between-study heterogeneity parameter across the network and, for continuous outcomes, one Gaussian residual standard deviation shared across all studies. Different component classes or subnetworks may plausibly have different heterogeneity, and unequal true residual variances distort study precision and network weighting.”
What was wrong with it. The source said the single residual standard deviation is shared across all studies and that it biases both the point estimate and its interval. GPT-5.6 Sol read choxos/cpaic@9d150e9 and narrowed both: the shared sigma covers IPD rows only, since aggregate arms use supplied se_agd and the two-stage routes fit study-specific regressions, and residual heteroskedasticity misallocates precision and invalidates model-based uncertainty without by itself implying biased mean-effect coefficients under a correct conditional mean model. The literature auditor added that a common tau is the standard default in netmeta and multinma too.
Dissent. codex dissented (overstated): When random effects are requested, C1-C3 are supported for cML-NMR: every Stan family declares at most one tau and applies tau[1] to all study-arm deviations; no component-, class-, or subnetwork-indexed heterogeneity scale exists. C4 and C6 require qualification. The Gaussian cML-NMR likelihood has one sigma across all IPD observations, but aggregate arms use their separately supplied se_agd values, and cMAIC/cSTC fit study-specific regressions rather than sharing this sigma. Unequal IPD residual variances do misrepresent relative precision, efficient weighting, and model-based uncertainty. However, heteroskedasticity alone does not imply biased mean-effect coefficients when the conditional mean model is correct, contrary to the report’s narrative. The priority is conditional on material variance or heterogeneity differences and the amount of evidence each affected stratum contributes.
- Literature and prior-art check
- open / unverifiable. Every claim here is about cpaic’s internals (single heterogeneity parameter, single residual standard deviation, absence of structured or study-specific scales). cpaic is not on CRAN and I found no public repository, so I have nothing to read and this belongs to the source-code auditor. On the literature side the statistical assertions are textbook and safe: a common tau borrows strength across possibly non-exchangeable parts of a network, and forcing a common residual variance misallocates precision weights. I did not retrieve a specific paper on comparison-specific or class-specific heterogeneity variances in network meta-analysis in this session, so I will not cite one; note only that a common tau is the near-universal default in netmeta and multinma too, so this is a field-level convention rather than a cpaic idiosyncrasy, and the entry would be stronger if it said so. The identifiability tension the entry raises (too few studies per stratum in sparse disconnected networks) is genuine and, as far as I found, unaddressed.
- GPT-5.6 Sol, technical and source-code lens
- open / overstated. When random effects are requested, C1-C3 are supported for cML-NMR: every Stan family declares at most one tau and applies tau[1] to all study-arm deviations; no component-, class-, or subnetwork-indexed heterogeneity scale exists. C4 and C6 require qualification. The Gaussian cML-NMR likelihood has one sigma across all IPD observations, but aggregate arms use their separately supplied se_agd values, and cMAIC/cSTC fit study-specific regressions rather than sharing this sigma. Unequal IPD residual variances do misrepresent relative precision, efficient weighting, and model-based uncertainty. However, heteroskedasticity alone does not imply biased mean-effect coefficients when the conditional mean model is correct, contrary to the report’s narrative. The priority is conditional on material variance or heterogeneity differences and the amount of evidence each affected stratum contributes.
- Grok 4.5, recency lens
- open / supported. cpaic 0.1.0 (README, cmlnmr Rd, NEWS) still fits one common tau and one network-wide Gaussian residual sigma; no component-class, subnetwork, or study-specific residual hierarchy is implemented. Broader NMA has class-effect and hierarchical heterogeneity models (e.g. class-effects NMA practice literature), but those are not component-PAIC residual or component-structured tau models and do not close the stated cpaic gap. No CRAN/GitHub change in multinma/maicplus/netmeta/cpaic was found that adds study-specific residual scales inside component ML-NMR. Queries: cpaic tau sigma; multinma study-specific residual; structured heterogeneity component NMA 2024–2026; class-effects NMA residual variance. Claims C1–C6 are faithful software/limitation statements; the methodological problem remains open as stated.
- Decision path
-
R5:default-open → R7:counter-and-support-both-present
Source
Derived from C:L1371, C:L1372, scope:package-specific default resting on a field-level NMA modeling question, scope:package-specific of the reviewed corpus.