Edge influence is approximate for multi-arm and random effects
This entry describes a limitation of a specific implementation rather than of the field. It is catalogued because the underlying methodological problem is general, and because a reference implementation is where such problems become concrete and checkable.
No, the refuting sentence fails; the registered verdict is not fit for ranking. The diagonal influence equaled the correct hat row for two-arm networks (difference 8e-15) and ranked studies well there with tau fixed at 0 (mean Spearman with refitted variance importance 0.95 to 0.97; top study missed in 5% to 6%). With three-arm studies and tau fixed, Spearman stayed at 0.90 to 0.92 but the top study was missed in 28% to 46% of networks; in a post hoc rerun the diagonal’s choice was a three-arm study in almost all misses. With tau re-estimated, mean Spearman was 0.51 to 0.70, including two-arm networks, and the top study was missed in 20% to 59%. Read the study below.
Statement
edge_influence() in cpaic, an unreleased development R package on GitHub (choxos/cpaic@9d150e9) that a reader cannot install from CRAN, asks whether a given IPD edge contributes to a requested target contrast. Its diagonal-weight calculation is exact for two-arm common-effect models with fixed weights and approximate for multi-arm studies, because the underlying frequentist CNMA fit uses within-study covariance; under random effects the fitted tau-squared is included in the weights, so the residual approximation comes from not re-estimating tau under perturbation rather than from ignoring shrinkage. Exact contribution machinery already exists for CNMA component effects and for study importance in NMA; what those do not cover is contribution to a target-specific, population-adjusted cross-gap contrast.
Why it is open
The approximation drops the correlation structure introduced by multi-arm trials and the re-estimation of heterogeneity under perturbation, both of which redistribute information across edges in ways a first-order calculation misses, and either can change an influence ranking. The diagnostic matters precisely because it exposes edges with excellent effective sample size that nonetheless carry no weight for the cross-gap contrast, a question standard MAIC overlap and weight diagnostics cannot answer, so an incorrect ranking undermines the one tool that answers it. The published exact decompositions operate on component effects in an unadjusted CNMA, not on a target-specific population-adjusted contrast, so they cannot simply be called.
What has been tried
| Work | What it contributes |
|---|---|
| Yang et al. 2026 | gives an exact decomposition of which studies and comparisons contribute to each component effect in CNMA using pseudo-paths and the component-level hat matrix; does not cover population-adjusted target-specific contrasts |
| Rucker et al. 2020, study importance | defines an exact, uniquely defined per-study influence measure for a specified NMA estimate as the reduction in that estimate’s variance when the study is added, for one-stage and two-stage models |
| Papakonstantinou et al. 2018 | establishes the flow-network proportion-contribution matrix showing how much each direct treatment effect contributes to each NMA estimate |
| cpaic 0.1.0 (development version, choxos/cpaic@9d150e9) | unreleased development R package on GitHub, not on CRAN; provides edge_influence() to test whether an IPD edge contributes to a requested target contrast, using inverse-variance weights that include the fitted tau-squared and are approximate for multi-arm studies |
Probable solution or research direction
Extend the existing exact contribution machinery, the flow and pseudo-path decomposition for component effects and the variance-reduction definition of study importance, to a target-specific population-adjusted contrast, rather than deriving a new influence measure from scratch. Where an exact extension is not available, compute influence from the model Jacobian and Hessian or by posterior perturbation of the observed edge, including re-estimation of heterogeneity. Report the approximation status alongside each ranking, since the diagnostic is already exact in the two-arm common-effect case.
Our study
A diagonal-weight influence ranked studies correctly only with two-arm studies and heterogeneity held fixed
The question put to it. Does the diagonal-weight influence that cpaic’s edge_influence() reports rank studies the way refitted leave-one-study-out references do, once multi-arm covariance and re-estimated heterogeneity enter?
What it found. No, the refuting sentence fails; the registered verdict is not fit for ranking. The diagonal influence equaled the correct hat row for two-arm networks (difference 8e-15) and ranked studies well there with tau fixed at 0 (mean Spearman with refitted variance importance 0.95 to 0.97; top study missed in 5% to 6%). With three-arm studies and tau fixed, Spearman stayed at 0.90 to 0.92 but the top study was missed in 28% to 46% of networks; in a post hoc rerun the diagonal’s choice was a three-arm study in almost all misses. With tau re-estimated, mean Spearman was 0.51 to 0.70, including two-arm networks, and the top study was missed in 20% to 59%.
Results.
- Post hoc: when the top study disagreed at tau 0, the diagonal’s choice was three-arm in 98% to 100% of networks against 18% to 25% for the reference’s, and its reference importance was a median 15% to 21% below the maximum (within 10% in 18% and 37% of misses).
- In 8-study two-arm networks with tau 0.3, 259 of 365 studies with zero diagonal influence moved the estimate when removed, through the re-estimated tau alone.
- Spearman with the absolute change in the estimate was 0.50 to 0.59 in every cell, including the exact case: influence measures leverage, not realized change.
Design.
- Estimand
- Rank agreement between each study’s diagonal-weight influence on the D versus A contrast and its refitted leave-one-study-out variance importance
- Methods compared
- cpaic diagonal-weight influence (commit cf27b1a), leave-one-study-out refit of the correct model, tau re-estimated
- Data-generating mechanism
- Random four-treatment networks of 8 or 16 studies, two-arm only or with three-arm studies, tau 0 to 0.3; diagonal influence against leave-one-study-out variance importance and estimate change with tau re-estimated. 12 cells, 500 networks each.
- Replicates
- 500 networks per cell, 12 cells
What this does not answer. Standard network meta-analysis rather than component networks with population adjustment; study-level deletion; cpaic’s formula reimplemented, not called; no fixed-tau reference; a block-weighted hat matrix was not tested; 120 of 6000 networks with singular refits dropped. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/CMP-24-edge-influence-approximation · Protocol: registered before the run
Verification
Verdict. Partially addressed. Part of this problem is addressed by existing work; the entry names which part.
The source said. “cpaic’s edge_influence() diagnostic asks whether a given IPD edge actually contributes to the requested target contrast, but its computation is approximate in multi-arm and random-effects settings, so influence rankings may not match exact fitted influence.”
What was wrong with it. The source treated edge influence as a novel diagnostic with no prior art and said random-effects shrinkage is ignored. The literature auditor found three established exact methods, including a component-level decomposition whose flow-weighted sum matches the CNMA component estimate within numerical tolerance (doi:10.1186/s12874-026-02766-8; doi:10.1186/s12874-020-01075-y; doi:10.12688/f1000research.14770.3); GPT-5.6 Sol found that the implementation does include the fitted tau-squared in its weights and is exact for two-arm common-effect models.
Dissent. codex dissented (open): C1 and C2 are correct with qualification. The source explicitly describes the diagonal-W calculation as approximate for multi-arm studies because discomb() uses within-study covariance. For two-arm common-effect models with W fixed, the displayed linear influence is exact. Under random effects, cpaic does include the fitted tau-squared in W; remaining approximation arises if exact perturbation influence is defined to include re-estimation of tau, not because random-effects weighting is ignored. Such omissions can change rankings. C3 is algebraically possible and is demonstrated by the package tests: ESS measures within-edge weight dispersion, whereas target influence is a network-design leverage quantity and can be zero. C4 is correct because ordinary MAIC overlap and weight diagnostics do not evaluate downstream leverage for a requested cross-gap contrast. This is a useful diagnostic refinement, but not a defect in the fitted estimator itself.
- Literature and prior-art check
- partially-solved / supported-with-caveat. The claim ‘Standard MAIC diagnostics do not answer whether an adjusted edge contributes to the requested target contrast’ is correct - effective sample size says nothing about network position. But the entry treats edge influence as a novel diagnostic and cites no prior art, and there is a lot. Rucker et al. 2020 define study importance exactly, not approximately, as the variance reduction from adding a study, ‘uniquely defined, easily calculated’, for both one-stage and two-stage NMA; Papakonstantinou et al. 2018 give the flow-decomposition proportion-contribution matrix; and, most directly, Yang et al. 2026 extend contribution analysis to component NMA using pseudo-paths and the component-level hat matrix, reporting that ‘The flow-weighted sum of pseudo-path-derived estimates matched exactly (within numerical tolerance) the overall component effect estimated by the CNMA model’. So exact evidence-contribution decomposition for component effects exists as of January 2026. What is not solved is contribution to a target-specific, population-adjusted cross-gap contrast. The approximation claim about cpaic itself is a code question. Cites doi:10.1186/s12874-026-02766-8: Provides an exact decomposition of which studies and comparisons contribute to each component effect in CNMA; does not cover population-adjusted, target-specific contrasts.. Cites doi:10.1186/s12874-020-01075-y: Gives an exact, uniquely defined per-study influence measure for a specified NMA estimate, defined for one-stage and two-stage models.. Cites doi:10.12688/f1000research.14770.3: The established flow-network proportion-contribution machinery that the component version generalises..
- GPT-5.6 Sol, technical and source-code lens
- open / supported-with-caveat. C1 and C2 are correct with qualification. The source explicitly describes the diagonal-W calculation as approximate for multi-arm studies because discomb() uses within-study covariance. For two-arm common-effect models with W fixed, the displayed linear influence is exact. Under random effects, cpaic does include the fitted tau-squared in W; remaining approximation arises if exact perturbation influence is defined to include re-estimation of tau, not because random-effects weighting is ignored. Such omissions can change rankings. C3 is algebraically possible and is demonstrated by the package tests: ESS measures within-edge weight dispersion, whereas target influence is a network-design leverage quantity and can be zero. C4 is correct because ordinary MAIC overlap and weight diagnostics do not evaluate downstream leverage for a requested cross-gap contrast. This is a useful diagnostic refinement, but not a defect in the fitted estimator itself.
- Decision path
-
R3:partially-solved(n=1) → R11:caveated(n=2)
How the evidence falls in time
2 papers in the reviewed corpus bear on this problem, published 2017 to 2025. Partial progress is on the record, and later work still calls this open. Both belong in view.
| Year | Says | Paper |
|---|---|---|
| 2017 | partly addresses it | Deriving percentage study weights in multi-parameter meta-analysis models: with application to meta-regression, network meta-analysis and one-stage individual participant data models |
| 2025 | confirms it is open | A leave-one-out algorithm for contribution analysis in component network meta-analysis |
Source
Derived from C:L1383, C:L1287-1289, scope:package-specific diagnostic of the reviewed corpus.