No certainty-of-evidence framework rates component effects
Statement
GRADE for network meta-analysis and its CINeMA implementation are defined for a treatment contrast: within-study bias is aggregated over the studies contributing to that contrast, imprecision is judged against a clinically important range for it, incoherence comes from its loops, and indirectness from the studies informing it. A component main effect has none of those referents. It is a linear functional of the whole design matrix, no single study estimates it, every contributing study carries several other components, and the decision usually concerns a combination of components whose certainty is not a function of the individual components’ ratings. Reporting-bias machinery has the same problem: comparison-adjusted funnel plots, Egger-type tests and ROB-ME all operate on treatment comparisons and have no component-level analogue. Applied reviews therefore improvise or omit: one built a bespoke modified GRADE anchored on each direct comparison’s proportional contribution to the component estimate and deferred the justification to unpublished work, one attempted GRADE, found it highly subjective for a component synthesis and abandoned it in deviation from its protocol, one judged GRADE inappropriate and drew no certainty conclusions at all, and others applied CINeMA to the standard network contrasts while leaving the component estimates, which are the primary output, unrated. A Cochrane protocol and a Cochrane overview that used component network meta-analysis both state the position in terms: there is very little guidance and no consensus on the preferred method for evaluating the certainty of component network meta-analysis evidence, and consensus is still needed on how to assess certainty and possible publication bias. The reporting-bias domain is the part of the framework with no component-level input at all: funnel plots, comparison-adjusted funnel plots, Egger-type regressions, network selection models and ROB-ME all require a study-level effect estimate paired with its standard error for a single treatment contrast, while a component main effect is a linear functional of the whole design matrix that no single study estimates, so there is no effect-versus-precision pair to plot, regress or model. Applied component analyses therefore either omit the assessment entirely or borrow judgments made at the pairwise contrast level in contributing reviews and assume they transfer to the component coefficients, and what is needed is a definition of the small-study or selection-mechanism null at component level together with evidence on how selective publication of whole multicomponent interventions propagates through an additive design matrix into individual component effects.
Why it is open
The unit a certainty framework rates has to be the unit the evidence attaches to, and for component models these differ: trials randomize regimens, the model reports components, and until recently there was no way even to say how much of a component estimate came from which comparison. That input has now arrived, since Yang et al. 2026 (doi:10.1186/s12874-026-02766-8) define pseudo-paths and a flow allocation that decompose a component effect into study-level contributions, but a contribution measure is not a rating rule and supplies only two of the six domains. Two domains resist transfer for structural reasons rather than for want of effort: incoherence has no loop analogue at the component level, because the additivity constraint replaces the loop as the testable restriction and cannot be tested at all in a disconnected network, and imprecision needs a clinically important range for a component increment, a quantity nobody elicits. Rating a combination is a further step, since it is a joint statement about several component effects plus the assumption that they add, and no rule composes component ratings into a rating for the package that would actually be delivered. Because component analyses are commissioned to say which parts of a complex intervention to fund, an unrated primary output is a decision-relevant hole rather than a documentation gap, and two reviews of the same evidence would grade it differently or not at all.
What has been tried
| Work | What it contributes |
|---|---|
| Nikolakopoulou et al. 2020, PLOS Medicine | Defines CINeMA, the six-domain confidence framework for network meta-analysis, and keys the within-study-bias and indirectness judgments to the percentage contribution matrix. Every judgment is made for a treatment contrast; nothing in it is defined for a component effect or a component combination. |
| Brignardello-Petersen et al. 2018, Journal of Clinical Epidemiology | Advances the GRADE approach for rating certainty in every pairwise comparison of a network, including when the indirect estimate need not be rated and how incoherence is handled at the comparison level. All four advances are stated for a contrast. |
| Puhan et al. 2014, BMJ | The original GRADE working group approach to rating the quality of treatment effect estimates from a network meta-analysis, from which the contrast-level structure that component models cannot inherit derives. |
| Papakonstantinou et al. 2018, F1000Research | Derives the proportion contribution matrix that CINeMA uses to attribute study-level bias and indirectness to a network contrast. It is defined over direct comparisons, so it returns nothing usable for a component parameter. |
| Yang et al. 2026, BMC Medical Research Methodology | Extends the shortest-path contribution approach to component network meta-analysis through pseudo-paths and a flow-allocation algorithm, giving the first quantitative decomposition of a component effect into study contributions, with the flow-weighted sum reproducing the fitted component effect. Supplies an input a certainty framework needs, not the framework. |
| Rucker, Petropoulou and Schwarzer 2020, Biometrical Journal | Defines additive and interaction component network meta-analysis and implements both in netmeta, so the estimand that needs a certainty rating is precisely specified even though no rating rule exists for it. |
| O’Malley et al. 2025, Cochrane Database of Systematic Reviews (protocol CD016145) | States that at present there is very little guidance and no consensus on the preferred method for evaluating the certainty of component network meta-analysis evidence, and proceeds without one. |
| Hartmann-Boyce et al. 2021, Cochrane Database of Systematic Reviews (CD013229.pub2) | Cochrane overview and network meta-analysis of behavioral smoking-cessation interventions using component models; states that consensus is still needed on how to manage, evaluate and present component analyses within Cochrane reviews, explicitly including methods for assessing certainty and possible publication bias. |
| Nunns et al. 2025, Journal of Medical Internet Research | systematic review with component network meta-analysis of digital weight-loss interventions; records that because of the nature of component network meta-analysis it was unable to test for the presence of publication bias, so the component estimates it reports carry no reporting-bias assessment |
| Hartmann-Boyce et al. 2022, Obesity | component network meta-analysis of behavioral weight management program characteristics; states that there is no way to test for publication bias in component network meta-analysis and that it cannot be ruled out, an independent statement of the same absence in one of the largest applied component analyses |
| Chaimani and Salanti 2012, Research Synthesis Methods | the comparison-adjusted funnel plot, the network-level small-study-effects diagnostic in routine use; it plots each study’s treatment-contrast effect against its precision after centering on the comparison-specific summary, so it requires a study-level estimate of a single contrast and has no component-level analogue |
| Mavridis et al. 2014, Statistics in Medicine | a selection model for publication bias across a full network, modeling the probability that a study is published as a function of its contrast-level effect and precision; the selection mechanism is parameterized on treatment comparisons, so there is no component-level version to fit |
| Page et al. 2023, BMJ (ROB-ME) | structured assessment of risk of bias due to missing evidence in a synthesis; operates on the synthesis of one treatment comparison and supplies no rule for judging missing evidence about an incremental component effect |
Probable solution or research direction
Start from what the contribution machinery now makes available: define a component-level analogue of the percentage contribution matrix from the pseudo-path decomposition, then key within-study bias and indirectness to it exactly as CINeMA does for contrasts. Replace the loop-based incoherence domain with an additivity domain that downgrades on the magnitude of the nested lack-of-fit statistic where additivity is testable, and applies a fixed downgrade where the network is disconnected and the assumption cannot be tested at all. Elicit clinically important ranges for component increments so imprecision has a referent, or state in the instrument that the domain is unratable and require that to appear in the summary of findings. Derive a composition rule that turns a set of component ratings plus the additivity judgment into a rating for a delivered combination, and require component reviews to publish a summary-of-findings table for the combinations they recommend. Test the resulting instrument for inter-rater reliability on the published component reviews that improvised their own ratings, which provide a ready comparison set with known disagreement.
Verification
Verdict. Confirmed open. Two independent readers proposed this and the reviewer accepted both, holding that component effects do not map onto contrast-level CINeMA contributions and that contribution calculations alone supply no rules for rating a component effect or a delivered combination. The two supporting quotes are direct statements of absence by a Cochrane protocol and by a Cochrane overview that used component network meta-analysis, corroborated by five applied reviews that each improvised a bespoke rating or omitted one.
Revised by the full-text reading. Applied full-text reading.
- merged-in
- absorbed a proposal the reading raised separately: No reporting-bias or small-study-effects diagnostic for component estimates 5 citation(s) added
Source
Derived from reading:cnma-01, reading:cnma-04 of the reviewed corpus.