No sensitivity or reporting standard for discretionary component coding
Statement
The component design matrix is fixed by discretionary coding decisions taken before any model is fitted. Graded features such as dose, session length, contact frequency and program duration are forced into binary indicators, sparse components are merged into related ones, and components present in nearly every arm are dropped because they generate no contrast. Every dropped or merged component remains part of the interventions actually delivered, so its effect is absorbed into the reference arm or into the retained coefficients, which then answer a different question from the one the report states. The reductions are large and documented: the BE:COME program reduced its expert-selected component list to 11 component areas by excluding anything ambiguous, unsuitable for a dichotomy or showing too little variation between programs, dropping frequency of weight monitoring outright because every intervention arm monitored weight, and then reduced those 11 to 7 on an identifiability screen, so that pre-assessment, multidisciplinary delivery, program duration and personalized physical activity advice left the model while remaining in every delivered program; an earlier component review of psychological preparation for surgery collapsed a seventh category, hypnosis, into relaxation because too few trials used it, and reported no analysis of what the merged coefficient then estimates. No rule, sensitivity analysis or reporting convention governs these decisions, and no published component network meta-analysis reports how its conclusions would change under a different coding. What is missing is both a statement of the estimand a component coefficient targets under a given coding and a procedure for propagating the coding choice into the reported intervals.
Why it is open
The choices are made before any model is fitted, so no model can price them: the likelihood conditions on the design matrix and the discarded alternatives never enter it, and propagating them requires refitting over a space of codings that somebody must first enumerate and that nobody has defined. Two requirements conflict. A coding fine enough to preserve what was delivered makes the design rank-deficient or weakly identified, and a coding coarse enough to be estimable changes the estimand, so the identifiability screens that force the coarsening are doing their job and the resulting shift has nowhere to be recorded. The estimand itself is unstated: what a component coefficient means when the reference arm silently contains the dropped content has not been written down, so there is no target for a sensitivity analysis to be a sensitivity about. Dichotomization of graded predictors is long known to be harmful in ordinary regression, but here it is not a nuisance-variable issue, because the dichotomized quantity is the estimand.
What has been tried
| Work | What it contributes |
|---|---|
| Gregg et al. 2026, Health Technology Assessment (BE:COME) | documents the component selection process for a behavioral weight management component network meta-analysis, excluding components that were ambiguous, unsuitable for a dichotomy or insufficiently variable, and records the exclusion reasons component by component |
| Jaiswal et al. 2025, Health Technology Assessment | the component analysis built on that selection; reduces the 11 shortlisted components to 7 on identifiability grounds, dropping pre-assessment, multidisciplinary delivery, program duration and personalized physical activity advice, with no sensitivity analysis over the retained coding |
| Freeman et al. 2018, Journal of Clinical Epidemiology | component network meta-analysis of psychological preparation for surgery that collapsed the Cochrane review’s seventh component, hypnosis, into relaxation because of the low number of trials including it; the merge is stated but its effect on what the relaxation coefficient estimates is not analyzed |
| Rucker, Petropoulou and Schwarzer 2020, Biometrical Journal | defines the treatment-by-component matrix as an analyst-supplied input to the model; the coding enters as data and is never estimated, so nothing in the fit records its arbitrariness |
| Petropoulou et al. 2023, BMC Medical Research Methodology | supplies the estimability screening that forces components and interactions out of the model, the mechanism that converts an identifiability problem into an unrecorded estimand change |
| Michie et al. 2013, Annals of Behavioral Medicine | the Behaviour Change Technique Taxonomy v1 of 93 hierarchically clustered techniques, a standard vocabulary in which a candidate component list and its reduction could be expressed and versioned |
| Hoffmann et al. 2014, BMJ (TIDieR) | requires interventions to be described in enough detail to replicate them, including materials, procedures, provider, mode, intensity and tailoring; the reporting precedent a component coding standard would extend to the synthesis stage |
| Royston, Altman and Sauerbrei 2006, Statistics in Medicine | documents the cost of dichotomizing continuous predictors in regression, including loss of information and cut-point dependence; the general result whose component-model analogue has not been quantified |
Probable solution or research direction
State the estimand explicitly: define the component coefficient as a contrast between two regimens that differ in the coded component and agree on the rest of the coded design, with the uncoded content left as it was in the contributing arms, and require the report to name the content absorbed into the reference. Make the coding a versioned, published object: the full candidate component list, the rule applied to each one, retained, dichotomized at a stated cut, merged into which component, or dropped for which reason, and the identifiability verdict that forced any reduction, expressed in an existing intervention ontology and intervention-reporting vocabulary. Run the coding multiverse: refit over a prespecified set of alternative codings, different cuts for graded components, alternative merges, and restoration of near-universal components as a shared term, and report the range of each component effect and of the resulting hierarchy, as analytic multiverses are already reported for evidence-set assembly. Quantify the cost by simulation: generate data with graded component effects, code them under the rules seen in practice, and report the bias in the retained coefficients as a function of the dose spread within a component and the prevalence of the dropped ones.
Verification
Verdict. Confirmed open. One reader proposed it and the reviewer accepted it as a genuine estimand problem, distinct from ontology availability, component misclassification and post hoc interaction selection, while noting that its universal claims would ultimately need a broader review of component network meta-analysis practice; the evidence is one detailed applied program plus a second component review that merged a component for sparsity.
Source
Derived from reading:cnma-06 of the reviewed corpus.