Measurement error and component misclassification are ignored
Partly. When the bridge is real, coding the shared component as a distinct agent removed that trial’s bridging information and, when it hit both bridging trials (up to 9.5% of replicates), made the target non-estimable, which a rank screen detects exactly; otherwise the estimate stayed unbiased with nominal coverage. The miscoding that left the row space intact, dropping an internal component, biased the target by about 0.8 SE and cut coverage to 0.83 given a miscoding. Probabilistic coding restored coverage (0.967 or more); coding sensitivity did too but was the whole line whenever a plausible coding broke the bridge. Read the study below.
Statement
Component PAIC as implemented takes covariates as observed values with no calibration or latent measurement-error model, and derives the component matrix by deterministically parsing treatment labels with no probability model for component assignment. Because a bridge exists exactly when two subnetworks share a component, a coding error can change the design matrix, its rank, and its row space, and so can create or remove shared-component paths rather than merely perturbing a coefficient. Not every miscoding changes connectivity, and measurement error in a variable that is not an effect modifier need not bias the estimand, so the concern is conditional on which covariate or component is uncertain.
Why it is open
Component coding is currently a deterministic analyst decision with no uncertainty attached, so the network topology is treated as known when it is inferred from trial reports. Covariate measurement error additionally biases the weighting step and attenuates the interaction terms that carry effect modification, and the direction depends on the unknown error structure, so a post hoc correction has nothing to key on. The mature general machinery for this problem, regression calibration, SIMEX, and Bayesian measurement-error and misclassification models, has not been applied to population-adjusted or component synthesis in any work the auditors could retrieve.
What has been tried
| Work | What it contributes |
|---|---|
| cpaic 0.1.0 (development version, choxos/cpaic@9d150e9) | unreleased development R package on GitHub, not on CRAN; takes covariates as observed with no calibration or latent measurement-error model, and builds the component matrix by deterministic label parsing with no probability model for assignment |
Probable solution or research direction
Introduce probabilistic component coding so that assignment uncertainty propagates into the design matrix, the row space, and therefore the estimand, and report how often the cross-gap contrast becomes non-estimable under plausible codings. Add covariate calibration models for effect modifiers using existing measurement-error machinery, prioritizing the modifiers that actually load the requested contrast. Because the concern is conditional, the first deliverable is a diagnostic that identifies which coding and measurement uncertainties would matter for a given target contrast.
Our study
Miscoding the shared component of a bridged network costs identifiability, not accuracy; miscoding an internal component biases the bridged estimate
The question put to it. When the component coding of a disconnected network bridged by a shared component is wrong, does the error perturb the estimate or change what is estimable, and do sensitivity or probabilistic coding give honest intervals?
What it found. Partly. When the bridge is real, coding the shared component as a distinct agent removed that trial’s bridging information and, when it hit both bridging trials (up to 9.5% of replicates), made the target non-estimable, which a rank screen detects exactly; otherwise the estimate stayed unbiased with nominal coverage. The miscoding that left the row space intact, dropping an internal component, biased the target by about 0.8 SE and cut coverage to 0.83 given a miscoding. Probabilistic coding restored coverage (0.967 or more); coding sensitivity did too but was the whole line whenever a plausible coding broke the bridge.
Results.
- Connectivity-changing miscoding: coverage 0.943 to 0.945 given a miscoding and estimability; non-estimable 0.4% to 9.5%.
- Connectivity-preserving miscoding: bias -0.095 to -0.105, coverage 0.826 to 0.868 given a miscoding (second null control failed).
- A false bridge (a distinct agent coded as the shared component) is not identified under the true coding and was not simulated.
Design.
- Estimand
- Effect of component B across the gap
- Methods compared
- deterministic coding with a rank screen, coding sensitivity over the plausible set, probabilistic coding (prior-weighted mixture)
- Data-generating mechanism
- Two subnetworks linked only by component A; subnetwork-2 codings miscoded independently with probability 0 to 0.3, either coding the shared component as a distinct agent (connectivity-changing) or dropping an internal component (connectivity-preserving); deterministic, sensitivity and probabilistic coding. 7 cells, 2000 replicates each.
- Replicates
- 2000 per cell, 7 cells
What this does not answer. One hand-built network, fixed-effect contrast-level fits, no covariate adjustment, synergy or Bayesian arm; independent miscoding within a small plausible set. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/CMP-06-component-miscoding-and-connectivity · Protocol: registered before the run
Verification
Verdict. Partially addressed. Auditors agreed the problem is open and no counterevidence surfaced. Reopened by the full-text reading: 1 finding from 1 paper survived both the reader and an independent second reviewer, so the verdict moves from confirmed-open to partially-addressed, on a single finding.
The source said. “Component methods ignore measurement error in covariates and misclassification in how interventions are coded into components. Component misclassification can create an apparent bridge between subnetworks where none exists, or destroy a real one.”
What was wrong with it. The source stated the absence at field level and implied any miscoding changes the bridge. GPT-5.6 Sol confirmed the absence at choxos/cpaic@9d150e9 but scoped it to that package, and qualified the mechanism: not every miscoding changes connectivity, and measurement error does not bias every estimand; the literature auditor searched and found no measurement-error or misclassification method for MAIC, STC, ML-NMR, or CNMA and no source asserting the absence, so recorded it as unverifiable at field level.
Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).
- Verdict changed
- confirmed-open -> partially-addressed 1 surviving finding from: Dismantling cognitive-behaviour therapy for panic disorder: . New citations: 10.1017/S0033291717003919
- Literature and prior-art check
- open / unverifiable. I searched PubMed and the web for measurement error or misclassification methods in MAIC, STC, ML-NMR or CNMA and found nothing that models covariate measurement error in the weighting step or treats component coding probabilistically. The general statistical machinery (regression calibration, SIMEX, Bayesian measurement-error models, misclassification priors) is of course mature, but I retrieved no application to population-adjusted or component synthesis, so I cannot supply a locator. The claim is a ‘no method exists’ claim and no source I found asserts the absence, so per calibration it is unverifiable rather than supported. The substantive observation that misclassifying a component changes network topology rather than a coefficient is a genuine and, as far as I can tell, unpublished point; it is the strongest part of the entry. Entry has no prior_work at all, which is a presentation weakness rather than a factual error.
- GPT-5.6 Sol, technical and source-code lens
- open / supported-with-caveat. C1 is source-confirmed at 9d150e9. Covariates enter cMAIC, cSTC, and cML-NMR as observed values without a calibration or latent measurement-error model, while build_C_matrix() deterministically parses the supplied labels with no probability model for component assignment. C2 is correct: changing a component assignment changes C and can consequently create or remove shared-component paths, alter rank, or change the row space, although not every miscoding changes connectivity. The cpaic-specific part of C3 is supported; the claim about every existing component NMA/PAIC package is outside this audit. Measurement error is not guaranteed to bias every estimand, for example when the mismeasured variable is not an effect modifier, but it can attenuate interaction estimates, leave true imbalance after weighting, and invalidate transport. This is decision-critical when coding or modifier measurement is genuinely uncertain; for carefully curated, effectively error-free inputs it is a lower-priority roadmap item.
- Grok 4.5, recency lens
- open / supported. Searched for probabilistic component coding, misclassification models that randomize the CNMA design/combination matrix, and covariate measurement-error correction inside component NMA/PAIC. General measurement-error and exposure-misclassification methods exist for meta-analysis and IPD (e.g., Jrsm work on misclassification adjustment), but none were found that (a) treat component assignment as uncertain topology or (b) ship in netmeta, multinma, gemtc, MBNMAdose, or a public cpaic. Queries: ‘component network meta-analysis misclassification’, ‘probabilistic component coding NMA’, ‘measurement error effect modifiers PAIC CNMA’, plus 2024–2026 journal and package scans. No merged PR or CRAN release notes close this gap. C1–C3 therefore remain open as stated; confidence is moderate because absence-of-evidence for a niche software feature is harder to prove exhaustively than a positive locator.
- Decision path
-
R5:default-open → R11:caveated(n=1)
How the evidence falls in time
32 papers in the reviewed corpus bear on this problem, published 2018 to 2026. Partial progress is on the record, and later work still calls this open. Both belong in view.
This problem is recurrent: three or more papers spanning at least eight years assert it independently. Sustained restatement by authors who mostly do not cite each other is the strongest evidence this reading can offer that a gap is real rather than one group’s framing.
| Year | Says | Paper |
|---|---|---|
| 2018 | partly addresses it | Dismantling cognitive-behaviour therapy for panic disorder: a systematic review and component network meta-analysis |
| 2018 | confirms it is open | Component network meta-analysis identifies the most effective components of psychological preparation for adults undergoing surgery under general anesthesia |
| 2018 | confirms it is open | An innovative and comprehensive technique to evaluate different measures of medication adherence: The network meta-analysis |
| 2019 | confirms it is open | Dismantling, personalising and optimising internet cognitive-behavioural therapy for depression: a study protocol for individual participant data component network meta-analysis |
| 2020 | confirms it is open | Interventions for adults with a history of complex traumatic events: the INCiTE mixed-methods systematic review |
| 2021 | confirms it is open | Non-pharmacological interventions for preventing delirium in hospitalised non-ICU patients |
| 2021 | confirms it is open | Understanding the effectiveness and underlying mechanisms of lifestyle modification interventions in adults with learning disabilities: protocol for a mixed-methods systematic review |
| 2022 | confirms it is open | Impact of program characteristics on weight loss in adult behavioral weight management interventions: systematic review and component network meta-analysis |
| 2022 | confirms it is open | Psychological interventions for inflammatory bowel disease: a systematic review and component network meta-analysis protocol |
| 2022 | confirms it is open | Pharmacological and non-pharmacological treatments for restless legs syndrome in end-stage kidney disease: a systematic review and component network meta-analysis (supplementary file and main article) |
| 2022 | confirms it is open | Complex speech-language therapy interventions for stroke-related aphasia: the RELEASE study incorporating a systematic review and individual participant data network meta-analysis |
| 2023 | confirms it is open | Impact of pharmacological interventions on intrapulmonary shunt during one-lung ventilation in adult thoracic surgery: a systematic review and component network meta-analysis |
| 2024 | confirms it is open | Lifestyle modification interventions for adults with intellectual disabilities: systematic review and meta-analysis at intervention and component levels |
| 2024 | confirms it is open | Exploring the Effectiveness of Self-Management Interventions in Type 2 Diabetes: A Systematic Review and Network Meta-Analysis |
| 2024 | confirms it is open | Components and Delivery Formats of Cognitive Behavioral Therapy for Chronic Insomnia in Adults: A Systematic Review and Component Network Meta-Analysis |
| 2024 | confirms it is open | Network meta-analysis examining efficacy of components of cognitive behavioural therapy for insomnia |
| 2024 | confirms it is open | Comparing interventions for early psychosis: a systematic review and component network meta-analysis |
| 2024 | confirms it is open | Pharmacological Treatments in Heart Failure With Mildly Reduced and Preserved Ejection Fraction: Systematic Review and Network Meta-Analysis |
| 2024 | confirms it is open | Comparative effects of behaviour change techniques using eHealth and mHealth in promoting dietary behaviour: protocol for a systematic review and component network meta-analysis |
| 2024 | confirms it is open | Effects of non-pharmacological interventions for adults with subjective cognitive decline: a network meta-analysis and component network meta-analysis |
| 2025 | confirms it is open | Pain science education and exercise interventions for people with knee or hip osteoarthritis: a systematic review, content and meta-analysis |
| 2025 | confirms it is open | Understanding the effectiveness and underlying mechanisms of lifestyle modification interventions in adults with learning disabilities: a mixed-methods systematic review |
| 2025 | confirms it is open | Dismantling behavioural weight management interventions: component network meta-analysis of randomised controlled trials and real-world services |
| 2025 | confirms it is open | Deconstructing resuscitation training for healthcare providers: a protocol for a component network meta-analysis |
| 2025 | confirms it is open | Impact of pulmonary rehabilitation programme design on effectiveness in COPD: a systematic review and component network meta-analysis |
| 2025 | confirms it is open | Dismantling task-sharing psychosocial interventions to personalize care for people affected by common mental disorders in poor resource settings |
| 2025 | confirms it is open | Gone with the Protocol: Searching for unpublished documents from unregistered clinical trials |
| 2026 | confirms it is open | Components of cognitive-behavioral interventions to prevent the onset of depression among the high-risk population: a systematic review and component network meta-analysis |
| 2026 | confirms it is open | Effectiveness of interventions to increase vaccine uptake: component network meta-analysis |
| 2026 | confirms it is open | Cognitive Behavioral Interventions for Children and Adolescents With Overweight or Obesity: A Systematic Review and Component Network Meta-Analysis |
| 2026 | confirms it is open | Mapping components of behavioural weight management interventions using electronic survey and component selection by expert consensus: the BE:COME Study |
| 2026 | confirms it is open | BEhavioural Weight Management: COMponents of Effectiveness (BE:COME) Synopsis |
Source
Derived from C:L1380, scope:field-level (component coding and covariate measurement); not addressed in cpaic of the reviewed corpus.