Total bias is non-additive, so one-at-a-time analyses mislead
No. An omitted confounder and misclassification of the comparator’s outcome interacted by up to 0.16 on the log odds ratio scale: with both at their largest the joint bias was about half the sum of the separate biases, and with the confounder reversed the joint bias had the opposite sign to that sum. Measurement error in a matched covariate combined nearly additively with either. The joint bias exceeded the largest single-mechanism bias in 22 of 36 combinations, so one-at-a-time analyses neither add up to nor bound it. Read the study below.
Statement
Bias mechanisms interact on nonlinear estimand scales, so total bias need not equal the sum of population, bridge and measurement bias, and a set of one-at-a-time curves cannot establish joint robustness, though surviving each curve does not prove that cancellation is what saved the estimate. This now has direct evidence in the external-control setting: Ackerman et al. 2024 (doi:10.3389/fdsfr.2024.1423493) show that misclassified progression events and irregular assessment times occurring together may generate bias greater than the sum of their parts. Joint multiple-bias machinery also already exists outside population adjustment: Brendel, Torres and Arah (doi:10.1093/ije/dyad001) adjust uncontrolled confounding, selection bias and misclassification simultaneously via imputation and regression weighting, with a CRAN implementation, so the PAIC-specific gap is composition of those bias models with the population-adjustment step rather than multiple-bias composition itself. The joint region also has no reportable form. Threshold and tipping-point machinery computes a decision-invariant region in as many dimensions as there are perturbed quantities, but no scalar summary, visualization or reporting convention exists once more than two or three data points, contrasts or bias parameters move at once; the obstruction is stated to be display and interpretation rather than derivation. Candidate low-dimensional functionals such as the minimum-norm violation vector under a stated metric, the region’s inradius, or the fraction of a prespecified plausible bias set that preserves the decision have not been proposed, evaluated or standardized, so the joint robustness statement this entry shows to be necessary is currently one that cannot be printed.
Why it is open
Nonlinear link functions and marginalization make the map from bias parameters to the marginal estimand non-additive, so single-parameter curves do not span the joint bias surface, and cancellation is invisible by construction because each curve is computed with the other mechanisms held at their assumed null. Sequential corrections are order-dependent when mechanisms interact, so the output of one correction is not a valid input to the next, yet sequential correction is what current practice does. A joint bias model composed with the population-adjustment step requires a dependence structure among nonidentified parameters that neither the source IPD nor the aggregate comparator can inform, so that specification lands entirely on elicitation, and computing a joint surface costs far more than one-at-a-time curves. Attribution shares inherit the nonidentification of their inputs and describe a chosen model rather than recovering an underlying quantity.
What has been tried
| Work | What it contributes |
|---|---|
| Brendel, Torres and Arah 2023, Simultaneous adjustment of uncontrolled confounding, selection bias and misclassification in multiple-bias modelling, International Journal of Epidemiology 52:1220-1230 | Provides the joint, non-sequential multiple-bias machinery this entry asks for, adjusting uncontrolled confounding, selection bias and misclassification simultaneously via imputation and regression weighting as an alternative to one-bias-at-a-time adjustment. It has no population-adjustment step. |
| multibias R package (CRAN) | CRAN implementation of the simultaneous multiple-bias adjustment above, cited by the literature auditor as counter-evidence to the claim that no software composes several bias models jointly. |
| Ackerman et al. 2024, Measurement error and bias in real-world oncology endpoints when constructing external control arms, Frontiers in Drug Safety and Regulation | Measures non-additive interaction between two bias mechanisms in an external control arm, converting the entry’s non-additivity assertion into evidence. |
| cpaic, unreleased development package on GitHub (not on CRAN), commit 9d150e9 | Varies one common bound on component main-effect drift in bridge_fragility(), explicitly excluding interaction drift, and refits prior scales in prior_sensitivity(). It exports no general joint-bias QBA surface, so the entry’s request for a joint surface in this package is a roadmap feature rather than a defect. |
| Phillippo, Dias, Ades, Didelez and Welton 2018, Sensitivity of treatment recommendations to bias in network meta-analysis, Journal of the Royal Statistical Society Series A 181:843-867 | Derives bias-adjustment thresholds and invariant regions for a network meta-analysis recommendation, and states that the obstruction in higher dimensions is not deriving them but visualizing and interpreting them. Its worked outputs are one-dimensional threshold intervals and a two-dimensional polygon, so the joint region a multi-parameter bias analysis produces has no published reporting form. |
| Phillippo et al. 2019, Threshold Analysis as an Alternative to GRADE for Assessing Confidence in Guideline Recommendations Based on Network Meta-analyses, Annals of Internal Medicine 170:538-546 | Puts threshold analysis into guideline practice as a confidence assessment, with the nmathresh package on CRAN. What it delivers to a committee is per-datum invariant intervals, which is the one-dimensional case; it supplies no summary of a joint region over several simultaneously perturbed quantities. |
Probable solution or research direction
Compose the existing joint multiple-bias machinery with the population-adjustment step rather than reinventing it: take simultaneous imputation and weighting adjustment for confounding, selection and misclassification and make it operate on a weighted or standardized PAIC estimator, computed from an explicitly defined ideal target trial, a decision estimand and a causal diagram for each departure. Report joint and mechanism-specific results together so reinforcement and cancellation are visible, and state the dependence assumption used, including independence where that is the intended scenario. Present order-robust or Shapley-style summaries as descriptive allocations conditional on the assumed bias model, stating the fixed parameter vector or the joint distribution they average over.
Our study
Do bias mechanisms in an unanchored MAIC combine additively?
This study was designed against DIA-13 and bears on this entry in part. What it does not answer is stated below.
The question put to it. When an omitted confounder, comparator outcome misclassification and covariate measurement error act together on an unanchored MAIC, is their joint bias the sum of their separate biases, and does a one-at-a-time analysis bound it?
What it found. No. An omitted confounder and misclassification of the comparator’s outcome interacted by up to 0.16 on the log odds ratio scale: with both at their largest the joint bias was about half the sum of the separate biases, and with the confounder reversed the joint bias had the opposite sign to that sum. Measurement error in a matched covariate combined nearly additively with either. The joint bias exceeded the largest single-mechanism bias in 22 of 36 combinations, so one-at-a-time analyses neither add up to nor bound it.
Results.
- Refuting sentence fails: median interaction 30% of the summed separate biases where two or three mechanisms acted.
- Confounder by misclassification: interaction up to 0.163; other pairs at most 0.005.
- Joint bias beyond the largest single-mechanism bias in 22 of 36 combinations; mechanisms partly cancelled in 16.
Design.
- Estimand
- Marginal log odds ratio, B versus A, in the target (unanchored)
- Methods compared
- unanchored MAIC limit under combinations of three bias mechanisms
- Data-generating mechanism
- Population-level numerical study: exact bias of the MAIC limit over 45 combinations of three mechanisms, with interaction terms by inclusion-exclusion.
- Replicates
- Population-level calculation, 45 combinations, exact quadrature
What this does not answer. Population-level only; no bias-analysis procedure, false reassurance or tipping sets; three mechanisms; selection and correlated parameters not varied. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/DIA-13-biases-that-act-together · Protocol: registered before the run
Question. When several bias mechanisms act together on an unanchored MAIC, does a one-at-a-time threshold report predict a decision flip as well as a minimum-norm distance or the fraction of an elicited bias distribution that preserves the decision, and does the minimum-norm answer depend on the metric?
Design. DIA-13’s three-mechanism bias model on an unanchored MAIC; elicited box and copula distribution with independent, positive or negative dependence; true bias drawn from the elicited law or a wider one; one-at-a-time, box, ball and preservation-fraction summaries scored against the decision flip. 24 cells, 2000 replicates each.
The protocol is registered before any result is seen: read it.
Verification
Verdict. Partially addressed. Auditors agreed the problem is open and no counterevidence surfaced. Reopened by the full-text reading: 1 finding from 1 paper survived both the reader and an independent second reviewer, so the verdict moves from confirmed-open to partially-addressed, on a single finding.
The source said. “Total bias does not in general equal the sum of population, bridge and measurement bias: mechanisms interact and can cancel, so an estimate that survives every one-at-a-time sensitivity curve may be robust only by accident. A multiple-bias analysis must specify one joint bias model with preserved parameter dependence rather than independent after-the-fact corrections, and order-robust or Shapley-style attribution of total bias is description, not identification.”
What was wrong with it. The literature auditor found the claim that no software composes several bias models jointly wrong as written: the multibias package on CRAN implements simultaneous adjustment of three bias mechanisms (doi:10.1093/ije/dyad001), and what is missing is composition with the population-adjustment step. GPT-5.6 Sol narrowed three further claims: one-at-a-time analysis cannot establish joint robustness rather than proving cancellation, a multiple-bias analysis should state and explore dependence rather than preserve an unknown dependence (independence is a legitimate scenario and a deterministic grid needs no joint distribution), and Shapley attribution requires a joint parameter distribution only for expected or conditional summaries, not at a fixed bias-parameter vector.
Dissent. literature dissented (supported): Claims 1, 3, 4, 7 and 8 gain a real empirical citation the entry lacks: Ackerman et al. 2024 show, in exactly the external-control setting, that ‘When misclassified progression events and irregular assessment times occur simultaneously, they may generate bias that is greater than the sum of their parts’. That is a directly measured demonstration of non-additivity between two bias mechanisms in an ECA, and converts the entry’s assertion from theory to evidence. Claim 17 (‘No software composes several bias models jointly with the population-adjustment step’) survives, but should be worded precisely, because software that composes several bias models jointly does exist: the multibias R package on CRAN implements Brendel, Torres and Arah’s simultaneous adjustment of uncontrolled confounding, selection bias and misclassification via imputation and regression weighting (Int J Epidemiol 2023;52:1220-1230), explicitly as an alternative to sequential one-at-a-time correction. What is missing is composition with the population-adjustment step, not multiple-bias composition itself. Claims 15 and 16 (Shapley-style attribution is description, not identification) are correct.
Revised by the full-text reading. Applied full-text reading.
- merged-in
- absorbed a proposal the reading raised separately: High-dimensional decision-invariant regions lack reportable summaries 2 citation(s) added. The drafter disagreed with placing it here: Recorded as a merge, as directed, and the placement is defensible since QBA-22 is where the need for a joint statement is established. The scope is nonetheless wider than QBA-22: the reporting obstruction applies to any multi-parameter invariant region, including threshold analysis over data points and contrasts, which perturbs evidence rather than bias mechanisms, and the component drift surface in CMP-23. If the region-summary question is later worked on in its own right, it will not sit comfortably inside an entry about the non-additivity of bias mechanisms.
- Verdict changed
- confirmed-open -> partially-addressed 1 surviving finding from: Applying Quantitative Bias Analysis to Epidemiologic Data, S. New citations: 10.1007/978-3-030-82673-4
- Literature and prior-art check
- open / supported. Claims 1, 3, 4, 7 and 8 gain a real empirical citation the entry lacks: Ackerman et al. 2024 show, in exactly the external-control setting, that ‘When misclassified progression events and irregular assessment times occur simultaneously, they may generate bias that is greater than the sum of their parts’. That is a directly measured demonstration of non-additivity between two bias mechanisms in an ECA, and converts the entry’s assertion from theory to evidence. Claim 17 (‘No software composes several bias models jointly with the population-adjustment step’) survives, but should be worded precisely, because software that composes several bias models jointly does exist: the multibias R package on CRAN implements Brendel, Torres and Arah’s simultaneous adjustment of uncontrolled confounding, selection bias and misclassification via imputation and regression weighting (Int J Epidemiol 2023;52:1220-1230), explicitly as an alternative to sequential one-at-a-time correction. What is missing is composition with the population-adjustment step, not multiple-bias composition itself. Claims 15 and 16 (Shapley-style attribution is description, not identification) are correct. Cites 10.1093/ije/dyad001: Provides the joint (non-sequential) multiple-bias machinery and CRAN implementation (multibias) that claim 11 asks for, but without any population-adjustment step, so the PAIC-specific gap in claim 17 stands.. Cites 10.3389/fdsfr.2024.1423493: Empirical demonstration of non-additive interaction between two bias mechanisms in an external control arm; supports claims 1, 3, 4 and 7 with evidence rather than assertion..
- GPT-5.6 Sol, technical and source-code lens
- open / overstated. C1-C4, C6, C8-C11, and C13 are correct as possibility or conditional claims: if B(eta)=T(eta)-T(0), isolated mechanism biases need not sum unless T is additive, while a first-order Taylor expansion gives C2. C7 should say that one-at-a-time analysis cannot establish joint robustness, not that cancellation logically explains survival of each isolated curve. C12 should require stating and exploring dependence, not preserving an unknown dependence: independence can be a legitimate scenario, and a deterministic grid needs no joint distribution. C14 is strong causal-analysis practice, not a mathematical necessity. C15 is correct that Shapley allocation does not identify bias from data, but Shapley values exactly decompose a chosen model-based total. C16 is false in general: at a fixed bias-parameter vector, coalition values and an ordering average suffice; a joint distribution is needed only for expected or conditional Shapley summaries. C17 is a field-wide absence claim that cpaic source cannot establish. At cpaic 9d150e9, bridge_fragility() varies one scalar box bound on component main-effect drift and explicitly excludes interaction drift, while prior_sensitivity() only refits prior scales; there is no general joint-bias QBA export. Thus the cpaic gap is open, but C5 describes a roadmap feature rather than a correctness defect and the overall very-high-priority framing is inflated.
- Decision path
-
R5:default-open → R9:substantiated-single-overstatement:codex
How the evidence falls in time
8 papers in the reviewed corpus bear on this problem, published 2016 to 2026. Partial progress is on the record, and later work still calls this open. Both belong in view.
This problem is recurrent: three or more papers spanning at least eight years assert it independently. Sustained restatement by authors who mostly do not cite each other is the strongest evidence this reading can offer that a gap is real rather than one group’s framing.
| Year | Says | Paper |
|---|---|---|
| 2016 | confirms it is open | A threshold analysis assessed the credibility of conclusions from network meta-analysis |
| 2017 | confirms it is open | Sensitivity of treatment recommendations to bias in network meta-analysis |
| 2018 | confirms it is open | Allowing for informative missingness in aggregate data meta-analysis with continuous or binary outcomes: Extensions to metamiss |
| 2020 | confirms it is open | A Review of Two Regulatory Approved Anti-CD19 CAR T-Cell Therapies in Diffuse Large B-Cell Lymphoma: Why Are Indirect Treatment Comparisons Not Feasible? |
| 2021 | partly addresses it | Applying Quantitative Bias Analysis to Epidemiologic Data, Second Edition (Fox, MacLehose, Lash; Springer, Statistics for Biology and Health, 2021) |
| 2023 | confirms it is open | A Critical Appraisal of Matching-Adjusted Indirect Comparisons in Spinal Muscular Atrophy |
| 2026 | confirms it is open | Global sensitivity analysis for studies extending inferences from a randomized trial to a target population |
| 2026 | confirms it is open | Target Trial Emulation Meta-Analysis of Benzodiazepines For Out-of-Hospital Status Epilepticus in Adults |
Source
Derived from C:L635-643, C:L782-784, C:L786-794, C:L2144 of the reviewed corpus.