Cross-environment invariance as a screen for bridges and modifiers
A stability screen across observed trials cannot confirm a bridge and refutes it rarely. With 4 or 10 trials, drift visible across them that biased the gap estimate by 0.28 was flagged in only 20% to 24% of analyses, and drift confined to the gap never above the null rate; passing the screen did not improve coverage of the reported analyses. Screening marginal log odds ratios raised false abstention under a valid bridge from 0.11 to 0.19 to 0.24 through non-collapsibility; screening conditional or standardized effects removed that. Read the study below.
Statement
No invariant-causal-prediction screen for bridge or component assumptions exists in evidence synthesis, so the proposal stands. The nearest published precedent is Tanboga (arXiv:2604.18646), a transport meta-analysis with a precision-weighted sign-stability diagnostic and a two-condition abstention rule that declines to report a pooled estimate when stability is not supported; it operates within a connected evidence base and is a single-author preprint, so it does not certify a bridge across a disconnected gap. Two qualifications belong in the claim: a trial, era, country or subnetwork is not automatically a valid environment, since that requires harmonized variables, suitable exogenous shifts and adequate within-environment treatment support, and strong instability should flag a bridge for sensitivity analysis rather than trigger automatic rejection, because sampling error, measurement change, case mix and effect-scale non-collapsibility produce instability too.
Why it is open
The bridging assumption that matters concerns the gap where no data exist, so every stability check is necessarily performed somewhere else; passing the check is consistent with the bridge failing exactly where it is used, which makes the procedure able to refute but never to confirm. Invariance methods also need enough distinct environments to separate stable structure from noise, and evidence networks supply very few trials. The environment assumptions themselves, that studies differ in the right way and that an invariant set exists, cannot be verified from the network alone.
What has been tried
| Work | What it contributes |
|---|---|
| Tanboga, Stable Transport Meta-Analysis for Heterogeneous Cardiovascular Trials: A Nuisance-Anchor Framework with a Sign-Stability Diagnostic (2026) | Nearest published precedent for invariance-style screening with abstention in evidence synthesis: a precision-weighted sign-stability diagnostic with a two-condition abstention rule. Operates within a connected evidence base and says nothing about disconnected-gap bridges. |
Probable solution or research direction
Build invariance-based screening that estimates treatment or component effects separately by environment while letting baselines vary, and that flags bridges whose effects are unstable where they can be checked. Establish what counts as a valid environment in a trial network, given harmonization, exogenous shift and treatment support, and characterize how many environments are needed before the signal is usable. Present the output as screening and calibration rather than validation, routing failures into sensitivity analysis rather than into a pass/fail gate.
Our study
An invariance screen for a bridging assumption, scored as a classifier
The question put to it. Does instability across observed trials predict failure of a bridge used in a gap with no data, and does a screen on marginal effects fire under a valid bridge through non-collapsibility?
What it found. A stability screen across observed trials cannot confirm a bridge and refutes it rarely. With 4 or 10 trials, drift visible across them that biased the gap estimate by 0.28 was flagged in only 20% to 24% of analyses, and drift confined to the gap never above the null rate; passing the screen did not improve coverage of the reported analyses. Screening marginal log odds ratios raised false abstention under a valid bridge from 0.11 to 0.19 to 0.24 through non-collapsibility; screening conditional or standardized effects removed that.
Results.
- Primary confirmed: under a valid bridge with strong prognosis, marginal minus conditional abstention 0.077 (SE 0.016) with 4 trials and 0.113 (SE 0.017) with 10; standardized within 0.03 of conditional.
- Visible drift of 0.15 per unit (gap bias about 0.28): abstention 0.20 to 0.24; of 0.3 per unit (gap bias about 0.56): 0.50 to 0.67.
- Gap-only drift: abstention within Monte Carlo error of the valid-bridge rate while the gap estimate was biased by up to 0.60.
- Coverage among analyses that passed the screen differed from coverage among all analyses by at most 0.011.
- Controls: identical trials 0.099 to 0.104; collapsible risk-difference model, marginal screen 0.117; large drift 0.990 or more.
Design.
- Estimand
- Conditional log odds ratio in the gap environment
- Methods compared
- Cochran’s Q screen on marginal log odds ratios, on conditional log odds ratios, on standardized marginal log odds ratios
- Data-generating mechanism
- K trials with differing covariate laws and a gap environment; Cochran’s Q screen with abstention on marginal, conditional and standardized log odds ratios; valid bridge, drift visible in the trials, and drift confined to the gap; 20 cells and 3 controls, 1000 replicates each.
- Replicates
- 1000 per scenario, 23,000 in total
What this does not answer. One event-rate level, no measurement shift, Cochran’s Q only; environments valid by construction. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/ADJ-09-a-screen-that-can-refute-and-never-confirm · Protocol: registered before the run
Verification
Verdict. Confirmed open. Auditors agreed the problem is open and no counterevidence surfaced.
The source said. “Treating each trial, era, country or subnetwork as an environment, one can search for treatment-by-covariate or component-effect structures that remain stable while nuisance baselines vary, and reject or down-weight a candidate bridge whose effect varies strongly where it can be checked. The report labels invariant causal prediction speculative and calls the limitation of invariance-based bridge screening serious.”
What was wrong with it. The source said a candidate bridge should be rejected or down-weighted when its effect varies across environments; GPT-5.6 Sol narrowed that to flagging and sensitivity analysis, because instability has several innocent sources. The literature auditor also named the nearest precedent, arXiv:2604.18646, which the source did not cite.
- Literature and prior-art check
- open / supported. I found no invariant-causal-prediction screen for bridge or component assumptions in evidence synthesis, so the entry stands. The generic machinery exists (Peters, Buhlmann & Meinshausen’s invariant causal prediction; ‘Active Invariant Causal Prediction: Experiment Selection through Stability’, NeurIPS 2020), and one 2026 preprint does implement a stability screen inside transport meta-analysis: Tanboga, ‘Stable Transport Meta-Analysis for Heterogeneous Cardiovascular Trials: A Nuisance-Anchor Framework with a Sign-Stability Diagnostic’ (arXiv:2604.18646), which ‘incorporates a precision-weighted sign-stability diagnostic with a two-condition abstention rule to avoid reporting a single pooled estimate when stability is not supported’, evaluated in a pre-specified ADEMP simulation. That is the entry’s screen-and-abstain idea, but applied to pooling across connected trials, not to certifying a bridge across a disconnected gap; it is a single-author unrefereed preprint. The entry’s central logical point, that a stability check performed where data exist cannot license the gap where they do not, is correct and is not contradicted by anything I retrieved. Cites arxiv:2604.18646: Does NOT resolve this entry. It is the nearest published precedent for invariance-style screening with abstention in evidence synthesis, and the entry should cite it as such; it operates within a connected evidence base and says nothing about disconnected-gap bridges..
- GPT-5.6 Sol, technical and source-code lens
- open / supported-with-caveat. C1 and C2 are plausible for an adaptation of invariance methods, but ordinary invariant causal prediction targets invariant conditional laws or parent sets, not treatment-by-covariate interactions automatically. C3, C4, C7, and C8 capture the main limitations: observed invariance can be tested, while the causal environment assumptions and extrapolation across an unobserved gap cannot be established from those tests. C5 requires harmonized variables, suitable exogenous shifts, and enough within-environment treatment support; a trial, era, country, or subnetwork is not automatically a valid environment. Under C6, strong instability should flag the bridge and enter sensitivity analysis, not trigger automatic rejection or down-weighting without accounting for sampling error, measurement changes, case mix, and effect-scale non-collapsibility. C9 accurately reports the label. The adaptation and calibration problem remains open; its seriousness is conditional on using such a screen to support a decision-grade bridge.
- Grok 4.5, recency lens
- open / supported. Invariant causal prediction is a real multi-environment method for stable predictors/interactions, but aggressive search did not surface a peer-reviewed ITC/NMA/PAIC implementation that screens component or treatment bridges and is evaluated as operational practice. The report’s core epistemic claims match the method’s known limits: stability can be checked only where environments exist, so a pass never confirms the gap where the bridge is used; few trials supply few environments; environment assumptions are untestable from the network alone. That package of limitations means the problem is still open as stated (speculative screening tool, not validation). No RSM/SIM/MDM/JCE online-first 2024–2026 paper, arXiv stat.ME hit, or package PR closed invariance-based bridge screening for evidence networks. Queries: invariant causal prediction network meta-analysis; invariant prediction multi-trial effect modification; ICP transportability trials.
- Decision path
-
R5:default-open → R12:supported
Source
Derived from C:L1646-1647, C:L1659, C:L1708-1712 of the reviewed corpus.