Structured elicitation is not applied to the cross-gap contrast
Statement
When the likelihood does not identify a cross-gap contrast, an explicitly justified prior can supply external identifying information under the Bayesian model, though it does not create likelihood identification; Goring et al. 2016 already describe inserting a distribution for the unknown treatment effect as the recommended approach for disconnected networks. Structured elicitation is likewise codified rather than missing: the NIHR reference protocol of Bojke, Soares and Claxton covers expert selection, level of elicitation, fitting, and pooling, and the STEER resources operationalize it for health care decision making. Software defaults are modeling conventions that may dominate a flat likelihood direction; calling a hidden default prior undocumented expert judgment overstates it, since a default need not represent anyone’s elicited belief. What is not standard is applying either apparatus to the cross-gap quantity itself, and reporting prior-only alongside posterior estimates so a reader can see how much of the answer is prior.
Why it is open
In a weakly identified disconnected model the prior determines the flat direction of the likelihood, so the number a reader sees can be mostly prior, and without a prior-only estimate beside the posterior there is no way to tell how much. The elicitation protocols that exist were built for health-economic parameters rather than for cross-gap treatment contrasts, which are harder to phrase in clinically interpretable terms and harder to check against an expert’s experience. Prior-data conflict diagnostics are also ill-posed for a parameter with no likelihood contribution across the gap, since there is nothing for the prior to conflict with.
What has been tried
| Work | What it contributes |
|---|---|
| Goring et al. 2016 | establishes inserting a distribution for the unknown treatment effect as the currently recommended approach for disconnected networks, so the elicited-prior bridge is textbook rather than a proposal |
| Bojke, Soares and Claxton, NIHR reference protocol for structured expert elicitation (Health Technology Assessment 25.37, 2021) | supplies the structured-elicitation protocol the source asked for, covering expert selection, level of elicitation, fitting, and pooling |
| Jankovic et al. 2025 (STEER) | open-access resources operationalizing structured expert elicitation for health care decision making |
| cpaic 0.1.0 (development version, choxos/cpaic@9d150e9) | unreleased development R package on GitHub, not on CRAN; records the complete prior specification on every fitted object and implements prior-only sampling, parameter-level prior-posterior comparison, and prior-sensitivity refits, but no structured cross-gap elicitation and no automatic prior-only versus posterior contrast table |
Probable solution or research direction
Apply the existing elicitation protocols to the cross-gap quantity: elicit it in clinically interpretable terms, separate location from uncertainty, and aggregate multiple experts where the context supports it rather than as a blanket requirement. Make prior-only versus posterior reporting a routine output; cpaic already supplies prior-only sampling, prior-posterior comparison, and prior-sensitivity refits, so the missing pieces are an automatic prior-only versus posterior contrast table and the elicitation step in front of it. Note that cpaic is an unreleased development package on GitHub and is not installable from CRAN, so even that partial capability is not available to a general user.
Verification
Verdict. Partially addressed. Part of this problem is addressed by existing work; the entry names which part.
The source titled this. “Expert elicitation as a bridge, and the hidden default prior” The title’s hidden-default-prior framing was rejected as rhetorical, and structured elicitation protocols were found already codified rather than open.
The source said. “When the data do not identify a cross-gap effect, expert information may be an appropriate source of identification provided elicitation is transparent: elicit clinically interpretable quantities, separate location from uncertainty, use multiple experts with structured aggregation, report prior-only and posterior estimates, and perform conflict and sensitivity analyses. A hidden default prior is still expert judgment, only undocumented.”
What was wrong with it. The source presented transparent elicitation as an open direction; the literature auditor found it already established, at doi:10.1002/jrsm.1204, https://www.ncbi.nlm.nih.gov/books/NBK571051/, and doi:10.1177/0272989X251343013, so the claim narrows to the cross-gap application. GPT-5.6 Sol rejected the hidden-default-prior framing as rhetorical, treated multiple experts and structured aggregation as context-dependent rather than universal requirements, and found that the cpaic development package already records the complete prior specification and provides prior-only sampling, prior-posterior comparison, and prior-sensitivity refits, though not structured cross-gap elicitation.
Dissent. codex dissented (overstated): C1 is defensible if identification means identification under external prior information, not identification by the likelihood. C2 is too rhetorical: a default prior encodes modeling judgment and may determine a flat-likelihood direction, but it need not represent expert elicitation or be undocumented. C3 turns good practices into universal requirements; multiple experts and structured aggregation are context-dependent, while prior-only versus posterior comparison is a reporting diagnostic rather than part of elicitation itself. C4 is an unverified routine-practice claim. cpaic already records complete priors and supplies prior-only sampling, prior-posterior plots, and prior-sensitivity refits, partially addressing the transparency problem.
Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).
- Verdict left unchanged
- verdict left unchanged Later work reports progress, but on the chronology review the second reviewer judged it ‘different-question’ at high confidence: The later paper operationalizes structured elicitation for a population-drift parameter in a connected but sparse network. That is a related transportability question, not elicitation of the unidentified cross-gap contrast in a disconnected network.
- Literature and prior-art check
- partially-solved / supported-with-caveat. prior_work is empty, which misses that the elicited-prior bridge is the textbook approach, not a proposal. Goring et al. 2016: ‘A currently recommended approach is to insert a distribution for the unknown treatment effect(s) into a network meta-analysis model of treatment effect.’ The elicitation half of the entry’s requirement list is also already codified: the Bojke, Soares and Claxton NIHR reference protocol for structured expert elicitation in health-care decision-making (2021) covers expert selection, level of elicitation, fitting and pooling; an ISPOR task force good-practices report on structured elicitation protocols and the open-access STEER resources (Jankovic et al. 2025) followed. So claims 1 and 3 restate established guidance rather than identifying a gap. What is defensibly open is narrower and, in my view, the real contribution here: cross-gap contrasts are rarely the elicited quantity, and prior-only estimates are not reported alongside posteriors so readers cannot see how much of the answer is prior. Claim 4 is an absence claim I cannot verify positively. Cites 10.1002/jrsm.1204: Establishes the elicited-distribution bridge as the already-recommended approach for disconnected networks. Cites https://www.ncbi.nlm.nih.gov/books/NBK571051/: Supplies the existing structured-elicitation protocol the entry asks for, covering expert selection, level of elicitation, fitting and pooling. Cites 10.1177/0272989X251343013: Current open-access operationalization of structured elicitation for HTA, showing the practice is standardized outside evidence synthesis.
- GPT-5.6 Sol, technical and source-code lens
- partially-solved / overstated. C1 is defensible if identification means identification under external prior information, not identification by the likelihood. C2 is too rhetorical: a default prior encodes modeling judgment and may determine a flat-likelihood direction, but it need not represent expert elicitation or be undocumented. C3 turns good practices into universal requirements; multiple experts and structured aggregation are context-dependent, while prior-only versus posterior comparison is a reporting diagnostic rather than part of elicitation itself. C4 is an unverified routine-practice claim. cpaic already records complete priors and supplies prior-only sampling, prior-posterior plots, and prior-sensitivity refits, partially addressing the transparency problem. Cites choxos/cpaic@9d150e9cad88f2e6fbcd94dbb92c3dee3b7c2dca: Records the full default prior specification and implements prior-only sampling, parameter-level prior-posterior comparison, and prior-sensitivity refits. It does not implement structured cross-gap elicitation or an automatic prior-only versus posterior contrast table..
- Decision path
-
R3:partially-solved(n=2) → R9:substantiated-single-overstatement:codex
How the evidence falls in time
3 papers in the reviewed corpus bear on this problem, published 2018 to 2024. The most recent paper to touch this reports progress on it, and every paper calling it open is older.
Does the later work answer the earlier work? Put to an independent reviewer with the older and newer claims side by side, it judged that the later work is about a different question and leaves the earlier gap untouched (high confidence). The later paper operationalizes structured elicitation for a population-drift parameter in a connected but sparse network. That is a related transportability question, not elicitation of the unidentified cross-gap contrast in a disconnected network. What remains: Structured elicitation of the cross-gap contrast itself in a disconnected network, including prior-only versus posterior reporting.
| Year | Says | Paper |
|---|---|---|
| 2018 | confirms it is open | Rapid methods including network meta-analysis to produce evidence in clinical decision support: a decision analysis |
| 2019 | confirms it is open | Creating historical controls using data from a previous line of treatment - two non-standard approaches |
| 2024 | partly addresses it | Sharing information across patient subgroups to draw conclusions from sparse treatment networks |
Source
Derived from C:L1142-1152 of the reviewed corpus.