Committees get no one-page statement of what an estimate rests on
Source note. This entry derives in part from a source document whose inline citations were ChatGPT interface tokens rather than references, and so resolve to nothing. Claims below are attributed to that document and were checked independently where possible; anything that could not be independently sourced is marked in the verification trail.
Statement
No agency template requires a single committee-facing page that states, together, what is being estimated, for which population, what connects disconnected evidence, where overlap is weak, which results are prior- or bridge-driven, and how much assumption violation would change the decision. Parts of this already exist: NICE DSU TSD 7 provides a pro forma reviewer checklist covering the decision problem, evidence structure, assumptions, sensitivity and absolute outcomes, TSD 18 requires the target population for the decision problem to be stated explicitly, and threshold analysis (10.7326/M18-3542) already reports how much the evidence could change before the recommendation changes. What is missing is that none of it is consolidated into one required artifact, and that no method supplies the data-driven versus prior-driven versus bridge-driven decomposition the source asks for.
Why it is open
The answers are spread across a long technical appendix and several separate checklists, and no template forces them into one place where inconsistencies would become visible; because the artifact is never required, its absence is never penalized. Committees include clinicians and lay members who cannot extract these answers from a statistical appendix, so translation falls on whoever is least equipped to do it. The decomposition into data-driven, prior-driven and bridge-driven contributions has no unique additive definition, so it must be operationally defined before it can be required.
What has been tried
| Work | What it contributes |
|---|---|
| Phillippo et al. 2019 (NICE technology appraisals review) | Reviews NICE appraisals using population adjustment; reports that committees received such results with caution and looked for greater cost effectiveness to minimize decision risk, and that 89% (16/18) of applications were unanchored. |
| Jiang et al. 2025 | Shows that without a clearly defined target population the results of an indirect comparison lack applicability for guiding decisions, sourcing the requirement that a committee summary name the population. |
| Phillippo et al. 2019 (threshold analysis for guideline developers) | Presents threshold analysis as a way to tell decision makers how much the evidence could change before the recommendation changes, which is the sensitivity-hierarchy element of the proposed page. |
| NICE DSU Technical Support Document 7 (2012) | Consolidates many committee-relevant questions into a pro forma checklist covering target population, treatments, evidence structure, assumptions, sensitivity and absolute outcomes. Read by the code auditor from a document corpus at the given commit. |
Probable solution or research direction
Assemble the page from what exists rather than design a new framework: take the TSD 7 checklist items and the TSD 18 population-adjustment requirements, add overlap diagnostics, a plainly stated bridge, the parameter provenance for any quantified residual bias, and a decision-sensitivity line reported as a threshold analysis. Define operationally what counts as prior- or bridge-driven before requiring it to be reported. Treat the page as a required complement to the technical appendix rather than a substitute for it.
Verification
Verdict. Partially addressed. Part of this problem is addressed by existing work; the entry names which part.
The source said. “No appraisal process requires a committee-facing one-page assumptions map, diagnostics summary and sensitivity hierarchy, so the nine questions that decide whether an ITC can support a decision are never answered in one place: what is being estimated, for which population, which comparisons are randomized, what connects disconnected evidence, which variables are assumed to remove population differences, where overlap is weak, what is data-driven rather than prior- or bridge-driven, which residual biases were quantified and from what parameter sources, and how much assumption violation changes the decision. Interpretability therefore rests on the individual reviewer, and methods that are easy to narrate are favored over methods whose assumptions are weaker but whose explanation is harder.”
What was wrong with it. The source asserted, without attribution, that HTA committees remain skeptical of weakly justified unanchored analyses; the literature auditor sourced it to Phillippo et al. 2019 (10.1017/S0266462319000333), which found that committees received population-adjusted results with caution and looked for greater cost effectiveness to minimize decision risk. GPT-5.6 Sol found the questions already partly consolidated by the TSD 7 checklist and TSD 18, and rejected as unsupported the claims that one page is inherently more informative than an appendix, that MAIC is popular because it is easy to narrate, and that narrative simplicity implies stronger assumptions.
Dissent. literature dissented (open): I found no reporting instrument, agency template or published proposal for a one-page committee-facing assumptions map answering these nine questions, and Farinasso et al. 2025 and Truong et al. 2023 both document that the underlying information is not reported consistently anywhere, let alone in one place. Two of the entry’s sub-claims are now sourceable rather than unsourced. Phillippo et al. 2019 (IJTAHC, doi 10.1017/S0266462319000333) reviewed NICE TAs and found ‘Appraisal committees received results of population-adjusted analyses with caution and typically looked for greater cost effectiveness to minimise decision risk’, supporting HTA skepticism toward weakly justified unanchored analyses; 89% (16/18) of those applications were unanchored. Jiang et al. 2025 supports the specific need to state the target population plainly, since failing to do so permits opposing conclusions from the same data. Caveat: the ‘sensitivity hierarchy’ element is partly pre-existing, since Phillippo et al. 2019 (Ann Intern Med, doi 10.7326/M18-3542) proposed threshold analysis precisely to tell committees how robust a recommendation is. The claims that MAIC is popular because it is easier to narrate, and that narrative simplicity hides stronger assumptions, are plausible but I found no source measuring either. codex dissented (overstated): A one-page committee artifact of the exact proposed form may be absent, but it is incorrect that the relevant questions are never consolidated or that no template exists. TSD 7 already provides a pro forma decision-facing checklist, and TSD 18 adds PAIC-specific reporting for target population, overlap, weights, uncertainty, and systematic error. Neither source makes such a page mandatory or supplies a complete data-versus-prior-versus-bridge decomposition, so a communication gap remains. C3 and C5 are unsupported comparative claims: one page is not inherently more informative, and fastest route is not a statistical conclusion. C7-C9 are reasonable reporting requirements, although the amount that is data-, prior-, or bridge-driven may lack a unique additive decomposition and must be operationally defined. C10 is an unsupported causal claim. C11 is false generally because assumption strength depends mainly on anchored status, identification, overlap, and model restrictions, not narrative simplicity or computational complexity. C12 is an institutional-prevalence claim outside this audit.
- Literature and prior-art check
- open / supported-with-caveat. I found no reporting instrument, agency template or published proposal for a one-page committee-facing assumptions map answering these nine questions, and Farinasso et al. 2025 and Truong et al. 2023 both document that the underlying information is not reported consistently anywhere, let alone in one place. Two of the entry’s sub-claims are now sourceable rather than unsourced. Phillippo et al. 2019 (IJTAHC, doi 10.1017/S0266462319000333) reviewed NICE TAs and found ‘Appraisal committees received results of population-adjusted analyses with caution and typically looked for greater cost effectiveness to minimise decision risk’, supporting HTA skepticism toward weakly justified unanchored analyses; 89% (16/18) of those applications were unanchored. Jiang et al. 2025 supports the specific need to state the target population plainly, since failing to do so permits opposing conclusions from the same data. Caveat: the ‘sensitivity hierarchy’ element is partly pre-existing, since Phillippo et al. 2019 (Ann Intern Med, doi 10.7326/M18-3542) proposed threshold analysis precisely to tell committees how robust a recommendation is. The claims that MAIC is popular because it is easier to narrate, and that narrative simplicity hides stronger assumptions, are plausible but I found no source measuring either. Cites 10.1017/S0266462319000333: Sources the previously unsourced claim that HTA bodies remain skeptical of weakly justified unanchored analyses.. Cites 10.1017/rsm.2025.10: Sources the claim that a committee-facing summary must state for which population the estimate applies..
- GPT-5.6 Sol, technical and source-code lens
- partially-solved / overstated. A one-page committee artifact of the exact proposed form may be absent, but it is incorrect that the relevant questions are never consolidated or that no template exists. TSD 7 already provides a pro forma decision-facing checklist, and TSD 18 adds PAIC-specific reporting for target population, overlap, weights, uncertainty, and systematic error. Neither source makes such a page mandatory or supplies a complete data-versus-prior-versus-bridge decomposition, so a communication gap remains. C3 and C5 are unsupported comparative claims: one page is not inherently more informative, and fastest route is not a statistical conclusion. C7-C9 are reasonable reporting requirements, although the amount that is data-, prior-, or bridge-driven may lack a unique additive decomposition and must be operationally defined. C10 is an unsupported causal claim. C11 is false generally because assumption strength depends mainly on anchored status, identification, overlap, and model restrictions, not narrative simplicity or computational complexity. C12 is an institutional-prevalence claim outside this audit. Cites choxos/NICE-DSU-TSD-RAG@75abcb9c226a9f8fa7a0a4ffeba42874f1712ecb: Consolidates many committee-relevant questions into a pro forma checklist covering the target population, treatments, evidence structure, assumptions, sensitivity, and absolute outcomes.. Cites choxos/NICE-DSU-TSD-RAG@75abcb9c226a9f8fa7a0a4ffeba42874f1712ecb: Adds population-adjustment-specific reporting requirements and explicitly requires the decision target population..
- Decision path
-
R3:partially-solved(n=1) → R9:substantiated-single-overstatement:codex
How the evidence falls in time
9 papers in the reviewed corpus bear on this problem, published 2013 to 2026. Partial progress is on the record, and later work still calls this open. Both belong in view.
It is also being closed in pieces: partial results come from more than one paper in more than one year, each covering a different part.
| Year | Says | Paper |
|---|---|---|
| 2013 | partly addresses it | Evidence synthesis for decision making 7: a reviewer’s checklist |
| 2014 | partly addresses it | Evaluating the quality of evidence from a network meta-analysis |
| 2017 | partly addresses it | Approaches to interpreting and choosing the best treatments in network meta-analyses |
| 2019 | partly addresses it | Threshold Analysis as an Alternative to GRADE for Assessing Confidence in Guideline Recommendations Based on Network Meta-analyses |
| 2023 | partly addresses it | The use of network meta-analysis in updating WHO living maternal and perinatal health recommendations |
| 2024 | confirms it is open | A systematic survey of 200 systematic reviews with network meta-analysis (published 2020-2021) reveals that few reviews report structured evidence summaries |
| 2024 | confirms it is open | Physicians found an interactive tool displaying structured evidence summaries for multiple comparisons understandable and useful: a qualitative user testing study |
| 2024 | confirms it is open | The impact of Cochrane Reviews that apply network meta-analysis in clinical guidelines: A systematic review |
| 2026 | confirms it is open | Network meta-analyses in IBD: pitfalls and promise for clinicians. |
Source
Derived from A:L78, A:L221, C:L2450-2464 of the reviewed corpus.