No reporting standard covers what population adjustment needs
Source note. This entry derives in part from a source document whose inline citations were ChatGPT interface tokens rather than references, and so resolve to nothing. Claims below are attributed to that document and were checked independently where possible; anything that could not be independently sourced is marked in the verification trail.
Statement
Generic baseline-covariate reporting is already mandated: CONSORT 2025 item 25 requires a table of baseline demographic and clinical characteristics for each group, and CONSORT presents itself as a minimum set of items for a trial report (doi:10.1371/journal.pmed.1004587). What no standard specifies is what population adjustment needs: which covariates, dispersion measures, any joint or correlation structure, and reporting on the analysis set that matches the reported effect estimate. In practice the adjustment set is therefore bounded by what the comparator publication happened to print; a review of 18 NICE technology appraisals using population adjustment lists availability among the bases on which covariates were selected (Phillippo et al. 2019, doi:10.1017/S0266462319000333).
Why it is open
The analyst does not control the comparator publication, so the moments needed for balancing are supplied or withheld by a third party with no stake in the later comparison. A CONSORT-style baseline table does not close the gap, because what is required varies with the estimand and the model, and marginal means and proportions can be insufficient when joint distributions, nonlinear terms or interactions matter. No journal, regulator or registry mandates that PAIC-specific increment, and how often it is missing has not been quantified, though reviews of PAIC practice report inconsistent and incomplete covariate reporting (Truong et al. 2023, doi:10.1002/jrsm.1653). The nearest thing to a PAIC reporting standard governs the analyst’s own reporting of the comparison rather than what the comparator publication must carry (Ishak et al. 2025, doi:10.1007/s40273-025-01503-1).
What has been tried
| Work | What it contributes |
|---|---|
| Phillippo et al. 2019, International Journal of Technology Assessment in Health Care | Review of 18 NICE technology appraisals using population adjustment; reports that covariates were included on the basis of availability, expert opinion, effective sample size, statistical significance or cross-validation, sourcing the availability-bounded adjustment set. |
| CONSORT 2025 statement, PLOS Medicine | Trial reporting standard whose item 25 mandates a baseline demographic and clinical characteristics table per group; establishes that a mandated minimum baseline reporting set does exist, without specifying what population adjustment needs. |
| Harari et al. 2023, Research Synthesis Methods | Network meta-interpolation: adjusts for effect modification using subgroup analyses reported in aggregate publications, with standard errors, instead of treating the reported covariate set as a fixed constraint. |
| Truong et al. 2023, Research Synthesis Methods | Methodological review of PAIC practice documenting inconsistent reporting and incomplete covariate information; cited by the recency auditor as support that partial reporting is a live obstacle, without giving a prevalence estimate for the specific summaries PAIC needs. |
| Ishak et al. 2025, PharmacoEconomics | Framework for reliable, transparent and reproducible PAICs; the closest existing thing to a PAIC reporting standard, but it governs the analyst’s reporting rather than the comparator publication’s covariate summaries. |
Probable solution or research direction
Define and benchmark the PAIC-specific increment over CONSORT, namely which covariates, which dispersions, any joint or correlation structure, and reporting on the analysis set matching the reported effect estimate, and pursue it as an addition to trial reporting guidance rather than as an analyst-side workaround. Quantify how often that increment is missing, which no auditor found measured, so the case for a standard rests on evidence rather than assertion. In the meantime, methods that build effect-modifier adjustment out of the subgroup information aggregate publications do report already exist and should be extended (network meta-interpolation, Harari et al. 2023, doi:10.1002/jrsm.1608); treating unreported summaries as missing data propagates uncertainty only once identifying assumptions, external information or priors are added, so partial identification and sensitivity analysis are the honest fallback.
Verification
Verdict. Unverifiable. Auditors disagreed and the disagreement was evidence-bearing on both sides.
The source titled this. “No minimum covariate-reporting standard for aggregate-data trials” The title says no minimum covariate-reporting standard exists, but CONSORT 2025 item 25 already mandates a baseline characteristics table for each group.
The source said. “The covariate summaries that population adjustment needs are frequently absent from the comparator publication, or reported for a different analysis set, and no benchmarked minimum reporting set exists that would require them. Which covariates can be balanced is therefore decided by what a journal printed rather than by what causal reasoning requires.”
What was wrong with it. The claim was unattributed in the source and stated flatly that no benchmarked minimum reporting set exists; the literature auditor read CONSORT 2025 and found item 25 mandates baseline data (doi:10.1371/journal.pmed.1004587), so only the narrower PAIC-specific version survives, and it is now sourced to Phillippo et al. 2019 (doi:10.1017/S0266462319000333). GPT-5.6 Sol added that there is no universal finite set of summaries every PAIC requires, because the requirement depends on the estimand, the effect-modification basis, the link and outcome models and the weighting method.
Dissent. literature dissented (overstated): This entry is source_unsourced, so I judged it against the literature. I read the CONSORT 2025 full text. It does contain a mandated baseline-covariate reporting item, item 25 ‘Baseline data’, and the statement describes itself as ‘a minimum set of items to be included in a report of a randomised trial’, endorsed by ICMJE, WAME and CSE, with the same item present as CONSORT 2010 item 15. Claim 2 as written (‘No benchmarked minimum reporting set of baseline covariate summaries currently exists’) is therefore contradicted at face value. What CONSORT does not do is specify which covariates, require dispersion measures, require any joint or correlation structure, or require alignment with the analysis set used for the reported effect estimate; that narrower version survives and I found no PAIC-benchmarked standard. Claim 1 is supportable and I can source it: Phillippo et al. 2019 IJTAHC reviewed 18 NICE technology appraisals using population adjustment and found availability was among the bases on which covariates were selected. Harari et al. 2023 network meta-interpolation exists precisely because full effect-modifier summaries are not uniformly reported. Claim 3 remains without a prevalence estimate.
- Literature and prior-art check
- open / overstated. This entry is source_unsourced, so I judged it against the literature. I read the CONSORT 2025 full text. It does contain a mandated baseline-covariate reporting item, item 25 ‘Baseline data’, and the statement describes itself as ‘a minimum set of items to be included in a report of a randomised trial’, endorsed by ICMJE, WAME and CSE, with the same item present as CONSORT 2010 item 15. Claim 2 as written (‘No benchmarked minimum reporting set of baseline covariate summaries currently exists’) is therefore contradicted at face value. What CONSORT does not do is specify which covariates, require dispersion measures, require any joint or correlation structure, or require alignment with the analysis set used for the reported effect estimate; that narrower version survives and I found no PAIC-benchmarked standard. Claim 1 is supportable and I can source it: Phillippo et al. 2019 IJTAHC reviewed 18 NICE technology appraisals using population adjustment and found availability was among the bases on which covariates were selected. Harari et al. 2023 network meta-interpolation exists precisely because full effect-modifier summaries are not uniformly reported. Claim 3 remains without a prevalence estimate. Cites doi:10.1017/S0266462319000333: Supplies a real reference for claim 1, converting an unsourced assertion into a sourced one: in 18 NICE appraisals using population adjustment, availability was one of the reported bases for covariate selection.. Cites doi:10.1002/jrsm.1608: Existing partial response to the proposed direction: builds effect-modifier adjustment out of whatever subgroup information aggregate publications do report, with standard errors, rather than treating the reported set as a fixed constraint.. Cites doi:10.1007/s40273-025-01503-1: Nearest existing thing to a PAIC reporting standard, but it governs the analyst’s own reporting of the comparison, not the covariate summaries the comparator publication must carry; so it does not close the gap the entry describes..
- GPT-5.6 Sol, technical and source-code lens
- open / unverifiable. The technical core of C2 is correct: absent or incompatible target-population summaries can prevent balancing of prespecified effect modifiers or make PAIC strongly assumption-dependent. C1 contains no substantive proposition. C3 is an uncited absence claim and C4 is an uncited frequency claim, so neither can be verified under the specified technical-only, no-literature lens. Moreover, there is no universal finite collection of summaries that every PAIC requires; requirements depend on the estimand, effect-modification basis, link and outcome models, and weighting method. Marginal means and proportions may be insufficient when joint distributions, nonlinear terms, or interactions matter. Treating unreported summaries as missing data propagates uncertainty only after adding identifying assumptions, external information, or priors; otherwise sensitivity analysis or partial identification is required. The reporting limitation remains open, but its alleged prevalence and the nonexistence of standards are not established here.
- Grok 4.5, recency lens
- open / supported. Inverted search looked for a benchmarked minimum covariate-reporting set that would make aggregate-data publications PAIC-ready. Methodological reviews of PAIC practice document inconsistent reporting and incomplete covariate information as live obstacles (Serret-Larmande et al. 2023, J Clin Epidemiol; Truong et al. 2023, Res Synth Methods, doi:10.1002/jrsm.1653), supporting C2 and C4. CONSORT and trial reporting norms require baseline characteristic tables, but they do not mandate the full set of moments PAIC needs (means and SDs for continuous covariates on the analysis set of interest, consistent analysis populations, and joint/correlation structure). PRISMA-NMA update work (Veroniki et al. 2025 scoping/update papers) targets NMA reporting gaps (nodes, consistency, certainty of evidence) and is not a minimum AD covariate package for population adjustment. NICE DSU TSD 18 and EU HTA 2024 quantitative synthesis guidance describe what analysts need for MAIC/STC/ML-NMR but do not create a journal/registry mandate for publishers of comparator AD trials. No 2024–2026 standard was found that benchmarks a minimum reporting set and closes the reporting failure as a solved policy problem. C3 therefore still holds: the constraint remains what journals print. Treat incompletely reported AD as missing data with propagated uncertainty remains a proposed analyst-side response, not a solved standard.
- Decision path
-
R5:default-open → R7:counter-and-support-both-present
How the evidence falls in time
21 papers in the reviewed corpus bear on this problem, published 2012 to 2026. Every paper found here asserts the problem is open; none reports progress.
This problem is recurrent: three or more papers spanning at least eight years assert it independently. Sustained restatement by authors who mostly do not cite each other is the strongest evidence this reading can offer that a gap is real rather than one group’s framing.
| Year | Says | Paper |
|---|---|---|
| 2012 | confirms it is open | A systematic review with attempted network meta-analysis of asthma therapy recommended for five to eighteen year olds in GINA steps three and four |
| 2015 | confirms it is open | Design, Analysis, and Reporting of Crossover Trials for Inclusion in a Meta-Analysis |
| 2017 | confirms it is open | Management of people with early- or very early-stage hepatocellular carcinoma: an attempted network meta-analysis |
| 2017 | confirms it is open | Pharmacological interventions for acute hepatitis B infection: an attempted network meta-analysis |
| 2017 | confirms it is open | Pharmacological interventions for acute hepatitis C infection: an attempted network meta-analysis |
| 2017 | confirms it is open | Pharmacological interventions for alcoholic liver disease (alcohol-related liver disease): an attempted network meta-analysis |
| 2017 | confirms it is open | Pharmacological interventions for non-alcohol related fatty liver disease (NAFLD): an attempted network meta-analysis |
| 2017 | confirms it is open | Pharmacological interventions for primary biliary cholangitis: an attempted network meta-analysis |
| 2017 | confirms it is open | Pharmacological interventions for primary sclerosing cholangitis: an attempted network meta-analysis |
| 2020 | confirms it is open | Summarising salient information on historical controls: A structured assessment of validity and comparability across studies |
| 2021 | confirms it is open | Individual Participant Data Meta-Analysis: A Handbook for Healthcare Research (Riley, Tierney, Stewart, eds.; Wiley Series in Statistics in Practice, 2021) |
| 2022 | confirms it is open | Geometric approaches to assessing the numerical feasibility for conducting matching-adjusted indirect comparisons |
| 2022 | confirms it is open | Definitions matter: Heterogeneity of COVID-19 disease severity criteria and incomplete reporting compromise meta-analysis |
| 2022 | confirms it is open | Reporting and data sharing level for COVID-19 vaccine trials: A cross-sectional study |
| 2024 | confirms it is open | Implementing Multilevel Network Meta-Regression for Time-To-Event Outcomes: A Case Study in Relapsed Refractory Multiple Myeloma |
| 2024 | confirms it is open | Feasibility of Indirect Treatment Comparisons Between Niraparib Plus Abiraterone Acetate and Other First-Line Poly ADP-Ribose Polymerase Inhibitor Treatment Regimens for Patients with BRCA1/2 Mutation-Positive Metastatic Castration-Resistant Prostate Cancer |
| 2024 | confirms it is open | Transitivity, coherence, and reliability of network meta-analyses comparing proximal humerus fracture treatments: a meta-epidemiological study |
| 2024 | confirms it is open | Which clinical trial designs and statistical approaches have been used in assessments of orphan maintenance by the European Medicines Agency between 2012 and 2022? A cross-sectional study |
| 2024 | confirms it is open | Twenty years of network meta-analysis: Continuing controversies and recent developments |
| 2025 | confirms it is open | Evaluating the feasibility of a network meta-analysis comparing treatment options in polycythemia vera |
| 2026 | confirms it is open | Effectiveness of interventions to increase vaccine uptake: component network meta-analysis |
Source
Derived from A:L63, A:L75 of the reviewed corpus.