Privacy guarantees and governance for shared trial-level data
Privacy noise is not negligible at usable budgets for trial-sized targets: at epsilon 1 and 150 patients it was 8% to 60% of the estimate’s variance and ignoring it cut coverage to 0.88 to 0.94; at 40 patients to 0.73. Its variance is known exactly, and adding it restored coverage in every scenario, conservatively where noise dominated. Releasing second moments pays only at larger budgets; at small ones it made MAIC infeasible in up to 87% of replicates. Read the study below.
Statement
Differential privacy would let data holders publish richer target-population summaries under a formal guarantee, and adjacent work exists: differentially private covariate balancing propagates privacy noise into weighted treatment-effect estimators (arXiv:2410.14789), FedECA delivers federated inverse-probability-weighted external-control analysis without pooling individual data (arXiv:2311.16984), and methods exist for choosing the privacy parameter for trial data release (PMC11031247). None calibrates a budget for the target moments PAIC needs, and no PAIC estimator treats released summaries as noisy inputs and propagates that noise into the transported effect. Two claims need narrowing: a correctly calibrated guarantee does not cease to hold because an arm is small, with utility rather than validity the casualty, and the assertion that no governance framework exists in evidence synthesis was not verifiable by any auditor.
Why it is open
Trial arms are small, so aggregate quantities computed on them have high sensitivity: gradients, sufficient statistics or likelihood contributions from a few dozen patients need large noise to be protected, and standard federated privacy analyses assume per-site samples far larger than a 40-patient arm. The guarantee is bought with injected noise, so extra moments arrive already degraded, and PAIC estimators must propagate that noise into the effect rather than treating the summaries as fixed; nobody has established what privacy budget yields target moments precise enough to change a PAIC conclusion. Governance is the second constraint, since cross-sponsor analysis requires legal agreement about who runs the code, who sees intermediate quantities and who is accountable.
What has been tried
| Work | What it contributes |
|---|---|
| Ohnishi & Awan (arXiv:2410.14789), differentially private covariate balancing | Privatizes propensity-score and balancing steps and propagates privacy noise into weighted treatment-effect estimators with privacy-utility rates. Does not calibrate a budget for the target moments PAIC needs. |
| FedECA (Nature Communications 2025; arXiv:2311.16984) | Federated inverse-probability-weighted external-control analysis for time-to-event outcomes without pooling individual data, with discussion of differentially private variants. |
| PMC11031247 (data-driven choice of the privacy parameter for clinical trial data release) | Addresses privacy-parameter selection for trial data release, the nearest existing answer to the budget-calibration question, but not for PAIC target moments. |
| PMC8449619 (scoping review of differential privacy in health research) | Surveys differential privacy applications in health research, establishing the privacy-utility trade-off and the state of adoption. |
Probable solution or research direction
Characterize the privacy attack surface of evidence synthesis when trial arms are small, and design release mechanisms and noise calibration for exactly the target moments PAIC needs in that regime, extending the differentially private balancing work that already propagates privacy noise into weighted estimators. Build PAIC estimators that treat released summaries as noisy inputs and carry the uncertainty into the transported effect. Develop governance conventions for cross-sponsor analyses covering code custody, intermediate quantities and accountability, and document what already exists before claiming a vacuum.
Our study
Target moments released under differential privacy
The question put to it. When a target trial releases covariate moments under differential privacy, how much does ignoring the added noise cost MAIC coverage at realistic arm sizes and budgets, does propagating its known variance repair it, and when do extra released moments stop paying for their share of the budget?
What it found. Privacy noise is not negligible at usable budgets for trial-sized targets: at epsilon 1 and 150 patients it was 8% to 60% of the estimate’s variance and ignoring it cut coverage to 0.88 to 0.94; at 40 patients to 0.73. Its variance is known exactly, and adding it restored coverage in every scenario, conservatively where noise dominated. Releasing second moments pays only at larger budgets; at small ones it made MAIC infeasible in up to 87% of replicates.
Results.
- Refuting sentence fails: coverage ignoring the noise was as low as 0.883 at epsilon of at least 1 and targets of at least 150.
- Propagating the known noise variance gave coverage of at least 0.926 in every private scenario, the minimum matching the non-private analysis’s own misspecification; it reached 1.000 where noise dominated.
- Releasing means and second moments with a split budget left no feasible weights in 45% to 87% of replicates at 40 to 150 patients and epsilon 0.5 to 1.
- Under quadratic modification at epsilon 4 and 150 patients second moments cut RMSE from 0.229 to 0.171.
Design.
- Estimand
- Target-population mean difference, A versus C, at the target trial’s own covariate moments
- Methods compared
- MAIC to non-private moments, MAIC to private moments, noise ignored, MAIC to private moments, noise variance propagated
- Data-generating mechanism
- MAIC to Laplace-privatized target means or means and second moments, 48 cells crossing target size (40 to 500), privacy budget, released statistics and linear or quadratic effect modification; non-private, noise-ignored and noise-propagated analyses. 1000 replicates per cell.
- Replicates
- 1000 per scenario, 48,000 in total
What this does not answer. Laplace mechanism only; Gaussian mechanism, released correlations and overlap not varied. Results are conditional on feasible weights. Budget choice and governance are outside the study. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/ADJ-15-noise-whose-variance-is-known-exactly · Protocol: registered before the run
Verification
Verdict. Partially addressed. Auditors agreed the problem is open and no counterevidence surfaced. Reopened by the full-text reading: 2 findings from 2 papers survived both the reader and an independent second reviewer, so the verdict moves from confirmed-open to partially-addressed.
The source said. “Differential privacy is proposed so data holders can publish richer target-population summaries, including more moments, correlations and joint structure, under a formal guarantee, while the report separately names privacy attacks arising from small trial arms and the governance of cross-sponsor analyses as key open questions. Together these determine what can safely be released and who is accountable for the result.”
What was wrong with it. The source implied that small trial arms defeat formal privacy guarantees and that no cross-sponsor governance framework exists. GPT-5.6 Sol pointed out that a correctly calibrated guarantee still applies at small n and that the universal governance claim is not established, and the literature auditor could not verify it either; Grok 4.5 supplied the adjacent differentially private causal work the source did not cite.
Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).
- Verdict changed
- confirmed-open -> partially-addressed 2 surviving findings from: Arbitrated Indirect Treatment Comparisons; Identifying back pain subgroups: developing and applying app. New citations: 10.1007/s43441-026-00976-2, 10.3310/pgfar04100
- Literature and prior-art check
- open / supported. I found differential privacy applied to clinical trial data sharing, including a data-driven method for choosing the privacy parameter for trial data release (PMC11031247) and a scoping review of DP in health research (PMC8449619), plus the standard precedent that published aggregate summaries alone permitted large-scale reconstruction of the 2010 US Census. Nothing I retrieved releases DP-protected target-population moments and propagates the injected noise through a PAIC estimator, which is the entry’s specific gap, and nothing establishes what privacy budget still yields target moments precise enough to change a PAIC conclusion. The privacy-utility trade-off, the requirement to account for injected noise, and the elevated re-identification risk of statistics computed on small arms are all standard and documented. The governance claim (‘no such framework exists in evidence synthesis’) is a no-method-exists assertion; I found no source stating the absence, so treat that specific sentence as unverified rather than verified.
- GPT-5.6 Sol, technical and source-code lens
- open / overstated. C1, C2, C4, C6, and C8 are technically sound with conditions. A differential-privacy guarantee requires a defined neighboring unit, bounded contributions, a specified mechanism, and privacy accounting; releasing more or higher-dimensional moments generally consumes privacy budget or increases error. Small arms worsen sensitivity and utility, but a correctly calibrated formal guarantee does not cease to apply merely because n is small. Attack feasibility also does not follow from sample size alone. Privacy-induced randomness should normally be propagated for unconditional inference, although an explicitly conditional analysis can condition on the released value. Cross-sponsor governance is a legitimate implementation question, but the universal claim that no framework exists is not established technically. This remains open PAIC-specific work, but it is a decision-grade blocker only when privacy-constrained sharing is required.
- Grok 4.5, recency lens
- open / supported-with-caveat. Inverted-prior search found adjacent DP-plus-causal and federated ECA work, but not a solution to the PAIC-specific package as stated. Differentially private covariate balancing (Ohnishi and Awan, arXiv:2410.14789) shows how to privatize propensity-score / balancing steps and propagate privacy noise into weighted ATE/WATE estimators with privacy-utility rates; FedECA (Nature Communications 2025; arXiv:2311.16984) delivers federated IPTW for time-to-event external control arms without pooling IPD and discusses DP variants. Neither calibrates a privacy budget for richer target-population moments (means, correlations, joints) that classic AgD PAIC needs, nor builds PAIC estimators that treat those released moments as noisy inputs, nor characterises the small-arm (tens of patients) attack surface specific to evidence-synthesis statistics, nor supplies a cross-sponsor governance framework for code custody and intermediate quantities. Gradient-leakage and federated DP literature confirm C5–C6 as general facts, not as ITC closures. C1–C4 are standard DP properties (supported as framing), not evidence that the PAIC adaptation is done. C7 remains open. C8 is a report-internal maturity label and was not independently auditable beyond consistency with the sparse PAIC-DP literature. Queries run: differential privacy + (MAIC|PAIC|NMA|indirect comparison|target moments); FedECA DP; DP covariate balancing causal inference; small-sample federated gradient leakage; ISPOR/EUnetHTA/NICE federated HTA governance 2024–2026.
- Decision path
-
R5:default-open → R9:substantiated-single-overstatement:codex
How the evidence falls in time
4 papers in the reviewed corpus bear on this problem, published 2015 to 2026. The most recent paper to touch this reports progress on it, and every paper calling it open is older.
It is also being closed in pieces: partial results come from more than one paper in more than one year, each covering a different part.
Does the later work answer the earlier work? Put to an independent reviewer with the older and newer claims side by side, it judged that the later work answers a component of it, leaving a named part (high confidence). The later work provides practical governance and no-transfer arrangements for trial data, addressing part of the earlier disclosure-and-governance concern. Neither provides a formal privacy guarantee or the required statistical treatment of privacy-induced release noise in PAIC. What remains: A formal differential-privacy budget calibrated to PAIC-required target moments, plus a PAIC estimator that propagates released-summary noise into the transported effect.
| Year | Says | Paper |
|---|---|---|
| 2015 | confirms it is open | Network meta-analysis incorporating randomized controlled trials and non-randomized comparative cohort studies for assessing the safety and effectiveness of medical treatments: challenges and opportunities |
| 2016 | partly addresses it | Identifying back pain subgroups: developing and applying approaches using individual patient data collected within clinical trials |
| 2025 | confirms it is open | A critical assessment of matching-adjusted indirect comparisons in relation to target populations |
| 2026 | partly addresses it | Arbitrated Indirect Treatment Comparisons |
Source
Derived from C:L1646-1647, C:L1665, C:L1728-1730, C:L1735-1736 of the reviewed corpus.