Survival pseudo-IPD reconstruction uncertainty is not propagated
No, the refuting sentence fails: neither ensemble behaved as a multiple imputation. For RMST to 24 months at a 6-month risk table the variance calibration ratio (one for an imputation) was 31.8 to 44.3 for six analyst variants and 2.3 to 11.2 for 20 observation-model draws, the registered ‘both far from one’ outcome. Across cells the draws overstated the error of RMST (2.1 to 13.1) and 12-month survival (1.1 to 3.5) and understated it for Weibull survival at 48 months (0.15 to 0.77; 90% intervals covered the true-data value in 42% to 77%). Reconstruction error was small against sampling error (single-reconstruction RMSE 2% to 8% of the sampling SD for RMST, 11% to 26% for 48-month survival), so population coverage stayed at 0.91 to 0.98. Read the study below.
Statement
Component survival models condition on pseudo-IPD reconstructed from digitized Kaplan-Meier curves and at-risk tables as if the event and censoring times were observed, with no reconstruction-error layer, and the implemented route further rejects event-count and person-time summaries as insufficient for its exact individual survival likelihood. Reconstruction uncertainty now has some machinery behind it: RESOLVE-IPD (arXiv 2511.01785, November 2025) extracts coordinates and explicit censoring marks from vectorized figures, drops the uniform-censoring assumption, and generates ensembles of valid labelings so that a downstream meta-analysis propagates that uncertainty. It is a preprint and its propagated uncertainty is over subgroup label assignment rather than over pixel-level curve reading or at-risk-table rounding, so it narrows the gap without closing it, and no measurement model of this kind is implemented in component PAIC.
Why it is open
The reconstruction pipeline turns an image plus a risk table into an exact-looking event and censoring dataset, and every downstream likelihood then treats those synthetic records as real observations. Existing work addresses fidelity and subgroup-label ambiguity, not a measurement model for pixel-level curve reading or at-risk-table rounding, so the dominant error sources still have no distribution attached. In oncology this compounds with an exact individual survival likelihood that is deliberately more demanding than event-count summaries, so the model is precise about data whose provenance is uncertain.
What has been tried
| Work | What it contributes |
|---|---|
| RESOLVE-IPD 2025 | reconstructs individual patient data from vectorized figures with explicit censoring marks, removes the uniform-censoring assumption, and generates labeling ensembles so subgroup reconstruction uncertainty propagates into a meta-analysis |
| cpaic 0.1.0 (development version, choxos/cpaic@9d150e9) | unreleased development R package on GitHub, not on CRAN; supports exact individual survival likelihood contributions with right, left, and interval censoring and delayed entry, rejects event-count and person-time summaries, and contains no reconstruction-error layer |
Probable solution or research direction
Generate multiple reconstructions and pool across them, using the ensemble machinery that already exists for subgroup labeling as the template, and extend it to pixel-level curve reading and at-risk-table rounding. Build a Bayesian measurement model for those two error sources and propagate it into the posterior rather than conditioning on one reconstruction. Report how much of the reported precision survives the propagation, since that is the number a decision maker needs.
Our study
Reconstruction ensembles were not imputations: their spread overstated the error in restricted mean survival and understated it in the extrapolated tail
This study was designed against OUT-13 and bears on this entry in part. What it does not answer is stated below.
The question put to it. When individual data are reconstructed repeatedly from a published Kaplan-Meier curve, does the spread of the ensemble match the true reconstruction error, so that pooling it like multiple imputation is valid?
What it found. No, the refuting sentence fails: neither ensemble behaved as a multiple imputation. For RMST to 24 months at a 6-month risk table the variance calibration ratio (one for an imputation) was 31.8 to 44.3 for six analyst variants and 2.3 to 11.2 for 20 observation-model draws, the registered ‘both far from one’ outcome. Across cells the draws overstated the error of RMST (2.1 to 13.1) and 12-month survival (1.1 to 3.5) and understated it for Weibull survival at 48 months (0.15 to 0.77; 90% intervals covered the true-data value in 42% to 77%). Reconstruction error was small against sampling error (single-reconstruction RMSE 2% to 8% of the sampling SD for RMST, 11% to 26% for 48-month survival), so population coverage stayed at 0.91 to 0.98.
Results.
- The analyst variants’ mean overstated RMST by 0.17 to 0.19 months in every cell; a post hoc rerun traced it to sparse digitization (error 0.29 to 0.34 months with 30 points, 0.15 to 0.16 with 60, 0.06 with 120), and their spread is that ordering, not uncertainty.
- Rubin pooling widened intervals by at most 1% with the draws; the variants widened them by up to 23% only for 48-month survival without a risk table.
- Positive control failed (12-month survival RMSE 13% of the sampling SD in the coarse, no-table, clustered cell against a registered 25%): milestone survival was not materially affected in this design.
- The null control (monthly table) passed on 161 of 300 replicates; 139 were lost to a 20-second wall-clock guard that fired under a machine load average of 500 to 900.
Design.
- Estimand
- Variance calibration ratio of a reconstruction ensemble: mean between-reconstruction variance times (1 + 1/M) over the variance of the ensemble mean’s error against the true data
- Methods compared
- single Guyot reconstruction, analyst variants (30, 60, 120 points; event total given or not), observation-model draws (pixel and censoring positions redrawn)
- Data-generating mechanism
- Simulated trial data rendered as a pixel-rounded Kaplan-Meier figure with or without a risk table, reconstructed by Guyot’s algorithm; analyst-variant and observation-model ensembles scored by the ratio of their spread to the true reconstruction error for RMST, milestone and extrapolated survival. 13 cells, 300 replicates each.
- Replicates
- 300 per cell, 13 cells
What this does not answer. The draws are not a posterior: Guyot’s event-count rounding is held fixed and there is no likelihood for the published ordinates. One arm and no downstream comparison, so propagation into a population-adjusted or component comparison (CMP-17) is not measured; no human digitization error; one sample size. The mechanism analyses are post hoc. Peer review has not been done.
Peer review. Not yet done. The result stands on the study alone until it is.
Read it: PDF · Markdown · OpenDocument · Code and data: studies/OUT-13-an-ensemble-that-is-not-an-imputation · Protocol: registered before the run
Question. When both arms of a comparator trial are reconstructed from a published Kaplan-Meier figure and carried into an anchored comparison through a joint Weibull likelihood or anchored Kaplan-Meier contrasts, does treating the reconstruction as observed data keep nominal coverage for late and extrapolated survival differences, and how much of the reported precision would honest propagation leave?
Design. An individual-data A versus C trial and a B versus C trial published as a Kaplan-Meier figure with a 6-monthly or no risk table, reconstructed by Guyot’s algorithm; joint Weibull likelihoods with study-specific or common shapes and anchored Kaplan-Meier and Cox contrasts, each fitted to the true data, a single reconstruction and a 10-draw observation-model ensemble, at 250 or 1000 per arm. 7 cells, 500 replicates each.
The protocol is registered before any result is seen: read it.
What this project measured
OUT-13 measured whether repeated reconstructions from a published Kaplan-Meier curve carry the right uncertainty. They did not, and the direction depended on the functional. Observation-model draws, which redraw each step’s pixel position and each censoring’s time within its risk-table interval, overstated the reconstruction error of RMST to 24 months by a factor of 2 to 13 and understated it for Weibull survival extrapolated to 48 months (ratio 0.15 to 0.77; their 90% intervals covered the true-data value in 42% to 77% of replicates). The error itself concentrated in the tail: a single reconstruction’s RMSE was 2% to 8% of the sampling SD for RMST and 11% to 26% for 48-month survival.
Not answered: how that tail error propagates into a population-adjusted or component comparison. OUT-13 used one arm and no downstream model, so the propagation this entry asks about is untested; an ensemble built from these draws would not repair it, since it understates exactly the component that matters there.
Verification
Verdict. Partially addressed. Part of this problem is addressed by existing work; the entry names which part.
The source said. “Pseudo-IPD reconstructed from digitized Kaplan-Meier curves and at-risk tables is treated as if it were observed data. Digitization and reconstruction error are therefore not propagated into the reported uncertainty.”
What was wrong with it. The source said reconstruction error has no distribution to propagate. The literature auditor found RESOLVE-IPD (arxiv:2511.01785), which quantifies and propagates reconstruction ambiguity into a meta-analysis, though only for subgroup labeling and as a preprint; Grok 4.5 agreed it is partial prior art rather than a resolution, and GPT-5.6 Sol scoped the software-absence claim to choxos/cpaic@9d150e9 rather than to component PAIC software generally.
- Literature and prior-art check
- partially-solved / supported-with-caveat. The premise that reconstructed pseudo-IPD is normally treated as observed data is correct and widely true, but the claim that reconstruction uncertainty has ‘no distribution to propagate’ is now dated. RESOLVE-IPD (arXiv 2511.01785, submitted November 2025, revised January 2026) is a framework whose stated purpose is high-fidelity reconstruction plus ‘uncertainty-aware subgroup meta-analysis’: VEC-KM extracts coordinates and explicit censoring marks from vectorised figures to minimise digitisation error, CEN-KM removes the uniform-censoring assumption, and MAPLE ‘generates ensembles of mathematically valid labelings, facilitating a propagating meta-analysis that quantifies and reflects uncertainty from subgroup reconstruction’. Two honest caveats: it is a preprint, and its propagated uncertainty is over subgroup label assignment rather than over pixel-level curve reading or at-risk-table rounding, so it narrows rather than closes the entry’s gap. The specific claim that no such measurement model exists in component PAIC software remains true. Cites arxiv:2511.01785: Quantifies reconstruction uncertainty and propagates it into a downstream meta-analysis, and removes the uniform-censoring assumption; covers subgroup-label uncertainty rather than digitisation and at-risk-table error..
- GPT-5.6 Sol, technical and source-code lens
- open / overstated. C1, C2, and C4 are source-confirmed. cmlnmr() accepts supplied or reconstructed event/censoring rows, passes their times and statuses directly into the survival likelihood, contains no reconstruction-error layer, and explicitly rejects event-count/person-time summaries for that intended exact individual likelihood. C3 is established only for cpaic. Neither this repository nor the cited package claim supports the ecosystem-wide assertion that no Bayesian measurement model exists in any component PAIC software, and the broader statement that there is no accepted measurement model is outside this source-code audit. The problem is potentially decision-critical when reconstructed pseudo-IPD materially drives the result, but irrelevant to analyses using genuinely observed individual survival data.
- Grok 4.5, recency lens
- open / supported. cpaic still requires reconstructed event/censor rows and treats them as ordinary individual contributions; README and man/cmlnmr.Rd explicitly say uncertainty from KM reconstruction is not propagated. Adjacent 2025 work improves fidelity and some uncertainty handling but does not solve this problem for component PAIC: RESOLVE-IPD (arXiv:2511.01785) reduces digitization/censoring bias and propagates subgroup-label ensembles into meta-analysis, yet it is not a Bayesian measurement model for pixel/risk-table error inside a cML-NMR likelihood, nor integrated with cpaic. IPDfromKM/KM-GPT remain point-reconstruction pipelines. No Bayesian measurement model for KM curve plus at-risk table uncertainty in component PAIC software was found. Queries: RESOLVE-IPD; IPDfromKM uncertainty propagation; Bayesian measurement model Kaplan-Meier digitization; cpaic pseudo-IPD. Status remains open; RESOLVE is partial prior art for ensemble pooling of reconstruction ambiguity, not a resolution of cpaic’s conditioning on one pseudo-IPD.
- Decision path
-
R3:partially-solved(n=1) → R9:substantiated-single-overstatement:codex
How the evidence falls in time
1 paper in the reviewed corpus bears on this problem, published 2014. Every paper found here reports progress; none asserts the problem is open.
| Year | Says | Paper |
|---|---|---|
| 2014 | partly addresses it | Network meta-analysis of (individual patient) time to event data alongside (aggregate) count data |
Source
Derived from C:L1375, C:L1309-1311, scope:field-level (all survival PAIC using digitized curves); applies to cpaic’s survival path of the reviewed corpus.