Identification, estimation, computation and decision are conflated
Statement
Identification, estimation, computation and decision relevance are distinct, and every specific claim the source makes is true: a faster sampler adds no identifying information, double robustness still requires positivity for its target, measured balance does not address unmeasured effect modification, a component model imposes rather than proves additivity, and a narrow interval says nothing about whether the target population was the right one. The classification itself is not missing. Petersen and van der Laan (doi:10.1097/EDE.0000000000000078) set out the staged separation of causal question, identification and statistical estimation and named the confusions that follow from conflating them, and the Causal Roadmap (doi:10.1017/cts.2023.635) operationalizes it as an itemized, prespecified process developed with regulators and industry. What has not been done is the ITC and PAIC instantiation: no published mapping of MAIC, STC, ML-NMR, node-splitting and disconnected-network methods onto the four problem classes.
Why it is open
Progress in the field is delivered mainly as new estimators and faster samplers, which are estimation and computation advances, while the binding constraint in most applied comparisons is identification or decision relevance. A more sophisticated estimator produces a more polished output, so the appearance of improvement is decoupled from whether the causal quantity was identified at all. The general roadmap exists but is not written in the vocabulary of indirect comparisons, so an analyst choosing between MAIC, STC and ML-NMR gets no help from it in naming which class the binding constraint belongs to. Two qualifications survive: identification is always relative to explicit structural assumptions, and practical near-positivity is not the same thing as structural positivity.
What has been tried
| Work | What it contributes |
|---|---|
| Petersen and van der Laan 2014, Causal models and learning from data: integrating causal modeling and statistical estimation, Epidemiology | presents a systematic approach integrating causal modeling with statistical estimation and highlights the common points of confusion that occur when the stages are conflated in practice |
| Dang et al. 2023, A causal roadmap for generating high-quality real-world evidence, Journal of Clinical and Translational Science | operationalizes the separation of causal question, estimand and identifying assumptions from estimation and from sensitivity analysis as an explicit, itemized, iterative prespecification process; not yet specialized to ITC or PAIC |
Probable solution or research direction
Instantiate the existing roadmap for ITC and PAIC rather than propose a new classification: map MAIC, STC, ML-NMR, node-splitting and disconnected-network methods onto identification, estimation, computation and decision relevance, and state for each what it can and cannot repair. Require an analysis to name the class its binding constraint falls in before a method is chosen, and accept added model structure only when it is scientifically defensible and adequately informed by data. Since the framework is already endorsed elsewhere, the deliverable is a specialization and a reporting checklist, not a new conceptual apparatus.
Verification
Verdict. Partially addressed. Part of this problem is addressed by existing work; the entry names which part.
The source said. “Identification, estimation, computation and decision problems are logically distinct, and treating a problem in one class with a solution from another is a systematic error in the ITC literature. A faster sampler does not cure non-identification, a doubly robust estimator does not cure positivity failure, and a narrow interval does not establish that the target population was the right one.”
What was wrong with it. The source presented the four-way separation as absent from the field. The literature auditor found it standardized outside indirect comparison since 2014 (doi:10.1097/EDE.0000000000000078) and operationalized in the Causal Roadmap (doi:10.1017/cts.2023.635), leaving only the ITC instantiation, and the source-code auditor went further and voted that this is established analysis hygiene and a reporting checklist rather than an unresolved research problem.
- Literature and prior-art check
- partially-solved / supported-with-caveat. Every claim_to_check here is definitionally true and I found nothing contradicting any of them. But the entry frames the four-way separation as a missing classification, and a mature, widely adopted framework already does exactly this outside the ITC literature. Petersen and van der Laan (Epidemiology 2014) present ‘A systematic approach for the integration of causal modeling with statistical estimation’ and explicitly ‘highlight some common points of confusion that occur when causal modeling techniques are applied in practice’. The Causal Roadmap paper (Dang et al., J Clin Transl Sci 2023, 26 authors including Dahabreh, Stuart, FDA and industry statisticians) operationalizes it as ‘an explicit, itemized, iterative process that guides investigators to prespecify study design and analysis plans’, with the causal question, estimand and identifying assumptions in steps 1-3, estimation in step 4 and sensitivity analysis in step 5. That is the entry’s proposed direction, already standardized and endorsed. What is not done is the ITC/PAIC instantiation: no published mapping of MAIC, STC, ML-NMR, node-splitting and disconnected-network methods onto the four problem classes. Publish it as that narrower gap, citing the Roadmap, not as an unaddressed conceptual hole. Cites 10.1097/EDE.0000000000000078: Establishes the separation of causal question, identification and statistical estimation as an explicit staged framework and names the confusions that follow from conflating them. Cites 10.1017/cts.2023.635: Provides the standardized, regulator-engaged process that separates estimand and identification from estimation and from decision relevance, i.e. the classification the entry proposes; not yet specialized to ITC/PAIC.
- GPT-5.6 Sol, technical and source-code lens
- not-a-problem / supported-with-caveat. C1-C4 correctly distinguish identification, finite-sample estimation, computation, and decision relevance relative to a stated estimand and assumption set. C5-C10 are also correct: faster sampling does not add identifying information; double robustness still requires positivity for its target; measured balance does not address unmeasured effect modification; a component model imposes rather than proves additivity; precision does not establish target-population relevance; and complexity does not imply dominance. The qualifications are that identification is always relative to explicit structural or model assumptions and practical near-positivity differs from structural positivity. This is established analysis hygiene and a useful reporting checklist, not an unresolved methodological research problem. Claims about how often the ITC literature commits these errors and the asserted priority are outside the technical lens.
- Decision path
-
R3:partially-solved(n=1) → R11:caveated(n=2)
Source
Derived from C:L205, C:L263-272 of the reviewed corpus.