No agreed rule turns an NMA posterior into a recommendation
Statement
Converting a fitted network meta-analysis into a treatment recommendation is done by conventions that do not share a decision objective. Ranking on highest expected value is optimal for a risk-neutral decision maker and ignores uncertainty; SUCRA and P-score blend mean effect and uncertainty in a way that answers no stated decision question; and GRADE’s instruction to prefer the more certain of two conflicting estimates can return an incoherent recommendation set. The gap was named in a 2024 review of the field as the need for a definition of optimality that accommodates multiple treatment recommendations while penalizing uncertainty, whether the evaluative function is efficacy, cost-effectiveness or something else. A proposal now exists: loss-adjusted expected value, with a two-stage rule that first identifies treatments superior to the reference and then those within a minimally important difference of the best, compared against GRADE and three probability-based rankings on stylized examples and 10 network meta-analyses used in NICE guidelines, where an expected-value decision maker would recommend 4 to 14 treatments and the loss-adjusted rule 0 to 3 fewer (median 2), and where GRADE failed to recommend the highest expected-value treatment in 3 of the 10 cases. What is not settled is the rest of the problem: the proposal is confined to risk aversion and excludes a risk-seeking posture, it is not extended to net benefit or cost-effectiveness, alternative approaches to risk aversion in the wider literature were not compared, and no guidance body has adopted any rule. No certainty or confidence label has an established mapping onto the ranking it is supposed to inform, so a committee reading a hierarchy alongside a set of GRADE ratings is still combining them by judgment.
Why it is open
The rule sits between a fitted synthesis model and the recommendation a committee issues, and neither the statistical methodology literature nor the appraisal guidance claims that interface, so nothing obliges either to converge on an answer. Penalizing uncertainty requires a loss function, and eliciting one for a population-level guideline decision is contested; the alternatives, namely mean-variance trade-offs, rules capping the probability of a poor outcome, and explicit risk-aversion parameters, all introduce inputs whose values are to some extent arbitrary, which is why none has been taken up in this literature. Risk posture may itself depend on baseline health, with evidence that individuals are risk-seeking at low levels of baseline health, so a single risk-averse rule may be wrong exactly where the decisions are gravest. The evaluative function that matters for reimbursement is net benefit rather than efficacy, and moving a decision rule onto that scale requires the economic model as well as the network, which puts it outside what a synthesis paper normally carries.
What has been tried
| Work | What it contributes |
|---|---|
| Ades et al. 2024, Twenty years of network meta-analysis: Continuing controversies and recent developments, Research Synthesis Methods | States the gap directly: further research is needed to find a definition of optimality that accommodates multiple treatment recommendations while penalizing uncertainty, whether decisions rest on efficacy, cost-effectiveness or another evaluative function. |
| Ades et al. 2025, Treatment recommendations based on network meta-analysis: Rules for risk-averse decision-makers, Research Synthesis Methods | Introduces loss-adjusted expected value and a two-stage rule using a minimally important difference, defines properties of a valid ranking under uncertainty, and compares against GRADE and three probability rankings on 10 NICE network meta-analyses; it excludes risk-seeking postures, does not compare mean-variance or maximum-probability alternatives, and states that extension to net benefit and to baseline-dependent risk posture is ongoing research. |
| Salanti et al. 2011, Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial, Journal of Clinical Epidemiology | Introduces SUCRA, the incumbent convention that mixes effect size and uncertainty without reference to a decision problem. |
| Rucker and Schwarzer 2015, Ranking treatments in frequentist network meta-analysis works without resampling methods, BMC Medical Research Methodology | Gives the P-score, the frequentist counterpart, so the same objection applies in both frameworks. |
| Brignardello-Petersen et al. 2020, GRADE approach to drawing conclusions from a network meta-analysis using a minimally contextualised framework, BMJ | Specifies how GRADE turns network estimates into categorized conclusions using certainty ratings and a threshold, the incumbent rule that the loss-adjusted comparison finds can privilege more uncertain treatments; the partially contextualized companion is doi 10.1136/bmj.m3907. |
| Claxton 1999, The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies, Journal of Health Economics | Establishes the decision-theoretic argument that adoption decisions should follow expected net benefit rather than inferential thresholds; the precedent on the net-benefit scale that a network decision rule would have to connect to. |
| Epstein 2019, Beyond the cost-effectiveness acceptability curve: The appropriateness of rank probabilities for presenting the results of economic evaluation, Health Economics | Examines whether rank probabilities are an appropriate presentation of economic evaluation results, the same objection posed on the cost-effectiveness side. |
| Chalkou et al. 2023, Decision Curve Analysis for Personalized Treatment Choice between Multiple Options, Medical Decision Making | Supplies a decision-analytic route for choosing among multiple options with an explicit threshold, developed for prediction rather than for network synthesis output. |
| Naci et al. 2014, Evidence-Based Prescribing: Combining Network Meta-Analysis With Multicriteria Decision Analysis to Choose Among Multiple Drugs, Circulation: Cardiovascular Quality and Outcomes | Attaches an explicit multicriteria value function to network output, showing one way of naming an objective and the weights it requires. |
| Dias et al. 2013, Evidence Synthesis for Decision Making 6: Embedding Evidence Synthesis in Probabilistic Cost-Effectiveness Analysis, Medical Decision Making | Shows how a network posterior enters a probabilistic cost-effectiveness model, the pathway through which a loss-adjusted rule would have to be extended to net benefit. |
| Phillippo et al. 2018, Sensitivity of treatment recommendations to bias in network meta-analysis, Journal of the Royal Statistical Society Series A | Defines decision-invariant thresholds for a recommendation, which presuppose a decision rule and therefore inherit whichever convention is chosen. |
Probable solution or research direction
Test the existing proposal rather than add another: run loss-adjusted expected value, plain expected value, SUCRA, P-score and the GRADE contextualized frameworks over a common set of simulated networks with known truth, and score them on realized decision error and on the size of the recommended set, a comparison nobody has run. Extend the loss adjustment to the net-benefit scale, where the evaluative function is already defined and the cost-effectiveness literature supplies the loss, so the rule can be used for reimbursement rather than only for efficacy ordering. Let the risk penalty depend on baseline health rather than fixing it, and report whether the recommendation set is stable to that dependence, since the fixed-penalty version may be wrong for life-threatening conditions. Fix a reporting convention that names the decision rule and its parameters alongside the recommendation, so that a recommendation set can be reproduced from the posterior by someone who did not fit the model.
Verification
Verdict. Partially addressed. One reader proposed this and the reviewer accepted it as a well-posed gap at the posterior-to-recommendation interface, quoting the 2024 review that names the missing definition of optimality. It is partially addressed because the same group published a concrete rule in 2025, evaluated on 10 NICE networks; what remains is that the rule is untested against realized decision error, confined to risk aversion, not extended to net benefit, and adopted by nobody.
Revised by the full-text reading. Applied 2026-07-26 full-text reading of the consolidated library (687 papers).
- Citation corrected
- normalized a citation year: ‘Chalkou et al. 2022, Decision Curve Analysis for Personalized Treatment Choice between Multiple Options, Medical Decision Making’ -> ‘Chalkou et al. 2023, Decision Curve Analysis for Personalized Treatment Choice between Multiple Options, Medical Decision Making’ 10.1177/0272989X221143058 is cited with two different years across the registry. CrossRef gives 2022 online and 2023 in print; the issue year is used throughout.
Source
Derived from reading:nma-11 of the reviewed corpus.