Responder analysis of continuous trial outcomes

Converts continuous outcomes (mean change, SD and sample size per arm, across studies) into responder proportions and a range of between-arm effect measures: the risk difference (RD), risk ratio (RR), odds ratio (OR) and number needed to treat (NNT), under a parametric model for the change scores. Responders are defined by a minimal important difference (MID) threshold (the cut-point or "dichotomization" approach of Anzures-Cabrera, Sarpatwari and Higgins, 2011). For a threshold-free alternative see responder_cles().

Usage

# S3 method for class 'meta'
responder_analysis(data, mid, ...)

# S3 method for class 'rma'
responder_analysis(
  data,
  mid = NULL,
  ...,
  baseline,
  direction = c("higher", "lower"),
  measure = NULL,
  sd_ref = NULL,
  dist = c("normal", "lognormal", "t"),
  df = NULL
)

# Default S3 method
responder_analysis(data, ...)

responder_analysis(data, ...)

# S3 method for class 'data.frame'
responder_analysis(
  data,
  mid,
  direction = c("higher", "lower"),
  method = c("individual", "weighted", "unweighted", "median", "smd"),
  se_method = c("delta", "binomial"),
  pooling = c("fixed", "random"),
  control = c("matched", "median"),
  tau_method = c("DL", "REML"),
  dist = c("normal", "lognormal", "t"),
  df = NULL,
  mid_sd = 0,
  ci_type = c("wald", "logit"),
  ci_method = c("wald", "hksj", "hksj_adhoc"),
  conf_level = 0.95,
  rd_weights = c("sample_size", "inverse_variance"),
  ...
)

Arguments

data

A data frame with one row per study and columns study, change_e, sd_e, n_e, change_c, sd_c, n_c and optionally trial (see sample_responder_data); or a fitted meta-analysis object (see the "Meta-analysis objects" section).

mid

Single finite number: the minimal important difference threshold.

...

Further arguments for the method. Unknown arguments are an error.

baseline

For a metafor rma object (must be named): the control responder risk, as a probability in (0, 1) or a list with mean and sd (and optionally n). See responder_from_effect().

direction

"higher" (a larger change indicates response) or "lower".

measure

For a metafor rma object: "SMD" or "MD", overriding the measure stored in the object.

sd_ref

For a metafor rma object fitted on mean differences: the reference within-arm SD on the original scale.

dist

Change-score distribution: "normal" (default), "lognormal" or "t". Sampling variances use the distribution's own third and fourth moments.

df

Degrees of freedom when dist = "t" (a single number greater than 2).

method

Methods to compute: any of "individual", "weighted", "unweighted", "median", "smd". Defaults to the first four.

se_method

Per-study variance model for "individual": "delta" (default) propagates the sampling uncertainty of the reported mean and SD through the change-score model; "binomial" uses the pseudo-binomial p (1 - p) / n, which treats the model-based probability as an observed proportion (the version 0.1.0 default).

pooling

"fixed" (default) or "random" effects, for the "individual", "weighted" and "smd" methods.

control

Baseline-risk rule for the summary methods: "matched" (default) or "median". See the "Baseline risk" section.

tau_method

Between-study variance estimator when pooling = "random": "DL" (DerSimonian-Laird, default) or "REML" (needs the metafor package; falls back to DL with a warning if unavailable).

mid_sd

Optional standard deviation of the MID threshold; when > 0 its uncertainty is added to the effect-measure variances (once, after pooling).

ci_type

"wald" (default) or "logit". With "logit" the weighted method builds its risk-difference interval by the MOVER method from logit-scale arm intervals (allowing for their correlation), so it stays within [-1, 1]. It applies to the weighted method and to responder_rd_individual(); pooled individual and smd intervals are formed on the scale that is pooled. A nonzero mid_sd forces Wald intervals, with a warning.

ci_method

Random-effects interval method: "wald" (Normal, default), "hksj" (Hartung-Knapp-Sidik-Jonkman) or "hksj_adhoc" (HKSJ with the ad hoc floor, never narrower than Wald; recommended for few studies).

conf_level

Confidence level (default 0.95).

rd_weights

Fixed-effect weights for the pooled risk difference: "sample_size" (default, n_e n_c / (n_e + n_c)) or "inverse_variance".

Value

A responder_result: a data frame with one row per requested method and columns method, pooling, k, p_e, p_c, rd/rd_lb/rd_ub, rr/rr_lb/rr_ub, or/or_lb/or_ub, nnt/nnt_lb/nnt_ub, var_rd (the variance behind the risk-difference interval), and the heterogeneity statistics tau2, i2, q, q_p, pi_lb, pi_ub (prediction interval on the risk-difference scale). Proportions and risk differences are on the proportion scale; multiply by 100 for percentages. The analysis settings and the per-study statistics are kept as attributes and used by the print(), summary() and plot() methods (responder_result) and by responder_sof().

Methods

individual

Dichotomize each study, then pool the per-study effect measures (fixed or random effects). The recommended method: every comparison stays within a randomized study. The per-study variance follows se_method. Ratio measures are computed from exact log probabilities, so a study deep in a probability tail cannot be pinned to an artificial ratio of 1.

weighted

Pool the control-arm mean and the within-study mean difference separately (inverse variance, fixed or random effects per pooling), set the experimental mean to their sum, pool each arm's SD as the within-study pooled SD, then dichotomize. Variances come from the multivariate delta method, including the covariance between the pooled control mean and the pooled mean difference. Anchoring on the within-study difference keeps the randomized comparison intact (pooling the two arms separately, as in version 0.1.0, can produce a Simpson's-paradox bias when allocation ratios differ across studies).

unweighted, median

Dichotomize the arithmetic mean or the median of the study means and SDs. Descriptive summaries with no variance model: intervals are NA.

smd

Pool the standardized mean difference (Hedges' g), bridge to an odds ratio through the logistic distribution (lnOR = (pi / sqrt(3)) g; Hasselblad and Hedges, 1995; Chinn, 2000), and combine with the pooled control responder risk to recover risks. The odds ratio interval is the pooled g interval transformed (so ci_method applies); the risk difference and risk ratio intervals also carry the uncertainty of the estimated control risk. Not included by default.

Baseline risk

For the weighted method the control proportion comes, by default, from the pooled control mean (control = "matched"). Set control = "median" to take the baseline from the median control arm instead (as in the Sofi-Mahmudi 2024 simulation study); the experimental mean is then that median control mean plus the pooled within-study mean difference. For unweighted and median, control = "median" takes the control proportion from the median control arm. Because the median control arm carries no sampling-variance model, control = "median" reports point estimates only. The individual and smd methods pool per-study contrasts and ignore control.

Pooling and intervals

Heterogeneity statistics (q, i2, tau2) are inverse-variance based and refer to the scale that was pooled: the risk difference for individual and Hedges' g for smd. With fixed effects the risk difference is pooled with sample-size weights n_e n_c / (n_e + n_c) by default (rd_weights = "sample_size"), which a single study whose responder probabilities sit near 0 or 1 (and so has a tiny model-based variance) cannot dominate; the risk ratio and odds ratio use inverse-variance weights on the log scale. A warning is raised whenever one study carries more than half of the weight for a pooled measure (with three or more studies).

For random effects, ci_method = "hksj_adhoc" (Hartung-Knapp-Sidik-Jonkman with the max(1, q) floor of Rover, Knapp and Friede, 2015) is recommended when there are few studies: plain "hksj" can be much narrower than the Wald interval when the studies agree closely. Prediction intervals use t_{k-2} and the variance that built the confidence interval, for every tau_method, and are NA for fewer than three studies.

A nonzero mid_sd treats the threshold as uncertain. Because the MID is shared by every study, its uncertainty is added once, after pooling, through the derivative of the pooled estimate with respect to the MID; it does not change the weights or the point estimate.

Multi-arm trials

If data has a trial column, rows with the same trial value and an identical control arm (change_c, sd_c, n_c) are treated as comparisons that share one control group, and the control sample size is divided equally among them (Cochrane Handbook, section 23.3.4) so the control patients are not counted twice. A message reports when this happens.

Meta-analysis objects

responder_analysis() also accepts a fitted continuous meta-analysis saved in an R object instead of a data frame. A meta object (a metacont from the meta package) carries the per-arm summaries, so it is converted to the data-frame form (dropping excluded studies) and analyzed with the full method menu; its pooling settings (common or random effects, a DL or REML tau-squared, a Hartung-Knapp interval with or without the ad hoc rule, and the confidence level) are used unless you override them, and settings respondeR cannot reproduce are reported in a warning. A metafor rma object carries only the pooled relative effect, so it is converted with responder_from_effect() and you must supply a named baseline control responder risk (and sd_ref for a mean-difference model). The object is read with base R, so the source package need not be attached.

References

Sofi-Mahmudi A (2024). Identifying an optimal strategy for converting pain as a continuous outcome to a responder analysis. Master's thesis, McMaster University. https://hdl.handle.net/11375/30210

Thorlund K, Walter SD, Johnston BC, Furukawa TA, Guyatt GH (2011). Pooling health-related quality of life outcomes in meta-analysis: a tutorial and review of methods for enhancing interpretability. Research Synthesis Methods, 2(3), 188 to 203. doi:10.1002/jrsm.46

Anzures-Cabrera J, Sarpatwari A, Higgins JPT (2011). Expressing findings from meta-analyses of continuous outcomes in terms of risks. Statistics in Medicine, 30(25), 2967 to 2985. doi:10.1002/sim.4298

Rover C, Knapp G, Friede T (2015). Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Medical Research Methodology, 15, 99. doi:10.1186/s12874-015-0091-1

Altman DG, Deeks JJ (2002). Meta-analysis, Simpson's paradox, and the number needed to treat. BMC Medical Research Methodology, 2, 3. doi:10.1186/1471-2288-2-3

See also

responder_rd_individual(), responder_cles(), responder_proportions(), responder_curve(), responder_sof()

Examples

responder_analysis(sample_responder_data, mid = 1)
#> Responder analysis (respondeR)
#>   MID = 1 (a higher change is a response); k = 3 studies; fixed
#>   effects; delta per-study variance
#>   Percentages; 95% confidence intervals.
#> 
#>           Method   PE   PC                  RD                  RR
#>       Individual    -    - 25.5 (20.0 to 31.0) 2.15 (1.79 to 2.58)
#>    Weighted mean 47.6 22.1 25.5 (20.0 to 31.1) 2.16 (1.80 to 2.59)
#>  Unweighted mean 47.7 22.8                24.9                2.09
#>           Median 48.7 21.5                27.2                2.26
#>                   OR                   NNT
#>  3.20 (2.45 to 4.18) NNTB 3.9 (3.2 to 5.0)
#>  3.21 (2.46 to 4.19) NNTB 3.9 (3.2 to 5.0)
#>                 3.09              NNTB 4.0
#>                 3.46              NNTB 3.7
#> 
#> Use as.data.frame() for every column, summary() for details, plot() for a forest plot.

# Random-effects individual method with the small-k interval:
responder_analysis(sample_responder_data, mid = 1,
  method = "individual", pooling = "random", ci_method = "hksj_adhoc")
#> Responder analysis (respondeR)
#>   MID = 1 (a higher change is a response); k = 3 studies; random
#>   effects; delta per-study variance
#>   Percentages; 95% confidence intervals.
#> 
#>      Method PE PC                  RD                  RR                  OR
#>  Individual  -  - 25.4 (11.5 to 39.3) 2.15 (1.43 to 3.22) 3.19 (1.72 to 5.91)
#>                    NNT
#>  NNTB 3.9 (2.5 to 8.7)
#> 
#>   individual: I-squared 19%, tau-squared 0.000631, Q 2.48 (p 0.290); prediction interval -26.5 to 77.4
#> 
#> Use as.data.frame() for every column, summary() for details, plot() for a forest plot.