When users choose to estimate and evaluate ITR under cross-validation, the package implements Algorithm 1 from Imai and Li (2023) to estimate and evaluate ITR. For more information about Algorithm 1, please refer to the this page.
Instead of specifying the split_ratio argument, we
choose the number of folds (n_folds). We present an example
of estimating ITR with 3 folds cross-validation. In practice, we
recommend using 10 folds to get a more stable model performance.
| Input | R package input | Descriptions |
|---|---|---|
| Data Z = {Xi, Ti, Yi}i = 1n | treatment = treatment, form = user_formula, data = star_data |
treatment is a character string specifying
the treatment variable in the data; form is a
formula specifying the outcome and covariates; and a dataframe
data |
| Machine learning algorithm F | algorithms = c("causal_forest") |
a character vector specifying the ML algorithms to be used |
| Evaluation metric τf | PAPE, PAPD, AUPEC, GATE | By default |
| Number of folds K | n_folds = 3 |
n_folds is a numeric value indicating the
number of folds used for cross-validation |
| … | budget = 0.2 |
budget is a numeric value specifying the
maximum percentage of population that can be treated under the budget
constraint |
library(evalITR)
# estimate ITR
set.seed(2021)
fit_cv <- estimate_itr(
treatment = treatment,
form = user_formula,
data = star_data,
algorithms = c("causal_forest"),
budget = 0.2,
n_folds = 3)
#> Evaluate ITR with cross-validation ...The output will be an object that includes estimated evaluation metric τ̂F and the estimated variance of τ̂F for different metrics (PAPE, PAPD, AUPEC).
# evaluate ITR
est_cv <- evaluate_itr(fit_cv)
#> Cannot compute PAPDp
# summarize estimates
summary(est_cv)
#> ── PAPE ────────────────────────────────────────────────────────────────────────
#> estimate std.deviation algorithm statistic p.value
#> 1 0.2 0.76 causal_forest 0.26 0.79
#>
#> ── PAPEp ───────────────────────────────────────────────────────────────────────
#> estimate std.deviation algorithm statistic p.value
#> 1 2.5 0.65 causal_forest 3.8 0.00012
#>
#> ── PAPDp ───────────────────────────────────────────────────────────────────────
#> Cannot compute PAPDp
#>
#> ── AUPEC ───────────────────────────────────────────────────────────────────────
#> estimate std.deviation algorithm statistic p.value
#> 1 1.3 1.6 causal_forest 0.83 0.41
#>
#> ── GATE ────────────────────────────────────────────────────────────────────────
#> estimate std.deviation algorithm group statistic p.value upper lower
#> 1 -92.1 59 causal_forest 1 -1.566 0.12 23 -207
#> 2 68.7 59 causal_forest 2 1.155 0.25 185 -48
#> 3 20.4 59 causal_forest 3 0.346 0.73 136 -95
#> 4 14.8 59 causal_forest 4 0.250 0.80 131 -101
#> 5 6.4 102 causal_forest 5 0.063 0.95 206 -193