Compare models accuracy

Source: R/compare_accuracy.R

Computes some common model accuracy indices of several different models at once, allowing model comparison.

compare_accuracy(..., rank_by = NULL, quiet = FALSE)

Arguments

...

A list of models. The models can be of the same or of different classes, including lvmisc_cv class.
rank_by

A character string with the name of an accuracy index to rank the models by.
quiet

A logical indicating whether or not to show any warnings. If FALSE (the default) no warnings are shown.

Value

A data.frame with a model per row and an index per column.

Examples

m1 <- lm(Sepal.Length ~ Species, data = iris)
m2 <- lm(Sepal.Length ~ Species + Petal.Length, data = iris)
m3 <- lm(Sepal.Length ~ Species * Petal.Length, data = iris)
compare_accuracy(m1, m2, m3)

#>   Model Class   R2 R2_adj    AIC    BIC  MAE  MAPE RMSE
#> 1    m1    lm 0.62   0.61 231.45 243.49 0.40 6.77% 0.51
#> 2    m2    lm 0.84   0.83 106.23 121.29 0.27 4.71% 0.33
#> 3    m3    lm 0.84   0.83 106.77 127.84 0.27 4.67% 0.33

if (require(lme4, quietly = TRUE)) {
  mtcars <- tibble::as_tibble(mtcars, rownames = "cars")
  m1 <- lm(Sepal.Length ~ Species, data = iris)
  m2 <- lmer(
    Sepal.Length ~ Sepal.Width + Petal.Length + (1 | Species), data = iris
  )
  m3 <- lm(disp ~ mpg * hp, mtcars)
  cv3 <- loo_cv(m3, mtcars, cars)
  compare_accuracy(m1, m2, cv3, rank_by = "AIC")
}

#> Warning: Not all models have the same response variable.
#> Warning: Not all models are of the same class.
#> Warning: Some models were refit using maximum likelihood.
#>   Model              Class   R2 R2_adj R2_marg R2_cond    AIC    BIC   MAE
#> 1    m2            lmerMod   NA     NA    0.81    0.95  97.74 112.79  0.25
#> 2    m1                 lm 0.62   0.61      NA      NA 231.45 243.49  0.40
#> 3   cv3 lvmisc_cv_model/lm 0.78   0.76      NA      NA 359.54 366.87 50.34
#>     MAPE  RMSE
#> 1  4.29%  0.31
#> 2  6.77%  0.51
#> 3 26.04% 68.25