Persistence-relative hindcast forecast error — compute_hindcast

Compare forecasts with holding the last observation available at the forecast origin constant. Values below one indicate smaller absolute errors than persistence. This uses the evaluation-period convention of Carvalho et al., rather than a training-period scaling denominator.

Usage

compute_hindcast_mase(data, log = TRUE)

Arguments

data: Matched forecast rows containing obs, pred and baseline. The baseline must use observations available at the forecast origin.
log: Use log-scale errors (default); observations and baselines must then be positive. Use FALSE for proportions or mean modeled age.

Value

A number, or NA when there are no valid pairs or persistence has zero error. See the score tables from fit_hindcast() for counts and reasons.

Details

Numerator and denominator use the same forecast cases: $$\mathrm{MASE}=\sum_i|z_i-\widehat z_i|/\sum_i|z_i-z_{0,i}|.$$ For multiple horizons the baseline remains fixed at the origin. This is an extension of the paper's one-year calculation. Predictions are conditional medians, as in existing log-RMSE calculations; this is not a probability score.

References

Carvalho et al. (2021). A cookbook for using model diagnostics in integrated stock assessments. Fisheries Research 240, 105959. doi:10.1016/j.fishres.2021.105959 .

Examples

compute_hindcast_mase(data.frame(obs = c(2, 4), pred = c(2.1, 3.8),
                                baseline = c(1, 2)))
#> [1] 0.07219495