Skip to contents

Compute hindcast RMSE (observed vs projected)

Source: R/stats.R
compute_hindcast_rmse.Rd

Calculates the root–mean–squared error (RMSE) between observed values and one–step–ahead projections from a hindcast run (e.g., a data frame from hindcasts$obs_pred$catch or hindcasts$obs_pred$index).

Usage

compute_hindcast_rmse(data, log = TRUE)

Arguments

data

A long-format data frame with columns (e.g., from obs_pred in fit_hindcast() output):

  • year: integer assessment year.

  • age: integer model age.

  • obs: numeric observed value for the given year × age.

  • pred: numeric projected value for the given year × age.

  • fold: integer or numeric label for the hindcast peel (typically the terminal year used in that run).

  • is_proj: logical; TRUE for projection rows (the one–step–ahead predictions), FALSE otherwise.

log

Logical; if TRUE (default), compute RMSE on the log scale. Zeros in obs/pred are converted to NA before logging.

Value

A single numeric RMSE. Returns NaN (a missing numeric value) if there are no non-missing matched forecast errors.

Details

Matches projected rows (is_proj == TRUE) with available historical observations by year and age, also using type and survey when present. Include type when combining catch and index tables, and survey when combining surveys. Conflicting observations for the same key cause an error. Repeated copies of the same observation across folds count only once per prediction. Each matched forecast receives equal weight. Matched rows from result$forecast_pairs$index are also accepted directly.

On the natural scale, $$\mathrm{RMSE}=\sqrt{\frac{1}{K}\sum_i(Y_i-\widetilde Y_i)^2}.$$ With log = TRUE, both values are logged first. Log RMSE measures departures on a relative scale and is often more useful when catch and survey units differ. It still combines errors from different series and is not a likelihood score. Zeros and missing pairs are omitted for log RMSE; zeros are retained on the natural scale. Predictions beyond the available observations do not contribute. All projection rows are eligible; use fit_hindcast() for one-year horizons or subset the input to the desired forecast horizon.

Examples

if (FALSE) { # \dontrun{
# Suppose `hindcasts <- fit_hindcast(fit, folds = 3)`
# Overall RMSE for the index series on the log scale
compute_hindcast_rmse(hindcasts$obs_pred$index, log = TRUE)

# RMSE for catch on the natural scale
compute_hindcast_rmse(hindcasts$obs_pred$catch, log = FALSE)
} # }