Retrospective fits and observation hindcasts

Refit models after peeling terminal years. Hindcasts additionally forecast held-out observations, conditional on observed removals by default.

Usage

fit_retro(
  fit,
  ...,
  folds = 2,
  start_from_fit = FALSE,
  hindcast = FALSE,
  proj_settings = NULL,
  grad_tol = NULL,
  progress = TRUE,
  globals = NULL
)

fit_hindcast(
  fit,
  ...,
  folds = 2,
  start_from_fit = FALSE,
  hindcast = TRUE,
  proj_settings = list(n_proj = 1, n_mean = 1, catch = "observed"),
  grad_tol = NULL,
  progress = TRUE,
  globals = NULL
)

# S3 method for class 'tam_retro'
print(x, ...)

Arguments

fit, ...: One or more tam_fit objects. Name additional models for distinct labels. All other controls must be named.
folds: Number of peels, leaving at least two historical years per fit.
start_from_fit: Use each supplied fit to initialize its peels?
hindcast: Project peeled fits? Defaults to FALSE for fit_retro() and TRUE for fit_hindcast().
proj_settings: Hindcast settings: n_proj (default 1), n_mean (default 1), and either catch = "observed" (default) or one non-negative F_mult. Partial lists inherit these defaults; supplying F_mult clears catch. Numeric catch targets and control vectors are not accepted in hindcasts. Forecasts are shortened to available validation years. Ordinary retrospectives reject projection settings. NULL uses hindcast defaults when hindcast = TRUE. Ordinary model projections retain scalar/vector numeric catch and F controls; see prepare_tam().
grad_tol: Numerical check tolerance; defaults to each fit's stored value.
progress: Show progress using progressr?
globals: Global names passed to furrr workers. Configure your own future plan.
x: A tam_retro object.

Value

A tam_retro list with fits nested by model and terminal year; stacked pop and obs_pred with model, fold and horizon (zero historically); mohns_rho; detailed scores; matched forecast_pairs; fold_diagnostics; and resolved projection_controls. Failed references make rho unavailable. All-failed runs still return diagnostics.

Details

Only observed years determine folds. Historical references have no forecasts. Held-out indices and catch compositions do not enter fitting; observed-catch conditioning uses only aggregate removals. Forecast biology uses training years and n_mean.

Forecast assumptions. catch = "observed" is available only in hindcast evaluation (fit_hindcast() or fit_retro(hindcast = TRUE)), not ordinary fit_tam() projections. It uses actual total catch weight in each held-out year, calculated from complete raw catch-at-age and catch weights (stock weights if no catch weights are supplied), summed across all fleets with their own catch weights. Forecast stock and catch weights are averaged separately by type and fleet using training years only. This conditions prediction on removals that actually occurred; held-out surveys and catch compositions do not enter fitting. Agreement with the conditioning catch total is not forecast skill. Missing catch/weight coverage fails that fold; it is not imputed. Results do not represent forecasts of future removals. An alternative is proj_settings = list(F_mult = 1), retaining the F-at-age estimated at each fold's terminal year. Any scalar multiplier applies to that terminal profile at every available horizon, rather than compounding annually. Controls therefore follow each fold's shifting calendar window.

Printed summary. One row per model shows Mohn's rho for SSB, recruitment and average F, relative to the unpeeled reference fit. Hindcasts also show log_RMSE (pooled survey-age log prediction errors), MASE (persistence-relative absolute log error, averaged across ages within survey, surveys within horizon, then horizons), and n (survey observation-forecast pairs used for log-RMSE). scored_folds counts folds with eligible survey predictions; failed_folds counts failed numerical/projection attempts, including a failed reference. The unpeeled reference does not contribute forecast scores. A failed reference makes rho unavailable; missing forecast pairs or persistence baselines make the corresponding scores unavailable. Catch-composition and detailed survey, age and horizon scores remain in scores. Unequal validation coverage is flagged and no automatic ranking is produced.

Survey log-RMSE pools observation errors. MASE compares absolute log errors with persistence on identical cases; see compute_hindcast_mase(). Overall MASE averages ages within survey, surveys within horizon, then horizons. Aggregate indices require complete, consistent age coverage. Catch proportions and mean modeled age (plus age coded by its lower bound) are scored separately on their natural scale; genuine zeros are retained. Overlapping forecasts are distinct origin/horizon cases, not independent observations. These are conditional point-forecast diagnostics, not probability scores.

Compare equivalent observations and age definitions. No ranking is produced. Missing baselines or zero persistence error make MASE unavailable. Failed numerical fits remain in fits, but not in stacked results or scores. Supplied fits and fitted objectives are unchanged.

Parallel execution

Set the execution plan before calling either function, for example future::plan(future::multisession, workers = 4). Without a parallel plan, folds run sequentially. Neither function changes your plan or worker count. Use future::plan(future::sequential) afterwards to shut down the workers.

Examples

if (FALSE) { # \dontrun{
future::plan(future::multisession, workers = 4)
retro <- fit_retro(fit, alternative = other_fit, folds = 5)
hind <- fit_hindcast(fit, folds = 10,
  proj_settings = list(n_proj = 3, n_mean = 3, catch = "observed"))
print(hind)
hind$scores
vis_tam(hind)
vis_tam(retro, model = names(retro$fits)[1])
future::plan(future::sequential)
} # }