Fitting and Validating Loss Models

Model fitting estimates parameters; validation asks whether the fitted model is useful for unseen insurance decisions. These are separate steps.

1. Audit the sample

Before fitting, answer:

1. Are amounts at a common price and currency level?
2. Are losses ground-up or payments after deductible and limit?
3. Are limit observations censored or genuinely equal to the limit?
4. Are small losses absent because of truncation or because none occurred?
5. Are claims independent, or clustered within events and policyholders?
6. Was the sample selected using information unavailable at prediction time?

The likelihood must describe the observation process, not only the latent ground-up loss.

2. Likelihood contributions

For exact independent observations $x_1,\ldots,x_n$ with density $f(x;\theta)$,

Left truncation at threshold $d$

If only claims with $X>d$ enter the dataset and their ground-up values are observed, each contribution is

Ignoring the denominator treats a selected large-loss sample as representative of all losses.

Right censoring at limit $u$

An exact value below $u$ contributes $f(x_i;\theta)$; an observation known only to satisfy $X\ge u$ contributes $\bar F(u;\theta)$. Replacing every limited payment by an exact loss of $u$ understates the ground-up tail.

Deductible payment data

If recorded payment is $Y=X-d$ conditional on $X>d$, derive the density of $Y$ from the conditional loss distribution. Do not fit $Y$ as if it were ground-up $X$.

3. Estimation methods

4. Fit candidates, then test unseen data

The executable example generates a fixed synthetic lognormal sample. It compares Exponential, Gamma, and lognormal fits using training AIC and held-out negative log score.

The synthetic generator makes lognormal plausible, but a finite holdout need not rank it first every run. That sampling variation is part of the lesson.

5. Diagnostics answer different questions

Use logarithmic survival axes to make tail differences visible.

6. AIC and goodness-of-fit boundaries

For a model with $k$ estimated parameters,

Lower AIC compares estimated information loss among models fitted to the same observations and likelihood basis. It is not an absolute fit certificate, and AIC values from differently truncated samples are not directly comparable.

The ordinary one-sample Kolmogorov–Smirnov null distribution assumes the reference distribution is fully specified. If parameters are fitted from the same data, textbook p-values are generally invalid. Use an appropriate correction or parametric bootstrap:

1. fit the model and calculate the observed statistic;
2. simulate many samples from the fitted model;
3. refit the model to each sample;
4. recompute the statistic;
5. compare the observed statistic with this simulated reference distribution.

7. Parameter and model uncertainty

A fitted 99.5% quantile $Q(.995;\widehat\theta)$ is a point estimate. Quantify uncertainty with profile likelihood, bootstrap, posterior draws, and/or alternative plausible families. For a high layer, model spread can dominate sampling error within one chosen family.

Practice

1. A dataset records only claims above £10,000. Which likelihood adjustment is required?
2. A policy pays £100,000 for every ground-up claim at or above its limit. Is £100,000 an exact ground-up loss?
3. Why should a time-dependent portfolio use a chronological holdout rather than a random split?