Uncertainty and Diagnostics
Uncertainty and Diagnostics
A reserve estimate is conditional on data and assumptions. A complete analysis asks both how variable can future emergence be? and what if the model structure is wrong?
1. Separate four uncertainties
| Source | Meaning | Example | Typical treatment |
|---|---|---|---|
| Process | future claims vary even with known parameters | next year's payments differ randomly | stochastic model or simulation |
| Parameter | factors and distribution parameters are estimated | Dev 2→3 uses one origin year | standard error, bootstrap, sensitivity |
| Model | another plausible structure gives another answer | calendar inflation breaks age-only development | alternative models and scenarios |
| Operational/data | recorded information is incomplete or unstable | reopened claims or system migration | reconciliation, controls, data-quality allowance |
One confidence interval rarely captures all four.
2. Mack chain ladder: what it adds
Mack's distribution-free framework keeps chain-ladder mean structure while estimating conditional variance under assumptions such as:
with origin years independent. It yields standard errors for projected reserves without assuming a full claim distribution.
3. Bootstrap: preserve the workflow
A reserving bootstrap typically:
- fits an incremental mean model;
- computes and scales residuals;
- resamples residuals to create pseudo-triangles;
- refits development parameters;
- simulates future process outcomes;
- stores total reserve and origin-year results.
Refitting in each replicate captures parameter variation. Simulating future increments captures process variation. Merely sampling final point estimates captures neither correctly.
4. Backtest by valuation date
For each historical diagonal :
- hide all observations after ;
- fit the method using only information available at ;
- predict the next diagonal or later ultimate;
- compare with subsequently observed values;
- repeat over several .
Useful measures include signed error, absolute percentage error where denominators are stable, interval coverage, and error by development age. Reserve backtesting must avoid look-ahead bias: factor selections and exclusions should reflect what could have been known at the historical valuation date.
Tiny backtest example
Using only AY 2020 and 2021 to estimate gives
It predicts AY 2022 Dev 1 paid of , versus £260m observed. The next-diagonal error is £5.5m, or 2.1% of observed. One good prediction is not validation; the example shows the information boundary.
5. Residual patterns diagnose structure
| Pattern | Interpretation to investigate |
|---|---|
| residuals trend across origin years | portfolio mix, exposure, or underwriting change |
| residuals align on calendar diagonals | inflation, legislation, catastrophe, or operations |
| variance rises faster than fitted mean | heteroscedasticity or large-loss mixture |
| paid and incurred residuals diverge | settlement and case-reserve processes changed |
| youngest years fail repeatedly | prior or development pattern is stale |
Residuals are prompts for investigation, not mechanical deletion rules.
6. Scenario table for the reserve committee
| Scenario | Change from baseline | Question answered |
|---|---|---|
| factor selection | use trimmed or credibility-weighted link ratios | sensitivity to influential cells |
| tail | add plausible tail factors | cost beyond observed development |
| inflation | increase future severity/calendar effect | sensitivity to claims inflation |
| large loss | model separately or cap for factor selection | concentration risk |
| prior | vary ELR in BF | dependence on external expectation |
| settlement speed | shift payment timing without changing ultimate | paid-pattern instability |
Present the rationale and probability status: a deterministic stress is not a percentile unless linked to a probabilistic model.
7. Current research: richer time and claim information
Recent work develops two complementary directions. State-space models make aggregate reserving dynamics and sampling distributions explicit. Individual-claim approaches use transaction histories, claim status, covariates, and survival methods. Examples include Selukar's state-space workflow (2025), the CAS Claim Life Cycle Model (2024), the ReSurv claim-count framework (2025), and a 2026 preprint connecting chain ladder with individual-claim prediction.
The teaching conclusion is measured:
- individual data can explain reporting, closure, reopening, and payment timing;
- richer data do not remove selection bias, censoring, or operational drift;
- aggregate triangles remain valuable for reconciliation and benchmarking;
- recent preprints are research-frontier evidence, not settled professional standards.
8. Minimum reserve report
A reviewer should be able to reproduce:
- data scope and valuation date;
- triangle transformations and exclusions;
- factor/prior/tail selections;
- central estimate by origin year;
- process and parameter uncertainty where modelled;
- alternative methods and scenarios;
- backtest results and known failure modes;
- judgement, ownership, and limitations.
Practice
- Classify “future claim amounts vary around a correct expected value.”
- Classify “a repair-cost shock affects all cells on the latest diagonal.”
- Why is fitting on the full triangle and then pretending to backtest an earlier diagonal invalid?
Answers
- Process uncertainty.
- Primarily model/calendar uncertainty; parameter and process effects may accompany it.
- The selected factors have already seen the held-out observations, so the error is optimistically biased by information leakage.
Sources and further reading
- Mack, T. (1993), “Distribution-Free Calculation of the Standard Error of Chain Ladder Reserve Estimates,” ASTIN Bulletin, 23(2), 213–225.
- England, P. D. and Verrall, R. J. (2002), “Stochastic Claims Reserving in General Insurance,” British Actuarial Journal, 8(3), 443–518.
- CAS — The Development and Use of a Claim Life Cycle Model (2024)
- CAS — Claim Counts Prediction Using Individual Data with ReSurv (2025)
- Selukar — Stochastic Claims Reserving Using State Space Modeling (2025 preprint)
- Richman and Wüthrich — From Chain-Ladder to Individual Claims Reserving (2026 preprint)