(Un)Fit for purpose

Academics lay great store by anonymous peer review and in openly publishing their results.  There are good reasons for this — anonymous peer review allows expert third parties (usually two) to challenge assumptions without fear of retribution, while open publishing allows others to test things and find their limitations.  For example, the model from Lee & Carter (1992) has been thoroughly researched over the past two decades, and its limitations are well known.  However, the Lee-Carter model has stood the test of time in no small part because these limitations have been publicly documented by other researchers.

One advantage of all this open publication is that major problems can be spotted and brought to people's attention.  A forthcoming paper by Currie (2014) performs a considerable service to the actuarial community by describing how to fit the eight projection models evaluated in Cairns et al (2009).  A useful side-effect of this is that two previously published models have been shown to be unusable for mortality projections. The two models are the Renshaw-Haberman model (also known as M2) and the CBD model with age-modulated cohort effects (also known as M8).  The details are technical and somewhat lengthy, and thus beyond the scope of a short article such as this — the interested reader can consult Currie (2014).  However, we can summarise by saying that there are tremendous problems in fitting these two models.  Each model has its own highly specific fitting issues, but the conclusion is the same: neither model can be safely used for mortality forecasting.

What does this mean for actuaries?  The immediate corollary is that they should not forecast using M2 or M8, nor do they need to consider these models when exploring model risk.  However, there are two further important points.  The first is that actuaries should rely on openly published models from peer-reviewed academic literature.  Not only does the peer-review process screen out obviously unsuitable materials, but open publication in academic journals allows further scrutiny afterwards, as with Currie (2014).  Using a model which has not been peer-reviewed and openly published is risky, because problems are much less likely to be found.

The second corollary is that it is important not to rely on any single model for mortality projections.  If you use a handful of different models to explore model risk, then you are much less likely to be caught out if one of those models proves to be unsuitable.