Back to the future with Whittaker smoothing

(Aug 7, 2015)

Many actuaries will be familiar with Whittaker smoothing (1923) but few will be aware of the close connection between this early method and the method of P-splines. The purpose of this blog is to explain this connection.

Figure 1 is a typical plot of log(mortality) of the kind of data we might want to smooth; here we use UK male data for 2011 for ages 2 to 30 taken from the Human Mortality Database.

Figure 1: Crude and Whittaker smoothed log mortality rates for UK males ages 2 to 30 in 2011.

Let \(y_i = \log(d_i/e_i), i = 1, \ldots, n\), be our data where the \(d_i\) are the observed deaths at age \(i\) and the \(e_i\) are the corresponding central exposures, and let \(\mu_i, i = 1, \ldots, n\), be the candidate smooth values…

Read more

Tags: Whittaker smoothing, splines, P-splines, penalty function

Forecasting with penalty functions - Part III

(Jun 2, 2015)

This is the last of my three blogs on forecasting with penalties. I discussed the 1-d case in the first blog and the 2-d case in the second. Here we discuss some of the properties of 2-d forecasting. Some readers may find some of my remarks surprising, even paradoxical.

In our first blog we used the Lee-Carter model as an example where a time series is used to forecast mortality. The method is (a) estimate the parameters in the model by fitting the model to suitable data and (b) forecast a subset of the parameters with a suitable time series. The fit to data, by definition, does not depend on the forecast horizon. This is a familiar and attractive property; we will refer to this as the invariance property. It is easy to overlook…

Read more

Tags: forecasting, splines, P-splines, penalty function, mortality crossover

Forecasting with penalty functions - Part II

(Mar 18, 2015)

Our first blog in this series of three looked at forecasting log mortality with penalties in one dimension, i.e. forecasting with data for a single age. We now look at the same problem, but in two dimensions. Figure 1 shows our data. We see an irregular surface sitting on top of the age-year plane. Just as in the 1-d case, we see an underlying smooth surface, and it is this surface that we wish both to estimate and to forecast.

Figure 1: Crude log mortality rates for Australian males ages 50-90 over 1960-2010.

In the 1-d case we saw (a) how a basis of B-splines sat under the data, and (b) that each B-spline had a coefficient associated with the peak of its B-spline. The same idea applies in 2-d.  Figure 2 shows (for a reduced…

Read more

Tags: forecasting, splines, P-splines, penalty function, mortality crossover

Forecasting with penalty functions - Part I

(Feb 2, 2015)

There is much to say on the topic of penalty forecasting, so this is the first of three blogs.  In this blog we will describe penalty forecasting in one dimension; this will establish the basic ideas.  In the second blog we will discuss the case of most interest to actuaries: two-dimensional forecasting.  In the final blog we will discuss some of the properties of penalty forecasting in two dimensions.

Forecasting with penalties is very different from forecasting with the more familiar time series methods, so let us begin with a time-series example.  The Lee-Carter model assumes that:

\(\log \mu_{i,j} = \alpha_i + \beta_i \kappa_j\qquad(1)\)

where \(\mu_{i,j}\) is the force of mortality at age \(i\) in year…

Read more

Tags: forecasting, splines, P-splines, penalty function

Effective dimension

(Feb 19, 2014)

Actuaries often need to smooth mortality rates.  Gompertz (1825) smoothed mortality rates by age and his famous law was a landmark in this area.  Figure 1 shows the Gompertz model fitted to CMI assured lives data for ages 20-90 in the year 2002. The Gompertz Law usually breaks down below about age 40 and a more general smooth curve would be appropriate.  However, a more general smooth curve would obviously require more parameters than the two for the simple Gompertz model.

Figure 1.  Crude mortality rates (black dots) with fitted Gompertz line (solid red) on a logarithmic scale.

When we switch to a forecasting scenario, Figure 2 shows both a straight line and a parabola fitted to years 1947-2002 for age 70.  Clearly,…

Read more

Tags: effective dimension, splines, P-splines


(Nov 28, 2012)

Graduation is the process whereby smooth mortality rates are created from crude mortality rates.  Smoothness is an important part of graduation, but another is the extrapolation of mortality rates to ages at which data may be unreliable or even non-existent.  An example would be pension-fund work where reliable mortality-experience data might be available up to age 100, but where actuaries required mortality rates up to age 120 (say) for the purposes of calculating pension reserves.  An illustration of this dual smoothing and extrapolation is given in Figure 1.

Figure 1. log(mortality) by age for males in United States of America.  Crude rates shown for ages 20-100, together with graduated rates outside…

Read more

Tags: graduation, extrapolation by age, smoothing, splines

Find by key-word

Find by date

Find by tag (show all )