Back to the future with Whittaker smoothing

(Aug 7, 2015)

Many actuaries will be familiar with Whittaker smoothing (1923) but few will be aware of the close connection between this early method and the method of P-splines. The purpose of this blog is to explain this connection.

Figure 1 is a typical plot of log(mortality) of the kind of data we might want to smooth; here we use UK male data for 2011 for ages 2 to 30 taken from the Human Mortality Database.

Figure 1: Crude and Whittaker smoothed log mortality rates for UK males ages 2 to 30 in 2011.

Let $$y_i = \log(d_i/e_i), i = 1, \ldots, n$$, be our data where the $$d_i$$ are the observed deaths at age $$i$$ and the $$e_i$$ are the corresponding central exposures, and let $$\mu_i, i = 1, \ldots, n$$, be the candidate smooth values…

Forecasting with penalty functions - Part III

(Jun 2, 2015)

This is the last of my three blogs on forecasting with penalties. I discussed the 1-d case in the first blog and the 2-d case in the second. Here we discuss some of the properties of 2-d forecasting. Some readers may find some of my remarks surprising, even paradoxical.

In our first blog we used the Lee-Carter model as an example where a time series is used to forecast mortality. The method is (a) estimate the parameters in the model by fitting the model to suitable data and (b) forecast a subset of the parameters with a suitable time series. The fit to data, by definition, does not depend on the forecast horizon. This is a familiar and attractive property; we will refer to this as the invariance property. It is easy to overlook…

Forecasting with penalty functions - Part II

(Mar 18, 2015)

Our first blog in this series of three looked at forecasting log mortality with penalties in one dimension, i.e. forecasting with data for a single age. We now look at the same problem, but in two dimensions. Figure 1 shows our data. We see an irregular surface sitting on top of the age-year plane. Just as in the 1-d case, we see an underlying smooth surface, and it is this surface that we wish both to estimate and to forecast.

Figure 1: Crude log mortality rates for Australian males ages 50-90 over 1960-2010.

In the 1-d case we saw (a) how a basis of B-splines sat under the data, and (b) that each B-spline had a coefficient associated with the peak of its B-spline. The same idea applies in 2-d.  Figure 2 shows (for a reduced…

Forecasting with penalty functions - Part I

(Feb 2, 2015)

There is much to say on the topic of penalty forecasting, so this is the first of three blogs.  In this blog we will describe penalty forecasting in one dimension; this will establish the basic ideas.  In the second blog we will discuss the case of most interest to actuaries: two-dimensional forecasting.  In the final blog we will discuss some of the properties of penalty forecasting in two dimensions.

Forecasting with penalties is very different from forecasting with the more familiar time series methods, so let us begin with a time-series example.  The Lee-Carter model assumes that:

$$\log \mu_{i,j} = \alpha_i + \beta_i \kappa_j\qquad(1)$$

where $$\mu_{i,j}$$ is the force of mortality at age $$i$$ in year…