Deduplication and annuities

Deduplication is an important step in data preparation for mortality modelling (or any other kind of modelling for that matter). If people in your data set have multiple benefit records, then the crucial independence assumption for statistical modelling in invalidated. An effective algorithm for identifying duplicates is described in a paper presented to the Institute of Actuaries.

The problem of duplicates is a major issue for annuity portfolios, where it is very common for people to have multiple policies. On average I expect around 1.2 annuities per person, although this is obviously portfolio-specific. I also find that the average number of annuities per person tends to increase with age. This might partly be product-driven, as older annuitants are more likely to be self-employed holders of old-style s226 policies, whereas at younger ages there are more CPAs from personal pensions. However, the increase in average policies per person with age also appears to be correlated with their socio-economic status and wealth. This makes deduplication even more important, as the duplicate problem is linked to one of the important variables you are trying to analyse.

I did once hear a claim from a many-tentacled consultancy that the independence assumption wasn't important for large portfolios. This is obviously untrue, since an average of 1.2 annuities per person ruins the independence assumption equally for large and small data sets. Indeed, large portfolios tend to exhibit ever-greater extremes. In one very large annuity portfolio I analysed in 2005, I found a man with no fewer than 32 seperate annuities.




Find by key-word


This blog brings together two pieces of work. The first ... Read more
In a previous post we considered the advantages of repurposing ... Read more
A fundamental assumption underlying most modern presentations of mortality modelling ... Read more
Stephen Richards
Stephen Richards is the Managing Director of Longevitas
Deduplication in Longevitas
Longevitas users can control all aspects of deduplication including switching it off in the Deduplication tab in the Configuration area. There are ten different deduplication schemes that you can choose to apply, depending on what data you have available.