Choosing between models

(Aug 13, 2008)

In any model-fitting exercise you will be faced with choices. What shape of mortality curve to use? Which risk factors to include? How many size bands for benefit amount? In each case there is a balance to be struck between improving the model fit and making the model more complicated.

Our preferred method of measuring model fit is the log-likelihood function, but this on its own does not take account of model complexity. For example it is usually possible to make a model fit better - i.e. increase the log-likelihood value - by adding extra parameters and risk factors. But is this extra complexity justified? Are those extra parameters and risk factors earning their keep in the model?

There are a number of different…

Read more

Tags: AIC, log-likelihood, model fit

Choosing between models - a business view

(Aug 13, 2008)

We discussed how we use the AIC to choose between models. The standard definition of the AIC is:

AIC = -2 * log-likelihood + 2 * number of parameters

However, this is a statistician's view of a model, where the only criterion for including a parameter is whether it is statistically significant. A business view might be different, as each extra parameter in a system will cost you money. IT systems have to be specified, programmed, tested and maintained, for example, and IT staff are not cheap. Each extra parameter might therefore cost you £5,000 in development costs (say), so you might be inclined to only include parameters if they are really significant. One way of doing this is to increase the penalty for the number…

Read more

Tags: AIC, log-likelihood, model fit

Find by key-word


Find by date


Find by tag (show all )