Underflow

(Sep 14, 2010)

Earlier I described a problem in mathematical computing for mortality modelling.  This was where an intermediate step resulted in a number too big for the computer to handle, causing the entire calculation to overflow and fail.  The cause is due to the compromises inherent in how computers deal with real numbers, and the solution lies in a bit of careful programming.

Unfortunately, computer representation of real numbers also involve compromises in accuracy as well as scale.  A particular problem area lies in subtracting numbers which differ by a very small amount.  On each computer there is usually a limit number, sometimes denoted epsilon, whereby 1+epsilon or 1-epsilon is still distinguishable fromů

Read more

Tags: floating point, machine arithmetic, precision

Find by key-word


Find by date


Find by tag (show all )