Roundoff Error
Roundoff error is a form of noise that pervades all calculations performed on computers (other than those dealing strictly with integers). Roundoff error occurs because computers can only use finite strings of digits to represent any given number, while many numbers are often not representable by finite strings. Numbers as commonplace as or 1/3, for example, cannot be perfectly represented as finite strings: 1/3 = .3333333 . . . (the 3's repeat forever). If one represents 1/3 using, say, five digits of precision (1/3 = 3.3333 x 10-1), a roundoff error has been introduced of 1/3 - (3.3333 x 10-1) = 3.333. . . x 10-6. Such errors may seem small, but they can and do accumulate in long chains of calculations. In some studies, including simulations of chaotic phenomena (physical processes extremely sensitive to small changes in their initial conditions), roundoff error may even lead to completely erroneous outcomes. What is more, since roundoff error may accumulate differently on different computers, completely different erroneous outcomes might be obtained on different machines.
An example of error accumulation is as follows. Consider the calculation 1/3 + 1/3 = 1/6. If 1/3 is represented first as 3.3333 x 10-1, then 1/3 + 1/3 is given by 3.3333 x 10-1 + 3.3333 x 10-1 = 6.6666 x 10-1.
Not only does this result have a roundoff error of 6.6666 . . . x 10-6, which is double the error for either of the two operands, but if one were to calculate 1/6 by dividing 6.0000 directly into 1.0000 one would obtain an answer (6.6667 x 10-1) with a different roundoff error: 1/6 - (6.6667 x 10-1) = -3.3333 . . . x 10-6. The most powerful computer thus calculates "1/3 + 1/3 = 1/6" less precisely than a sixth-grader armed with a pencil (assuming the sixth-grader knows how to add fractions).
There are two remedies for roundoff error, and they must be employed in parallel. One is brute numerical force: if enough bits are used, the roundoff error can usually be made small enough to not matter. It is for this reason that an optional double-precision form of floating-point arithmetic (a form using about twice as many bits) is mandated by such standards as the IEEE (Institute of Electrical and Electronic Engineers) Standard for Binary Floating-Point Arithmetic (Standard 754-1985). Variables of this type are thus available to programmers of all machines that do floating-point arithmetic. The other remedy is to choose a good technique for rounding. In the example above, rounding 1/6 to 6.6667 x 10-1 involved application of the rounding method always taught in schools, the "round to nearest" rule: If the digit after the rightmost digit to be retained is 5 or larger, increment the rightmost digit to be retained; if it is less than 5, do not increment. Thus 1.345 rounds up to 1.35, 1.346 also rounds up to 1.35, and 1.344 rounds down to 1.34. Alternatively, one could simply drop all extra digits, regardless of what they are (the "always round down" rule). By this method (also called chopping or truncation), even 1.346 rounds down to 1.34. Though it sounds crude some computers actually do chop, as it involves no logical operations and is therefore fast.
The problem with the usual round-to-nearest rule is that it is unfair. In computer calculations, results can and sometimes do land exactly halfway between two representable numbers, and by the round-to-nearest rule these halfway results will always be rounded up. Rounding up therefore occurs more often than rounding down, introducing a slight bias or slant. This bias can be overcome in at least two ways: (1) Round halfway numbers up or down, whichever gives the nearest even number. This will result in rounding up and down about equally often. This "round-to-even" rule is employed in many computers. (2) Decide which way to round on the basis of a random (or pseudorandom) number. This latter method has the advantage that the effects of roundoff error can be varied from one program run to the next, allowing the changing effects of roundoff error (if any) to be observed. Unlike other rounding techniques random rounding is strictly a research tool, and is not built into processor hardware.
This is the complete article, containing 698 words
(approx. 2 pages at 300 words per page).
