*Macmillan Science Library: Mathematics*. Copyright © 2001-2006 by Macmillan Reference USA, an imprint of the Gale Group. All rights reserved.

This section contains 938 words(approx. 4 pages at 300 words per page) |

## Precision

Precision depends on the unit used to obtain a measure. The smaller the unit, the more precise the measure. Consider measures of time, such as 12 seconds and 12 days. A measurement of 12 seconds implies a time between11.5 and 12.5 seconds. This measurement is precise to the nearest second, with a maximum potential error of 0.5 seconds. A time of 12 days is far less precise. Twelve days suggests a time between 11.5 and 12.5 days, yielding a potential error of 0.5 days, or 43,200 seconds! Because the potential error is greater, the measure is less precise. Thus, as the length of the unit increases, the measure becomes less precise.

The number of decimal places in a measurement also affects precision. A time of 12.1 seconds is more precise than a time of 12 seconds; it implies a measure precise to the nearest tenth of a second. The potential error in12.1 seconds is 0.05 seconds, compared with a...

This section contains 938 words(approx. 4 pages at 300 words per page) |