Skip Navigation

The Computer Journal 1975 18(3):231-233; doi:10.1093/comjnl/18.3.231
© 1975 by British Computer Society
This Article
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (2)
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Hunter, G.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

A quantitative measure of precision

G. Hunter *

York University, Department of Chemistry, 4700 Keele Street, Downsview M3J 1P3, Ontario, Canada

The precision {zeta}b of a real number is defined quantitatively in terms of the fractional error in the number, and the base of the arithmetic in which it is represented. The definition is an extension of the traditional rough measure of precision as the number of significant digits in the number. In binary arithmetic the integral part of {zeta}b is the number of binary digits required to store the number. Conversion of the precision from one base to another (such as binary/decimal) is discussed, and applied to consideration of the intrinsic precision of input/output routines and floating point arithmetic.

Received April 1974.

* York University, Department of Chemistry, 4700 Keele Street, Downsview M3J 1P3, Ontario, Canada


Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?




Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.