View Single Post
  #5  
Old 06-21-2005, 04:33 PM
Siegmund Siegmund is offline
Senior Member
 
Join Date: Feb 2005
Posts: 415
Default Re: Got my calculator to lie

Actually, no, it's extremely rare to find a floating point calculator that knows anything at all about how many digits are accurate in the final answer. A cheap calculator carries no extra digits at all beyond what it displays. HPs, if memory serves, always carry 13 or 14, but only display however many you tell it to display. (Having it display all of them and not hide any is a good way to make yourself aware of when roundoff errors look like they might be happening.)

Any calculator will spit back an incorrect value without giving you a warning, if you abuse it in the right way. Calculating a number very close to one, so that it saves .999999 as (it thinks) significant digits, and then subtracting from 1 is a fine way to cause it to mis-estimate its significant figures. It was sort of a fun game, among the math majors when I was in college, to find things that would trick Mathematica into giving wrong answers (tricking calculators and Excel into giving wrong answers was too easy to be much a challenge.) The good news is that the Wolfram staff played the game too, and it got much harder to fool them with each new version that was released.

The HP48 gives 37,000,037. It, obviously, carried three more, but not six more, digits than the 15C did.

Just lucky that the 32 digits carried in the Windows calculator happen to have been sufficient this time around. But it isn't that the Windows calculator is all that much smarter!

While we're on the subject, the variance formula, E(X^2)-E(X)^2, is notorious for giving fewer than expected (often zero) significant figures on data sets that feature a mismatch between magnitude and spread of the observations, very large numbers (millions) of observations. Statistics theory books don't warn you about it - the formula IS correct, if you work it out exactly - but the numerical programming books give you dire warnings to use a recurrence relation to update variance as you go, not add up the sum of squares and do it at the end.
Reply With Quote