Got my calculator to lie

[BruceZ]

I was under the impression that calculators are supposed to store enough hidden digits so that the digits on the display are accurate. My HP15C evaluates this:

1 / [1 - (1000000/1000001)^(1/37)]

as 37,037,037.04 when it should be 37,000,019. Excel and the windows calculator get it right.

What policy will guarantee that all digits are significant?

[zerosum]

Interesting. My HP 10B evaluates

1 / [1 - (1000000/1000001)^(1/37)]

as 37,000,037.

[uuDevil]

HP32SII: 37,000,037

Edit: Also, TI89: 37,000,023.310015

[BruceZ]

[ QUOTE ]
HP32SII: 37,000,037

Edit: Also, TI89: 37,000,023.310015

[/ QUOTE ]

37,000,037 is what you would get if did the 1 - first, and then replace the ^ with *. So you would have 1 / (1/1000001 * 1/37).

[Siegmund]

Actually, no, it's extremely rare to find a floating point calculator that knows anything at all about how many digits are accurate in the final answer. A cheap calculator carries no extra digits at all beyond what it displays. HPs, if memory serves, always carry 13 or 14, but only display however many you tell it to display. (Having it display all of them and not hide any is a good way to make yourself aware of when roundoff errors look like they might be happening.)

Any calculator will spit back an incorrect value without giving you a warning, if you abuse it in the right way. Calculating a number very close to one, so that it saves .999999 as (it thinks) significant digits, and then subtracting from 1 is a fine way to cause it to mis-estimate its significant figures. It was sort of a fun game, among the math majors when I was in college, to find things that would trick Mathematica into giving wrong answers (tricking calculators and Excel into giving wrong answers was too easy to be much a challenge.) The good news is that the Wolfram staff played the game too, and it got much harder to fool them with each new version that was released.

The HP48 gives 37,000,037. It, obviously, carried three more, but not six more, digits than the 15C did.

Just lucky that the 32 digits carried in the Windows calculator happen to have been sufficient this time around. But it isn't that the Windows calculator is all that much smarter!

While we're on the subject, the variance formula, E(X^2)-E(X)^2, is notorious for giving fewer than expected (often zero) significant figures on data sets that feature a mismatch between magnitude and spread of the observations, very large numbers (millions) of observations. Statistics theory books don't warn you about it - the formula IS correct, if you work it out exactly - but the numerical programming books give you dire warnings to use a recurrence relation to update variance as you go, not add up the sum of squares and do it at the end.

[uuDevil]

[ QUOTE ]
[ QUOTE ]
HP32SII: 37,000,037

Edit: Also, TI89: 37,000,023.310015

[/ QUOTE ]

37,000,037 is what you would get if did the 1 - first, and then replace the ^ with *. So you would have 1 / (1/1000001 * 1/37).

[/ QUOTE ]

I'm a little rusty w/ RPN, but the key sequence

1000000[ENTER]1000001[DIVIDE]37[1/x][y^x][+/-]1[+][1/x]

gives a result of 37000037 on my HP 32SII.

[BabyJesus]

my TI-83+ just ran off 37,000,023.31

[shday]

Same answer an my TI-80: 37000037

aside: the Python interpreter gives 37000018.952451177 (the same as Microsoft Excel)

[2+2 wannabe]

my HP 48G+ gave 37,000,037

[seabass74]

My TI-92 came back with 37000023.310015, I'm embarrassed I even tried it