How confident can Little Johnny be in Confidence Intervals??

[Lexander]

Just adding a few thoughts.

A CI is not quite the same as a pure probability calculation. If all the underlying assumptions used to calculate the CI are correct, the actual estimated parameter will be within the interval 95% of the time if you are using an alpha of .05.

A CI will be wrong 5% of the time in this situation even if everything else you calculated is completely accurate. You have reason to suspect this sample is extreme. Your understanding of Poker sense tells you so. The CI calculation doesn't know any of this. It assumes the sample is just some random sample and makes no such assumptions.

Another way to think of it is to imagine every Poker player in the world performing this same calculation. What should happen is that about 95% of the calculated intervals should accurately estimate the actual rate of all those players. Little Johnny just happens to be one of those 5%.

As an aside, an alpha of 0.05 turns out to work pretty well in many cases. The lower the alpha, the less Type I errors you make but the increase in Type II errors means your Statistical Power decreases too quickly. Such is the tradeoff.