[chris_a]

I've seen a lot of stats posts lately and some conclusions being drawn about some stats. I felt like I should post this so that people on micro know about it. I know this is more of a probability forum post, but it directly affects posts in micro so I think it's appropriate here. Generally speaking anyone who comments about a players stats should know about this so that they don't give bad information based on very low sample size (for instance about someone's blind stealing tendencies that don't converge for a while.)

The fact is, every stat. estimate that's a "% of time" stat. has its own standard deviation. We can determine, based on the number of times such an event happens, what range we can expect the true number to be in.

Generally:
If you have an event, that happens with probability p, and it has happened N times, then you can figure out the standard deviation of the estimated percentage by:

Std. Dev. = sqrt(p*(1-p)/N)

For example if someone calculated there VPIP after 10,000 hands at 20 VPIP. That means that 20% of the time they voluntarily put money into the pot. Then,

Std. Dev. = sqrt(0.2*0.8/10000) = 0.004 = 0.4% They can roughly expect their true VPIP to be within 2 std. dev. of the 20VPIP estimate about 95% of the time. Thus they can expect that their VPIP is between 19.2 and 20.8. You can see that VPIP does converge quickly as this is pretty accurate.

Now, what about a number like "Went to SD %"? You can't compute the standard deviation of this number from the total number of hands. You have to compute it from the number of times the player has seen the flop. You can get that number from the More Detail... button on the general tab in poker tracker. It's listed as the 5th number down "Saw Flop All Hands". So for instance if your number of hands is 10k maybe your saw flop number will be around 3000.

Thus to compute the std. dev. of "Went to SD %" you would take the estimate say it's 35% and N = 3000 (from the previous paragraph). You'd get

Std. Dev. = sqrt(0.35*0.65/3000) = 0.0087 = 0.87%

As you can see it's slightly bigger than the Std. Dev. for VPIP. The expected 95% interval for Went to SD would be 33.3%-36.7%.

For one final example, we'll take the "Won $ at SD %" stat. For this one N is the number of showdowns you've gone to. This is given on the More Detail... page again under "Went to Showdown". An example is:

Went to Showdown: 41.65% (1320 times out of 3169)

The number you'd want to use in computing the std. dev. would be the 1320. So for the std. dev. for Won$@SD if the Won$@SD number was say 55% we'd have:

Std. Dev. = sqrt(0.55*0.45/1320) = 0.0137 = 1.37%

Thus the 95% confidence interval would be 52.3% to about 57.7%. So obviously this stat converges slower than VPIP or Went to SD.

Note: the confidence intervals assume that the stats are Gaussian, but they aren't really, they are binomial. However, after the sample sizes that we are talking about, the approximation becomes good.

Conclusion: You can use the above formula to get the std. dev. for any of these percentage stats, but you have to know the number of times such a thing happens and that affects the range of the %.

Ball Park Figures:
-Your VPIP should be within 1 percentage point of its true value (with 95% confidence) after about 5,000 hands.
-Your Went to SD should be within 1 percentage point of its true value after about 30,000 hands.
-Your Won$ at showdown should be within 1 percentage point of its true value after about 100,000 hands. Not coincidentally, this is about the same as when your win rate is starting to be really converged.

After 10k hands conclusion:
-VPIP is very well converged. It's known plus or minus ~0.8 percentage points.
-Went to Showdown is pretty well converged and you can draw rough conclusions about it. It's known plus or minus ~1.7 percentage points.
-Won $ at SD isn't very converged at all and probably not a lot of conclusions should be drawn about it unless it's in the mid 40%s or lower, or its higher than 60%. It's known plus or minus about 3 percentage points.