Standard Deviation of PT Stats (Light Math)

[chris_a]

I've seen a lot of stats posts lately and some conclusions being drawn about some stats. I felt like I should post this so that people on micro know about it. I know this is more of a probability forum post, but it directly affects posts in micro so I think it's appropriate here. Generally speaking anyone who comments about a players stats should know about this so that they don't give bad information based on very low sample size (for instance about someone's blind stealing tendencies that don't converge for a while.)

The fact is, every stat. estimate that's a "% of time" stat. has its own standard deviation. We can determine, based on the number of times such an event happens, what range we can expect the true number to be in.

Generally:
If you have an event, that happens with probability p, and it has happened N times, then you can figure out the standard deviation of the estimated percentage by:

Std. Dev. = sqrt(p*(1-p)/N)

For example if someone calculated there VPIP after 10,000 hands at 20 VPIP. That means that 20% of the time they voluntarily put money into the pot. Then,

Std. Dev. = sqrt(0.2*0.8/10000) = 0.004 = 0.4% They can roughly expect their true VPIP to be within 2 std. dev. of the 20VPIP estimate about 95% of the time. Thus they can expect that their VPIP is between 19.2 and 20.8. You can see that VPIP does converge quickly as this is pretty accurate.

Now, what about a number like "Went to SD %"? You can't compute the standard deviation of this number from the total number of hands. You have to compute it from the number of times the player has seen the flop. You can get that number from the More Detail... button on the general tab in poker tracker. It's listed as the 5th number down "Saw Flop All Hands". So for instance if your number of hands is 10k maybe your saw flop number will be around 3000.

Thus to compute the std. dev. of "Went to SD %" you would take the estimate say it's 35% and N = 3000 (from the previous paragraph). You'd get

Std. Dev. = sqrt(0.35*0.65/3000) = 0.0087 = 0.87%

As you can see it's slightly bigger than the Std. Dev. for VPIP. The expected 95% interval for Went to SD would be 33.3%-36.7%.

For one final example, we'll take the "Won $ at SD %" stat. For this one N is the number of showdowns you've gone to. This is given on the More Detail... page again under "Went to Showdown". An example is:

Went to Showdown: 41.65% (1320 times out of 3169)

The number you'd want to use in computing the std. dev. would be the 1320. So for the std. dev. for Won$@SD if the Won$@SD number was say 55% we'd have:

Std. Dev. = sqrt(0.55*0.45/1320) = 0.0137 = 1.37%

Thus the 95% confidence interval would be 52.3% to about 57.7%. So obviously this stat converges slower than VPIP or Went to SD.

Note: the confidence intervals assume that the stats are Gaussian, but they aren't really, they are binomial. However, after the sample sizes that we are talking about, the approximation becomes good.

Conclusion: You can use the above formula to get the std. dev. for any of these percentage stats, but you have to know the number of times such a thing happens and that affects the range of the %.

Ball Park Figures:
-Your VPIP should be within 1 percentage point of its true value (with 95% confidence) after about 5,000 hands.
-Your Went to SD should be within 1 percentage point of its true value after about 30,000 hands.
-Your Won$ at showdown should be within 1 percentage point of its true value after about 100,000 hands. Not coincidentally, this is about the same as when your win rate is starting to be really converged.

After 10k hands conclusion:
-VPIP is very well converged. It's known plus or minus ~0.8 percentage points.
-Went to Showdown is pretty well converged and you can draw rough conclusions about it. It's known plus or minus ~1.7 percentage points.
-Won $ at SD isn't very converged at all and probably not a lot of conclusions should be drawn about it unless it's in the mid 40%s or lower, or its higher than 60%. It's known plus or minus about 3 percentage points.

[Stuey]

Very nice post. Wish I had something to add but I would likely create confusion. Your post states some very important facts. I read GTAOT and it cleared up SD as it relates to BR management, and winrates. But it is easy to forget how the SD can skew the importance of a number. Thanks for the info.

[dozer]

Can you tell me what these mean, I've been waiting to ask someone that knows about standard deviation.

[jaxUp]

They basically mean that variance sucks ass. The longer version basically says that every hundred hands you play, you're on average, 15.6327 BBs from what you would expect to win.

(this is very possibly wrong...wait for validation).

[UncleSalty]

[ QUOTE ]
They basically mean that variance sucks ass.

[/ QUOTE ]

Variance in Texas Hold'em is God's greatest gift to skilled players. Without variance allowing absolute morons to score some very big wins on a semi-regular basis, we would very quickly run out of fish and end up playing each other. (Yikes.)

Say it with me kids..."I love variance".

Statistically speaking, your standard deviation is used to provide a level of confidence in your winrate. It's been a long time since I took a stats class, so I'm sure some of our resident math whizzes can provide more clarity.

Basically, you can be about 99% confident that your "true" winrate is within 3 standard deviations on either side of your current average winrate after 13K hands. I believe the confidence interval for just one SD is about 78% or so.

So, what it really means is that your 13K hand sample size doesn't tell you dick about your true winrate.

[jrz1972]

[ QUOTE ]
Basically, you can be about 99% confident that your "true" winrate is within 3 standard deviations on either side of your current average winrate after 13K hands. I believe the confidence interval for just one SD is about 78% or so.

[/ QUOTE ]

You're getting "standard deviation" confused with "standard error."

Chris had a thread a while back describing how to go from SD to SE and how to estimate the confidence interval of your winrate.

[UncleSalty]

Gotcha, thanks.

[chris_a]

[ QUOTE ]
They basically mean that variance sucks ass.

[/ QUOTE ]

This standard deviation isn't terrible at all for this win rate.

[ QUOTE ]
The longer version basically says that every hundred hands you play, you're on average, 15.6327 BBs from what you would expect to win.

[/ QUOTE ]

Yeah that's the intuitive feel though it's not exactly right mathematically.

After 100 hands, you have won some amount X. You were expected to win 3.5 BBs or whatever. But you won X.

Variance = expected value of (X-3.5)^2
Std. Dev. is square root of variance.

Sorry to be pedantic.

[yellowjack]

Ok, I'll try very hand to explain this in the simplest terms possible.

Getting the definitions of the way, standard deviation is a measure of our friend (and enemy), variance.

It means that on any other day, if you held the same cards but the community cards came out differently for all of your hands in the database:

You could easily lose 15.6327 BB/100 more from what you actually won because of horrible community cards coming up (bad luck).
Conversely, you could easily have won 15.6327 BBs more than what you have aactually won, due to great community cards coming up (good luck).

In fact, your true winrate for all your hands on average (average good and bad community cards) is

WR (+ and -) 15.6327 BBs/100
where WR is the winrate you happened to get for all your sessions.
i.e. PokerTracker says you won 3.2 BB/100 for all your sessions, then WR = 3.2

So your true winrate would be:
= 3.2 (+ and -) 15.6327 BBs/100
So the minimum true winrate is (3.2BB/100) - (15.6327BB/100) = -12.4327 BB/100
Your maximum true winrate is (3.2BB/100) + (15.6327BB/100) = 18.8327BB/100

The range of your true winrate is between -12.4327BB/100 and 18.8327BB/100.
You put as an interval like this (-12.4327, 18.8327). That's a huge range, because it doesn't even tell you if you're a winning or losing player yet!
Also, this huge range isn't even that accurate. We can only say that your true winrate is between (-12.4327, 18.8327 with a 68.4% level of confidence. We're 68.4% sure that your winrate is anywhere between there. So we're not even that sure. However we can say that your winrate is in (-28.0654, 34.4654) with a 95% level of confidence.

The theory behind the previous paragraph is something like this:

A standard deviation is used to show the variance (we've all heard that term, right?) of statistics of the same category, i.e. the statistic of hit avg. for all Yankee players.

In short, the sample mean (+ or -) 1 s.d. gives you a 68.4% probability that any randomly selected Yankee player has a hit avg. between the interval (mean-1sd, mean+1sd).

For more info that has minimal jargon, try this.

jaxup, reply as soon as you can so I can touch this up for other posters who are in the same boat as you.

[dozer]

Thanks for explaining it to me, I kinda understand it now. I would useThis to get a more acurate look at my true winrate, but I don't even have excel on my computer. I actually don't really care too much about my true winrate for now since I know that the sample size is small, I was just curious what the numbers in pokertracker meant under standard deviation.