confidence intervals?

[jasonHoldEm]

Total hands - 26315
Total hours - 431.13
BB/Hr - 1.46
stddev/hr - 13.04

I'm not sure if all the above are necessary info, but how do I figure out my "actual" win rate...isn't there some formula that says XX% of the time I will make between X and Y? Thanks.

J

[Jezebel]

Divide your SD by the sqaureroot of the number of hours played for your standard error. For the info you provided your standard error is .63

Your true winrate will be + or - one standard error 68% of the time.

Your true winrate will be + or - 1.3 standard error 80% of the time.

Your true winrate will be + or - 1.6 standard error 90% of the time.

Your true winrate will be + or - 2 standard errors 95% of the time.

So it appears that there is a 95% chance that your true win rate lies between .2 BB/hr and 2.72 BB/hr. The more hours you play the tigher this range will become.

[fluff]

Your can be 95% confident that your win rate falls between 0.229BB/hr and 2.69BB/hr.

Or between WR-1.96*SD/(Time^0.5) and WR+1.96*SD/(Time^0.5)

[jasonHoldEm]

Thanks guys!

[jds1201]

I've been racking my brain over the past few weeks trying to figure out how the relationship between my confidence in my win rate and the distribution of true win rates play together.

For example, let's say that I've got a win rate of 3BB/hr and a standard error of 1. The bell curve around my win rate says that I could be 95% confident that my true win rate is 3 +- 2BB/hr (so between 1 and 5).

What is puzzling me that it seems to me that the low end of the range (1) is much more likely than the high end of the range (5) because there are a lot more 1BB/Hr players out there than there are 5BB/Hr. Taking it a step further, I would expect almost all of the outlying 5% to be to the low side instead of the high side, not 2.5% on each side, yet the normal distribution around my result would seem to indicate that the chances of these results are equal.

I'm sure I'm missing something simple (I haven't worked with this stuff in a long time), but I just can't seem to figure out how this distribution interacts with the distribution of all players.

I hope this made some sense.

Thanks.

JDS

[Am Alert]

You are not missing anything, stop racking your brain. It is an overtly simplistic approach to apply normal distribution to every problem and expect reliable results.

[BruceZ]

yet the normal distribution around my result would seem to indicate that the chances of these results are equal.

In fact you are missing something very important and very subtle which many people miss regarding the definition of confidence intervals. The confidence interval does not tell you the probability that your win rate lies in that interval. Instead it gives the probability of obtaining your results if your win rate were in that interval. Your win rate would have to lie between 1 and 5 bb/hr in order to achieve your results or better with a probability of 95%. This means there is a 95% confidence that your win rate is between 1 and 5 bb/hr, but confidence and probability are two different things.

The win rate and standard deviation you computed are called maximum likelihood estimates. This is because they are the values which maximize the probability of obtaining your results, and not because they are the most likely estimates of those values. In fact, there are other intervals which give a 95% probability, but the one which is symmetrical to your computed win rate can be shown to be the shortest.

In order to compute the probability that your win rate lies within some interval, you need a different form of statistics called Bayesian statistics. In Bayesian estimation, you take into account any prior knowledge you already have about where your win rate is likely to lie by assuming a prior probability distribution of your win rate. This prior distribution would take into account the distribution of win rates among all players. If you think it is very unlikely that your win rate is above 5, then this would be reflected in the prior distribution. Your data is then used to update the prior distribution to yield a posterior distribution for the win rate which is a true probability distribution. The final distribution you get depends on the prior distribution you start with, so the results can be somewhat controversial. This contrasts with using maximum likelihood estimation and confidence intervals which depend only on the observed data.

[irchans]

Bruce,

You have some very important points in your last post most of which I understand and agree with. The idea of using Bayesian statistics is very good. I would think a normal a prior distribution of winrate with a standard deviation of 2 BB per sqrt hour and mean of 0 BB per hour would be reasonable. Then the ML estimator for win rate would be something like

estimate = WinRateObserved / ( 1 + 2*SDestimate/Sqrt[hours])


If instead we chose a uniform distribution say between -1000 BB per hour and +1000 BB per hour, then I wonder if we could then say that there is a 95% probability that the win rate is within 2 standard errors of the observed win rate.


I do have one question about your first paragraph.

[ QUOTE ]
The confidence interval does not tell you the probability that your win rate lies in that interval. Instead it gives the probability of obtaining your results if your win rate were in that interval. Your win rate would have to lie between 1 and 5 bb/hr in order to achieve your results or better with a probability of 95%.

[/ QUOTE ]

I think there must be a word or two wrong in those statements. If your win rate was 100 bb/hr, then you would achieve those "results or better." Am I missing something here?

Cheers,
Irchans

[irchans]

AmAlert,

I agree that "It is an overtly simplistic approach to apply normal distribution to every problem and expect reliable results." Stock market analysts make this mistake all the time. There are situations where normal distributions are good and there are situations where normal distributions are bad. I would think that a normal distribution would work well for estimating win rates of poker with hundreds of hours of data. What would be a better distribution? How much improvement could we expect if we used a better distribution?

Cheers,
Irchans

[BruceZ]

Your win rate would have to lie between 1 and 5 bb/hr in order to achieve your results or better with a probability of 95%. This means there is a 95% confidence that your win rate is between 1 and 5 bb/hr, but confidence and probability are two different things.

This first sentence was worded wrong; sorry if it caused confusion. What I intended to explain was that if your true win rate were to lie below 1 bb/hr, then there would have been less than a 2.5% probability of obtaining your result or better, and if your true win rate were to lie above 5 bb/hr, then there would have been less than a 2.5% probability of obtaining your win rate or worse. So if your true win rate lies outside of this interval from 1 to 5 bb/hr, then there would have been less than a 5% probability of obtaining your result, or a result which is farther from the true win rate. It is correct to describe this situation by stating that there is a 95% confidence that your true win rate lies between 1 and 5 bb/hr as stated. The important point is that the confidence interval describes the probability of your results given particular win rates, not the probability of the true win rate.

Another way to look at this is that before you played these hours, there actually was a 95% probability that your observed win rate would fall within 2 standard errors of your true win rate. However, after you play, and your win rate and standard error are numbers rather than variables, then you can no longer make this statement unless you substitute the word “confidence” for “probability”.