Re: Are Winrates Normally Distributed?
 

[ QUOTE ]

i think a more realistic hand distribution would be much closer to approaching normality.

[/ QUOTE ]

Where's Capt obvious?

Mine was just for example purposes.

BOTH will move towards normalcy one will just be quicker than the other.

[ QUOTE ]

lose 0sb with P1
lose 1sb with P2
lose .5sb with P3
lose 2 sbs with P4
lose 3sbs with P5
lose 4sbs with P6
lose 5sbs with P7
.
.
.
lose 12bbs with Pn

then the upside:
win 0 with Pa
win
.
.
.
win 55bbs with Pm


[/ QUOTE ]

This is the table I was recommending to be set up. Then run convolutions.

Just like we will never know our win rates to the .00001 per 100 we will never know exactly where the distribution becomes normal.
That's just part of statistics and the reason people use confidence intervals.

The defintion of 'enough' data is how much you have.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
It looks pretty close to normal here, but I think the way you did the sampling is not quite right.

You need to draw random samples from the total group of hands. Chopping them up into blocks is easier, but not appropriate. The way you have done it, we might find a different results using a different database.

[/ QUOTE ]

You probably want to draw largish random blocks of hands to randomize position as much as possible between the samples.

[/ QUOTE ]

OK, I figured out a way to do this in SPSS. I thought the data file would crash my computer but it doesn't

I have a data file with the amount I won/lost for 164,724 hands at 15/30. My win rate over these hands is a pitiful 1.13BB/100.

How large should the samples be and how many should I pull? I was thinking of selecting 10,000 samples of 1000 hands each.

Then I can I plot them and get the skewness, kurtosis, etc.

[/ QUOTE ]

I just ran 1000 samples of 1000 hands each from this list and from the graph and statistics, I am convinced that the win rates for these samples are normally distributed.

I can run a larger job overnight and then post some pretty pictures and stats for you.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
It looks pretty close to normal here, but I think the way you did the sampling is not quite right.

You need to draw random samples from the total group of hands. Chopping them up into blocks is easier, but not appropriate. The way you have done it, we might find a different results using a different database.

[/ QUOTE ]

You probably want to draw largish random blocks of hands to randomize position as much as possible between the samples.

[/ QUOTE ]

OK, I figured out a way to do this in SPSS. I thought the data file would crash my computer but it doesn't

I have a data file with the amount I won/lost for 164,724 hands at 15/30. My win rate over these hands is a pitiful 1.13BB/100.

How large should the samples be and how many should I pull? I was thinking of selecting 10,000 samples of 1000 hands each.

Then I can I plot them and get the skewness, kurtosis, etc.

[/ QUOTE ]

I think 1K hand chunks is probably large enough.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
Josh's graph with (only?) 1500 datapoints suggests otherwise.

[/ QUOTE ]

Actually it doesn't suggest otherwise. It just doesn't suggest anything. If you look closely at the BB/100 graph you will see that it is much too granular to draw any sort of conclusion on convergence properties (how closely it approximates normal). There are only 38 different points on the x-axis each of which have having frequency values between 0 and 11. To even have a shot at drawing a reasonable conclusion we would need something more like the BB/10 granularity at the BB/100 level. This would require a 1.5 million hand DB instead of a 150k hand DB.

I do agree with you, however, that this is interesting and is worth looking into. And I think that Josh's method (looking at empirical data) is the best way to do it because we don't have an accurate theoretical model for a hold'em probability distribution.

Re: Are Winrates Normally Distributed?
 

Max Power,

Isn't what you are describiing a sampling distribution of the mean? Doesn't that always produce a normal distribution even if the population distribution is not normal?

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
Max Power,

Isn't what you are describiing a sampling distribution of the mean? Doesn't that always produce a normal distribution even if the population distribution is not normal?

[/ QUOTE ]

i don't know what it's called, but his method will produce a normal distribution regardless of what the data is, if the samples are large enough.

Re: Are Winrates Normally Distributed?
 

I don't think that tells you anything particularly usefull. A distribution of randomly sampled means will necessarily converge to normality, but doesn't give you any new volatility information.

Re: Are Winrates Normally Distributed?
 

I think a better way to do this would be to run some ARMA models with component GARCH disturbances. This should be run on the time series of bb/x blocked returns. The confidence interval can be computed using http://i35.photobucket.com/albums/d1...stedCIcopy.jpg

with gamma1=skewness, gamma2=kurtosis, C(sub-alpha)=the standard normal Critical value corresponding to the confidence level.

Piece of Cake.

Re: Are Winrates Normally Distributed?
 

Josh, you have Arean open. I can see it. Save a huge text file, load it into input analyzer, and see what kind of p-values it gives you for all the different distributions it tests for.

FWIW, I tried to say this earlier and pete made me doubt it a little with his well-thought-out responses. However, I still want to do more tests on it.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
Max Power,

Isn't what you are describiing a sampling distribution of the mean? Doesn't that always produce a normal distribution even if the population distribution is not normal?

[/ QUOTE ]

You guys are correct. I just basically demonstrated the Central Limit Theorem.

It is not at all useful. It tells us what we already know, but I think that Josh's method was flawed and led him to the wrong conclusion.

The sampling distribution of win rates is normally distributed and it is appropriate to use that distribution to calculate confidence intervals. This is just an empirical demonstration.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]

It is not at all useful. It tells us what we already know, but I think that Josh's method was flawed and led him to the wrong conclusion.

[/ QUOTE ]

while josh's method is flawed in a purely mathematical sense, i think his random variable, profit/hand (or profit/orbit if you prefer), is close enough to a random variable that his results will converge to something very close to a normal distribution for N>100, if he has enough hands.

the major problem here, as has been pointed out, is that the posted graphs aren't based on nearly enough hands.

Re: Are Winrates Normally Distributed?
 

Hey,

MaxPower's post is the solution for all of the problems that we found with PT's sampling errors. It makes no difference how big your database is, because you can draw random 100 hand samples (picked one hand at a time) from it a near infinate number of times. The old way you could only have 1000 x 100 hand samples if you played 100k hands. The new way, there are 100000 nCr 100 ways to draw 100 hands samples (this number is [censored] huge, 200 nCr 100 is like 10^58). If you take millions of these, you could both get a better picture of what the distribution looks like and also figure out what your true standard deviation with a very large level of confidence.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
Hey,

MaxPower's post is the solution for all of the problems that we found with PT's sampling errors. It makes no difference how big your database is, because you can draw random 100 hand samples (picked one hand at a time) from it an infinate number of times. The old way you could only have 1000 x 100 hand samples if you played 100k hands. The new way, there are 100000 nCr 100 ways to draw 100 hands samples (this number is [censored] huge, 200 nCr 100 is like 10^58). If you take millions of these, you could both get a better picture of what the distribution looks like and also figure out what your true standard deviation with a very large level of confidence.

[/ QUOTE ]

how would this give you any new standard deviation data? i can't prove it off the top of my head, but i'm pretty sure this would give the same standard deviation PT would give you over the 100k hands.

EDIT: this assumes PT calculates SD on a per hand basis, which i believe it does, but i could be wrong.

Re: Are Winrates Normally Distributed?
 

Well you can't calculate SD accurately on a per hand basis because the distribution isn't close to normal. You will have a lot of hands that will lose a small amount and then a long tail on the positive side (it will look more like an F-curve or even a chi square distribution). The reason why we do it in 100 hands blocks is to get a more normal distribution.

I always figured that PT did it by sessions (or tables played) and not by # of hands. I remember one time I put a couple big sessions (maybe 500 hands each) and it gave some error like "not enough sessions to calculate SD". So I deleted them and put in 5 or 6 small 30-50 hand sessions and it calculated an SD for me. So while I'm not certain, what it might be doing is finding the SD of the corse of the session and then normalizing it to a BB/100 value. This seems like a very poor way to figure out SD so I could be wrong (and hope I'm wrong quite honestly).

Re: Are Winrates Normally Distributed?
 

woops, my example sucked. deleted. fixing it. attempt 2 here:

[ QUOTE ]
Well you can't calculate SD accurately on a per hand basis because the distribution isn't close to normal.

[/ QUOTE ]

the definition of standard deviation doesn't rely on a normal distribution in any way. you can calculate the standard deviation of any distribution. its meaning obviously varies depending on the distribution though.

Example:

1 hand:
90% -1
10% +9

SDa = sqrt(0.9*1+0.1*9) = sqrt(1.8)

100 hands:
90% -1
100% +9
SDb = sqrt(90*1 + 10*9) = sqrt(180)


and SDb = sqrt(100)*SDa

so i ask, what is the problem with calculating the SD on a per hand basis?

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
[ QUOTE ]
It looks pretty close to normal here, but I think the way you did the sampling is not quite right.

You need to draw random samples from the total group of hands. Chopping them up into blocks is easier, but not appropriate. The way you have done it, we might find a different results using a different database.

[/ QUOTE ]

You probably want to draw largish random blocks of hands to randomize position as much as possible between the samples.

[/ QUOTE ]

OK, I figured out a way to do this in SPSS. I thought the data file would crash my computer but it doesn't

I have a data file with the amount I won/lost for 164,724 hands at 15/30. My win rate over these hands is a pitiful 1.13BB/100.

How large should the samples be and how many should I pull? I was thinking of selecting 10,000 samples of 1000 hands each.

Then I can I plot them and get the skewness, kurtosis, etc.

[/ QUOTE ]

I just ran 1000 samples of 1000 hands each from this list and from the graph and statistics, I am convinced that the win rates for these samples are normally distributed.

I can run a larger job overnight and then post some pretty pictures and stats for you.

[/ QUOTE ]

Max,
Can you do this with sets of 100 hands? I understand that once the sample sizes get large enough the sampling distribution will be about normal, but I'm wondering how close the bb/100 statistic gets us there.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]

Max,
Can you do this with sets of 100 hands? I understand that once the sample sizes get large enough the sampling distribution will be about normal, but I'm wondering how close the bb/100 statistic gets us there.

[/ QUOTE ]

this is a good suggestion.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
[ QUOTE ]

Max,
Can you do this with sets of 100 hands? I understand that once the sample sizes get large enough the sampling distribution will be about normal, but I'm wondering how close the bb/100 statistic gets us there.

[/ QUOTE ]

this is a good suggestion.

[/ QUOTE ]

I don't think so. 100 hands is 10 orbits of full ring, and having a 100 hand block with 3 more button hands or 5 more Big Blind hands is very significant. I think you need large chunks in order to mitigate that.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]

Max,
Can you do this with sets of 100 hands? I understand that once the sample sizes get large enough the sampling distribution will be about normal, but I'm wondering how close the bb/100 statistic gets us there.

[/ QUOTE ]

this is a good suggestion.

[/ QUOTE ]

I don't think so. 100 hands is 10 orbits of full ring, and having a 100 hand block with 3 more button hands or 5 more Big Blind hands is very significant. I think you need large chunks in order to mitigate that.

[/ QUOTE ]

You just made an argument for running the sims with 100 hand sets.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
[ QUOTE ]
[ QUOTE ]

Max,
Can you do this with sets of 100 hands? I understand that once the sample sizes get large enough the sampling distribution will be about normal, but I'm wondering how close the bb/100 statistic gets us there.

[/ QUOTE ]

this is a good suggestion.

[/ QUOTE ]

I don't think so. 100 hands is 10 orbits of full ring, and having a 100 hand block with 3 more button hands or 5 more Big Blind hands is very significant. I think you need large chunks in order to mitigate that.

[/ QUOTE ]

that's a good point. rather than selecting hands at random, he could do something like filtering for 10-handed games only and selecting 10 hands from each position for each 100 hand sample. posting in the cutoff (or anywhere outside of the blinds) screws things up too, so those hands could be ignored. unfortunately that will get rid of a lot of the hands, since they're probably not nearly all at full 10-handed tables. there's a number of things similar to this that you could do to improve the approximation - this is probably the simplest, but as i said, you lose a bunch of data.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]

You just made an argument for running the sims with 100 hand sets.

[/ QUOTE ]

i think his point was that the samples don't provide a realistic approximation of 100 hand blocks since you could easily have, for example, 15 big blind hands, which you almost never would in an actual 100 hand block, so the distributions wouldn't be entirely comparable.

i don't think it really makes much of a difference though.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
[ QUOTE ]

You just made an argument for running the sims with 100 hand sets.

[/ QUOTE ]

i think his point was that the samples don't provide a realistic approximation of 100 hand blocks since you could easily have, for example, 15 big blind hands, which you almost never would in an actual 100 hand block, so the distributions wouldn't be entirely comparable.

i don't think it really makes much of a difference though.

[/ QUOTE ]

Oh I get it now. Yeah that complicates things a bit.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
[ QUOTE ]

You just made an argument for running the sims with 100 hand sets.

[/ QUOTE ]

i think his point was that the samples don't provide a realistic approximation of 100 hand blocks since you could easily have, for example, 15 big blind hands, which you almost never would in an actual 100 hand block, so the distributions wouldn't be entirely comparable.

i don't think it really makes much of a difference though.

[/ QUOTE ]


Picking hands randomly is a better way to to control for these extraneous factors.

I could run 100 hand blocks, but I'm playing poker right now and it will crash my machine. I'll do it at work.

I don't know why you guys are so hung up on 100 hands. It is just an arbitrary number.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]

Picking hands randomly is a better way to to control for these extraneous factors.


[/ QUOTE ]

why do you say that? it's a better way to control for factors like game conditions and tilt, but i don't see how it's a better way to control for position/blinds.

[ QUOTE ]

I could run 100 hand blocks, but I'm playing poker right now and it will crash my machine. I'll do it at work.

I don't know why you guys are so hung up on 100 hands. It is just an arbitrary number.

[/ QUOTE ]

blame it on pokertracker pat.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]


I don't know why you guys are so hung up on 100 hands. It is just an arbitrary number.

[/ QUOTE ]

PT does everything in BB/100, so I'd like to know the significance of this stat over certain sample sizes.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
It looks pretty close to normal here, but I think the way you did the sampling is not quite right.

You need to draw random samples from the total group of hands. Chopping them up into blocks is easier, but not appropriate. The way you have done it, we might find a different results using a different database.

[/ QUOTE ]

Exactly.

It looks like the effect that you're seeing is caused by the fact that playing badly costs you more than playing well earns you. Your $/hand should still be normally distributed, but not if you break it up into temperal blocks because you're more likely to see effects of tilt or playing poorly (no offense intended) or bad tables, etc.

Re: Are Winrates Normally Distributed?
 

OK, I did run 10,000 samples of 100 hands each last night and I can post the results. But before I do, I want to make a point. I do not think the results of my simulation have any practical implications.

Why do we keep track of BB/100 and SD/100? We use these to determine our bankroll needs, how much we can expect to win (confidence intervals), how long one can break even, etc.

I don't know if Mason Malmuth was the first to apply these concepts, but he certainly popularized it. I assume that BB/hour and SD/hour were used in order to simply record keeping and computation. It could have been done per hand, but then you would need to keep track of how many hands you played.

With the advent of internet multitabling, BB/hour was replaced by BB/100, but once again the choice of 100 hands was arbitrary.

We could keep track of win rate and SD on a per hand basis and it would work just as well.

Obviously, the win rate for individual hands are not normally distributed (since you win/lose zero for a majority of you hands), but that does not matter.

What matters is your total sample size, how you compute your test statistic is not important (as long as it is accurate and consistent). We could make it BB/hand, BB/10, BB/1000, or BB/134 and it wouldn't matter.

So if I play 20,000 hands and my BB/100 is 1.5, I know that the sampling distribution for samples of that size is normally distributed.

The fact that the sd is based on 100 hands is also irrelevant, because we use the standard error to compute confidence intervals and that takes the number of hands played into account.

I'm not certain about the bankroll formulas, but I'm pretty sure that it is the same concept.

It has been about 8 years since I studied stats and I am a little out of practice, so please correct me if I am wrong here.

I will post the results of the simulation for those that are interested.

Re: Are Winrates Normally Distributed?
 

OK, this is 10,000 random saples of 100 hands each drawn from 164,724 hands at 15/30 with a win rate of 1.13BB/100.

First, the Descriptive Statistics:

The skeweness is positive and the ratio of the skewness to the standard error of the skewness indicates that it it different from the normal distribution. The distribution is positively skewed - as you can see the mean is higher than the median.

The same is true for the kurtosis. The value are more closely clustered about the mean than in a normal distribution.
http://i3.photobucket.com/albums/y56...scriptives.jpg

This is a test of normality. The significance value indicates that this distribution differs significantly from normal.


http://i3.photobucket.com/albums/y56.../normality.jpg


These are the extreme sample values:
http://i3.photobucket.com/albums/y56...remevalues.jpg

This is a histogram of the distribution with a normal curve superimposed:
http://i3.photobucket.com/albums/y56...J/a096890f.jpg


These are some additional fun plots for the geeks out there:

http://i3.photobucket.com/albums/y56...J/9e85ceb4.jpg
http://i3.photobucket.com/albums/y56...J/00a2c629.jpg
http://i3.photobucket.com/albums/y56...J/11b6c615.jpg

Re: Are Winrates Normally Distributed?
 

excellent post...i was thinking of the "normality test" but couldn't think of the kolmogorov smirnov name...

PS- even in their self named tests, the russions represent smirnov [img]/images/graemlins/smile.gif[/img]

Barron

Re: Are Winrates Normally Distributed?
 

from looking at the q-q plot, it seems that the upper tail of the poker data is fatter than the gaussian, and the lower is thinner. shouldn't this result in large downswings happening less often than a normal assumption would predict?

i need to spend some more time thinking about this, the repetition of trials inherent in poker makes this more complicated than a standard VaR calculation.

Re: Are Winrates Normally Distributed?
 

Nice work Max, thanks a lot.

[ QUOTE ]
So if I play 20,000 hands and my BB/100 is 1.5, I know that the sampling distribution for samples of that size is normally distributed.

[/ QUOTE ]

Don't the numbers you posted indicate that this is not the case? Sampling distributions are going to approximate normal with n sufficiently large. What your numbers seem to show is that n=100 is not large enough.

Correct me if I'm wrong, but you've shown that the sampling distribution for bb/100 is decidedly not normal. What I want to know is how does this affect confidence intervals?

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
Nice work Max, thanks a lot.

[ QUOTE ]
So if I play 20,000 hands and my BB/100 is 1.5, I know that the sampling distribution for samples of that size is normally distributed.

[/ QUOTE ]

Don't the numbers you posted indicate that this is not the case? Sampling distributions are going to approximate normal with n sufficiently large. What your numbers seem to show is that n=100 is not large enough.

Correct me if I'm wrong, but you've shown that the sampling distribution for bb/100 is decidedly not normal. What I want to know is how does this affect confidence intervals?

[/ QUOTE ]

If your BB/100 is based on exactly 100 hands, then the sampling distribution is not normal. If it based on thousands of hands, then it will be.

The BB/100 is just a measure, don't get hung up on it. We can measure height in feet, inches, centimeters, etc. Regardless of the units we use your height stays the same and the distribution stays the same.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
Nice work Max, thanks a lot.

[ QUOTE ]
So if I play 20,000 hands and my BB/100 is 1.5, I know that the sampling distribution for samples of that size is normally distributed.

[/ QUOTE ]

Don't the numbers you posted indicate that this is not the case? Sampling distributions are going to approximate normal with n sufficiently large. What your numbers seem to show is that n=100 is not large enough.

Correct me if I'm wrong, but you've shown that the sampling distribution for bb/100 is decidedly not normal. What I want to know is how does this affect confidence intervals?

[/ QUOTE ]

There is no such thing as sampling distribution for BB/100 (the measure), only a sampling distribution for samples of size X (in this case, 100). If I did the same analysis using BB/hand or BB/1000 or bb/21, I would find the exact same results.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
from looking at the q-q plot, it seems that the upper tail of the poker data is fatter than the gaussian, and the lower is thinner. shouldn't this result in large downswings happening less often than a normal assumption would predict?

i need to spend some more time thinking about this, the repetition of trials inherent in poker makes this more complicated than a standard VaR calculation.

[/ QUOTE ]

These are based on 100 hand samples, so in order win a lot or lose a lot you probably need to play some big pots. Typically , you win more bets when you take down a big pot than you lose bets when you lose a big pot (I am talking about multiway pots here). So the big wins should be bigger than the big losses.

This has got me curious. I will run another one using a very large sample size and see what it looks like.

Re: Are Winrates Normally Distributed?
 

[ QUOTE ]
from looking at the q-q plot, it seems that the upper tail of the poker data is fatter than the gaussian, and the lower is thinner. shouldn't this result in large downswings happening less often than a normal assumption would predict?

[/ QUOTE ]

yes, but not really. what it really suggests is that really bad 100-hand stretches would happen less often than a normal distribution would predict. big downswings occur over 10,000 or more hands, and the sample size should reflect that if you want to know how likely big downswings are. over that many hands though, it should be very very close to a normal distribution, considering that this is quite close as well (even though everyone's saying it's not for some reason... just look at the damn graph, it's pretty close).

anyway, i say "yes, but not really" because the chart with 10,000 samples should be shaped somewhere between this and a normal distribution. it should differ from a normal distribution in the same way, though not by nearly as much. (someone correct me if i've misunderstood on this)

on the other hand, while the shape suggests that really terrible downswings should happen less than in a normal distribution, it also suggests that pain-in-the-ass break-even streaks should be longer on average than a normal distribution suggests, since that's what will happen when you don't get any of those rare, winrate boosting surges that are illustrated on the far right of the graph. you can see this by noting that the graph is "fatter" on the left side of the mean than the right side (the bars near the mean are above the normal on the left side, and below the normal on the right side).

edited some errors. i dont know why i can't type what i'm thinking the first time.

Re: Are Winrates Normally Distributed?
 

I think the way this excellent line research will show is that BB/100 is both:

-approximately normal, but
-signifigantly different than normal.