There is no such thing as a confidence interval for sit-n-go's.

[bball904]

I simply can't take it anymore. There are so many discussions of statistical measures on this forum that are all completely off based, it is ridiculous.

I remember the first time I opened Aleo's spreadsheet and saw confidence intervals. My initial reaction was to laugh. Results of poker tournaments are not random variables. The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

A confidence interval can not be computed under the basis of these conditions. Period. The confidence intervals that are being quoted are a lot like saying that based on the last 100 years, I have a 95% confidence that the Chicago Cubs will win between 54 and 111 games this year. It is kinda cute, but means nothing.

[Irieguy]

In theory, you are absolutely correct... and I used to get frustrated by this concept as well.

But Aleo did some very important and underrated work when he compared emperical results to those predicted by assumptions of a normal distribution. What he found is that over enough trials (and "enough" is surprisingly few), SNG results for a standard winning player resemble a normal distribution with remarkable similarity. In fact, by the time you get to a few hundred, you can barely tell the difference.

What that means is that you can apply statistical measures for normal distributions to SNG results and make reliable inferences if you have enough data. There hasn't been much discussion about it since, but I found it remarkably important at the time and didn't thank Aleo appropriately, I fear.

I am not a statistician, so it's quite possible that I am completely misguided by what I've learned. But I have a lot of experience deciphering statistical measures applied to biologic models... something that was formerly felt to be impossible but is now the basis for evidence-based medicine, and I think there is a similar utility here (with regard to an imperfect application working surprisingly well.)

Irieguy

[Xhiggy]

the results of a SNG ARE random, but the DISTRIBUTION of your finishes isn't. (if they weren't random, all your results would be the same). the experiment is just like throwing a 4-sided die, but each side does not have the same probability of coming up. (where side 1 is a win, side 2 is 2nd place, side 3 is 3rd place, side 4 is a loss).

your SKILL does not make these results free of luck, it only shifts the "probabilities of certain outcomes". whereas one player might have a 15% chance of winning (getting side 1 to come up), another player might have only a 10% of that happening. the more skillful you are, the more "weighted" your die is, but it doesn't mean there's an "less luck" in determining the outcome (based on the weighted die).

the statistical analysis is used to try and determine how accurate your guess of your percentages is. if you think side 1 comes up for you 15% of the time, etc., you can see what the confidence interval is and such.
the statistics are meaningful.

[eastbay]

Dig up Aleo's work comparing a normal distribution with an ROI distribution about a mean, and refute it for us. That would be a worthwhile contribution.

eastbay

[AleoMagus]

Just out of curiosity, what are some examples of data samples that you think can be subject to this sort of statistical analysis?

Lets take for granted purely random processes like the throwing of dice, or flipping coins. What else?

Regards
Brad S

[bball904]

[ QUOTE ]
Just out of curiosity, what are some examples of data samples that you think can be subject to this sort of statistical analysis?


[/ QUOTE ]

Inferential statistics are used to draw inferences about a population from a sample. Many examples would be manufacturing related. For example, the weight or dimensions of any particular widget coming off an assembly line. Also, behavioral studies, such as what does the effect of drinking 12 Heineken's have on one's ability to correctly spell their name in the snow. That's probably a bad example, but you get the idea.

[rachelwxm]

[ QUOTE ]
[ QUOTE ]
Just out of curiosity, what are some examples of data samples that you think can be subject to this sort of statistical analysis?


[/ QUOTE ]

Inferential statistics are used to draw inferences about a population from a sample. Many examples would be manufacturing related. For example, the weight or dimensions of any particular widget coming off an assembly line. Also, behavioral studies, such as what does the effect of drinking 12 Heineken's have on one's ability to correctly spell their name in the snow. That's probably a bad example, but you get the idea.

[/ QUOTE ]

I think you infer that poker cannot be subject to mean ROI analysis? It is true that your poker skill, along with the slowly improvement of the field do affect your long term trend of your ROI, for example if you play one SNG per day, confidence level can never mean to you. But for guys that play 300-500 game a month assming your skill does not change that dramatically, you can actually infer some meaning results.

[ericlambi]

[ QUOTE ]
Inferential statistics are used to draw inferences about a population from a sample. Many examples would be manufacturing related. For example, the weight or dimensions of any particular widget coming off an assembly line. Also, behavioral studies, such as what does the effect of drinking 12 Heineken's have on one's ability to correctly spell their name in the snow. That's probably a bad example, but you get the idea.

[/ QUOTE ]

and

[ QUOTE ]
I remember the first time I opened Aleo's spreadsheet and saw confidence intervals. My initial reaction was to laugh. Results of poker tournaments are not random variables. The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

A confidence interval can not be computed under the basis of these conditions. Period. The confidence intervals that are being quoted are a lot like saying that based on the last 100 years, I have a 95% confidence that the Chicago Cubs will win between 54 and 111 games this year. It is kinda cute, but means nothing.


[/ QUOTE ]

If you actually believe these things, you either didn't take the time to think this through, or you need to go back to statistics school.

This is the exact same analysis done to evaluate past performance (and predict future performance) of mutual fund portfolio managers and day traders. If you believe there is no skill involved in making investments that outperform the general market, you are wrong. If you believe these professionals aren't continually evaluating their own job performance and finding ways to improve, you are wrong. If you believe that this analysis sort of analysis is not appropriate in this situation, you are VERY wrong.

As a matter of course, when predicting the expected return of investments given past data, you never have the luxury of hundreds of data points. You are lucky if you have N=20. Yet useful insight is gathered every day from this sort of data.

[AleoMagus]

Ok, I just said the other day I'd stop talking about stats on here but I can't help myself. It's a problem.

So anyways, I was inclined at first to not take this post too seriously. Lots of people come on here who have no idea about what the stats really mean, and I am sometimes inclined to believe that anyybody who questions the stats is just another person who doesn't want to believe them because they want to think that after 72 SNGs they can be absolutely positive that they are gonna keep their 46% ROI.

But, as I often do, I clicked on your name and read a few of your other posts. You seem to know a little about stats (and actually seemed to trust them up until this post [img]/images/graemlins/confused.gif[/img]). Then I saw this:

[ QUOTE ]
I took a few statistics classes on the way to my statistics degree

[/ QUOTE ]

Which I will assume is the truth.

This would mean that you have much more formal stats background than me. This also makes me want to take your objections a bit more seriously.

From what I have already shown in the past, (and from CLT from what I hear), SNG results do approximate a normal distribution. Rather, if we think of each of our respective finishes probabilities as the actual probability that we will finish in that place in any random SNG, then the confidence calculations should mean something.

What I'm guessing that you have a problem with then, is the thought that in any SNG, our odds of finishing 1st,2nd,3rd,Other are equal to our past frequency of that finish.

This makes some sense to doubt. After all, I have before expressed my opinion about SNG results bunching together based on factors that might affect large groups of SNGs played, even though each SNG 'seems' independent of one another. Obviously, if I am in a bad mood, drunk, sick, tired, etc..., the stats are less meaningful. Similarly, if 12,000 people are on party or 65,000 people are on Party, this too affects whole bunches of SNGs. Of course, if this is your criticism, it applies to all SNG stats. ROI, ITM, etc...

Another problem is the fact that skill changes over time and this has been talked about before. It might not make too much sense to reference SNG stats that are a year old, or from our first few hundred SNGs played. Even something like a dramatic alteration of SNG play (such as after finding these forums, or reading a book) might devalue past SNG results in such a way as to make confidence calculations far less meaningful.

All that said, and even more considered, I still think that you are wrong. The confidence calculations must have some merit, even if less than in the strictest sense of throwing dice and making confidence claims. It's true that a lot of things affect my future SNG results and my skill/tilt/other factors makes the results distribute in some way other than purely random. Still, if I was forced to bet on the kind of distribution my (or another person's) results would take the shape of over the next 100,200,etc SNGs, my past performance would be a huge factor in making this bet. If a solidly established confidence interval indicated a 33% chance that I'd make over $1000 in the next X SNGs and someone gave me 5-1 odds that I wouldn't make that much, I think I'd take that bet. Sure, other factors would weigh in, but assuming such factors as family emergencies, personal high points, skill progression/regression, tilt, etc... to all make up the 'noise' behind the resulting distribution, the analysis seems well founded.

Don't the social sciences make statistical calculations of this sort all the time? We hear about statistical calculations all the time involving suicide rates, car crash rates, workplace injury rates, and other social phenomena and these calculations come complete with confidence intervals. Surely the data cannot be considered totally random and is analagous to something like SNG poker results. Are these kinds of statistical analyses all flawed?

Regards
Brad S

[ReDeYES88]

I have a feeling that this thread is going to get good, and at the same time make my poor little brain explode.

can't wait