There is no such thing as a confidence interval for sit-n-go's.

[bball904]

I simply can't take it anymore. There are so many discussions of statistical measures on this forum that are all completely off based, it is ridiculous.

I remember the first time I opened Aleo's spreadsheet and saw confidence intervals. My initial reaction was to laugh. Results of poker tournaments are not random variables. The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

A confidence interval can not be computed under the basis of these conditions. Period. The confidence intervals that are being quoted are a lot like saying that based on the last 100 years, I have a 95% confidence that the Chicago Cubs will win between 54 and 111 games this year. It is kinda cute, but means nothing.

[Irieguy]

In theory, you are absolutely correct... and I used to get frustrated by this concept as well.

But Aleo did some very important and underrated work when he compared emperical results to those predicted by assumptions of a normal distribution. What he found is that over enough trials (and "enough" is surprisingly few), SNG results for a standard winning player resemble a normal distribution with remarkable similarity. In fact, by the time you get to a few hundred, you can barely tell the difference.

What that means is that you can apply statistical measures for normal distributions to SNG results and make reliable inferences if you have enough data. There hasn't been much discussion about it since, but I found it remarkably important at the time and didn't thank Aleo appropriately, I fear.

I am not a statistician, so it's quite possible that I am completely misguided by what I've learned. But I have a lot of experience deciphering statistical measures applied to biologic models... something that was formerly felt to be impossible but is now the basis for evidence-based medicine, and I think there is a similar utility here (with regard to an imperfect application working surprisingly well.)

Irieguy

[Xhiggy]

the results of a SNG ARE random, but the DISTRIBUTION of your finishes isn't. (if they weren't random, all your results would be the same). the experiment is just like throwing a 4-sided die, but each side does not have the same probability of coming up. (where side 1 is a win, side 2 is 2nd place, side 3 is 3rd place, side 4 is a loss).

your SKILL does not make these results free of luck, it only shifts the "probabilities of certain outcomes". whereas one player might have a 15% chance of winning (getting side 1 to come up), another player might have only a 10% of that happening. the more skillful you are, the more "weighted" your die is, but it doesn't mean there's an "less luck" in determining the outcome (based on the weighted die).

the statistical analysis is used to try and determine how accurate your guess of your percentages is. if you think side 1 comes up for you 15% of the time, etc., you can see what the confidence interval is and such.
the statistics are meaningful.

[eastbay]

Dig up Aleo's work comparing a normal distribution with an ROI distribution about a mean, and refute it for us. That would be a worthwhile contribution.

eastbay

[AleoMagus]

Just out of curiosity, what are some examples of data samples that you think can be subject to this sort of statistical analysis?

Lets take for granted purely random processes like the throwing of dice, or flipping coins. What else?

Regards
Brad S

[AleoMagus]

Ok, I just said the other day I'd stop talking about stats on here but I can't help myself. It's a problem.

So anyways, I was inclined at first to not take this post too seriously. Lots of people come on here who have no idea about what the stats really mean, and I am sometimes inclined to believe that anyybody who questions the stats is just another person who doesn't want to believe them because they want to think that after 72 SNGs they can be absolutely positive that they are gonna keep their 46% ROI.

But, as I often do, I clicked on your name and read a few of your other posts. You seem to know a little about stats (and actually seemed to trust them up until this post [img]/images/graemlins/confused.gif[/img]). Then I saw this:

[ QUOTE ]
I took a few statistics classes on the way to my statistics degree

[/ QUOTE ]

Which I will assume is the truth.

This would mean that you have much more formal stats background than me. This also makes me want to take your objections a bit more seriously.

From what I have already shown in the past, (and from CLT from what I hear), SNG results do approximate a normal distribution. Rather, if we think of each of our respective finishes probabilities as the actual probability that we will finish in that place in any random SNG, then the confidence calculations should mean something.

What I'm guessing that you have a problem with then, is the thought that in any SNG, our odds of finishing 1st,2nd,3rd,Other are equal to our past frequency of that finish.

This makes some sense to doubt. After all, I have before expressed my opinion about SNG results bunching together based on factors that might affect large groups of SNGs played, even though each SNG 'seems' independent of one another. Obviously, if I am in a bad mood, drunk, sick, tired, etc..., the stats are less meaningful. Similarly, if 12,000 people are on party or 65,000 people are on Party, this too affects whole bunches of SNGs. Of course, if this is your criticism, it applies to all SNG stats. ROI, ITM, etc...

Another problem is the fact that skill changes over time and this has been talked about before. It might not make too much sense to reference SNG stats that are a year old, or from our first few hundred SNGs played. Even something like a dramatic alteration of SNG play (such as after finding these forums, or reading a book) might devalue past SNG results in such a way as to make confidence calculations far less meaningful.

All that said, and even more considered, I still think that you are wrong. The confidence calculations must have some merit, even if less than in the strictest sense of throwing dice and making confidence claims. It's true that a lot of things affect my future SNG results and my skill/tilt/other factors makes the results distribute in some way other than purely random. Still, if I was forced to bet on the kind of distribution my (or another person's) results would take the shape of over the next 100,200,etc SNGs, my past performance would be a huge factor in making this bet. If a solidly established confidence interval indicated a 33% chance that I'd make over $1000 in the next X SNGs and someone gave me 5-1 odds that I wouldn't make that much, I think I'd take that bet. Sure, other factors would weigh in, but assuming such factors as family emergencies, personal high points, skill progression/regression, tilt, etc... to all make up the 'noise' behind the resulting distribution, the analysis seems well founded.

Don't the social sciences make statistical calculations of this sort all the time? We hear about statistical calculations all the time involving suicide rates, car crash rates, workplace injury rates, and other social phenomena and these calculations come complete with confidence intervals. Surely the data cannot be considered totally random and is analagous to something like SNG poker results. Are these kinds of statistical analyses all flawed?

Regards
Brad S

[ReDeYES88]

I have a feeling that this thread is going to get good, and at the same time make my poor little brain explode.

can't wait

[jcm4ccc]

You are absolutely correct. As far as credentials (since it was brought up in other posts), I have a Ph.D. in psychology and am employed as a psychometrician for the American Board of Anesthesiology. Here is a post I made a few weeks back, in regard to a similar post about standard deviations of SnGs.



I don’t think you should use standard deviations for SINGLE SNGs. You should combine multiple SNGs to use standard deviation. A standard deviation is only useful when you have a normal distribution of data. For example, let’s assume that:

• The mean height of the American male is 70 inches.
• The standard deviation is 5 inches.
• The data is normally distributed.

The last statement is crucial. It allows you to use z-tables calculate all sorts of things, such as:

68% of American males are between 65 and 75 inches
99% of American males are taller than 55 inches.

Etc etc etc.

SNGs do not have a normal distribution of data (there are only 4 possible outcomes), so you can’t make similar calculations. For example, here is the data that you described:

• Lose $11 60% of the time
• Win $9 13% of the time
• Win $19 13% of the time
• Win $39 14% of the time

• The mean profit is $2.50
• The standard deviation is $16.36

If you had a normal distribution of data, then the following would be true:

• 34% of the time your profit would be between -$13.86 and $2.50 (the difference between the mean and one standard deviation below the mean).

However, in reality, you are making between -$13.86 and $2.50 around 60% of the time (the times you are out of the money).
One way to use standard deviations for SNGs is to combine multiple results until you approximate a normal curve. Twenty-five SNGs seems to be large enough to do that. I used Excel to simulate 30,000 different sets of 25 SNGs, using the following parameters:

• Lose $11 60% of the time
• Win $9 13% of the time
• Win $19 13% of the time
• Win $39 14% of the time
• All results are independent of each other (in other words, the fact that you just won an SNG has no bearing on whether or not you win the next SNG).

The results were:

Mean: $63.14
Standard Deviation: $92.19



Here is a frequency chart of the 30,000 sets. The first column is the amount of winnings over 25 SNGs. The second column is how frequently this occurred. As you can see, the data centers around 65 and spreads out in a nice distribution. I have a graph that shows the normal distribution of this data, but I couldn’t figure out how to paste it:


$ winnings # occurrences
-235 2
-225 3
-215 2
-205 6
-195 13
-185 23
-175 19
-165 41
-155 71
-145 79
-135 111
-125 142
-115 191
-105 251
-95 295
-85 356
-75 494
-65 500
-55 598
-45 755
-35 823
-25 894
-15 944
-5 1022
5 1170
15 1144
25 1242
35 1265
45 1267
55 1264
65 1203
75 1254
85 1236
95 1187
105 1182
115 972
125 1035
135 855
145 844
155 747
165 703
175 593
185 534
195 452
205 388
215 316
225 302
235 258
245 208
255 160
265 119
275 86
285 86
295 78
305 47
315 41
325 46
335 21
345 14
355 8
365 12
375 8
385 4
395 9
405 2
425 2
465 1
Total 30000




You can use this data and z-tables to ascertain a number of useful facts about playing a set of 25 SNGs:


• You will lose money 25% of the time
• 38% of the time, you will gain between $17 and $109 (one-half standard deviation above and below the mean)
• 16% of the time, you will gain more than $155 (one standard deviation above the mean)

[bball904]

Aleo,

First let me say that my post was in no way directed toward you. I respect your contributions on this board very much. It was directed more toward the assinine posts such as this that I come across all too often.

[ QUOTE ]
I really question all the posters on here that believe 200 SNGs is a blip and may not represent your true capability. I have been playing SNGs for a while, but just created a spreadsheet to actually track my results this weekend. I've played 43 SNGs this weekend, here are some statistics:

Profit/SNG: $30.67
Std. Deviation: $76.63
t-value (95% confidence interval, DOF=42): 2.018

True profitability at SNGs with 95% confidence:

$7.09 <= Profit/SNG <= $54.26

If you see something wrong with my statistical analysis, please feel free to post. If you think statistics are some kind of hokum, and that this can't possibly be correct just because you don't believe it - don't bother to respond.


[/ QUOTE ]

I haven't checked the math in this example because it's not worth my time, but assuming it is correct, and assuming that there is statistical merit in sng confidence interval calculations, we'd have to give this poster the benefit of the doubt and tell him to quit his day job based on 43 tournaments.

I understand that my original post probably came across as offensive to you, and I apologize for that. I was just reading back through several old posts like this one above last night and it was therapeutic for me to make that post at the time. I will take the time to give a more detailed response to your questions sometime in the next few days.

[asofel]

[ QUOTE ]
I have a feeling that this thread is going to get good, and at the same time make my poor little brain explode.

can't wait

[/ QUOTE ]

its already close for me, and I was a math/cs double major [img]/images/graemlins/wink.gif[/img]