There is no such thing as a confidence interval for sit-n-go\'s.
 

I simply can't take it anymore. There are so many discussions of statistical measures on this forum that are all completely off based, it is ridiculous.

I remember the first time I opened Aleo's spreadsheet and saw confidence intervals. My initial reaction was to laugh. Results of poker tournaments are not random variables. The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

A confidence interval can not be computed under the basis of these conditions. Period. The confidence intervals that are being quoted are a lot like saying that based on the last 100 years, I have a 95% confidence that the Chicago Cubs will win between 54 and 111 games this year. It is kinda cute, but means nothing.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

In theory, you are absolutely correct... and I used to get frustrated by this concept as well.

But Aleo did some very important and underrated work when he compared emperical results to those predicted by assumptions of a normal distribution. What he found is that over enough trials (and "enough" is surprisingly few), SNG results for a standard winning player resemble a normal distribution with remarkable similarity. In fact, by the time you get to a few hundred, you can barely tell the difference.

What that means is that you can apply statistical measures for normal distributions to SNG results and make reliable inferences if you have enough data. There hasn't been much discussion about it since, but I found it remarkably important at the time and didn't thank Aleo appropriately, I fear.

I am not a statistician, so it's quite possible that I am completely misguided by what I've learned. But I have a lot of experience deciphering statistical measures applied to biologic models... something that was formerly felt to be impossible but is now the basis for evidence-based medicine, and I think there is a similar utility here (with regard to an imperfect application working surprisingly well.)

Irieguy

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

the results of a SNG ARE random, but the DISTRIBUTION of your finishes isn't. (if they weren't random, all your results would be the same). the experiment is just like throwing a 4-sided die, but each side does not have the same probability of coming up. (where side 1 is a win, side 2 is 2nd place, side 3 is 3rd place, side 4 is a loss).

your SKILL does not make these results free of luck, it only shifts the "probabilities of certain outcomes". whereas one player might have a 15% chance of winning (getting side 1 to come up), another player might have only a 10% of that happening. the more skillful you are, the more "weighted" your die is, but it doesn't mean there's an "less luck" in determining the outcome (based on the weighted die).

the statistical analysis is used to try and determine how accurate your guess of your percentages is. if you think side 1 comes up for you 15% of the time, etc., you can see what the confidence interval is and such.
the statistics are meaningful.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

Dig up Aleo's work comparing a normal distribution with an ROI distribution about a mean, and refute it for us. That would be a worthwhile contribution.

eastbay

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

Just out of curiosity, what are some examples of data samples that you think can be subject to this sort of statistical analysis?

Lets take for granted purely random processes like the throwing of dice, or flipping coins. What else?

Regards
Brad S

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

Ok, I just said the other day I'd stop talking about stats on here but I can't help myself. It's a problem.

So anyways, I was inclined at first to not take this post too seriously. Lots of people come on here who have no idea about what the stats really mean, and I am sometimes inclined to believe that anyybody who questions the stats is just another person who doesn't want to believe them because they want to think that after 72 SNGs they can be absolutely positive that they are gonna keep their 46% ROI.

But, as I often do, I clicked on your name and read a few of your other posts. You seem to know a little about stats (and actually seemed to trust them up until this post [img]/images/graemlins/confused.gif[/img]). Then I saw this:

[ QUOTE ]
I took a few statistics classes on the way to my statistics degree

[/ QUOTE ]

Which I will assume is the truth.

This would mean that you have much more formal stats background than me. This also makes me want to take your objections a bit more seriously.

From what I have already shown in the past, (and from CLT from what I hear), SNG results do approximate a normal distribution. Rather, if we think of each of our respective finishes probabilities as the actual probability that we will finish in that place in any random SNG, then the confidence calculations should mean something.

What I'm guessing that you have a problem with then, is the thought that in any SNG, our odds of finishing 1st,2nd,3rd,Other are equal to our past frequency of that finish.

This makes some sense to doubt. After all, I have before expressed my opinion about SNG results bunching together based on factors that might affect large groups of SNGs played, even though each SNG 'seems' independent of one another. Obviously, if I am in a bad mood, drunk, sick, tired, etc..., the stats are less meaningful. Similarly, if 12,000 people are on party or 65,000 people are on Party, this too affects whole bunches of SNGs. Of course, if this is your criticism, it applies to all SNG stats. ROI, ITM, etc...

Another problem is the fact that skill changes over time and this has been talked about before. It might not make too much sense to reference SNG stats that are a year old, or from our first few hundred SNGs played. Even something like a dramatic alteration of SNG play (such as after finding these forums, or reading a book) might devalue past SNG results in such a way as to make confidence calculations far less meaningful.

All that said, and even more considered, I still think that you are wrong. The confidence calculations must have some merit, even if less than in the strictest sense of throwing dice and making confidence claims. It's true that a lot of things affect my future SNG results and my skill/tilt/other factors makes the results distribute in some way other than purely random. Still, if I was forced to bet on the kind of distribution my (or another person's) results would take the shape of over the next 100,200,etc SNGs, my past performance would be a huge factor in making this bet. If a solidly established confidence interval indicated a 33% chance that I'd make over $1000 in the next X SNGs and someone gave me 5-1 odds that I wouldn't make that much, I think I'd take that bet. Sure, other factors would weigh in, but assuming such factors as family emergencies, personal high points, skill progression/regression, tilt, etc... to all make up the 'noise' behind the resulting distribution, the analysis seems well founded.

Don't the social sciences make statistical calculations of this sort all the time? We hear about statistical calculations all the time involving suicide rates, car crash rates, workplace injury rates, and other social phenomena and these calculations come complete with confidence intervals. Surely the data cannot be considered totally random and is analagous to something like SNG poker results. Are these kinds of statistical analyses all flawed?

Regards
Brad S

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

I have a feeling that this thread is going to get good, and at the same time make my poor little brain explode.

can't wait

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

You are absolutely correct. As far as credentials (since it was brought up in other posts), I have a Ph.D. in psychology and am employed as a psychometrician for the American Board of Anesthesiology. Here is a post I made a few weeks back, in regard to a similar post about standard deviations of SnGs.



I don’t think you should use standard deviations for SINGLE SNGs. You should combine multiple SNGs to use standard deviation. A standard deviation is only useful when you have a normal distribution of data. For example, let’s assume that:

• The mean height of the American male is 70 inches.
• The standard deviation is 5 inches.
• The data is normally distributed.

The last statement is crucial. It allows you to use z-tables calculate all sorts of things, such as:

68% of American males are between 65 and 75 inches
99% of American males are taller than 55 inches.

Etc etc etc.

SNGs do not have a normal distribution of data (there are only 4 possible outcomes), so you can’t make similar calculations. For example, here is the data that you described:

• Lose $11 60% of the time
• Win $9 13% of the time
• Win $19 13% of the time
• Win $39 14% of the time

• The mean profit is $2.50
• The standard deviation is $16.36

If you had a normal distribution of data, then the following would be true:

• 34% of the time your profit would be between -$13.86 and $2.50 (the difference between the mean and one standard deviation below the mean).

However, in reality, you are making between -$13.86 and $2.50 around 60% of the time (the times you are out of the money).
One way to use standard deviations for SNGs is to combine multiple results until you approximate a normal curve. Twenty-five SNGs seems to be large enough to do that. I used Excel to simulate 30,000 different sets of 25 SNGs, using the following parameters:

• Lose $11 60% of the time
• Win $9 13% of the time
• Win $19 13% of the time
• Win $39 14% of the time
• All results are independent of each other (in other words, the fact that you just won an SNG has no bearing on whether or not you win the next SNG).

The results were:

Mean: $63.14
Standard Deviation: $92.19



Here is a frequency chart of the 30,000 sets. The first column is the amount of winnings over 25 SNGs. The second column is how frequently this occurred. As you can see, the data centers around 65 and spreads out in a nice distribution. I have a graph that shows the normal distribution of this data, but I couldn’t figure out how to paste it:


$ winnings # occurrences
-235 2
-225 3
-215 2
-205 6
-195 13
-185 23
-175 19
-165 41
-155 71
-145 79
-135 111
-125 142
-115 191
-105 251
-95 295
-85 356
-75 494
-65 500
-55 598
-45 755
-35 823
-25 894
-15 944
-5 1022
5 1170
15 1144
25 1242
35 1265
45 1267
55 1264
65 1203
75 1254
85 1236
95 1187
105 1182
115 972
125 1035
135 855
145 844
155 747
165 703
175 593
185 534
195 452
205 388
215 316
225 302
235 258
245 208
255 160
265 119
275 86
285 86
295 78
305 47
315 41
325 46
335 21
345 14
355 8
365 12
375 8
385 4
395 9
405 2
425 2
465 1
Total 30000




You can use this data and z-tables to ascertain a number of useful facts about playing a set of 25 SNGs:


• You will lose money 25% of the time
• 38% of the time, you will gain between $17 and $109 (one-half standard deviation above and below the mean)
• 16% of the time, you will gain more than $155 (one standard deviation above the mean)

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

Aleo,

First let me say that my post was in no way directed toward you. I respect your contributions on this board very much. It was directed more toward the assinine posts such as this that I come across all too often.

[ QUOTE ]
I really question all the posters on here that believe 200 SNGs is a blip and may not represent your true capability. I have been playing SNGs for a while, but just created a spreadsheet to actually track my results this weekend. I've played 43 SNGs this weekend, here are some statistics:

Profit/SNG: $30.67
Std. Deviation: $76.63
t-value (95% confidence interval, DOF=42): 2.018

True profitability at SNGs with 95% confidence:

$7.09 <= Profit/SNG <= $54.26

If you see something wrong with my statistical analysis, please feel free to post. If you think statistics are some kind of hokum, and that this can't possibly be correct just because you don't believe it - don't bother to respond.


[/ QUOTE ]

I haven't checked the math in this example because it's not worth my time, but assuming it is correct, and assuming that there is statistical merit in sng confidence interval calculations, we'd have to give this poster the benefit of the doubt and tell him to quit his day job based on 43 tournaments.

I understand that my original post probably came across as offensive to you, and I apologize for that. I was just reading back through several old posts like this one above last night and it was therapeutic for me to make that post at the time. I will take the time to give a more detailed response to your questions sometime in the next few days.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
I have a feeling that this thread is going to get good, and at the same time make my poor little brain explode.

can't wait

[/ QUOTE ]

its already close for me, and I was a math/cs double major [img]/images/graemlins/wink.gif[/img]

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
[ QUOTE ]
I really question all the posters on here that believe 200 SNGs is a blip and may not represent your true capability. I have been playing SNGs for a while, but just created a spreadsheet to actually track my results this weekend. I've played 43 SNGs this weekend, here are some statistics:

Profit/SNG: $30.67
Std. Deviation: $76.63
t-value (95% confidence interval, DOF=42): 2.018

True profitability at SNGs with 95% confidence:

$7.09 <= Profit/SNG <= $54.26

If you see something wrong with my statistical analysis, please feel free to post. If you think statistics are some kind of hokum, and that this can't possibly be correct just because you don't believe it - don't bother to respond.


[/ QUOTE ]

I haven't checked the math in this example because it's not worth my time, but assuming it is correct, and assuming that there is statistical merit in sng confidence interval calculations, we'd have to give this poster the benefit of the doubt and tell him to quit his day job based on 43 tournaments.

[/ QUOTE ]

I'm certainly out of my league as far as statistics knowledge goes, but it would seem to me that the confidence intervals don't mean anything with such a small sample size. It's sort of like a baseball player who hits a homerun on the first day of the season, that would project for him to hit 162 home runs, which is ridiculous. However, if you instead see that he's hit 20 home runs at the halfway mark of the season, then 40 home runs hit for the season is a much more reasonable and and accurate projection.

It would seem to me that your problem doesn't really lie with Aleo's confidence intervals calculations, but more with people's misguided uses of it over laughably small samples. Of course, I could be looking at it too simplistically, but that's the way it seems to me.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
I simply can't take it anymore. There are so many discussions of statistical measures on this forum that are all completely off based, it is ridiculous.

I remember the first time I opened Aleo's spreadsheet and saw confidence intervals. My initial reaction was to laugh. Results of poker tournaments are not random variables. The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

A confidence interval can not be computed under the basis of these conditions. Period. The confidence intervals that are being quoted are a lot like saying that based on the last 100 years, I have a 95% confidence that the Chicago Cubs will win between 54 and 111 games this year. It is kinda cute, but means nothing.

[/ QUOTE ]

I am INTENSELY plebeian as far as stats go. This is more of an inquiry than a criticism...

But wouldn't the comparison to the Cubs record be a fallacious argument? I think the "skill factors" in a baseball situation are much more varied than in poker (considering over the Cubs' tenure, we have not only different teams, but entirely different rules as well), and most importantly, poker results are intrinsically related to the distribution of cards; and baseball does not have any random variable inherent in it (I'm sure I'm struggling with terms here. While the weather, or somebody's injury, may be considered a variable, what I mean is that there is no variable in which a standard distribution can actually be construed)

If I'm way wrong, or correct, I'd like to hear it: this stuff has always interested me, but I know very little about stats.

Another thing: If the results for a SNG can't fit into this realm due to the presence of skill, and tournament places not being "random variables," wouldn't it be silly to calculate a confidence interval for a cash game winrate as well? The distribution of your hands is random, but the results of them are very highly dependent on your skill relative to the field.

I think the general point I'm making here is that the vast majority of these statistical calculations seem to me to be a very interesting approximation, that is often close to correct in practice, but should be treated lightly due to the ever-changing environmental variables.

So, how dumb am I? [img]/images/graemlins/smile.gif[/img]

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[img]/images/graemlins/shocked.gif[/img] You must be running badly.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
I think the general point I'm making here is that the vast majority of these statistical calculations seem to me to be a very interesting approximation, that is often close to correct in practice

[/ QUOTE ]

Absolutely! I prefer to think of them as common sense intervals.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

On the contrary, I'm running quite well. 26.3% ROI in 285 55's in January. Can someone project how well I'll do if I can get 400 55's played in February? I'm dying to know...

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
It would seem to me that your problem doesn't really lie with Aleo's confidence intervals calculations, but more with people's misguided uses of it over laughably small samples.

[/ QUOTE ]

True that the frustration comes with the misguided use, but the fact remains that the calculations themselves are misguided.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
Just out of curiosity, what are some examples of data samples that you think can be subject to this sort of statistical analysis?


[/ QUOTE ]

Inferential statistics are used to draw inferences about a population from a sample. Many examples would be manufacturing related. For example, the weight or dimensions of any particular widget coming off an assembly line. Also, behavioral studies, such as what does the effect of drinking 12 Heineken's have on one's ability to correctly spell their name in the snow. That's probably a bad example, but you get the idea.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
I simply can't take it anymore. There are so many discussions of statistical measures on this forum that are all completely off based, it is ridiculous.

I remember the first time I opened Aleo's spreadsheet and saw confidence intervals. My initial reaction was to laugh. Results of poker tournaments are not random variables. The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

A confidence interval can not be computed under the basis of these conditions. Period. The confidence intervals that are being quoted are a lot like saying that based on the last 100 years, I have a 95% confidence that the Chicago Cubs will win between 54 and 111 games this year. It is kinda cute, but means nothing.

[/ QUOTE ]

Initially I agreed with you, but the more I read your comments, the less I agree. You seem to be implying that nothing of value can be predicted from your SnG stats. That is incorrect.

[ QUOTE ]
The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution.

[/ QUOTE ] Yes, but there are things you can do statistically to get around that fact. You can combine multiple SnGs into a stat that has a normal distribution. You can use binomial distributions for how often you get into the money, or how often you get first place. In other words, there are meaningful predictions you can make.

[ QUOTE ]
other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

[/ QUOTE ] Skill does not need to be analyzed. The basic assumption is that the underlying skill remains constant. A constant variable can be ignored in doing confidence intervals, etc. However, if your skill level has been increasing as you play more and more SnGs, then it would be difficult to factor that out in doing confidence intervals, etc.

If you will provide me with the following information, I can produce some stats for you:

1. Number of tournaments played
2. Number of 1st place finishes
3. Number of ITM finishes
4. Amount of money won in your first set of 25 SnGs, your second set of 25 SnGs, your third set of 25 SnGs, etc.

If you have only played a small number of SnGs at a particular level, then you will very large confidence intervals and the data will be less meaningful. But they will be correctly calculated.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

If I had read this post 20 years ago I would be able to give a proper answer here, but from my fading memory....

There is something called the central limit theorem that is terribly important in most stats but is generally brushed over. What it says is that the distribution of an average is normal irrespective of the distribution it is drawn from.

This is a fairly strange and counter intuitive result but is true.

Applied to SNG's what this means is that the observed average $ rate per game is an estimate for your true average rate. And what's more, this estimate is normally distributed having a mean of the observed average and a standard deviation of the standard deviation of the observed results divided by the square root of the number of obs.

So it is meaningful to construct a confidence interval around an average win rate in order to 'prove' that you are a winning player. If everyone did this then you would never get people posting averages based upon 30 results. The reason for this is that the observed standard deviation is always going to be relatively large compared with the average win. So there needs to be a large number of obs before root n becomes large enough to produce a distribution that is 95% +'ve. When people reply pointing out that you need 200+ results - this is why they are not kidding!

However your last point about things changing is also another much overlooked point. Remember, that your average is only an estimate drawn from an unknowable distribution of your results. I would suggest that this unknowable distribution is quite variant as you and your opponents change through time. What this means is that the confidence interval that is constructed may no longer be relevant as you play today.

Hence someone could calculate that they are 99% sure that they are a winning player. But in the strictest sense they are 99% that they were a winning player. Future results may or may not follow.

So finally to answer your original question, yes confidence intervals can be constructed. However the actual usefulness is, to me at least, marginal and probably more psychological. The amount of games that you would have to put in to get a tight confidence interval would be large. Having played this many you really should know whether you are a winner or loser.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

So what make you not confident about "confident level" at all? [img]/images/graemlins/cool.gif[/img]

my game is boring enough that I don't even wonder about those confidence level terms. [img]/images/graemlins/cool.gif[/img]

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

Your memory serves you well. I think some people have been misguided by the fact that their distribution of finishes is not normal. You dont need a normal distribution to calculate a confidence interval. As the number of tournaments grows, the results will begin to form a normal distribution. This will give an expected win rate per hour, avg finish, roi, etc. Each with a certain confidence interval. Will this help you in your next tournament? No, but it may help you convince your wife/parents that you can make money. It can help you decide whether to quit your day job.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
[ QUOTE ]
Just out of curiosity, what are some examples of data samples that you think can be subject to this sort of statistical analysis?


[/ QUOTE ]

Inferential statistics are used to draw inferences about a population from a sample. Many examples would be manufacturing related. For example, the weight or dimensions of any particular widget coming off an assembly line. Also, behavioral studies, such as what does the effect of drinking 12 Heineken's have on one's ability to correctly spell their name in the snow. That's probably a bad example, but you get the idea.

[/ QUOTE ]

I think you infer that poker cannot be subject to mean ROI analysis? It is true that your poker skill, along with the slowly improvement of the field do affect your long term trend of your ROI, for example if you play one SNG per day, confidence level can never mean to you. But for guys that play 300-500 game a month assming your skill does not change that dramatically, you can actually infer some meaning results.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

Excellent posts! I concur that your method of normalizing the data definitely does produce the desired effect of removing my main objection to the use of confidence intervals.

However, there is still one point that I strongly disagree with you.

[ QUOTE ]
Skill does not need to be analyzed. The basic assumption is that the underlying skill remains constant. A constant variable can be ignored in doing confidence intervals, etc.

[/ QUOTE ]

Your basic null assumption is a subjective statement that I do not agree with. If my skill level is a constant, why the heck am I spending time reading books and coming to 2+2? Also, it is not only your skill level that would need to be a constant, but the skill levels of your opponents as well.

I do have 1000 sng's at the 55 level. I will do the analysis for batchs of size 25 and 20 to see how normalized the data becomes with only 40 or 50 data points.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
You dont need a normal distribution to calculate a confidence interval.

[/ QUOTE ]

True, only if you want it to mean something.

[ QUOTE ]
As the number of tournaments grows, the results will begin to form a normal distribution.

[/ QUOTE ]

Wrong! However, jcm4ccc has shown that manipulating a large number of results into batches can begin to approximate a normal distribution.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
Excellent posts! I concur that your method of normalizing the data definitely does produce the desired effect of removing my main objection to the use of confidence intervals.

However, there is still one point that I strongly disagree with you.

[ QUOTE ]
Skill does not need to be analyzed. The basic assumption is that the underlying skill remains constant. A constant variable can be ignored in doing confidence intervals, etc.

[/ QUOTE ]

Your basic null assumption is a subjective statement that I do not agree with. If my skill level is a constant, why the heck am I spending time reading books and coming to 2+2? Also, it is not only your skill level that would need to be a constant, but the skill levels of your opponents as well.

I do have 1000 sng's at the 55 level. I will do the analysis for batchs of size 25 and 20 to see how normalized the data becomes with only 40 or 50 data points.

[/ QUOTE ]

Of course there is some long term trending in your ROIs. Just like stock market that I spend a lot of time on, simply because market is trending does not mean you cannot perform reliable statistical analysis on. [img]/images/graemlins/smile.gif[/img]

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
I simply can't take it anymore. There are so many discussions of statistical measures on this forum that are all completely off based, it is ridiculous.

I remember the first time I opened Aleo's spreadsheet and saw confidence intervals. My initial reaction was to laugh. Results of poker tournaments are not random variables. The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

A confidence interval can not be computed under the basis of these conditions. Period. The confidence intervals that are being quoted are a lot like saying that based on the last 100 years, I have a 95% confidence that the Chicago Cubs will win between 54 and 111 games this year. It is kinda cute, but means nothing.

[/ QUOTE ]

I'm no stats expert (just a few undergrad-level classes for my psych major), but it seems to me that forecasting SNG results is exactly analogous to the following problem:

You have a 10-sided die which is weighted so that certain faces come up more often. The exact amount of the weighting is unknown. The only data you are allowed to collect is by rolling the die over and over. With a sufficient number of rolls, you can have a high degree of confidence in the "weight" of each side of the die. At that point you can make meaningful predictions about future probability.

In the money/out of the money is the wrong way to approach this problem IMO. The die does not "know" which faces are assigned to winning or losing money. But if you know the weight of each face, you can easily make (probabilistic) predictions about the future monetary results.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

The OP has point in a way though. One has to be wary of assuming normal distributions in situations which clearly aren't. As an example, the null hypothesis concerning placings in SNG's must be that there is an equal chance of placing in either of 1st through 10th, hence a uniform distribution and not a normal one. (Aleo's ROI calcs are different though.) Then there are also situations where large samples can be assumed normally distributed by approximation but where smaller samples aren't. There are plenty of nice tests for that but it takes large samples. However, large in this context usually means something like 30 or 40. That's nothing.

Not that any of this would stop you from using stats on your results even in cases where you can't assume a normal distribution. After all, no one except maybe a professional statistician is better prepared for testing non-normal samples or using non-parametric tests than a psych grad. [img]/images/graemlins/wink.gif[/img]

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
using non-parametric tests [img]/images/graemlins/wink.gif[/img]

[/ QUOTE ]

Gee, we should rename this forum "abstract math" or "applied statistics" or something like that.

honestly, the analysis here always forced me to open some old dusted statistics books. [img]/images/graemlins/laugh.gif[/img]

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

I think everyone knows that conditions change over time and past performace is not a guarantee of future performance.

I think the bigger misuse of math around here has to do with analyzing individual hands. People routinely quote the EV of certain plays to the hundredth place when part of the calculation is what range of hands you put your opponant on.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
[ QUOTE ]
using non-parametric tests [img]/images/graemlins/wink.gif[/img]

[/ QUOTE ]

Gee, we should rename this forum "abstract math" or "applied statistics" or something like that.

honestly, the analysis here always forced me to open some old dusted statistics books. [img]/images/graemlins/laugh.gif[/img]

[/ QUOTE ]

The real problem is that I use my old stats and numerical analysis books to prop up my monitor... threads like this force me to pull them out, put them back. Very annoying.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

HOW I THINK OF STATS:

I find i can grasp the significance of statistcal data/preditions more easily if i can make a mental picture.

Picture a multi-sided dice (die). There is one side for every result you have logged.

If you play 10 tourneys, and get 1 win, 1 second, 1 3rd, you will have a 10 sided die, with one side a 1, one side a 2, one side a 3, and SEVEN sides are 0.

If you want to know the best statistical "guess" over your next 20 games, roll this 10-sided die 20 times. Sum up the results.

Now, lets say you have 100 games logged. There are 100 sides on our new die. Lets say there are 17 sides for 1st, 13 sides for 2nd, 15 sides for 3rd. And 55 sides for loss.

This is a much better predictor, to roll this 100-sided die.

Now, picture a 1000-sided die. This is better still.


I think the relevance of predictions increases as the dice evolves closer and closer to a sphere.

Good/bad?

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

Limpin'

You absolutely nailed it.

[ QUOTE ]
This is a much better predictor, to roll this 100-sided die.


[/ QUOTE ]

You just came across the key word in this whole discussion: predictor. Much like we predict the Patriots will roll the Eagles, and the Mets will still suck, and Sammy Sosa will struggle to hit 30 bombs without his juice; that's all the more we can hope to do with sng results because we are talking about a subject matter of skill based performance. Has anyone ever seen a confidence interval on the number of yards Tom Brady will throw for in the Super Bowl?

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
HOW I THINK OF STATS:

I find i can grasp the significance of statistcal data/preditions more easily if i can make a mental picture.

Picture a multi-sided dice (die). There is one side for every result you have logged.

If you play 10 tourneys, and get 1 win, 1 second, 1 3rd, you will have a 10 sided die, with one side a 1, one side a 2, one side a 3, and SEVEN sides are 0.

If you want to know the best statistical "guess" over your next 20 games, roll this 10-sided die 20 times. Sum up the results.

Now, lets say you have 100 games logged. There are 100 sides on our new die. Lets say there are 17 sides for 1st, 13 sides for 2nd, 15 sides for 3rd. And 55 sides for loss.

This is a much better predictor, to roll this 100-sided die.

Now, picture a 1000-sided die. This is better still.


I think the relevance of predictions increases as the dice evolves closer and closer to a sphere.

Good/bad?

[/ QUOTE ]

I don't agree with this. Your 1000-sided dice might have a little more precision. For example, your 100-sided dice has 17 sides for 1st, 13 sides for 2nd, and 15 sides for 3rd. If your 1000 sided dice has 170 sides for first, 130 sides for 2nd, and 150 sides for 3rd, then there is no difference. If your 1000 sided dice has 172 sides for first, 135 sides for 2nd, and 148 sides for 3rd, then that dice is a little more precise. But not much.

It's not the size of your dice that matters. It's how often you throw it.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
Limpin'

You absolutely nailed it.

[ QUOTE ]
This is a much better predictor, to roll this 100-sided die.


[/ QUOTE ]

You just came across the key word in this whole discussion: predictor. Much like we predict the Patriots will roll the Eagles, and the Mets will still suck, and Sammy Sosa will struggle to hit 30 bombs without his juice; that's all the more we can hope to do with sng results because we are talking about a subject matter of skill based performance. Has anyone ever seen a confidence interval on the number of yards Tom Brady will throw for in the Super Bowl?

[/ QUOTE ]

I don't agree that it is more difficult to predict something based on past experience just because there is a skill element involved. You wouldn't want to calculate a confidence interval on the number of yards Tom Brady will throw for in the Super Bowl because that is such a unique situation. But look at these stats for his throwing yardage by year:

2002: 3764 yards
2003: 3620 yards
2004: 3692 yards

You don't think you can make a meaningful prediction of his 2005 stats?

The only thing to worry about when skill is involved is if the skill level changes appreciably over time. But you can certainly make meaningful and fairly accurate predictions of future performance based on past performance.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
You don't think you can make a meaningful prediction of his 2005 stats?


[/ QUOTE ]

You just made my entire point. Yes, I absolutely can make a meaningful prediction, but it is just that. Science is not involved in making that "prediction".

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

Interesting discussion. The use of confidence intervals to evaluate the uncertainty of your estimated average win rate (per SNG) is clearly valid, as AgentSq thoughtfully describes in a post that should win this week’s award for Best First 2+2 Post. There is no need for the distribution of finishes to be normally distributed. Indeed, as jcm4ccc shows empirically, the estimated average win rate is already converging nicely toward a normal distribution with only 25 SNGs.

In the example bball904 gives, there’s nothing wrong with the CI the poster provides. (That poster uses the t distribution instead of the z [standard normal] distribution, but as the sample size grows, the t distribution converges to the z distribution – for the sample sizes we’re talking about it’s safe to essentially ignore the distinction.) While bball904 is largely off base in arguing that the calculations are misguided, he’s right on target that they are often over-interpreted - let’s look closer at the example he quotes:

win rate: $30.67, sd=76.63, n=43

95% CI: 7.09 to 54.25
But if we calculate some other CIs:
98% CI: 2.41 to 58.93
99% CI: -0.86 to 62.20

So even at the $31/SNG win rate, we’re not 99% confident that the player is even a winning player. This confirms our intuitive feeling that results of 43 SNGs are insufficient to draw useful conclusions (like, the guy should quit his day job), because the CI is always going to be too wide.

OTOH, if we had a guy with half the win rate, the same SD, and 10 times as many SNGs, we’d be much more confident that he’s a winner –

win rate $15.33, SD=76.63, n=430:

95% CI: 8.06 to 22.59
98% CI: 6.70 to 23.96
99% CI: 5.77 to 24.89

There are challenges, of course. The die-rolling examples of stillnotking and BigLimpin are useful analogies, but the SNG situation is more variable, in all the ways AleoMagus describes. But most of the ways the SNG situation departs from the ideal of “each SNG is an independent trial from the [static] population of all SNGs you might play” come out in the wash. Some of your past SNGs have been 4-tabling while drunk on Monday mornings with 12,000 players on Party and some have been sober, 1-tabling, on Saturday nights with 65,000. But that’s probably the distribution you’ll have in the future, too. The SNG situation is much more like the die-rolling scenario than the # of Cubs wins or # of Tom Brady passing yards situations are. (Why this is true is left as an exercise for the reader. [img]/images/graemlins/smile.gif[/img] ) And the information is much more useful, too.

There’s not much useful application to knowing the CI for Tom Brady’s long-run average SuperBowl yardage. For one thing, you don’t have a stake in it, and for another, Brady’s not going to play in enough more Super Bowls that you could profit from it if you did.

But it is useful to know have an idea of the CI for your SNG avg. win – the confidence interval doesn’t directly answer specific yes/no questions you might have, like, am I a winning player, or should I move up a level, but it does give some insight. And if the player in the 2nd example above (+$15/SNG) wants to ask himself, "Should I spend 40 hours/week over the next month to play 1000 SNGs or do something else with that time?" then it’s relevant for him to know that he can expect to be up somewhere between $6000 and $25000 ahead at the end, assuming he’s playing in the same general conditions as his last 430.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
It's not the size of your dice that matters. It's how often you throw it.

[/ QUOTE ]

I beg to differ. Its not that the bigger dice has more significant figures, its that the sample size to CONSTRUCT the dice is bigger, and therefore is more reflective of true probabilities.

For dramatic effect, say you play 2 games, and make a 2 sided dice (coin). It doesnt matter how many times you flip it.

All im saying is that as the dice evolves into a sphere, it likewwise evolves to a better predictor.

Theres a chance i could "make" (read: sample) a 100-sided die that has 35 first place finishes on it. Doesnt mean i am gonna finish 1st 35 times out of the next 100.

However, a 10000 sider is going to have to be damn close to true likelyhood.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
The use of confidence intervals to evaluate the uncertainty of your estimated average win rate (per SNG) is clearly valid, as AgentSq thoughtfully describes in a post that should win this week’s award for Best First 2+2 Post

[/ QUOTE ]

I agree the AgentSq has provided the most enlightening and most spot on correct post of this entire thread. I second his nomination. However, you didn't state his complete message. The CI's you can construct around your estimated win rate are really only valid for past results. The unresolved issue is that the uncontrolled conditions of all your future experiences are not relative to your past experiences for any confidence intervals you can construct.

[ QUOTE ]
jcm4ccc shows empirically, the estimated average win rate is already converging nicely toward a normal distribution with only 25 SNGs.


[/ QUOTE ]

I agree that jcm4ccc proved that sng results can be grouped so that they will effectly approximate a normal distribution over a large number of data sets. However, he showed this using 30,000 x 25 = 750,000 sng's. That's a few more than 25.

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

[ QUOTE ]
There is something called the central limit theorem that is terribly important in most stats but is generally brushed over. What it says is that the distribution of an average is normal irrespective of the distribution it is drawn from.

This is a fairly strange and counter intuitive result but is true.

Applied to SNG's what this means is that the observed average $ rate per game is an estimate for your true average rate. And what's more, this estimate is normally distributed having a mean of the observed average and a standard deviation of the standard deviation of the observed results divided by the square root of the number of obs.

[/ QUOTE ]


post '_'

post
'_'


'_' post


That is a post going over my head. Anyone have the appropriate animated smiley? [img]/images/graemlins/laugh.gif[/img]

Re: There is no such thing as a confidence interval for sit-n-go\'s.
 

I am 99% percent confident that average math SAT on these forums is at least one standard deviation above the average. [img]/images/graemlins/smirk.gif[/img] and equally confident that mine (720 taken 20+ years ago) is not.

Very interesting discussion.