There is no such thing as a confidence interval for sit-n-go's.

[ericlambi]

[ QUOTE ]
Inferential statistics are used to draw inferences about a population from a sample. Many examples would be manufacturing related. For example, the weight or dimensions of any particular widget coming off an assembly line. Also, behavioral studies, such as what does the effect of drinking 12 Heineken's have on one's ability to correctly spell their name in the snow. That's probably a bad example, but you get the idea.

[/ QUOTE ]

and

[ QUOTE ]
I remember the first time I opened Aleo's spreadsheet and saw confidence intervals. My initial reaction was to laugh. Results of poker tournaments are not random variables. The data distribution of 1st, 2nd, 3rd, other certainly does not represent a normal distribution. I even believe there is a degree of skill involved in the actual outcome of these poker tournaments. Skill is not really something that statisticians believe they can analyze to any degree of confidence.

A confidence interval can not be computed under the basis of these conditions. Period. The confidence intervals that are being quoted are a lot like saying that based on the last 100 years, I have a 95% confidence that the Chicago Cubs will win between 54 and 111 games this year. It is kinda cute, but means nothing.


[/ QUOTE ]

If you actually believe these things, you either didn't take the time to think this through, or you need to go back to statistics school.

This is the exact same analysis done to evaluate past performance (and predict future performance) of mutual fund portfolio managers and day traders. If you believe there is no skill involved in making investments that outperform the general market, you are wrong. If you believe these professionals aren't continually evaluating their own job performance and finding ways to improve, you are wrong. If you believe that this analysis sort of analysis is not appropriate in this situation, you are VERY wrong.

As a matter of course, when predicting the expected return of investments given past data, you never have the luxury of hundreds of data points. You are lucky if you have N=20. Yet useful insight is gathered every day from this sort of data.

[AleoMagus]

LOL! Well, I can tell you that I did not expect to come back to this thread and see 50 replies. Is anyone even still reading this far down.

And... I'm really dissapointed with the way in which this thread had unfolded. There have been some insightful posts, but there have also been some posts that made me cringe. I actually thought I understood why you might have a problem with the confidence interval calculations and I suppose that it had something to do with my own problems with them.

I think I was wrong. My own problems with those calculations remain far more pertinant than much of what has been brought up in this thread. Not only that, but lots of what has been brought up in this thread is also patently wrong.

I don't know where to start, but I'm going to give it a shot anyways. I do seriously doubt that many are still reading, but I also would have never guessed that there would be this many responses in this thread in 24 hours. (and this, after eastbay's almost simultaneous 'empirical equity study' post. Boy, do you guys have your SNG priorities mixed up)

Anyways...

I'll start with Jcm4ccc's post about the empirical evidence that a 25 SNG set forms approximates a normal distribution. I wish instead of simply quoting that thread directly, you would have linked to it instead, as I reponded in that thread and I think my observations are still relevant.

original post

so, clearly, you do not need to become obsessed with using 25 SNG samples, and treating each one like a single element of a larger sample. Once you hit 25-50 SNGs, the data starts to approximate a normal distribution and you can do all the usual calculations meaningfully. You demonstarted this empirically, and previous to this, I demonstrated this with pure math.

QED

this should adequately show the confusion present in this post:

[ QUOTE ]
Quote:
--------------------------------------------------------------------------------

As the number of tournaments grows, the results will begin to form a normal distribution.


--------------------------------------------------------------------------------



Wrong! However, jcm4ccc has shown that manipulating a large number of results into batches can begin to approximate a normal distribution.



[/ QUOTE ]

It is bball who is wrong here. As the number of tourneys grow, they do form normal distribution, and 'manipulating large numbers of results' is not necessary.

I never understood it at the time, but I think bozeman stated on 2+2 ages ago that this was true because of CLT. In any case, if the stipulations of CLT hold for SNG results, then the calculations are valid

What are these stipulations, and what exactly does CLT say?

Central Limit theorem: the fundamental statistical result that if a sequence of independent identically distributed random variables each has a finite variance, then as their number increases, their sum approaches a normally distributed random variable.

So, the grounds on which these calculations are most easily attacked are really on what I have already mentioned; SNG results may not be independent of one another and it is this fact that undermines the normal distribution assumption.

This is actually a criticism taht I agree with. SNG results are not independent. Past SNG results help us improve, put us on tilt, or are affected by similar and 'local' factors which can affect even large groups of SNG results.

All that Jcm4ccc's empirical evidence (and my combinatorial proof) show is that IF sng results are totally random and independent (like in a computer simulation) then the calculations will work for a sample large enough to approximate a normal distribution (about 25-50 SNGs).

We know this isn't true, but the better question is 'How not true'? That sounds silly now that I say it, but I'm still saying it.

Despite the local interdependence of SNG results, over time we still see what appears to be some kind of random distribution and although it is slightly more streaky (or less streaky depending on exact influences) than we might expect in a computer simulation, the normal distribution estimates do seem to be a good predictor. There is some kind of valid induction going on here, and MWC did a good job explaining why when he described SNG independence as eventually working out 'in the wash'. This all brings me to bball's statement:

[ QUOTE ]
I prefer to think of them as common sense intervals.

[/ QUOTE ]

and the totally bizarre later comment

[ QUOTE ]
Yes, I absolutely can make a meaningful prediction, but it is just that. Science is not involved in making that "prediction".

[/ QUOTE ]

to which I can only say... whatever [img]/images/graemlins/confused.gif[/img]. Just a meaningful prediction... but no such thing as a confidence interval... though it does make meaningful predictions.

If science is not involved, then what is? Is it valid reasoning? What about the superbowl yardage example? It seemed to show that in a game of skill, there would actually be a pretty narrow confidence interval and that statistical reasoning works pretty well. The same kind of statistical reasoning that you do not think applies in a 'scientific' sense.

At this point it appears we may be moving into the realm of a more philosophical discussion about the foundations of probability and Induction. Is this sort of induction valid? To this you can have your opinion and clearly adhere to some kind of subjectivist view of probability, but rest assured, this is not the mainstream view. Frequentists would still say that given no other information than past frequencies, the predictions that we make are still good.

Now usually, we do have other info. I don't seriously get drunk and stay awake for 36 hours after getting in a fight with my girlfriend, then log on to Party at 11:00 AM on a monday and think

"I've got a X% ROI quad tabling the $22s. I guess I'll play seeing as how I can expect $y profit this hour."

Obviously, local factors will greatly affect my results. Nobody doubts this. Given no other info however, such as when we look into a fairly long term future (the next month, six months, year) I can make some pretty reasonable predictions. Now maybe I will improve, and maybe a SNG book will get published that will wise up the majority of fish, but I don't know those things yet. When I do discover them, I can modify my predictions. Just because I later discover relevant info, doesn't mean that the predictions made at the time were wrong, just incomplete.

If you do not think that is scientific, you are on a slippery slope. Much of what we consider even hard science works this way. Besides, who cares what we call it? Is it rational? Is it valid?

I'll finish by just asking one simple question. If my stats for my last 2000 SNGs say that in the next 100 SNGs, there is a 45% chance that I will make over $X, and you are REQUIRED to bet at that I will either do this, or not do this. You have no other info. What would you choose? Why? Is your decision rational? Is the confidence interval irrelevant?

Regards
Brad S

[eastbay]

Very patient post, Aleo. I've composed - and deleted - no less than 3 responses in this thread, simply because I didn't have patience to lay out the checkmate against the OP as you have done here. And frankly, it felt a bit like a lost cause.

As long as we're talking stats: interesting how the degree of emotion ("I can't stand it", "laugh", etc.) expressed in an argument and correctness of that argument are, in my experience, inversely correlated.

eastbay

[hansarnic]

Quick question, grateful if someone can answer in a language I can understand [img]/images/graemlins/wink.gif[/img]

I think we all acknowledge that at each level the SnG structure (& especially's Party's) has some natural constraints to sustainable max win / loss rates. This is obviously assuming that:

1) Your opponents are trying to win (however bad they are)
2) They understand the basic rules (3 pay, flush beats str8, etc.)

So it therefore strikes me that Confidence intervals must be flawed if someone's past results exceed these min / max rates. In fact surely there must be doubt if past results are very close to these parameters, especially where there aren't many past results.

All of this would point to the need to revise confidence interval calculations with the inherent SnG constraints somehow factored into predictions of future earnings.

Or have I misunderstood?

Thx in advance

[rachelwxm]

wow, I am amazed by your patience to writing such long post. Sorry if my post does not include any real content, so does the original one. [img]/images/graemlins/cool.gif[/img]

Sometimes i have trouble distinguish real post and humorous ones. [img]/images/graemlins/cool.gif[/img]

Oh, I am not a journal man any more?