Variance problem/question for analysis

[zipppy]

I normally post in STT, but I feel like you guys might be better equipped to answer this question (no offense to STT).

Over in the land of SitNGos, 500 tourneys is considered a good benchmark for determining your skill at any particular level buyin. After 500, positive and negative variance is thought to have evened out sufficiently enough to give an accurate picture of what your ROI can be long term (assuming you don't improve, etc.)

This benchmark, however, is for 10 person sitNgos, with top 3 paid (50%/30%/20%).

Here's the question:
What would be a good benchmark, or another way to ask: how long would it take for variance to even out (for the most part), if the sit and go was only 6 people with the same top 3 payout (50%/30%/20%)?

I think that it could be much lower since you're going to be ITM with a much higher frequency, but I'm not sure how much lower the benchmark would be. Even if you all have never encountered a question like this, I'd greatly appreciate it if anyone would be willing to analyze it and see if there are some ways to mathmatically come up with a reasonable estimate.

Two things:
1) If there is a better forum to ask, point me in that direction.
2) It's not just a random question; there's a site I play on with this payout structure. Also, I'd really like to be able to know how to solve this type of problem in order to do it myself in the future, as I'm often trying new formats. I could see this applying to MTTs as well, which could be useful, at least for me.


THANKS!
>>>ZIPPPY

[mosdef]

[ QUOTE ]
Over in the land of SitNGos, 500 tourneys is considered a good benchmark for determining your skill at any particular level buyin. After 500, positive and negative variance is thought to have evened out sufficiently enough to give an accurate picture of what your ROI can be long term

[/ QUOTE ]

this isn't true. in the STT forum it is generally understood that you need a few THOUSAND SNGs before you can estimate your "real" ROI based on observed ROI. look for a post by eastbay about it.

[zipppy]

[ QUOTE ]
[ QUOTE ]
Over in the land of SitNGos, 500 tourneys is considered a good benchmark for determining your skill at any particular level buyin. After 500, positive and negative variance is thought to have evened out sufficiently enough to give an accurate picture of what your ROI can be long term

[/ QUOTE ]

this isn't true. in the STT forum it is generally understood that you need a few THOUSAND SNGs before you can estimate your "real" ROI based on observed ROI. look for a post by eastbay about it.

[/ QUOTE ]

In any case, I'm still wondering how many it would take for 6 person STTs, and how to calculate it.

[mosdef]

hey,

if you follow the link to eastbay's calcs you should be able to replicate them (with a little computer programming) for a 6 max game using his methods. it's all done via simulation over various ranges of finish distributions, rather than slick probability theory.

[SumZero]

It depends on your distribution of wins. If in a 10 person 1-table SNG your results are 1st 12.6%, 2nd 13.4%, and 3rd 12.4% with OOTM 61.6% for instance then your ROI is 16.4% (these are my current results after 500+ SNG). And assuming a $5+.5 typical payout this means your EV is $0.90. To figure out your stdev = sqrt((126*(19.5-.9)^2+134*(9.5-.9)^2+124*(4.5-.9)^2+616*(-5.5-.9)^2)/1000) [you could use your real raw finishes rather than percentages and some people divide by N while others by N-1 for reasons that aren't really important in the context of any decent size sample] which is 8.96 in my example.

I don't know what typical results are for the 6 handed 1-2-3 model, but for sake of argument lets make them 10/6 times as frequent so that means 1st is now 21%, 2nd is 22.3%, 3rd is 20.7%, and OOTM is 36%. Now your EV, assuming the same buy in, is .9. You stdev is now sqrt((210*(9.5-.9)^2+223*(3.5-.9)^2+207*(.5-.9)^2+36*(5.5-.9)^2)/1000) = 5.64 in this example.

I think you can do a Q&D estimate saying that since 8.96 = 1.59 * 5.64 and since estimates of the mean converge at roughly a sqrt n (i.e., to get an estimate twice as good you need four times as many samples) then you are willing to have a 1.59 times as bad estimate which means roughly 2.5 [1.59^2] times fewer observations. So if 500 was really a good sample size for 10 person than I'd say 200 would work for the 6 person tables like you described assuming similar distributions to the ones I described above.

[pzhon]

[ QUOTE ]
Over in the land of SitNGos, 500 tourneys is considered a good benchmark for determining your skill at any particular level buyin. After 500, positive and negative variance is thought to have evened out sufficiently enough to give an accurate picture of what your ROI can be long term (assuming you don't improve, etc.)

[/ QUOTE ]
This type of folklore is passed from person to person without trasmitting understanding, while far more accurate statements are usually ignored.

<ul type="square">The standard deviation for your ROI after n tournaments is about 170%/squareroot(n).[/list]This allows you to determine how many tournaments you need to get a certain level of accuracy. You can see that you can get some useful information after only 100 tournaments, perhaps enough to conclude that you are probably a winning player, or enough to conclude that you have a lot ot learn. If you want your ROI to be accurate within the percent, you need to play about 100,000 tournaments.

For whatever reason, when the above is mentioned, people say, "No, you need 500 tournaments." Then someone will play the role of the wise veteran, and say, "No, you need over 1000 tournaments before your results mean anything."

[ QUOTE ]

What would be a good benchmark, or another way to ask: how long would it take for variance to even out (for the most part), if the sit and go was only 6 people with the same top 3 payout (50%/30%/20%)?

[/ QUOTE ]
The natural problem is to estimate the standard deviation.

The standard deviation is the squareroot of the variance. The variance is the average of the square of the differences between the result and the average result.

If you assume each place occurs with probability 1/6, the average result is to lose the rake. For convenience, assume this is a $10+1 SNG. The standard deviation is
sqrt(1/6 ((-10)^2+(-10)^2 + (-10)^2 + (2)^2 + (8)^2 + (20)^2)) = $11.31 = 1.03 buy-ins.

The analogue for normal 10-player SNGs is $16.73 = 1.52 buy-ins.

Winning players should see slightly larger standard deviations, which may explain why people see standard deviations closer to 1.7 buy-ins.

[ QUOTE ]

1) If there is a better forum to ask, point me in that direction.


[/ QUOTE ]
This is the right forum.

[ QUOTE ]

I could see this applying to MTTs as well, which could be useful, at least for me.


[/ QUOTE ]
See this post of mine which mentions the results of similar calculations for some MTTs.

[AaronBrown]

It always pays to start simple, then add complexity if necessary.

Suppose the only question were to determine the probability of being in the money. With 10 players, 30% will be in the money. After 500 hands, the standard deviation of times actually in the money for the average player is (500*0.3*0.7)^0.5 = 10.25. With 6 players, 50% will be in the money, so the same calculation is (500*0.5*0.5)^0.5 = 11.18.

With 10 players, the average player expects to be in the money 150 times, 10.25/150 = 6.8%. With 6 players it's 11.18/250 = 4.5%. So it would take 500*4.5/6.8 = 327 hands of six player tournaments to get the same relative error of chance of being in the money as 500 ten player tournaments.

Although ROI is a little more complicated than probability of being in the money, there's no obvious reason to think it changes the relative information based on sample size. If you think it might, it's possible to make a more complicated model, that could give you a more precise result (unless the assumptions are wrong, in which case it might do worse).

[AaronBrown]

The always reliable phzon and I crossed posts. His answer is obviously far more comprehensive about the specific question of ROI in tournaments. Mine is only an attempt to quantify your valid intuition that with more places in the money, you get more useful information out of each tournament.

[mosdef]

[ QUOTE ]
Although ROI is a little more complicated than probability of being in the money, there's no obvious reason to think it changes the relative information based on sample size.

[/ QUOTE ]

It pains me to disagree with my personal favorite 2+2 poster, but I think this is wrong. The variance of the r.v. ROI is much higher than the variance of the r.v. ITM. You need significantly more data to put a confidence interval around your ROI than you do for your ITM.

[ QUOTE ]
If you think it might, it's possible to make a more complicated model, that could give you a more precise result (unless the assumptions are wrong, in which case it might do worse).

[/ QUOTE ]

I think the best thing to do (provided you have the computing power at hand) is to run simulations for a given probability distribution of finishes and, for each set of n trials, see if you fall within various confidence intervals. This is tedious and not very elegant, but it works!

[zipppy]

[ QUOTE ]
[ QUOTE ]
Although ROI is a little more complicated than probability of being in the money, there's no obvious reason to think it changes the relative information based on sample size.

[/ QUOTE ]

It pains me to disagree with my personal favorite 2+2 poster, but I think this is wrong. The variance of the r.v. ROI is much higher than the variance of the r.v. ITM. You need significantly more data to put a confidence interval around your ROI than you do for your ITM.

[ QUOTE ]
If you think it might, it's possible to make a more complicated model, that could give you a more precise result (unless the assumptions are wrong, in which case it might do worse).

[/ QUOTE ]

I think the best thing to do (provided you have the computing power at hand) is to run simulations for a given probability distribution of finishes and, for each set of n trials, see if you fall within various confidence intervals. This is tedious and not very elegant, but it works!

[/ QUOTE ]

Thanks for all the replys!! All are very helpful, and are inspiring me to tackle probability head-on! [img]/images/graemlins/smile.gif[/img]