SnG Total winnings and ROI confidence intervals--Monte Carlo estimates

[Pokerscott]

I’ve seen a lot of questions like these:

I’ve played fifty $10 SnGs and I am up $200. Am I any good?
My EV over the past 200 SnGs is +15%. Am I any good?

These are interesting (and important!) questions that may have been addressed before, but if not, I have some data that might be interesting to people. Feel free to skip to the results table below if you don’t want to know the details of the analysis.

As many of you know, you can’t give black and white answers to these questions, but you can give a degree of confidence if you carefully define the baseline (null hypothesis) and have the relevant data.

Generally, calculating confidence intervals analytically is quite challenging (especially when your sample is small). Fortunately, you can almost always use Monte Carlo methods to estimate confidence intervals in lieu of analytic solutions. Since computing power is very cheap and coding up the null hypothesis is easy in this case, I thought I would go ahead and figure out the answers to those two common questions.

Here is the setup:
-Party Poker standard setup
--10 players
--Players all pay 1.1 units
--First, second, third paid off 5, 3, 2 units respectively

Null Hypothesis:
-An average player has an equal probability of finishing in each position (1-10)

Method:
Step 1 - determine the number of SnGs that you are interested in (for example 100 SnGs)
Step 2 - Assuming a 10% chance of finishing in each position generate a random number between 1-10 with equal probability. If the number is 1, 2 or 3 the person placed 1st, 2nd, 3rd in that SnG and received the appropriate payout.
Step 3 - Repeat that step 2 100 times keeping track of the total (net) winnings. You now have one possible scenario consistent with the null hypothesis (average player with equal chance of finishing in any place)
Step 4 – Repeat steps 2 and 3 a large number of times to figure out the distribution of results that is consistent with the null hypothesis (in my case I did this 50,000 times for each number of SnGs of interest)
Step 5 – determine the relevant percentiles for the distribution of results.

Results:

Here is the cumulative balances table


................CUMULATIVE BALANCE

..............Number of SnGs played
.......50...100....200.....300....500....1000
PRC
1% -30 -46 -72 -95 -133 -220
5% -24 -36 -58 -77 -110 -185
10% -20 -31 -50 -67 -97 -167
25% -13 -22 -37 -50 -75 -136
50% -5 -11 -21 -31 -51 -100
75% 3 1 -4 -11 -25 -65
90% 11 11 11 7 -2 -32
95% 15 18 20 19 13 -12
99% 25 31 38 40 40 26


And the corresponding ROI table


..............Number of SnGs played
.......50.......100.....200......300......500..... .1000
PRC
1% -0.546 -0.418 -0.327 -0.288 -0.242 -0.200
5% -0.436 -0.327 -0.264 -0.233 -0.200 -0.168
10% -0.364 -0.282 -0.227 -0.203 -0.176 -0.152
25% -0.236 -0.200 -0.168 -0.152 -0.136 -0.124
50% -0.091 -0.100 -0.096 -0.094 -0.093 -0.091
75% 0.055 0.009 -0.018 -0.033 -0.046 -0.059
90% 0.200 0.100 0.050 0.021 -0.004 -0.029
95% 0.273 0.164 0.091 0.058 0.024 -0.011
99% 0.455 0.282 0.173 0.121 0.073 0.024

What does it all mean?

Well, suppose you wanted to know if you are an ‘above-average’ player. If you have played 100 SnGs, then your winnings would need to be larger than 11 units to be 90% confident you are ‘better-than-average’. If you are playing $50/5 SnGs, then if you are up $450 over those 100 SnGs then you can be 90% confident you are an above average player.

The ROI chart is basically the same information transformed into ROI. With the same example, the ROI chart indicates you need to be +10% ROI or above to be 90% confident of being better than average. Since you are paying 1.1 units for each SnG and you played 100 SnGs, your total outlay is 110 units. A +10% ROI would be 11 units which is the same as the cumulative balance chart.

I am likely above average, but can I make money?
This is a little bit different since you would formally want to change the null hypothesis to be consistent with whatever breakeven results you are interested in.

As a reasonably close approximation, you could simply add back in the house cut to get the cumulative winnings associated with breakeven. Using the same example, you paid 0.1 units for each SnG you played for a total of 10 units if you played 100 SnGs. If you wanted to know if you are good enough to ‘break-even’ as opposed to merely being above average, add back in the house cut. Now your cumulative balance would have to be 21 units for a 90% confidence level of being able to overcome the house advantage.

While this approximation is likely ok for ROIs close to baseline (-9.09%), you would formally want to specify exactly how you are getting the particular ROI (e.g. probability of 1st, 2nd, 3rd) in order to get the appropriate distribution under the null.

Let me know if you have questions/comments,

PokerScott

PS sorry about the table formating. If I figure out a way to make it look halfway decent I will update it [img]/images/graemlins/mad.gif[/img]

[The Yugoslavian]

This is interesting. However, my meathod is to just plug crap into Aleo's spreadsheet and it does all the confidence evaluations for me, [img]/images/graemlins/wink.gif[/img].

Besides, I (like everyone else) am the greatest poker player in the history of poker -- so what's there not to be confident about! [img]/images/graemlins/grin.gif[/img]

Yugoslav

[wulfheir]

I made your tables a little more readable while I was watching oprah.

[Pokerscott]

Teach a man to fish on the table formating?

Did you paste an image from excel or something?

Pokerscott

[AleoMagus]

I recently devised a new spreadsheet which calculates the confidence that a player is, in fact, the greatest player in the history of poker

I plugged in what I know about you and it actually only came out to a mere 0.0023%

I came out at about 0.0048%

So clearly, there is a much better chance that I am the greatest player in the history of poker.

I hope this has cleared up any confusion

Regards
Brad S

[The Yugoslavian]

[ QUOTE ]
I recently devised a new spreadsheet which calculates the confidence that a player is, in fact, the greatest player in the history of poker

I plugged in what I know about you and it actually only came out to a mere 0.0023%

I came out at about 0.0048%

So clearly, there is a much better chance that I am the greatest player in the history of poker.

I hope this has cleared up any confusion

Regards
Brad S

[/ QUOTE ]

Nicely played. [img]/images/graemlins/grin.gif[/img]

Yugoslav

[Dominic]

this is all fascinating and great work and all, but if I really want to know if I'm an above average and/or winning player, I look at my bankroll after hundreds of SNGs:

if my bankroll is increasing, I'm above average and winning. If it's decreasing, I'm not.

[img]/images/graemlins/grin.gif[/img]

[Daliman]

TYVM for this info. I have now figured out for SURE that I am an above average player.

[The Yugoslavian]

[ QUOTE ]
TYVM for this info. I have now figured out for SURE that I am an above average player.

[/ QUOTE ]

Not according to ZJ. And ZJ is second only to God (Giga), right?

But yeah, I guess you're *probably* slightly above average. It's pretty close though I'm sure -- I'll let other (like Danielh) elaborate. [img]/images/graemlins/wink.gif[/img]

Yugoslav

[viennagreen]

I don't exactly understand these results-- and perhaps the problem I have is with your Null Hypothesis...

If I'm interpreting it correctly, after 1000 SNGs, if your ROI is -0.011 , you are 95% confident that you are an above average player?

I do understand that given your definition of "average player", it makes sense. But obviously there is a problem--- there is no utility in declaring yourself 95% confident that you are above average if you are down 12 buyins after 1000 SNGs....

That said, I think that using the Monte Carlo method is a an appropriate way of calculating confidence intervals.

I think though, that you can artificially define "above average" by picking a long-term average result, and then extrapolating backwards.

For example-- define "above average" as being UP 10 buy-ins after 1000 SNGs. Then calculate a linear distribution of results that would result in that definition. Then run your simulation.

Just my thoughts....