Variance, winrates, and expectation (long)

[beachbum]

We talk a lot about downswings in this forum, mainly because SH play has a bigger variance inherent to it compared to full ring play. After I hit my first 300 BB downswing last month, I almost felt relieved. My thoughts were, "well, it's going to happen eventually so might as well learn to deal with it." It was almost like I had popped my poker cherry, so to speak.

So that got me thinking on how normal this magnitude of a downswing is. We all talk about winrates, not being results-oriented, sample sizes, etc. alot. I thought it might be interesting to graph some different winrate and variance scenarios to better visualize these concepts. Check out the following graphs:

http://www.tourney.com/images/1BB.jpg http://www.tourney.com/images/2BB.jpg

The 2 graphs are basically the same. The only difference is that the first graph illustrates variance considering a 1 BB/100 winrate, and the second graph shows 2 BB/100. The axes are labeled showing big bets earned and # of hands played. Both graphs are built on the same scale so it's easy to compare the two. Also, I used a standard deviation of 15 BB/100 for both.

The black line in the center shows a consistent zero-variance winrate (for reference). The colored lines on both sides of the winrate line show 2 different sets of limits concerning variance. Basically, your results should fall within the outer bands 99% of the time, and fall within the inner bands 95% of the time.

From these graphs some interesting observations can be drawn.


Downswing Severity

Looking at these graphs, the assumption is that the player is playing at a certain "skill level" (whether at a 1 BB/100 or 2 BB/100). Of course, this skill level will never be constant through a sample number of hands. It will fluctuate based on the typical variables of table selection, seat selection, whether the player is tired, distracted, on tilt, etc.

Take a look at the 2 BB/100 graph. If a player played at a solid 2 BB skill level for a sample size of let's say 20,000 hands, he really shouldn't have a very big downswing. If the player was able to maintain this quality of play even through lots of bad beats and cold cards, only about 5% of the time should he experience a downswing of greater than 110 BB's.

However, if this player's play now degrades to a 1 BB skill level, his downswings can be much more brutal. The same 5% of the time his downswing can get past 200 BB's. But a smaller percent of the time the downswing can get to be pretty high even surpassing the magic 300 BB number.


Break-even streaks

I think the break-even streches really get more magnified as a player's play degrades. In the 2 BB graph, it's fairly unlikely that a player will have a breakeven streak of more than 20,000 hands. However, if he's only playing at a 1 BB level, this same percentage of the time this breakeven streak can surpass 80,000 hands. Hell, it could be possible to be down after 100,000 hands and still be an overall 1 BB player.

Seeing the difference in these 2 graphs makes me want to focus even more on improving my skill level. Even though a 1 BB winrate is solid, there will definitely be times that will shake your confidence, to say the least. This tends to be a snowball effect too as bad results tend to bring about bad play and further degrade a player's winrate to even a negative number if tilt becomes a problem.

I guess bottom line we should continue to focus on our quality of play instead of getting so caught up with winrates and money won. It'll make life a lot easier /images/graemlins/smile.gif.


Predictability of Income

I found it interesting to see that as the number of hands increases, the range of money won continues to increase. Now this is different from expected winrate. Obviously, we know what our winrate is with more accuracy the more hands we play.

But look for example at the 1 BB graph. After 100,000 hands, we know that we'll have made somewhere between 0 and 2000 BB's about 95% of the time. This is a pretty big range. For those of us that play poker for a source of income, this unpredictability can be a bit trying at times.

Consider a full time pro who plays 20,000 hands per month of 10/20 at a skill level of 1 BB. At the end of the year, 95% of the time his earn will be between $19,200 and $76,800. Although it's only a small percent chance, it will be hard to feel good about your year if you've only made low $20's and felt you've played decent poker. FWIW, change this winrate to 2 BB and this lower bound jumps to $67,200.


I just thought some of these observations were pretty interesting and wanted to share. Please correct me anywhere I'm wrong or made a mistake. If any of this has helped you become less results-oriented and more focused on improving, then it makes the time spent putting this post together worth it. Just don't sit at my tables /images/graemlins/wink.gif


Chris

[TheMetetron]

tl;dr


does that work in this forum?

[RunDownHouse]

[ QUOTE ]
Take a look at the 2 BB/100 graph. If a player played at a solid 2 BB skill level for a sample size of let's say 20,000 hands, he really shouldn't have a very big downswing.

[/ QUOTE ]
How do you explain all of the empirical evidence to the contrary? Tilt? How does a player running ultra-hot for 1k hands, then ultracold for 3k hands fit into this?

[Surfbullet]

[ QUOTE ]
[ QUOTE ]
Take a look at the 2 BB/100 graph. If a player played at a solid 2 BB skill level for a sample size of let's say 20,000 hands, he really shouldn't have a very big downswing.

[/ QUOTE ]
How do you explain all of the empirical evidence to the contrary? Tilt? How does a player running ultra-hot for 1k hands, then ultracold for 3k hands fit into this?

[/ QUOTE ]

This graph is a 95% + 99% confidence interval. Even the 2BB/100 winner could have a 40k breakeven stretch, according to the 99% CI. Short-term variance is much more severe than this pretty graph, as well.

Furthermore, a SD of 15 was used for this approximation. Most SH players on this forum experince a SD of between 16.5-19 depending on playstyle. This opens up the possibilities of running good/running bad to a far wider magnitude.

Surf

[Surfbullet]

Hey Beachbum,

Excellent post. I'd love it if you could run another set using a SD of 17 and a SD of 19, since a SD of 15 would be more indicative of, say, a 5/10 full ring player.

If you were to run those other 2 sets we should have them stickied at the top of the forum, with a notification "If you're going to make a downswing post, READ THIS FIRST."

Great job.

Sur

[krishanleong]

[ QUOTE ]
Great job.

[/ QUOTE ]

I agree. I'd also like to see the graphs for 17 and 19 STDV.

Krishan

[joseki]

I think you're underestimating the 1%.

Let's say you play a +2BB/100 game but still drop 100BB in your first 10k hands. Looking forward to your next 10k hands, you face a real possibility of losing another 100BB. Same with the next 10k.

Looking forward those events are very unlikely to occur back to back, but having it happen the first time does not reduce the probibility of it happening again (obviously).

All I'm saying is that those runner-runner two-outers do come in, despite them falling outside the 99% confidence interval.

[ QUOTE ]
I think you're underestimating the 1%.

Let's say you play a +2BB/100 game but still drop 100BB in your first 10k hands. Looking forward to your next 10k hands, you face a real possibility of losing another 100BB. Same with the next 10k.

Looking forward those events are very unlikely to occur back to back, but having it happen the first time does not reduce the probibility of it happening again (obviously).

All I'm saying is that those runner-runner two-outers do come in, despite them falling outside the 99% confidence interval.

[/ QUOTE ]

I think you're over-estimating the 1%. OP isn't looking ahead 10k hands at a time. You are giving a scenario that fits your argument (lose for 10k then lose for another 10k then lose for another 10k). Independently, these aren't near 1%, but cumulatively they are. Everytime you look at something AFTER it has already happened, you are cutting the odds of the whole outcome down by that factor.

I'm not the best with explaining math, so I hope that made some sense.

[joseki]

Also, dude, these graphs address the probibilty of future outcomes starting "now". Every time you're delt cards that's one "now". From the begining of each hand, if you know that your winrate will remain constant and you know how much variance to expect, then you can plot these curves to get a probabilistic view into the future.

From a different perspective, when you look back on 100k hands there are 100k obervations and you can expect 1k of them to fall outside the 99% CI.

Am I wrong?

[ QUOTE ]
Also, dude, these graphs address the probibilty of future outcomes starting "now".

[/ QUOTE ]

This is the only way to calculate future outcomes.

What evidence to the contrary? Very few people who are true 2bb winners will have had 20k hand downswings. The math doesn't lie. I see a lot of people who THINK they are 2bb winners who have downswings that "defy" the math. But it is the estimate of their ability that is suspect, not the math. And if you look at the graph, a 20k hand downswing should happen more than 1% of the time. That means that a given 2bb winner should see at least 1 such downswing every 2 million hands. OR, 50 such winners who each play 40k hands should expect to have at least one of them hitting such a downswing. When you have a forum with hundreds of good players posting, it is not unlikely that you will see several such "1% events".

-v

[ QUOTE ]
I see a lot of people who THINK they are 2bb winners who have downswings that "defy" the math. But it is the estimate of their ability that is suspect, not the math.

[/ QUOTE ]

Amen. The math gods are pleased.

[joseki]

[ QUOTE ]
[ QUOTE ]
Quote:
Also, dude, these graphs address the probibilty of future outcomes starting "now".

[/ QUOTE ]



This is the only way to calculate future outcomes.

[/ QUOTE ]

Nice post.

It just seems to me that people think of the 99% CI as a bound, when it clearly is not. There is no need for alarm if your results fall outside these lines on occasion.

Isn't the whole point of the OP to discount the likelyhood of large downswings? If so, I thought I could clarify how these downswings easily fit into this CI exercise.

[RunDownHouse]

[ QUOTE ]
Very few people who are true 2bb winners will have had 20k hand downswings.

[/ QUOTE ]
OK, I agree completely and think we just got mixed up on semantics. When you said "big" downswings, I misunderstood and took it as magnitude, not length, as you meant it.

[cartman]

[ QUOTE ]
these graphs address the probibilty of future outcomes starting "now". Every time you're delt cards that's one "now".

[/ QUOTE ]

This is true and crucial to the assessment of this type of situation. I think many are overlooking this. The fact that the -99% line in the bottom graph intersects 0 just short of the 40,000 hand mark doesn't mean that there is only a 1% chance that a 2BB/100 winner will have a breakeven stretch of that length at some point. It only means that he only has a 1% chance of breaking even on his VERY NEXT 40,000 HAND STRETCH STARTING RIGHT NOW! That is entirely different than having a 1% chance of ever breaking even for 40,000 hands.

Cartman

[MrBig30]

[ QUOTE ]
[ QUOTE ]
Great job.

[/ QUOTE ]

I agree. I'd also like to see the graphs for 17 and 19 STDV.

Krishan

[/ QUOTE ]

Me too. Like Surf said 15 is really low and those beautiful graphs would be more interesting to consider with a more normal shorthanded SD (FWIW mine seems to be about 16.5-17)

Other than that, great post.

[tallstack]

Very nice post. I do think, though, that you should not use this calculation method to address downswing severity.

[ QUOTE ]
Take a look at the 2 BB/100 graph. If a player played at a solid 2 BB skill level for a sample size of let's say 20,000 hands, he really shouldn't have a very big downswing. If the player was able to maintain this quality of play even through lots of bad beats and cold cards, only about 5% of the time should he experience a downswing of greater than 110 BB's.


[/ QUOTE ]

I think that there is a problem with reaching this conclusion from your data. The results show only end points for a given hand count, not the path taken to arrive at this end point. By this I mean at 20,000 hands it shows you the likelihood of being up or down a given amount on the 20,000th hand. It does not give you any information on the path taken to get there. Any poker player is going to quote his downswing based on how low he is from his highest peak. Thus, downswings are entirely path dependent and the information regarding the path taken to reach a given end point is not extractible from your type of calculation.

If we try and calculate maximum downswings for very large hand samples we can see more clearly that this type of calculation is not suitable. If we choose 100,000 hands for the 2BB/100 player for example, what would the maximum downswing with a xx% probability be? We can't look at the 100,000 hand total, because this will include all the winning periods before and after the maximum downswing. We also can't look at the maximum dip prior to 100,000 hands because this would show no larger dip for 100,000 hands than it would for say 50,000 hands or 500,000 hands. This cannot be correct. Intuitively, as sample sizes get larger the probability of a certain sized downswing must increase as well. Again, the information of the path taken to reach a hand count must be considered.

We could simulate a large sample of hands and run the simulation many times to get a measure of the downswings expected. I know that this has been done by several posters in the STT forum, and it likely has been done by someone in a limit forum, but I don't have a 2+2 link that I know of. Outside of simulating the path taken through a given number of hands, I do not think that we can conclude what an expected downswing should be.

Dave S

[Luzion]

Nice post sir. Well written, organized, informative, and detailed.

[beachbum]

[ QUOTE ]


Quote:
--------------------------------------------------------------------------------

Take a look at the 2 BB/100 graph. If a player played at a solid 2 BB skill level for a sample size of let's say 20,000 hands, he really shouldn't have a very big downswing.


--------------------------------------------------------------------------------


How do you explain all of the empirical evidence to the contrary? Tilt? How does a player running ultra-hot for 1k hands, then ultracold for 3k hands fit into this?

[/ QUOTE ]

Well, although SD converges fairly quickly for a player's style of play, in the short term it can increase quite a bit. This could be a result of a bit higher VPIP and WtSD in the short term. Also, short term theoretical winrates can be higher or lower than normal.

For instance, my hottest stretch was being up about 120 BB in ~300 hands. For these 300 hands, my SD was probably much higher than normal. Also, my theorectical winrate was maybe closer to 3 or 4 BB. However, I wouldn't be surprised if I never won this many BB's in such a small amount of hands ever again since it was <<1% chance.

[beachbum]

I think I can explain your concerns. Let's consider looking at a 20,000 hand sample within a 100,000 hand sample. Are there 5 20,000 samples we can choose from? Not really, there are 80,000 20,000 hand samples. 1-20,000, 2-20,001, 3-20,002, 4-20,003,...,79,999-99,998, and 80,000-99,999.

For illustration purposes, if we want to get exact, a 200 BB downswing will occur on the 2 BB graph at a 99.25% confidence interval right around 10,000 hands. Since there are 90,000 10,000 hand samples in total, there should be 0.75% of these samples where a downswing of that magnitude has occured. This results in 675 (0.75% x 90,000) ranges of 10,000 hands. Like you mentioned this is looking at some highest point in the graph and calculating from there.

Let's say a player goes on a rush and wins 100 BB's in 1000 hands. Then he cools off alot. There's a decent chance if you look at hand 1001-11,000, there will be a 200 BB downswing. Then look at immediately succeeding hand ranges like 1002-11,001, 1003-11,002, etc. and you might get hundreds of ranges where this downswing is 200BB.

However, if you just look at your results in consecutive 10,000 hand intervals, there's a very small chance you're going to have a downswing of this severity. I guess my point is, instead of blaming variance, turn the focus inward and see where our play has been suboptimal.

[Surfbullet]

Just to tack on to some other posters queries about 2-3BB/100 players running bad...

Schnieds has posted graphs of 30k breakeven stretches. Sxb who has a winrate of 3BB/100 at 10/20 over 300k+ hands, has posted a 650 BB downswing.

Davidross, longtime poker pro, who is estimated at 2+BB/100 has had a 600BB downswing that lasted almost 30k hands.

These things are rare, but they happen. The CI is by no means a bound(as another poster noted) and results outside of the 99% CI are not as uncommon as we would like to think, especially given how many hands of poker most play.

Surf

[tallstack]

I think I will be getting out of my league pretty soon here, but I still have a few comments.

[ QUOTE ]
I think I can explain your concerns. Let's consider looking at a 20,000 hand sample within a 100,000 hand sample. Are there 5 20,000 samples we can choose from? Not really, there are 80,000 20,000 hand samples. 1-20,000, 2-20,001, 3-20,002, 4-20,003,...,79,999-99,998, and 80,000-99,999.

[/ QUOTE ]

There certainly are 5 20,000 hand 'independent' samples. The 80,000 20,000 hand samples are about as dependent as can be. If we are going to normal distribution assumptions for our data then I believe that they need to be independent data points. Thus, using 80,000 20,000 hand samples is not valid. Using 5 20,000 hand samples would be valid, but 20,000 hand samples are likely too large to analyze a peak-to-valley downswing. It is better than using a single 100,000 sample, but not by much. We could use say 1000 100 hand samples and get a more refined answer, or we could take it down to a single hand level. My central argument is that this analysis method you have shown, while valid for determining a probable range of end values, is not suited to determining the probable range of paths taken to get there. If we want to determine swings along the way, we need to use a method that models paths within a sample size.

[ QUOTE ]
I guess my point is, instead of blaming variance, turn the focus inward and see where our play has been suboptimal.

[/ QUOTE ]

I agree with you here. /images/graemlins/smile.gif

[Justin A]

The fact that the -99% line in the bottom graph intersects 0 just short of the 40,000 hand mark doesn't mean that there is only a 1% chance that a 2BB/100 winner will have a breakeven stretch of that length at some point. It only means that he only has a 1% chance of breaking even on his VERY NEXT 40,000 HAND STRETCH STARTING RIGHT NOW! That is entirely different than having a 1% chance of ever breaking even for 40,000 hands.

Just thought that this should be repeated. Nice post cartman.

[beachbum]

[ QUOTE ]
I'd love it if you could run another set using a SD of 17 and a SD of 19, since a SD of 15 would be more indicative of, say, a 5/10 full ring player.

[/ QUOTE ]

LOL. So is my 14.6 SD at 5/10 6-max a little low? /images/graemlins/laugh.gif There's never been a question about my being risk averse.

Anyway, here ya go.


http://www.tourney.com/images/1BB17SD.jpg
http://www.tourney.com/images/2BB17SD.jpg

http://www.tourney.com/images/1BB19SD.jpg
http://www.tourney.com/images/2BB19SD.jpg

The 1 BB graph at 19 SD is just painful to look at.

[Surfbullet]

Thanks for doing that, beachbum.

It's very telling that after a mere 50k hands the same 1BB/100 player w/ 17 SD(mine is 16.8) could either be down 200BB or up 1250BB! That's a STAGGERING range, and really something to think about before anyone judges the quality of a person's play based solely on their winrate over a small hand sample.

Surf

[MrBig30]

Very nice. I have this thread bookmarked now, and I think whenever someone new complains about running bad he should be told to take a look at these graphs.... /images/graemlins/smile.gif

Love your work Beachbum.

It is very scary to look at the 1BB/100 WR on the 19SD graph. If your running bad it could nearly take a life time of poker to come out of it. A 1BB/100 WR with 19SD suxs. Its basically break even poker.

I'm thinking that you can't comprimise on your WR. Multi-tabling will have its dangers in decreasing your WR.

Trent.

[ggbman]

Excellent post here. I am not a math guy, so its always nice to have people simplify things for us common folk. I think the key thing here to take away is that very good players can have uncharecteristic downswings that are more severe than their peers.

The important thing to do is focus on styaing condifident and playing within the parameteres of your game. Obvioulsy, during a downswing, you might need to move down so as not to comprimise your bankroll. Gigabet said it well in one of his fews posts. "At some point, everyone will run worse than they ever thought was possible". This is very true, and for some people it will be in the form of a 500 or 600 BB downswing and for others it might be kick starting your 100/200 career with a 200BB downswing (not I! /images/graemlins/smile.gif) Anyway, it should be reassuring to people going therough these downswings that it's a normal part of variance and that it will go away after about 17 million hands.

Gabe