On confidence intervals

[elindauer]

Mike just dropped a monster post in which he says, to sum up, that limit poker is a joke because you have to play 25 million hands to know whether or not you're any good. He uses this argument to imply that the luck factor in limit hold 'em is very high.

I have now seen statements like this many times in many forms, and I've never seen anyone bother to debunk it, so I'm going to try to do it now. Here's the deal:

Let's say we look only at a player's results, and we attempt to determine if this player is a winning player. Fine. We check out his EV, we look at his standard dev, and we calculate a confidence interval. As Mike points out, correctly, we have to have a huge number of hands to determine somewhat conclusively that this player is a winner. Of course, this calculation uses no knowledge of poker whatsoever.

In real life though, we have much more information. Namely, we can look at the actual hands that were played, both by the player and by the opponents, to aid us in our estimate of how much money this player is making.

For example, if I see a player call 3 cold with A8, I don't need 2 million hands to tell me he's a losing player. I can do it in one. And if I'm playing with this guy, and he's losing money, then I'm winning it. Now I just look around the table, and look at all my opponents this way. If I can identify lots of obvious leaks, then I must be winning. It's that simple.

I've played about 30K hands of 15/30. Using confidence intervals, I can barely prove that I'm a break even player. But I know I'm a substantial winner. I know I'm not just on a two standard dev hot streak. I know this, because I ignore the mathematics of confidence intervals and use the much faster converging mathematics of poker. I know that calling 3 cold with A8 is making a hugely losing play, one that I don't make. I know that playing any 2 suited is a losing proposition. And I know a lot of things more subtle than that. I can see the poker mistakes that my opponents make, and I know that I am winning money from them.

This confidence interval stuff has been blown way out of proportion.


Good luck.
Eric

[Equal]

Good post.

[lil feller]

Excellent post. I think the popularity of online poker and PokerTracker have taken a lot of the feel out of many people's games. Nice to see somebody reminding us to remember that it isn't just about numbers...its a game, and we play it.

lf

[pfkaok]

Good post. Its certainly true that if you're a solid winning player you can spot obvious mistakes that others are making and know that you have to be winning money from them.

However, one reason that there is some merit to this million hand thing is that most players who aren't experts are unable to objectively judge the skill of themselves, or their opponents. Dr Al talks about this, and how most people are quick to assume that others are making mistakes just b/c they themselves wouldn't have played that hand in a particular manner. So a weak/tight player could easily conclude that a solid player is a total maniac b/c he sees him making sophistacated raises in spots that seem to the weak tight player as wreckless. Also somebody who's a fairly good player, but just too loose and aggressive might just think that he's playing with a bunch of predictable rocks, when in fact the other players are just being smarter and more selective than him.

This sample size police thing is important at times. When some player makes a post about how much they're killing the game after a few thousand hands, its much better that they're humbled, and learn that hot streaks can last much longer than beginners would ever expect. If a player never learns this then they're doomed to overestimate their skills, and most likely they'll play over their bankroll and get broke. Also, its nice to have comfort in the standard deviation numbers when you're running REALLY bad, then you can just try to review your play and make sure you're still playing solid. If you didn't know that 200BB swings happen, or that you can have 15-20k hands of breakeven poker even when playing well, then you'd totally freak out and lose all confidence whenever you were on the bottom end of variance.

Your method is certainly the way to go if you're playing live, or if you're only playing 1-2 games online. since your "true win rate" or accurate results won't be mathematically solid for years, then trying to figure confidence intervals to find if you're even a winning player would likely just play tricks with your mind. Also its much more effective for really good players who can objectively rate themselves and others.

[TStoneMBD]

this all goes without saying. i usually like your posts eric, but this topic really didnt deserve its own thread.

[Equal]

[ QUOTE ]
this all goes without saying. i usually like your posts eric, but this topic really didnt deserve its own thread.

[/ QUOTE ]

I beg to differ. It is a very valid point that sometimes gets lost in 2+2's over-emphasis on PT stats.

[SA125]

[ QUOTE ]
I've played about 30K hands of 15/30. Using confidence intervals, I can barely prove that I'm a break even player. But I know I'm a substantial winner.

[/ QUOTE ]

This reminds me of one of Mason's jokes last month in the internet magazine about telling the guy who busted out with AKo against 85o on his first hand in the WSOP that he just won a ton of money in the long run.

Seriously though, I hope you're right. I'm winning online but can't understand why I'm beating many good/tight players and losing to many of the any 2 card guys. I'm waiting about 30K hands for their luck to run out.

[Tommy Angelo]

Let's compare two players, Moe and Joe. Moe is an expert player and everyone says so. Joe is just some Joe.

They both play a million hands.

During the million hands, Joe played the same all the time. He was always fresh. He was good with game selection. His preflop game varied little. He is a boring, uncreative player, compared to Moe. After a million hands, Joe ends up even.

Moe plays great, better than almost everyone, most of the time. But when he tilts, he tilts huge and blows big. After a million hands, Moe is even.

We look at their stats and we learn nothing. They look at their stats and they learn nothing. They already know who they are and what they do. And they knew it 950,000 hands ago.

Tommy

[JimmyV]

Just wanted to pipe in and agree with those congratulating Eric on an excellent post. Helps remind me to stay focused on the small things -- each decision, and the extra two seconds to stay aware of position and reads.

Also worth noting that Tommy's intervention is not only charming and incisive as usual, but also helpful: I've probably assigned to tilt only half the tilt-fueled losses that have contributed to my long stretch of mediocrity recently. Some Moe in me maybe.

And, judging the population qualitatively, there's definitely some moe available on these 15/30 tables than I'm taking off of them.


JimmyV

[rigoletto]

[ QUOTE ]
Let's compare two players, Moe and Joe. Moe is an expert player and everyone says so. Joe is just some Joe.

They both play a million hands.

During the million hands, Joe played the same all the time. He was always fresh. He was good with game selection. His preflop game varied little. He is a boring, uncreative player, compared to Moe. After a million hands, Joe ends up even.

Moe plays great, better than almost everyone, most of the time. But when he tilts, he tilts huge and blows big. After a million hands, Moe is even.

We look at their stats and we learn nothing. They look at their stats and they learn nothing. They already know who they are and what they do. And they knew it 950,000 hands ago.

Tommy

[/ QUOTE ]

Tommy. I think you are wrong on two counts: a lot of people don't know what they are or what they are doing no matter how much experience they have. But when they do it is 999,995 hands ago.