On confidence intervals

[elindauer]

Mike just dropped a monster post in which he says, to sum up, that limit poker is a joke because you have to play 25 million hands to know whether or not you're any good. He uses this argument to imply that the luck factor in limit hold 'em is very high.

I have now seen statements like this many times in many forms, and I've never seen anyone bother to debunk it, so I'm going to try to do it now. Here's the deal:

Let's say we look only at a player's results, and we attempt to determine if this player is a winning player. Fine. We check out his EV, we look at his standard dev, and we calculate a confidence interval. As Mike points out, correctly, we have to have a huge number of hands to determine somewhat conclusively that this player is a winner. Of course, this calculation uses no knowledge of poker whatsoever.

In real life though, we have much more information. Namely, we can look at the actual hands that were played, both by the player and by the opponents, to aid us in our estimate of how much money this player is making.

For example, if I see a player call 3 cold with A8, I don't need 2 million hands to tell me he's a losing player. I can do it in one. And if I'm playing with this guy, and he's losing money, then I'm winning it. Now I just look around the table, and look at all my opponents this way. If I can identify lots of obvious leaks, then I must be winning. It's that simple.

I've played about 30K hands of 15/30. Using confidence intervals, I can barely prove that I'm a break even player. But I know I'm a substantial winner. I know I'm not just on a two standard dev hot streak. I know this, because I ignore the mathematics of confidence intervals and use the much faster converging mathematics of poker. I know that calling 3 cold with A8 is making a hugely losing play, one that I don't make. I know that playing any 2 suited is a losing proposition. And I know a lot of things more subtle than that. I can see the poker mistakes that my opponents make, and I know that I am winning money from them.

This confidence interval stuff has been blown way out of proportion.


Good luck.
Eric

[Equal]

Good post.

[lil feller]

Excellent post. I think the popularity of online poker and PokerTracker have taken a lot of the feel out of many people's games. Nice to see somebody reminding us to remember that it isn't just about numbers...its a game, and we play it.

lf

[pfkaok]

Good post. Its certainly true that if you're a solid winning player you can spot obvious mistakes that others are making and know that you have to be winning money from them.

However, one reason that there is some merit to this million hand thing is that most players who aren't experts are unable to objectively judge the skill of themselves, or their opponents. Dr Al talks about this, and how most people are quick to assume that others are making mistakes just b/c they themselves wouldn't have played that hand in a particular manner. So a weak/tight player could easily conclude that a solid player is a total maniac b/c he sees him making sophistacated raises in spots that seem to the weak tight player as wreckless. Also somebody who's a fairly good player, but just too loose and aggressive might just think that he's playing with a bunch of predictable rocks, when in fact the other players are just being smarter and more selective than him.

This sample size police thing is important at times. When some player makes a post about how much they're killing the game after a few thousand hands, its much better that they're humbled, and learn that hot streaks can last much longer than beginners would ever expect. If a player never learns this then they're doomed to overestimate their skills, and most likely they'll play over their bankroll and get broke. Also, its nice to have comfort in the standard deviation numbers when you're running REALLY bad, then you can just try to review your play and make sure you're still playing solid. If you didn't know that 200BB swings happen, or that you can have 15-20k hands of breakeven poker even when playing well, then you'd totally freak out and lose all confidence whenever you were on the bottom end of variance.

Your method is certainly the way to go if you're playing live, or if you're only playing 1-2 games online. since your "true win rate" or accurate results won't be mathematically solid for years, then trying to figure confidence intervals to find if you're even a winning player would likely just play tricks with your mind. Also its much more effective for really good players who can objectively rate themselves and others.

[TStoneMBD]

this all goes without saying. i usually like your posts eric, but this topic really didnt deserve its own thread.

[Equal]

[ QUOTE ]
this all goes without saying. i usually like your posts eric, but this topic really didnt deserve its own thread.

[/ QUOTE ]

I beg to differ. It is a very valid point that sometimes gets lost in 2+2's over-emphasis on PT stats.

[SA125]

[ QUOTE ]
I've played about 30K hands of 15/30. Using confidence intervals, I can barely prove that I'm a break even player. But I know I'm a substantial winner.

[/ QUOTE ]

This reminds me of one of Mason's jokes last month in the internet magazine about telling the guy who busted out with AKo against 85o on his first hand in the WSOP that he just won a ton of money in the long run.

Seriously though, I hope you're right. I'm winning online but can't understand why I'm beating many good/tight players and losing to many of the any 2 card guys. I'm waiting about 30K hands for their luck to run out.

[Tommy Angelo]

Let's compare two players, Moe and Joe. Moe is an expert player and everyone says so. Joe is just some Joe.

They both play a million hands.

During the million hands, Joe played the same all the time. He was always fresh. He was good with game selection. His preflop game varied little. He is a boring, uncreative player, compared to Moe. After a million hands, Joe ends up even.

Moe plays great, better than almost everyone, most of the time. But when he tilts, he tilts huge and blows big. After a million hands, Moe is even.

We look at their stats and we learn nothing. They look at their stats and they learn nothing. They already know who they are and what they do. And they knew it 950,000 hands ago.

Tommy

[JimmyV]

Just wanted to pipe in and agree with those congratulating Eric on an excellent post. Helps remind me to stay focused on the small things -- each decision, and the extra two seconds to stay aware of position and reads.

Also worth noting that Tommy's intervention is not only charming and incisive as usual, but also helpful: I've probably assigned to tilt only half the tilt-fueled losses that have contributed to my long stretch of mediocrity recently. Some Moe in me maybe.

And, judging the population qualitatively, there's definitely some moe available on these 15/30 tables than I'm taking off of them.


JimmyV

[rigoletto]

[ QUOTE ]
Let's compare two players, Moe and Joe. Moe is an expert player and everyone says so. Joe is just some Joe.

They both play a million hands.

During the million hands, Joe played the same all the time. He was always fresh. He was good with game selection. His preflop game varied little. He is a boring, uncreative player, compared to Moe. After a million hands, Joe ends up even.

Moe plays great, better than almost everyone, most of the time. But when he tilts, he tilts huge and blows big. After a million hands, Moe is even.

We look at their stats and we learn nothing. They look at their stats and they learn nothing. They already know who they are and what they do. And they knew it 950,000 hands ago.

Tommy

[/ QUOTE ]

Tommy. I think you are wrong on two counts: a lot of people don't know what they are or what they are doing no matter how much experience they have. But when they do it is 999,995 hands ago.

[elindauer]

Hi pfkaok,

You make some great points, and you're dead on. Confidence interval calculations certainly have merit. It's just that this math has been used as a claim that you can't know that you're winning unless you have a half million hands in your database, and that's just not true.

I particularly like your points that confidence interval calculations can be very surprising for noobies, and that it is easy for players with limited experience to misevaluate their opponents' mistakes. You got it. I think good advice here is that you should not consider players being too TIGHT, too AGGRESSIVE, or too PREDICTABLE to be making mistakes in your first year at least. Loose mistakes and passive mistakes are the ones you should count, until you're playing, say, 30-60 on a daily basis. I honestly still haven't figured out what mistakes a so-called "weak-tight" player is making if he isn't playing tight-passive. I suspect it's possible in a very tough high limit game and means a player who doesn't, for example, ever 3-bet light preflop or make similar moves with very weak cards in late position (see any mikel post for non-weak-tight play in a high limit game).

Of course, evaluating your own game requires a kind of brutal self-honesty that perhaps is hard to find. If you have any doubts and really want to know, get someone you respect to look at your hands and tell you how good you are. You'll find this much faster and almost as accurate as playing a million hands and looking at the results.


Good luck.
Eric

[elindauer]

Nice point Tommy. Even after you play the million hands, the confidence interval calculation only tells you your win rate, but doesn't give any insight into what you are doing right or wrong. A simple hand review after a few thousand hands tells you all of this. Forget confidence intervals. Study the hands.

Good luck.
Eric

[skp]

[ QUOTE ]
I've never seen anyone bother to debunk it, so I'm going to try to do it now

[/ QUOTE ]

What you said sounds like good debunking to me. It also lends credence to the adage "if you can't spot the sucker in 15 minutes, you are it".

This may sound like I am tooting my own horn, but there was a time when I used to play B&M and a bunch of us were sitting around having dinner and discussing probable win rates. I had everyone's hourly win rate pegged down pretty tight. And this was just based on observation for a few months while playing part time. What I was probably doing was gauaging what I thought everyone should be winnning based on thier poker actions. The fact that their actual results matched my guesses (or so they said) was probably a bit fluky but I think it illustrates the point you were making.

Incidentally, on losing players, I never have any idea how much they should theoretically lose. To me, that's just impossible to gauge properly.

[Dreamer]

You don`t need 25 million hands to know if a player is any good.
I wrote a spreadsheet which you can download for free http://dreamer5.topcities.com/index5.html

With say 100K hands and a 2BB/100 winrate, There is a 99% chance that the true winrate is between +0.481 BB/100 and +3.519 BB/100.

Even after 100K hands it would be HIGHLY UNLIKELY that this player has no skill. (he is beating the rake also!)
And a high probability that his true winrate is much closer to 2BB/100.

You certainly do NOT need 25 million hands to tell if anyone is a winning player unless their play is extremely marginal.

D.

[pfkaok]

[ QUOTE ]
I honestly still haven't figured out what mistakes a so-called "weak-tight" player is making if he isn't playing tight-passive.

[/ QUOTE ]

Well, I would consider a weak-tight player to be somebody who is at least a little bit too passive, and also gives up on hands that he should at least take to showdown. His biggest mistake might be folding too much on the turn and river. Like maybe he always fold to turn raises unless he has an absolute monster or soemthing. Folding too much can be a huge mistake if you're playing against overly aggressive players, like many of the LAGs I've come accross at party 15/30. Also, it seems that even the tighter, solid players in that game make lots of bluff and semibluff raises, so if your main mistake in that game was folding too much then you'd have a very tough time beating that game.

Your suggestion for having somebody else to look over some hands to evaluate is a very good idea, and I think its probably the best way to find leaks... especially if you can't find any obvious leaks in your pokertracker stats.

[elindauer]

[ QUOTE ]
I honestly still haven't figured out what mistakes a so-called "weak-tight" player is making if he isn't playing tight-passive.

[ QUOTE ]
Well, I would consider a weak-tight player to be somebody who is at least a little bit too passive, and also gives up on hands that he should at least take to showdown.

[/ QUOTE ]

[/ QUOTE ]


So... a weak tight player is someone who's too tight and too passive? Sounds a lot like tight-passive to me.

I'd put forth the following definition: a "weak tight" player is somebody who plays tight and aggressive poker, but plays in an ABC way where you can peg his range of hands very easily. He isn't easy to play against by any means, but he'll never 3-bet you with 55, he'll never check-call-check-raise on a total steal, etc. As such, you can get slightly better reads and make a few more marginal folds against this player than you could against the perfect TAG. In most games with loose and passive opposition, the weak tight players looks just like the tough TAG. The differences only show up when these players face off with other tight players.

Of course, with this definition, "weak" is misleading.

-Eric

[kem]

[ QUOTE ]
Mike just dropped a monster post in which he says, to sum up, that limit poker is a joke because you have to play 25 million hands to know whether or not you're any good. He uses this argument to imply that the luck factor in limit hold 'em is very high.

I have now seen statements like this many times in many forms, and I've never seen anyone bother to debunk it, so I'm going to try to do it now. Here's the deal:

Let's say we look only at a player's results, and we attempt to determine if this player is a winning player. Fine. We check out his EV, we look at his standard dev, and we calculate a confidence interval. As Mike points out, correctly, we have to have a huge number of hands to determine somewhat conclusively that this player is a winner. Of course, this calculation uses no knowledge of poker whatsoever.

In real life though, we have much more information. Namely, we can look at the actual hands that were played, both by the player and by the opponents, to aid us in our estimate of how much money this player is making.

For example, if I see a player call 3 cold with A8, I don't need 2 million hands to tell me he's a losing player. I can do it in one. And if I'm playing with this guy, and he's losing money, then I'm winning it. Now I just look around the table, and look at all my opponents this way. If I can identify lots of obvious leaks, then I must be winning. It's that simple.

I've played about 30K hands of 15/30. Using confidence intervals, I can barely prove that I'm a break even player. But I know I'm a substantial winner. I know I'm not just on a two standard dev hot streak. I know this, because I ignore the mathematics of confidence intervals and use the much faster converging mathematics of poker. I know that calling 3 cold with A8 is making a hugely losing play, one that I don't make. I know that playing any 2 suited is a losing proposition. And I know a lot of things more subtle than that. I can see the poker mistakes that my opponents make, and I know that I am winning money from them.

This confidence interval stuff has been blown way out of proportion.

[/ QUOTE ]

It might be a stretch to go from "some players at my table are playing some hands sub-optimally" to "i'm a winning player".. you're missing a few important steps, like showing the money is going to you and not 1800gambler who's sitting to your left, and also quantifying their mistakes to the point where it overcomes the rake.

[pfkaok]

well, you might be right about the aggresssion part, but to me, I still think that the weak part means that he'll fold too much. I don't know if thats the accepted definition, but from descriptions I hear about "weak/tight" players thats what i've gathered.

[roy_miami]

[ QUOTE ]
I honestly still haven't figured out what mistakes a so-called "weak-tight" player is making if he isn't playing tight-passive

[/ QUOTE ]

You'll never know with any certainty when a player makes a weak-tight mistake as opposed to a loose passive mistake for instance.

Loose passive mistakes are often witnessed at showdown, like a guy catching a set of 2's vs a preflop raisor on a AKQ-5-2 board in a small pot.

The only time you'll have evidence of a weak-tight mistake is if a) you're the one who made it or b) he shows his AA before he mucks on a board of Q94-T to a raise.

I would guess the most common weak-tight mistakes are folding drawing hands when the pot is laying you proper odds to continue. For example, folding 99 on a 8JQ board in a big pot for 1 bet on the flop. This players thoughts would go along the lines of-- My 9 outs may be dead to QQ and my ten outs may be dead to AK so rather than risk hitting my card and still lose a bunch of bets, I'll just go ahead and fold now.

I'm sure most of us are guilty of making margional weak-tight mistakes (I know I am) but as with everything in poker, so long as you're making fewer and more margional mistakes than your opponents, you should come out ahead.

[helpmeout]

If only it was that simple, the reason why people make mistakes is because they dont know any better.

I might think I am a good player but a world class player may look at my game and see heaps of mistakes that I dont know about.

As Tommy says you may tilt more often than others or play tired not play your "A" game.

Your "A" game may not even be that good but you think it is because you read 2+2 and think that this somehow makes you a good player.

Common sense tells me whether I am at least a winning player after 50k to 100k hands.

[twowords]

[ QUOTE ]

I'd put forth the following definition: a "weak tight" player is somebody who plays tight and aggressive poker, but plays in an ABC way where you can peg his range of hands very easily. He isn't easy to play against by any means...

[/ QUOTE ]

"Easy to play against" is actually the defining characteristic of a weak-tight player IMO. Whatever action he takes, you can immediately put him on a narrow range of hands. A TAG who mixes it up with a few fancy steals/semibuffs/bluffs will be much harder to read while the 60/25/2 LAG will be the hardest to read, he could have anything (although he probably has nothing hehe). You shouldn't pay off the weak tight guy much when he gets a hand, but you'll have to pay off the solid TAG more and pay off the LAG most often.

Anyone disagree with me here?

[Tommy Angelo]

"the reason why people make mistakes is because they dont know any better."

If only that were true, how much simpler my life would have been.

If it were true what you say, that my thousands of mistakes have been made because I didn't know any better, then 1) I wouldn't know that I'd made thousands of mistakes 2) I wouldn't have felt bad about any of them.

The thing that seperates poker from other games is that it is the only game in which almost every player makes many mistakes in spite of knowing better.

Tommy

[Tommy Angelo]

"I would guess the most common weak-tight mistakes are folding drawing hands when the pot is laying you proper odds to continue. For example, folding 99 on a 8JQ board in a big pot for 1 bet on the flop. This players thoughts would go along the lines of-- My 9 outs may be dead to QQ and my ten outs may be dead to AK so rather than risk hitting my card and still lose a bunch of bets, I'll just go ahead and fold now."

What a game we play! What you call a mistake, I call an achievement.

[The Dude]

Tommy, please. This box is killing me. Is it a grey box in front of a grey backdrop, or is it a hollow black frame in a grey cloud? I'm going to go insane looking at this thing.

[Gabe]

and like where is the front of the box?

[rory]

Yes.