ICM: Does it match the results?

[gumpzilla]

Earlier today, in a fit of procrastination, I started wondering about the ICM. It seems like the predictions of the ICM are used to justify much of the late game play in SNGs. It seems clear that empirically, this brand of analysis has got to be at least somewhat successful, given the success of people who practice its teachings. However, I wonder if you couldn't get similar results by assuming almost any model that recognized the fact that cash EV, unlike chip EV, does not scale linearly with the number of chips in your stack unless you're playing heads up. Such models stress the importance of survival, as there is a ceiling on how much you can gain (can't get higher than 50% of the prize pool) that makes it so that there is frequently much more room to drop than there is room to climb.

So I'm curious if anybody has a sense for how accurate ICM itself is. I remember a thread a while back where curtains was remarking that there are many situations where the ICM makes predictions that seem silly, and that it doesn't take things like the size or position of the blinds into account. Do the predictions that ICM makes about expected return on prize pool stack up against the empirical results? This seems like it would be incredibly hard to measure, because setting up enough basically similar situations to get a good sample size is going to require approximately a googol of SNGs.

One approach that occurs to me is to just consider the following: every time I hit the first hand at four handed, I calculate what ICM says my expected return is. I then compare the average return I actually get vs. the average of predicted returns from ICM. I don't like this method because it pays no attention to what is distinctive about each situation, and is almost certainly going to mix results from regimes where ICM does really well and regimes where it blows goats, leaving the overall result one of mediocre agreement, most likely.

I'll also admit that I'm lazy and have yet to really read the theoretical argument behind ICM, so I can't attack it from that angle yet. But is it used just out of convenience/qualitative correctness - that is, do we just need any kind of numbers that address cash vs. chip EV?

[eastbay]

[ QUOTE ]

So I'm curious if anybody has a sense for how accurate ICM itself is. I remember a thread a while back where curtains was remarking that there are many situations where the ICM makes predictions that seem silly,


[/ QUOTE ]

While true, most of those objections ring hollow (some more than others) on closer look. To pick an example, if 2 guys have 1 chip and 10 chips, and players 3 and 4 have the rest of the chips in play, and 10 chip guy is about to take the blind, you can say "ICM says guy with 10 chips has 10x equity of 1 chip guy! That's ridiculous!" While true, it's also irrelevant. You're not going to draw any conclusions for making plays that matter based on the relative equity of 1 chip guy and 10 chip guy. From the stacks who actually have decisions to make, both of the short stacks have near zero equity, and it doesn't matter if it's a pittance or 10xpittance.

[ QUOTE ]

and that it doesn't take things like the size or position of the blinds into account. Do the predictions that ICM makes about expected return on prize pool stack up against the empirical results? This seems like it would be incredibly hard to measure, because setting up enough basically similar situations to get a good sample size is going to require approximately a googol of SNGs.


[/ QUOTE ]

Exactly. I attempted to do this once before running into sample size realities. There's a combinatorial explosion of the possible chip stack distributions, and no one person's data could ever be used to fill it out in a meaningful way.

[ QUOTE ]

One approach that occurs to me is to just consider the following: every time I hit the first hand at four handed, I calculate what ICM says my expected return is. I then compare the average return I actually get vs. the average of predicted returns from ICM.


[/ QUOTE ]

And in the year 6984, you might get a nice dataset.

The problem is that ICM accounts for all chip stacks, not just your own. This is the main strength of the method so you can't just sweep it under the rug. And once you realize that if you have, say, 1000 chips, there's lots of different ways for the other chip stacks to be arranged, even if you break it up into fairly large ranges over which you'd lump data together. And then there's the cases where you have 2000 chips, etc. You need significant data for each possible combination, and there's just too many of them.

[ QUOTE ]

But is it used just out of convenience/qualitative correctness - that is, do we just need any kind of numbers that address cash vs. chip EV?

[/ QUOTE ]

It is used because

#1 - It's the best thing available for practical application for $EV

but more compelling is that

#2 - In the vast majority of cases the results from the kind of push/fold calculations you can do with it are remarkably consistent with expert players' intuition and experience.

There are no doubt caveats and more subtle effects that expert players can debate over: positional differences in short-stacked play where "lookahead" effects can be important,i.e., in bubble folding war situations. Strong nonlinear advantages of big stacks on the bubble against competent players where you can quickly build leads by picking up blinds. Etc. These come with experience and tend to be adjustments you can make based on the baseline ICM result. This sort of thing usually gets pretty subjective and controversial, but IMO it's pretty nice to have a starting point for expert discussion, rather than just a bunch of assertions about the right play, which is what prevailed on this forum for quite some time before Bozeman came along and led the way with this kind of quantitative approach.

eastbay

PS I have an idea about how to approach the subject from another direction. If I ever free up enough time to try my idea, I'll probably share it. Maybe. /images/graemlins/wink.gif

[Nottom]

Well, its pretty well know that ICM calculations have some problems.

An extreme example would be 4 players, 3 with 3333 chips and one with 1. ICM would give the 1 chip player basically 0 $ equity, but anyone who has actually played poker knows there is always a chance that 2 of the other stacks collide (KK pushes into AA for example, or much less at some tables). Also the fact that things like blinds/position aren't accounted for can mess up some of the math as the blinds get big. Also and probably more importantly, ICM doesn't account for the players'relative skill.

However ICM is generally accepted as the best easy method to calculate these sort of things, but I'd imagine there are other methods out there, some of which may even be better sometimes that if used could arrive at similar results.

There have been some exhaustive posts written about the ICM analysis (try http://www.bol.ucla.edu/~sharnett/ICM/info.html for some good links) which show it's pretty accurate compared to some other methods.

*Well eastbay already posted, and he knows more about ICM calcs than just about anybody here, so just read what he had to say /images/graemlins/tongue.gif

[gumpzilla]

[ QUOTE ]

This sort of thing usually gets pretty subjective and controversial, but IMO it's pretty nice to have a starting point for expert discussion, rather than just a bunch of assertions about the right play, which is what prevailed on this forum for quite some time before Bozeman came along and led the way with this kind of quantitative approach.


[/ QUOTE ]

Though pre-ICM is before my time, I definitely agree that having something quantitative that seems to give decent results is a big plus. The quantitative bent that 2+2 has is definitely what got me interested in the place and is the strongest thing it has going for it, to my mind.

[ QUOTE ]

PS I have an idea about how to approach the subject from another direction. If I ever free up enough time to try my idea, I'll probably share it. Maybe. /images/graemlins/wink.gif


[/ QUOTE ]

My interest is piqued.