optimizing calling all-in in a heads-up, all-in or fold poker model

[eastbay]

[ QUOTE ]
[ QUOTE ]
Because chipEV != $EV, amongst many other reasons.


[/ QUOTE ]
I've thought about this a fair bit and have a hunch that deviations of chipEV from $EV are rare and probably not that important in the grand scheme of things. Still, I'd love to know the shape of the chip EV/$EV curve--where are the nonlinearities, etc.


[/ QUOTE ]

You might be interested in a thread I started awhile ago that looked at a model where your decisions were call/fold with a known EV. Search for "call/fold model" and you should find it.

I think you'll find the results interesting. It can matter, but if it matters to a practical degree is possibly debatable. Probably the two most important variables are blind to stack ratio and skill of your opponent. That's pretty intuitive, I think.

[ QUOTE ]

[ QUOTE ]
In my initial formulation, the strategy remains fixed for the tournament - that's just currently a constraint of the strategy.

[/ QUOTE ]

Tsk tsk I do not approve.


[/ QUOTE ]

Yeah I don't like it either. But I'm writing some nontrivial software for this and I have to start somewhere. I'm actually thinking about going back to something like my original model, and optimizing the call/fold curve in N-parameters (rather than just the slope of a linear assumption). That's probably a better way to go.


[ QUOTE ]

[ QUOTE ]
If you can think of a good way to approach it, I'm certainly interested in it.

[/ QUOTE ]

I think you have to work backwards from a static end point. Say, the opponent is down to his last chip. No, that's not static because to solve the one-chip problem you have to know what to do in the two-chip problem in case he doubles up. So it is still dynamic, but finite at least. So what you have to do is come up with strategies for each of the possible chip combinations and then figure out how to vary them so that they eventually converge. I.e., given the one-chip strategy, the two-chip action is optimal, and given the two-chips strategy, the one-chip strategy is optimal,


[/ QUOTE ]

Hmm. I don't follow you.

[ QUOTE ]

It is moments like this that I am glad I have very limited programming skills because I would have wasted a lot of time on this kind of stuff.


[/ QUOTE ]

Yeah, it's time consuming, but I do this kind of stuff professionally so it's not ridiculously so. I had the genetic algorithm working in an evening's time. The tournament simulator (which could do full-table NLHE given strategy inputs), took longer. Encoding the rules of tournament play is much more difficult than you might think.

eastbay

[eastbay]

[ QUOTE ]
My guess: best 40% of hands.

[/ QUOTE ]

Pretty damn good guess. I'm getting around 38%, +/- maybe 2%. I still have some sample size issues to resolve.

Take a stab at the resulting tourney win rate?

eastbay

[eastbay]

[ QUOTE ]
The way I suggested can be solved, right? No need for simulation. Yes, that's right. Try the simplest case of each player has two chips, one chip for a single blind, and hands can be only high, medium or low. 1/3 probability of each. That is a problem you can write down and solve, probably by hand, certainly with Mathematica or something. Then extrapolate that to deeper stacks and more types of hands. There should be some algorithm to efficently solve that type of huge problem somewhere.

[/ QUOTE ]

That sounds pretty close to my original model. The crux is the range of EV that your opponent will hand you. If he occasionally hands you big +EV, it does pay to fold small +EV, especially when all your chips are at stake. If it doesn't hurt your stack much to lose, you should take any +EV. If he won't hand you "outlier" +EV often enough (often enough relative to the rate you are being blinded out), you have to take what you can get when you can get it.

eastbay

[eastbay]

[ QUOTE ]


If so, how will your simulation work up those "button" situation? Is it going simply to skip them, and leave Hero on the blind for the whole game, constantly puting a fixed amount of his stack in the pot? This is, obviously, very bad for him.


[/ QUOTE ]

We're not playing great poker from the button to only be breaking even there. But I think that's ok.

Think of it this way: we're trying to get averages over the "long run" - something close to an infinite sample. So whatever way a particular tourney played out, think of it as the average of an extremely large number of tournaments that happened to have exactly the same cards dealt when hero is on the BB (and since there's an infinte sample, there's an infinite number of those "coincidence" tourneys to average over). And our strategies from the SB were such that we break even on each "in between" hand when you average over the ensemble. Effectively, we don't have to play those hands at all, because we know that they average out to zero net chip exchange. Do I know how to specify a strategy that would actually work out like that? No, but I don't have to.

Does that make sense?

[ QUOTE ]

Or is it that each time hero is on the button, both opponents push (that's a good simulation of "even play")?


[/ QUOTE ]

I don't think that would work very well. On the first or second hand, the tournament will essentially be decided on a coin flip basis - somebody will be out or very very close to it. We won't ever have a chance to see the effects of our strategy from the BB if we do that.

[ QUOTE ]

I guess you can solve it by stating that both players are paying equal blinds on each hand.


[/ QUOTE ]

That's how my original "call or fold" model worked, but I don't think it's strictly necessary. I think it's ok to assume that play from the SB averages to net even results. I dunno, I could try an even ante case and see if it's much different.

[ QUOTE ]

2. When you ask: "what is the optimal play against this maniac?", and on the other hand state that both of you play dead even (when Hero is on the button), you are obviously saying that your only "advantage" against the opponent is in the range of hands you choose to call his all-ins with.
And this goes back to a variation on the eternal "System" debate, i.e., it is a question of how high is the blind comparing to your stack.


[/ QUOTE ]

Yep. As I mentioned in the original post (I think, didn't I?), the blind/stack ratio is one of the key parameters. I chose the number 10% just to start with something. The results definitely change dramatically as you change that number.

[ QUOTE ]

If the blind is very high (approaching 100% of your stack) you cannot, by definition, have any "optimal play" against this guy.


[/ QUOTE ]

Well it's maybe semantics but I disagree - by definition there must be an optimal play. It just might not give you any edge.

[ QUOTE ]

It's a crap shoot, and his play is optimal as yours (if you call). As your stack grows in proportion to the blind, you can wait for better and better (range of) hands to call. In the opposite example, when the blind is approaching 0% of your stack, your "optimal" play will be simply to wait for AA.

His play, in this case, is far from optimal as possible.


[/ QUOTE ]

Yep. Agree 100%.

But I think there's a range of blind to stack ratios that are pretty realistic for the end of a tournament, when play does largely become an all-in or fold situation. And I think that there is also a range where not only does play tend to be all-in or fold, but proper hand selection for calling still matters. It's more of a crapshoot than smaller blinds, but not a complete and total crapshoot.

So I'm interested in a matrix of results. On one axis is how tight he's being about pushing. On the other axis is the blind to stack ratio. Clearly if he's pushing everything when blinds are low compared to stacks, he's playing very stupidly, and it's probably not very useful information. But along one diagonal of the matrix his play is pretty reasonable and realistic - tighter when blinds are lower, and more aggressive when blinds are bigger.


[ QUOTE ]

Computing the range of hands to call with, for each proportion of blinds/stack, should be a fairly easy mathematic problem, IMO. I don't see why a simulation is needed here. Please elaborate.

PrayingMantis



[/ QUOTE ]

Because we're not just interested in a single hand, nor are we just interested in chipEV. We want to win the tournament. To win the tournament, we have to consider the rate the blinds are taking our chips. If you think you can come up with a closed form solution for figuring out which hands are optimal calling hands considering the rate we're losing blinds as well as what happens if we call and win, and don't cover our opponent, or call and lose and are not covered, I'll be impressed. Maybe it's possible but it seems pretty hard to me.

I'm also interested in getting building blocks in place to try more complex experiments - maybe 3-handed play.

eastbay

[Paul2432]

I don't think hands can be catagorized as "the best 40%". The non-transitive nature of certain hands is well known. For example, JTs beats 22 beats AKo beats JTs.

Instead, I think the hands should be listed explicitly. That said, off the cuff, I would say call a player who moves in 10x the blind with any hand with:

Any pair
Any Ace
K-x (x=9 or higher)
QT
QJ

Paul

[Bozeman]

"We're not playing great poker from the button to only be breaking even there. But I think that's ok.

Think of it this way: we're trying to get averages over the "long run" - something close to an infinite sample. So whatever way a particular tourney played out, think of it as the average of an extremely large number of tournaments that happened to have exactly the same cards dealt when hero is on the BB (and since there's an infinte sample, there's an infinite number of those "coincidence" tourneys to average over). And our strategies from the SB were such that we break even on each "in between" hand when you average over the ensemble. Effectively, we don't have to play those hands at all, because we know that they average out to zero net chip exchange. Do I know how to specify a strategy that would actually work out like that? No, but I don't have to.

Does that make sense?"

I think you are missing something here because your tourney win rate will depend not just on your win rate in the SB, but also your variance there. For example, the allin on every SB strategy against a player who will always call will mean that the tourney is over quick, so you should take any +EV situation in the BB. The same would not be true if you had a zero EV, low variance strategy for the SB. Your analysis is effectively a zero EV, zero variance SB strategy, if I understand correctly.

So your assumptions lead to a very nonphysical situation (I did once play in a tourney where they forgot to move the button once, but it has only happened to me once [img]/images/graemlins/smile.gif[/img] ) since they try to model $EV, but they ignore half of the hands.

"If you think you can come up with a closed form solution for figuring out which hands are optimal calling hands considering the rate we're losing blinds as well as what happens if we call and win, and don't cover our opponent, or call and lose and are not covered, I'll be impressed. Maybe it's possible but it seems pretty hard to me."

It seems like this would be amenable to difference (or differential) equation analysis. For given strategies, your $EV must be a function of your stack size, and $EV(x)=SUM(P(x->y)*$EV(y))=P(call&lose)*$EV(2x-1)+P(call&win)*$EV(2x)+P(fold)*$EV(x-bb) for the situation you proposed (also, $EV(x)=1 for x>=1 and 0 for x<=0). For any given strategy you could solve for $EV(x), or at least use successive approximation to get a reasonable number, and then you may be able to get an optimal strategy since there are several obvious aspects of an optimal strategy: you are more likely to call with a hand that is better headups against a random hand, and you are more likely to call as the blinds/stack ratio increases. You could certainly evaluate to find the best strategy of the type "always call with top A% of hands where A prop. to 1/stack". Does this make any sense?

Craig

[eastbay]

[ QUOTE ]
I don't think hands can be catagorized as "the best 40%". The non-transitive nature of certain hands is well known. For example, JTs beats 22 beats AKo beats JTs.

Instead, I think the hands should be listed explicitly. That said, off the cuff, I would say call a player who moves in 10x the blind with any hand with:

Any pair
Any Ace
K-x (x=9 or higher)
QT
QJ

Paul

[/ QUOTE ]

Each hand has a well-defined avg win rate against a randomly chosen hand. This is the rank order we're talking about.

This is a reasonable way to specify an answer for the situation specified because your opponent is doing exactly that: pushing in a random hand.

If your opponent was selecting hands, then what you're saying becomes important, and saying "top 40%" no longer makes a lot of sense.

eastbay

[eastbay]

[ QUOTE ]

I think you are missing something here because your tourney win rate will depend not just on your win rate in the SB, but also your variance there. For example, the allin on every SB strategy against a player who will always call will mean that the tourney is over quick, so you should take any +EV situation in the BB. The same would not be true if you had a zero EV, low variance strategy for the SB. Your analysis is effectively a zero EV, zero variance SB strategy, if I understand correctly.


[/ QUOTE ]

Yeah, I think you're right. I see that now. The "how long can we wait" variable in hand selection is not only a function of blinding out rate, but also the likelihood that you'll go out on the SB. Yeah, that does seem clear now. Thanks. And thanks to the OP for bringing it up.

So, what to do about it? We could assume not only a pushing strategy for our opponent, but also his calling strategy, and work on an optimal push/call counterstrategy of our own.



That's a lot of variables. Damn, poker is complicated, and we haven't even gotten a flop yet. [img]/images/graemlins/smile.gif[/img]

[ QUOTE ]

It seems like this would be amenable to difference (or differential) equation analysis. For given strategies, your $EV must be a function of your stack size,


[/ QUOTE ]

Given opponent's stack and fixed blind size. Yep. It occurs to me that blind structure (rate blinds increase) is another variable which affects the "how long can I wait around" component of strategy. But hell, fixed blinds is good enough for now.

[ QUOTE ]

and $EV(x)=SUM(P(x->y)*$EV(y))=P(call&lose)*$EV(2x-1)+P(call&win)*$EV(2x)+P(fold)*$EV(x-bb) for the situation you proposed (also, $EV(x)=1 for x>=1 and 0 for x<=0).

For any given strategy you could solve for $EV(x), or at least use successive approximation to get a reasonable number, and then you may be able to get an optimal strategy since there are several obvious aspects of an optimal strategy: you are more likely to call with a hand that is better headups against a random hand, and you are more likely to call as the blinds/stack ratio increases. You could certainly evaluate to find the best strategy of the type "always call with top A% of hands where A prop. to 1/stack". Does this make any sense?

Craig

[/ QUOTE ]

I like the general idea of your proposed form for a calling strategy. Although I think it should be ~T/S, where T is the total number of chips in the tourney. Then at least it goes from 0-1, rather then 0->inf.

But now I'm concerned about your (and OP's) observation that a calling strategy's optimality is a function of play from the SB. Possibly strongly so.

So how about this:

Otherwise full-scale, preflop all-in or fold tournament poker with a fixed blind. Given a blind size and an opponent's strategy (choosing from some reasonable set of benchmarks), can we formulate a counterstrategy with a small number of parameters, and optimize the parameter set for countering such an opponent?

Benchmark opponent strategies:

1) Always push, always call (a weak benchmark opponent)
2) Push top 50%, call top 50% (a better benchmark opponent)
3) Push top 75%, call top 25% (a "gap" player)

Some counter strategy assumed forms, off the top of my head, inspired by your calling strategy form:

1) Push top N%, call top N%. Optimize N for $EV.
2) Push top N%, call top M%. Optimize N,M (is there a gap? how big?)
3) Push N*S/T%, call N*T/S%, where S is your stack and T is total chips.
4) Push f(S/T)%, call g(T/S)%, optimize functions f,g over some low-dimensional parameterization.

(All "top N%" references mean avg win rate against a randomly chosen hand)

Is this going somewhere?

eastbay

[eastbay]

[ QUOTE ]
Then at least it goes from 0-1, rather then 0->inf.


[/ QUOTE ]

1/T -> 1, that is. (time for bed)

eastbay

[Mergualdo]

[ QUOTE ]
They happen all the time. I can't tell you how many times I have made money being on the bubble with a short stack by folding in a pos EV situation.

[/ QUOTE ]

How do you know you took the $-maximizing action? You guaranteed yourself a dinky payout by folding the positive EV. It may have been optimal to risk the dinky payoff in order to take a shot at one of the bigger payouts.