Simulated heads-up endgame in all-in or fold mode

[dana33]

After lurking for a couple of months, this is my first post to the Two Plus Two forums.

Inspired by eastbay's optimizing calling all-in in a heads-up, all-in or fold poker model (http://forumserver.twoplustwo.com/showthreaded.php?Cat=&Number=560157&page=&view=&sb =5&o=) posting, I decided to do my own simulations along the same general lines, but with a somewhat different approach. Here is what I have done so far:

1. Got Steve Brecher's C code off the Web for computing heads-up showdown probabilities for winning outright (Pwin) and splitting the pot (Ptie) based on two-card starting hold 'em hands. Hacked the code to run in batch mode and compute the 169x169 matrices for Pwin and Ptie for all possible combinations of hands. I had to do some combinatorics for this as well. E.g., to compute Pwin for Q9s versus KJo, you need to properly weight the different cases according to whether there is a shared suit between the two hands, and if so whether the K or J shares the suit with the first hand.

2. Computed P(hand), the probability of getting a particular two-card starting hand, and P(hand2|hand1), the conditional probability that a single opponent has hand2 given that you have hand1. Computing these is a matter of simple combinatorics.

3. Used these probabilities to drive simulations of a tournament heads-up endgame where both players are in "all-in or fold" mode due to the size of the blinds. Each player has a set of hands he will move in with from the SB, and another (typically smaller) set of hands with which he will call from the BB if the first player has not folded. If the SB player does not move in, he folds.

In eastbay's simulations, he assumed an opponent who always moved in, and optimized the specific set of hands with which one should call. I simplified things by ordering the 169 starting hands according to Pwin against a random hand, and then introducing parameters Pmovein and Pcall representing the percentage of these hands with which to move in or call. E.g., if Pmovein is 80% and Pcall is 60%, a player will move in with the top 80% of hands in this list, and call with the top 60%. In my simulations, both the "hero" and "villain" have their own values for these parameters, each of which I varied in 20% increments, but always keeping Pmovein >= Pcall.

The results so far for the case where both players start the endgame with equal sized stacks which are 10 times the size of the BB:

If "par" is winning the tournament half the time, it is easy to do much worse than par by being too tight. E.g., if your opponent only moves in with the top 40% of hands and calls with the top 20%, you can win up to 69% of the time with optimum counter-tactics. But it is hard to do much better than par if your opponent is sufficiently loose. It appears that moving in with the top 80% of hands and calling with the top 60% is a virtually unbeatable combination, in that your opponent can do no better than par in this case (within the statistical error of my simulations). This seems surprisingly loose, but that's what the simulation results say. (I would be interested in any game theoretic argument anyone has regarding whether these results are right or wrong.)

Against a complete maniac who pushes and calls with anything, my results are fairly flat from Pcall=20% to 80% (with a win percentage that varies only between 50% and 54%), using the best Pmovein in each case. But there does appear to be a maximum in the curve close to 40%, which is consistent with the results that eastbay reported. I'll need to increase the number of tournaments I simulate for each set of parameters in order to be more confident in these particular results.

As discussed in eastbay's thread, there are more sophisticated hand selection strategies one could use, such as making Pmovein and Pcall functions of the stack-to-blind ratio. I plan to try these once I find the time.

Comments welcome.

Dana

P.S. I want to specifically thank Steve Brecher for posting his code to the Web and eastbay for getting me started on this effort. Whether or not I learn anything about poker from the results, I had a lot of fun writing the code.

[eastbay]

[ QUOTE ]
After lurking for a couple of months, this is my first post to the Two Plus Two forums.


[/ QUOTE ]

Excellent.

[ QUOTE ]

Inspired by eastbay's optimizing calling all-in in a heads-up, all-in or fold poker model (http://forumserver.twoplustwo.com/showthreaded.php?Cat=&Number=560157&page=&view=&sb =5&o=) posting, I decided to do my own simulations along the same general lines, but with a somewhat different approach. Here is what I have done so far:

1. Got Steve Brecher's C code off the Web for computing heads-up showdown probabilities for winning outright (Pwin) and splitting the pot (Ptie) based on two-card starting hold 'em hands. Hacked the code to run in batch mode and compute the 169x169 matrices for Pwin and Ptie for all possible combinations of hands. I had to do some combinatorics for this as well. E.g., to compute Pwin for Q9s versus KJo, you need to properly weight the different cases according to whether there is a shared suit between the two hands, and if so whether the K or J shares the suit with the first hand.

2. Computed P(hand), the probability of getting a particular two-card starting hand, and P(hand2|hand1), the conditional probability that a single opponent has hand2 given that you have hand1. Computing these is a matter of simple combinatorics.


[/ QUOTE ]

Very nice. I spent some time doing the same things a few days ago. We should compare numbers. We can do that on or off-list, whichever.

I did mine at 3am so I'm sure I've made a mistake or two, but getting our numbers to agree should be good for both of our codes.

[ QUOTE ]

3. Used these probabilities to drive simulations of a tournament heads-up endgame where both players are in "all-in or fold" mode due to the size of the blinds. Each player has a set of hands he will move in with from the SB, and another (typically smaller) set of hands with which he will call from the BB if the first player has not folded. If the SB player does not move in, he folds.

In eastbay's simulations, he assumed an opponent who always moved in, and optimized the specific set of hands with which one should call. I simplified things by ordering the 169 starting hands according to Pwin against a random hand, and then introducing parameters Pmovein and Pcall representing the percentage of these hands with which to move in or call. E.g., if Pmovein is 80% and Pcall is 60%, a player will move in with the top 80% of hands in this list, and call with the top 60%. In my simulations, both the "hero" and "villain" have their own values for these parameters, each of which I varied in 20% increments, but always keeping Pmovein >= Pcall.


[/ QUOTE ]

Ok. I'll see your simulation and raise you one.

My idea is stick with the two-parameter strategy of a top N% to push, top M% to call, but to re-order the hands as they rank against typical "all-in" hands. It will be interesting to see if this makes much of a difference in the results. The results of the rank order itself might be of some interest.

[ QUOTE ]

The results so far for the case where both players start the endgame with equal sized stacks which are 10 times the size of the BB:

If "par" is winning the tournament half the time, it is easy to do much worse than par by being too tight. E.g., if your opponent only moves in with the top 40% of hands and calls with the top 20%, you can win up to 69% of the time with optimum counter-tactics. But it is hard to do much better than par if your opponent is sufficiently loose.


[/ QUOTE ]

If there's anything I've learned from these kinds of heads-up analyses, this is it. When blinds reach 10% of stack or greater, heads-up, it's always better to push too much than too little. Almost without bound this is true. Waiting for a "premium hand" is a total disaster. Pushing every single hand, no matter what, is a much stronger strategy for winning the tournament. Possibly counter-intuitive, but true.

[ QUOTE ]

It appears that moving in with the top 80% of hands and calling with the top 60% is a virtually unbeatable combination, in that your opponent can do no better than par in this case (within the statistical error of my simulations). This seems surprisingly loose, but that's what the simulation results say. (I would be interested in any game theoretic argument anyone has regarding whether these results are right or wrong.)

Against a complete maniac who pushes and calls with anything, my results are fairly flat from Pcall=20% to 80% (with a win percentage that varies only between 50% and 54%), using the best Pmovein in each case. But there does appear to be a maximum in the curve close to 40%, which is consistent with the results that eastbay reported. I'll need to increase the number of tournaments I simulate for each set of parameters in order to be more confident in these particular results.

As discussed in eastbay's thread, there are more sophisticated hand selection strategies one could use, such as making Pmovein and Pcall functions of the stack-to-blind ratio. I plan to try these once I find the time.

Comments welcome.

Dana

P.S. I want to specifically thank Steve Brecher for posting his code to the Web and eastbay for getting me started on this effort. Whether or not I learn anything about poker from the results, I had a lot of fun writing the code.

[/ QUOTE ]

I think we're just getting warmed up with this line of inquiry, and I do think the results mean something qualitatively, at least.

Your results would take on a whole new meaning if you made some graphs. If you don't have a good graphing program, send me the numbers and I'll throw them in Excel and post the results, if you want.

eastbay

[dana33]

Thanks for the great feedback. We seem to be thinking along the same lines -- e.g., reordering the hands based on how they do against other legitimate hands as opposed to random hands.

I need to run some more simulations in order to get the noise down before I can get any meaningful graphs, but I agree that it's a good idea and will do it once I get better data.

A newbie technical question: I don't have my own Web site where I could post a graph and link from here. Once I create a graph, is there any way to upload a GIF or Excel file to the bulletin boards here and embed it in a posting?

[eastbay]

[ QUOTE ]
Thanks for the great feedback. We seem to be thinking along the same lines -- e.g., reordering the hands based on how they do against other legitimate hands as opposed to random hands.


[/ QUOTE ]

See my results for any ace, any pair rankings, freshly posted.

[ QUOTE ]

I need to run some more simulations in order to get the noise down before I can get any meaningful graphs, but I agree that it's a good idea and will do it once I get better data.


[/ QUOTE ]

Isn't it sick how many samples are required for answers good to three decimal places? It's kind of disconcerting about the luck factor in poker in general, IMO.

[ QUOTE ]

A newbie technical question: I don't have my own Web site where I could post a graph and link from here. Once I create a graph, is there any way to upload a GIF or Excel file to the bulletin boards here and embed it in a posting?

[/ QUOTE ]

I think you'd have to find a free webhost that would allow it - there aren't too many of those left.

If you're gung ho, I have a webserver that we could use to collect some of these results in one place - I could give you an account on it.

eastbay

[allenciox]

I have done some related tests --- figuring out what are the best hands for calling all-in bets based on what percentage (5%,10%,15%,25%,40%,67%,100%) of bets an opponent has raised all-in with. I used a matrix of 169x169 matchups, but only had the ev of each matchup. Would it be possible for me to get your matrix that includes Pwin and Ptie? Then I can collaborate in this effort.

Thanks!

[dana33]

eastbay,

Please send me your email address in a private message and I will send you my matrices of probabilities so we can cross-check.

I have done some more extensive simulations and the conclusion that "push with top 80%; call with top 60%" is a hard to beat strategy (with a 10:1 stack:BB ratio) is holding up. Given this, I wonder how important the exact hand-ranking scheme is. Even though the top hands clearly change depending on whether you are up against a random versus a strong hand, do the bottom 20-40% of hands change that much?

[eastbay]

I'm interested on how rapidly this optimal strategy changes with blind/stack ratio.

If you make it 5%, is it a little different? A lot different?

I'm going to try to write a lean/mean simulator for this that can crank large samples in a short amount of time. My full scale NLHE simulator is far too slow (due to the generality) for really exploring this space.

eastbay

[eastbay]

[ QUOTE ]
eastbay,

Please send me your email address in a private message and I will send you my matrices of probabilities so we can cross-check.

I have done some more extensive simulations and the conclusion that "push with top 80%; call with top 60%" is a hard to beat strategy (with a 10:1 stack:BB ratio) is holding up. Given this, I wonder how important the exact hand-ranking scheme is.

[/ QUOTE ]

I have a hunch it will result in optimal strategy being a bit tighter, since you'll be facing "stronger" hands when you call, I think you're going to have to tighten up your calling standards.

That's a guess, though.

eastbay

[lorinda]

This old thread may be of interest.

Note I haven't seen the thread that this post relates to.

Heads up vs blind man (http://forumserver.twoplustwo.com/showflat.php?Cat=&Board=probability&Number=260939& Forum=probability&Words=lorinda&Match=Entire%20Phr ase&Searchpage=0&Limit=25&Old=1year&Main=259422&Se arch=true#Post260939)

Lori

[dana33]

[ QUOTE ]
I'm interested on how rapidly this optimal strategy changes with blind/stack ratio.

If you make it 5%, is it a little different? A lot different?


[/ QUOTE ]

It's quite a bit different. While 80%/60% push/call is nearly unbeatable for a 10% BB:stack ratio, the nearly unbeatable strategy for a 5% ratio is closer to 60%/40%. However, even here, 80%/60% is not all that bad -- about the best you can do against it is a 54% win rate (with equal starting stacks).

Of course, when the BB is only 5% of your stack size, you're probably not yet to the point that "move in or fold" is a winning approach, so the assumptions of the simulation start to break down.

Note that in my current simulations, the strategy remains fixed throughout. That is, the players do not change their strategies as their individual stacks grow or shrink. On my list of 57 other things to try with my simulator is your previous suggestion of making the percentage of hands played a function of a player's current stack size.

[eastbay]

[ QUOTE ]
[ QUOTE ]
I'm interested on how rapidly this optimal strategy changes with blind/stack ratio.

If you make it 5%, is it a little different? A lot different?


[/ QUOTE ]

It's quite a bit different. While 80%/60% push/call is nearly unbeatable for a 10% BB:stack ratio, the nearly unbeatable strategy for a 5% ratio is closer to 60%/40%. However, even here, 80%/60% is not all that bad -- about the best you can do against it is a 54% win rate (with equal starting stacks).

Of course, when the BB is only 5% of your stack size, you're probably not yet to the point that "move in or fold" is a winning approach, so the assumptions of the simulation start to break down.


[/ QUOTE ]

Good point.

Maybe more interesting is this: what happens 3-way? If you say that you will always fold if there are two people in the pot, and both your opponents are playing strategy X, what is the best counter strategy Y? Is it looser or tighter than heads-up? I think it's got to be quite a bit tighter, to account for the probability that your opponent might get knocked out first.

We'd have to assume a payout structure to do this sensibly based on EV. 50/30/20 seems standard enough.

eastbay

[dana33]

[ QUOTE ]

Maybe more interesting is this: what happens 3-way? If you say that you will always fold if there are two people in the pot, and both your opponents are playing strategy X, what is the best counter strategy Y? Is it looser or tighter than heads-up? I think it's got to be quite a bit tighter, to account for the probability that your opponent might get knocked out first.


[/ QUOTE ]

Oh man, I don't even want to think about generating the conditional probabilities I would need for 3-way action! Your simulator is probably better equipped than mine to handle this case.

On the stack:BB ratio and its effect on optimal strategy, I have done a bit more. I collected data for stack:BB ratios of 20:1, 10:1, and 5:1, and fitted a cubic model to each data set. Using each model, I ran a numerical optimization to determine the percent of hands to move in and call with, given an opponent's move in and calling percentages. By trial and error, I varied the opponent's strategy until the best you could do for counter-strategy would be to match his strategy -- meaning each of you would win half the time, and with any deviation from the strategy, you would lose more than half the time.

Based on this analysis, and still using the ranking of the hands against random hands, the unbeatable strategy in each case is:

5:1 stack:BB -- Move in with anything (100%), call with top 75%.

10:1 stack:BB -- Move in with top 70%, call with top 50%. (Note that this differs from the 80%/60% result I posted previously, which was based only on inspection of the data.)

20:1 stack:BB -- Move in with top 60%, call with top 30%.

I trust the trends in the above numbers more than the exact numbers themselves, and they certainly make qualitative sense. The bad news is that, unless your opponent plays much too loose or much too tight, the edge you get from optimal strategy is not that great. Once the blinds get big, it's a bit of a crapshoot no matter how you slice it.

[Im Just A Bill]

I must say, you guys have inspired me. I am relatively new to hold 'em and am just getting back into stats. This is my first post here. If you get some of this stuff posted, I would love to take a look. I could also provide some web space if needed.

Bill

[eastbay]

[ QUOTE ]
[ QUOTE ]

Maybe more interesting is this: what happens 3-way? If you say that you will always fold if there are two people in the pot, and both your opponents are playing strategy X, what is the best counter strategy Y? Is it looser or tighter than heads-up? I think it's got to be quite a bit tighter, to account for the probability that your opponent might get knocked out first.


[/ QUOTE ]

Oh man, I don't even want to think about generating the conditional probabilities I would need for 3-way action! Your simulator is probably better equipped than mine to handle this case.


[/ QUOTE ]

It's probably easiest to just shuffle and deal a real deck, and work from there. Or, for speed's sake, just draw randomly from an unshuffled deck.

eastbay

[ctv1116]

[ QUOTE ]

Oh man, I don't even want to think about generating the conditional probabilities I would need for 3-way action! Your simulator is probably better equipped than mine to handle this case.

On the stack:BB ratio and its effect on optimal strategy, I have done a bit more. I collected data for stack:BB ratios of 20:1, 10:1, and 5:1, and fitted a cubic model to each data set. Using each model, I ran a numerical optimization to determine the percent of hands to move in and call with, given an opponent's move in and calling percentages. By trial and error, I varied the opponent's strategy until the best you could do for counter-strategy would be to match his strategy -- meaning each of you would win half the time, and with any deviation from the strategy, you would lose more than half the time.

Based on this analysis, and still using the ranking of the hands against random hands, the unbeatable strategy in each case is:

5:1 stack:BB -- Move in with anything (100%), call with top 75%.

10:1 stack:BB -- Move in with top 70%, call with top 50%. (Note that this differs from the 80%/60% result I posted previously, which was based only on inspection of the data.)

20:1 stack:BB -- Move in with top 60%, call with top 30%.

I trust the trends in the above numbers more than the exact numbers themselves, and they certainly make qualitative sense. The bad news is that, unless your opponent plays much too loose or much too tight, the edge you get from optimal strategy is not that great. Once the blinds get big, it's a bit of a crapshoot no matter how you slice it.


[/ QUOTE ]

A few questions: when you say stack/BB ratio, do you mean total chips out there BB. For example in a 10 person SNG where everyone starts with 1000 chips, and blinds are 500/1000, is that 10:1 ratio? Or are you referring to BB in relation to your stack?

Does your stack position as compared to the other person affect your strategy at all?

[eastbay]

[ QUOTE ]

As discussed in eastbay's thread, there are more sophisticated hand selection strategies one could use, such as making Pmovein and Pcall functions of the stack-to-blind ratio. I plan to try these once I find the time.


[/ QUOTE ]

I tried a Sklansky/Chubukov type criterion for pushing in:

http://www.decf.berkeley.edu/~chubukov/rankings.html

and it did slightly worse than a fixed strategy that was more aggressive, no matter what fixed percentage calling strategy I attached to it.

Hmmm.

eastbay

[eastbay]

[ QUOTE ]

A few questions: when you say stack/BB ratio, do you mean total chips out there BB.
For example in a 10 person SNG where everyone starts with 1000 chips, and blinds are 500/1000, is that 10:1 ratio? Or are you referring to BB in relation to your stack?


[/ QUOTE ]

stack/BB means just that. stack over big blind.

[ QUOTE ]

Does your stack position as compared to the other person affect your strategy at all?

[/ QUOTE ]

No. These strategies are fixed for the tournament and are not a function of relative chip position.

I have tried using a push strategy which follows the Sklansky/Chubukov (Karlson) idea for positive EV plays, but it did worse and not better than some other strategies which are independent of stack size.

eastbay

[dana33]

[ QUOTE ]

A few questions: when you say stack/BB ratio, do you mean total chips out there BB. For example in a 10 person SNG where everyone starts with 1000 chips, and blinds are 500/1000, is that 10:1 ratio? Or are you referring to BB in relation to your stack?

Does your stack position as compared to the other person affect your strategy at all?

[/ QUOTE ]

So far, I have only looked in detail at the case where the stacks are equal when the heads-up match begins, in which case the ratio is of either player's stack to the BB. I.e., if the BB is 100 and both players have stacks of 1000, that is a 10:1 ratio.

I have run some simulations where the stack sizes differ -- e.g., your stack is 1000 and your opponent's is 2000 (with BB=100). I still need to do the full analysis of the data. But from inspecting the raw results, it appears that if your opponent uses the 60/30 strategy, which would be optimal for equal stacks of 2000 each, then the best counter-strategy is looser than the 70/50 optimal strategy for equal stacks of 1000 each. (I hope that sentence is intelligible!) Short answer: it appears that your stack position relative to your opponent does indeed affect your optimal strategy.

I'll post more once I find the time to analyze my results for unequal stack sizes.

[dana33]

[ QUOTE ]
I have run some simulations where the stack sizes differ -- e.g., your stack is 1000 and your opponent's is 2000 (with BB=100). I still need to do the full analysis of the data. But from inspecting the raw results, it appears that if your opponent uses the 60/30 strategy, which would be optimal for equal stacks of 2000 each, then the best counter-strategy is looser than the 70/50 optimal strategy for equal stacks of 1000 each. (I hope that sentence is intelligible!) Short answer: it appears that your stack position relative to your opponent does indeed affect your optimal strategy.


[/ QUOTE ]

Just to clarify: In the simulations I have run, the strategy is not dynamic. That is, when I say a strategy is optimal for this or that ratio of stack to BB size, I mean that it is the best FIXED strategy given the INITIAL ratio once heads-up play begins.

A dynamic strategy makes more sense in practice -- e.g., even if you start with a stack:BB ratio of 20:1, if your opponent steals a bunch of blinds such that you get down to a 5:1 ratio, you will surely need to loosen up at that point. At some point, I plan to enhance my simulation code to support such a dynamic strategy.

[dana33]

[ QUOTE ]
Does your stack position as compared to the other person affect your strategy at all?

[/ QUOTE ]

I have now analyzed my results where stack1:BB is 10:1 and stack2:BB is 20:1 at the start of the simulation. It turns out that the optimal fixed strategy for both players is identical, and is equal to the average of the optimal strategies for equal stacks for the 10:1 and 20:1 stack:BB cases.

That is, my prior results for equal stacks say that 70/50 push/call is the optimal strategy for a 10:1 ratio, and 60/30 is the optimal strategy for a 20:1 ratio. For the unequal stack case above, the optimal strategy is 65/40 for both players.

So, given the assumption of a fixed strategy and these results, it appears that the optimal strategy depends only on the mean stack size relative to the BB, not on the stack sizes relative to each other. I'll have to see how these results change once I relax the fixed strategy assumption.