Game Theory & Heads-Up poker (long)

Well, after reading through some of the "Programming Poker" thread below, I thought I'd wade in with something I figured out recently.


I notice at one point that Vince asks to see a proof that an optimal strategy can be discovered in a poker game. I can't give a mathematical proof of this (though David has repeatedly stated that one exists). I can, however, give you some analysis I did of a simplified heads-up game, to show how this type of optimal strategy would be found.


Basically, the game I'm using for a model (which I posted about in the general theory forum) is a two player game where each player antes one chip, the first player can then bet 2 chips or fold. If the first player bets, the second can either call or fold. The winner is then determined.


I define the following variables:


N = the total number of possible hands.


Cn = the probability of getting hand n.


Fn = the percentage of the time I will bet with hand n.


Xij = my opponent's expectation if I bet with hand i and he calls with hand j.


Yj = my opponent's total expectation if he always calls with hand j.


P = the probability I will bet. P = sum over n from 1 to N (sum (n),1,N) of Cn*Fn


Since my opponent's expectation if he always folds with a specific hand is -1, the optimal strategy is one for which all the Yj's are equal to -1.


Yj = (sum((n),1,N) of Cn*Fn*Xnj)/P


Since Xij and Cn are known once you specify the game, this leaves a system of N equations in N unknowns (any hands we can't make a profit against we can safely bet every time we have them, so this works out). Solving the equations gives you the optimal betting strategy for the first player to act.


I went through the calculation for a game with 3 possible hands:

AA, AK, KK. I'm imagining that these hands are dealt from an infinite deck of half aces and half kings, and that the winner is determined by dealing 2 more cards from the same deck and comparing hands. No straights or flushes, quads beat trips, which beat 2 pair. It is not possible to get worse than 2 pair.


Cn = (.25, .5, .25)


Xij = ( 0 -1.5 -1.5)


(+1.5 0 +1.5)


(+1.5 -1.5 0 )


This gives us:


(.25)(0)(F1) + (.5)(+1.5)(F2) + (.25)(+1.5)(F3) = -P (this one isn't possible, btw)


(.25)(-1.5)(F1) + (.5)(0)(F2) + (.25)(-1.5)(F3) = -P


(.25)(-1.5)(F1) + (.5)(+1.5)(F2) + (.25)(0)(F3) = -P


P = .25(F1) + .5(F2) + .25(F3)


So ignoring the case where our opponent has aces and setting F1, which represents when we have aces, equal to one, we have:


(-.375) - (.375)F3 = -((.25) + (.5)F2 + (.25)F3)


(-.375) + (.75)F2 = -((.25) + (.5)F2 + (.25)F3)


This all simplifies to:


-.125 - (.125)F3 = (-.5)F2


-.125 + (1.25)F2 = (-.25)F3


Substituting .25 + (.25)F3 for F2 in the second equation, we get:


-.125 + (5/16) + (5/16)F3 = (-.25)F3


(5/16)F3 = (3/16), so F3 = .6


Therefore F2 = .4, so if we always bet with aces, bet 60% of the time with kings and bet 40% of the time with AK, we have the optimal strategy. If our opponent folds any hand but aces, and calls with aces, we lose 1.5 5% of the time, an additional 1.5 3.75% of the time, and we lose 1 40% of the time (when we fold). So are total losses (in a hundred hands) are 7.5 + 4.625 + 40 = 52.125, and we win 1 45% of the time (when we bet and our opponent doesn't have aces). So overall we lose 7.125, but if our opponent mixes up his calling with hands other than aces, we have the same result. So acting first is a disadvantage.


I'd love to hear anyone's thoughts on this. Especially Vince's.


Lenny

More nonsense !

"Since my opponent's expectation if he always folds with a specific hand is -1, the optimal strategy is one for which all the Yj's are equal to -1."


I do not know a lot about a lot of things so maybe i do not know why the optimal strategy "is one for which all the Yj's are equal to -1."


I believe the optimal strategy is one for which player one realizes the greatest expectation.


"I went through the calculation for a game with 3 possible hands: AA, AK, KK. "


I'm sorry but I do not have any view on a game of this type. I do not do game theory exercises mainly because I do not know how to and do not wish , right now, to learn. For all I know your calc's are correct. My question is "What relevance is there to Casino poker games such as Holdem or stud?"


Vince

"I'm sorry but I do not have any view on a game of this type. I do not do game theory exercises mainly because I do not know how to and do not wish , right now, to learn."


If you don't want to learn then why do you respond with a request for a proof?


"For all I know your calc's are correct. My question is "What relevance is there to Casino poker games such as Holdem or stud?"


If you don't want to learn then...


In any case I'll give you the answer. Maybe it'll help you along the road in some other area where you do want to learn.


Solving simplier problems is often used as a stepping stone to solving harder ones. An example in poker is using showdown numbers as a way to compare hands eventhough this is clearly wrong in some (many!) circumstances.

"In any case I'll give you the answer. "


"An example in poker is using showdown numbers as a way to compare hands eventhough this is clearly wrong in some (many!) circumstances."


Well, it seems to me that you have something down pat here. Much to complex for little ol me to understand just what in the hell you are saying so I guess I'll pass.


"Solving simplier problems is often used as a stepping stone to solving harder ones."


Mike, You and Sklansky and Weideman and Glover and M... and Haley and etc... can solve all the simple stepping stone problems you want. Go ahead, and if you ever come up with something remotely relevant to a practicle game theory way of playing full game limit poker please enlighten me.


"Maybe it'll help you along the road in some other area where you do want to learn."


Til then your sarcasm just further emphasizes your arrogant ignorance.


Vince

****"I went through the calculation for a game with 3 possible hands: AA, AK, KK. "


I'm sorry but I do not have any view on a game of this type. I do not do game theory exercises mainly


because I do not know how to and do not wish , right now, to learn. For all I know your calc's are correct. My


question is "What relevance is there to Casino poker games such as Holdem or stud?" ****


I was trying to show you how an optimal strategy can be generated for a simple example, because it is very complicated to work out for heads-up hold'em or stud. However, what I was trying to show you was the method for devising such a strategy. The example at the bottom of a simple game was just to show that the formula I gave at the beginning worked.


So the formula, which determines the expectation for each of your opponent's possible hands based on the frequency with which you bet each possible hand is applicable to hold'em or stud. You have to adjust it to take into account raises, and you have to work backward, in all likelyhood, from the last round, but the idea is the same.


***I do not know a lot about a lot of things so maybe i do not know why the optimal strategy "is one for which all


the Yj's are equal to -1."


I believe the optimal strategy is one for which player one realizes the greatest expectation. ***


This may explain why you refuse to accept the arguments being presented. You are taking "optimal" to be an equivalent term to "greatest expectation", but that's not what it is. You should instead think of it as "unexploitable". That is why the idea is to get all of the Yj's (or at least as many as possible) to be equal to -1. You want your opponent's actions to make as little difference as possible. That is why when deciding how often you should bluff on the end, you want to bluff at a frequency which makes the odds against you bluffing equal to the pot odds your opponent is getting to call. Game theory optimal strategy would be something akin to that, but applied to all possible hands across all rounds of betting. This means that devising a heads-up hold'em GTO strategy is incredibly difficult, but possible in theory. It may never be done, but nonetheless, it could be done. Just like mapping out all the possible games of chess that could ever be played may never be done, but would still be possible given sufficient time. It may be impractical to come up with a GTO poker bot, but that doesn't make it impossible.


I'm still unsure whether a GTO strategy is possible in a multi-way game, but a rules-based adaptive program would probably work much better in a ring game anyway, since it is easier to make a program that plays better than an average field of opponents than it is to make one that can beat a world champ heads-up. Short of devising a GTO bot, I don't believe we are anywhere close to making a computer which could beat a top pro poker player in a heads-up match. It may eventually be done, but only once we have ai which is able to accurately mimic human thought patterns (perhaps a very elaborate neural net would be capable of learning to beat a human pro). This is not the same thing as chess, because in chess you can evaluate a particular position and it is always the same every time you see that position. In poker, you have to adapt to your opponent, and without having a GTO strategy, the computer would have almost no chance of keeping up with a pro. Again, this argument is based on current ai technology.

"You should instead think of it as "unexploitable". "


Yes.


"That is why when deciding how often you should bluff on the end, you want to bluff at a frequency which makes the odds against you bluffing equal to the pot odds your opponent is getting to call"


Game theory can be an effective tool for determining Bluffing frequency, I agree.


"but only once we have ai which is able to accurately mimic human thought patterns"


Maybe accurately "predict human behavior" is more correct. I'm not sure a computer that mimicked human thought (patterns) would be very good at anything. Especially if they used me as a model. If they use Sklansky they might be o.k. But who wants a computer that chases woman half his sons age.


Good post. Thanks.


vince