Stuck writing an AI

[ddubois]

My company is developing a poker game and I've written an AI for holdem SNG play that I'm reasonably proud of. (We've shoe-horned the code to also do NL ring game play, which it is not good at, and limit ring game play, which it is awful at, but that's not relevant for this post.)

Now they have decided they would really like to provide a single-player mode for all the game types we originally intended to be online-only. This includes Stud and Stud/8. Now, I've lurked this forum some, played a little bit of low-limit in the past, and perused Zee's book some, but really, I'm not good at this game. However, being the resident 'poker genius', the task falls to me unequivocally. And I am struggling as to how to attack it. Some of the strateges and code I used for hold'em might be somewhat transferable to Omaha, but I do not perceive that to be the case at all for stud.

I am looking for any advice anyone might have. An example of something that would be useful: Are there any kinds of pointing systems for stud or stud/8 starting hands? Or would you think I would be best served to try to categorize all starting hands into one of Zee's 16 categories (pg 13-23), and add rules based logic according to those passages of the book?

Generally speaking, any mathematical or formulaic way of looking at the game is the the easiest means of giving the computer some ability to play. Less good is rules-based logic. (This is why I was able to make the SNG AI solid; at large blinds it's mostly a pre-flop math game.)

[jon_1van]

Questions first ::

-Should your code play different ante structures well (known how to react properly to over ante, high ante, medium ante, small ante, and no ante games?

-Will your code adjust to opponents. IE can a player run over a tight bot or wait for good cards against a LAG bot.

[BTirish]

I think that, for the starting hands in both stud and stud/8, you have to do what you suggested, and use rules-based logic based on the kind of hand you have. Stud hi in particular is a very complex game on 3rd street, and there is no set of absolute hand values possible--a pair of Q's with an offsuit kicker is strong when the Q is the highest card on board and the kicker is live. A pair of Q's with the same live offsuit kicker with a K raising and an A reraising is an almost worthless hand and an automatic fold. In fact, in a tight ante structure, with an A and a K yet to act behind you, the same pair of Q's can easily be folded. Hand value also shifts dramatically based on how many people are in the hand. Pot equity does change in hold 'em based on how many opponents one is up against, but not as dramatically as in stud.

Chip Reese's section in Super System I might be a good resource. He lays out some general principles for premium pairs, middle pairs, 3 flushes, and 3 straights.

It's on the later streets that a computer could be programmed to make decisions mathematically. Obviously the computer could be flawless in keeping track of dead cards and of computing odds to make a hand. The trick is getting the computer to know how to count good outs, based on the opponents' boards.

[ddubois]

[ QUOTE ]
Should your code play different ante structures well

[/ QUOTE ]
This would not be necesary. For simplicity's sake, we would only be using one betting structure, and in fact, if it turns out that one structure tends to favor the AI's tendencies and/or mask it's weaknesses, I will ensure that to be the structure we use. For instance, one way to mask the computer's stupidity in hold'em is to make it play extra tightly pre-flop so that it's post-flop mistakes don't occur as often, and with good starters it doesn't get into bad situations as often post-flop. Also, we can make the blinds rise quickly, to get to the part of the game the computer is good at. I would venture a guess that some aspects of stud can be the same, i.e., some ante structure makes "tight-is-right"? Do you have a recommendation along these lines?

[ QUOTE ]
Will your code adjust to opponents. IE can a player run over a tight bot or wait for good cards against a LAG bot.

[/ QUOTE ]
Yea. Not as much as a good player would adjust, but the AI needs to at least be smart enough to not be obviously exploitable by the 'mad bluffer'. In hold'em I track the PFR, VPIP, and AF for all the players, and make some minimal adjustments on decisions made if the opponent's numbers are out of the ordinary. I'm going to have to do something like that with stud as well, although unfortunately I don't know what exactly I will track or what would be considered 'normal'. There's no PokerTracker for stud is there? So concepts like AF, VPIP and WTSD% are not typically discussed as-such in this forum, correct?

Honestly, my goals are not very high. I don't even need to make an AI that can beat a low-limit party table, just one that's not ridiculously awful. Has anyone played stud on Bicyle Casino or World Championship Poker for the Xbox?

[donkeyradish]

A key difference, as I'm sure you realised already, is that your hand in stud has varying appeal depending on what other cards are out

[jon_1van]

When opening a pot I'd suggest using output from 2 methods

double getProbAhead(Card[] myCards , Card[] 3rdUpCards, Car[] deadCards) - returns approximate probability that the hand you hold is currently best

double getProbSteal(same auguments as above) - returns approximate probability that the opponents left don't have a hand that is worth a call (requires assumputions about which hand the opponent will call with)

Now when you get the results of those 2 methods you can have your bot weight the results...For example

if(probAhead*coefficent1 + probSteal*coefficent2 > threshold) raise
else fold

The biggest problem you'll get is how to react to people who have already opened the pot.

For this situation I'd suggest ::
-making a method that will categorize every possible 3rd street hand as a hand that wants a multiway pot (draws)or a hand that wants a shorthanded pot (pairs)or a hand that doesn't care much about the number of opponents (flush draws with live big cards)

-Then when deciding how to proceed it can take things like expected number of opponent into account....which should be important.


For a structure I'd suggest a structure isomorphic to the 3/6 structure. The varience is high enough in this structure that you bots can't get crushed as bad as they would in a really tight 5/10 type structure.

-I think 3rd street play will be very critical if you want your bots to play well. I would suggest a creating a number of methods like
-probImAhead
-probStealWork
-probRaiseBehind
-probImBestAtEnd (maybe)

Then weight whatever these methods give and add up that wieghted sum. If the sum is greater than some threshold do the appropraite thing.

Good luck, keep us posted.

Did you compete in the ICCM competition?