Two Plus Two Older Archives

Two Plus Two Older Archives (http://archives2.twoplustwo.com/index.php)

- Poker Theory (http://archives2.twoplustwo.com/forumdisplay.php?f=13)

- - Poker AI (http://archives2.twoplustwo.com/showthread.php?t=56134)

I'm curious what people here might know about the current state of the art in Poker Artificial Intelligence programs. It seems to me that this is the biggest long term threat to online poker. Automating the interphase with the online poker site's software might require some specialized work on the installed code. But that aside, I can't help but think current state of the art poker AI programs could easily beat nearly all the games played online by humans. I doubt the sites themselves would want to be running such BOTs, but what's stopping ambitious gamers from plugging them into their Online play?

Anyone know the word going around on this?

Hi PairTheBoard,

I play NLH tournaments, and perhaps I'm underestimating expert systems (which is what you mean, not AI), but I just don't think a computer opponent would be much of a worry in the games I play. Perhaps it's because NLH offers a wider range of possibilities and rewards intuition and creativity more than limit hold'em or stud, but I just don't think it likely that even a state of the art expert system could be in the same league with a good NLH player.

Cris

I disagree, I think a well written computer program would be a disater to online poker.

A program that could customize its play based on opponents past play history, could use game theory optimally, and could play without any tilt-issues would destroy online poker, even NL. Sure some players might be able to beat it, but I would guess that even a moderate computer program would be capable of beating 90%+ of the players online.

Think of it like chess, most chess players even ones who are pretty good probably can't even beat a storebought chess program set to max difficulty. If the poker equivalent of Deep Blue (or whatever the top chess program) were to be developed, I would be quite worried.

Beating 90% of the players online might make you a breakeven player but hardly a winner.

You have little understanding of how difficult it would be to write a poker bot that could beat even 50% of the online players. The incomplete information is overwhelming. An exception would be if your bot played against the same 9 oponnents in a 10 handed ring game for 50,000 hands or so. Then it might become one of the best players in this particular game but would go back to being mediocre if several new players were introduced. This would only be true of an extremely advanced AI program far more advanced than even POKI.

[ QUOTE ]
Beating 90% of the players online might make you a breakeven player but hardly a winner.

[/ QUOTE ]

I think this is a huge exageration.

[ QUOTE ]
You have little understanding of how difficult it would be to write a poker bot that could beat even 50% of the online players. The incomplete information is overwhelming. An exception would be if your bot played against the same 9 oponnents in a 10 handed ring game for 50,000 hands or so. Then it might become one of the best players in this particular game but would go back to being mediocre if several new players were introduced. This would only be true of an extremely advanced AI program far more advanced than even POKI.

[/ QUOTE ]

I never said it would be easy, but I think you are vastly overestimating the ability of an average player. Someone could probably write a bot that just plays ABC by the book poker and it would probably be a winner. If the bot could be tied into a database similar to pokertracker and played max-tables 24/7 it would have a huge database of player information to work with. As for POKI, I think it would probably do pretty well in an online game.

Re: Poker AI - Prisoner\'s Dilema - Genetic Algorithm

I really don't know. I'm just throwing out ideas. There IS a difference between Poker and Chess. The Forte of chess programs like Deep Blue is their Vast memory of previously played "Book" openings, and their ability to compute all possible move combinations for many moves ahead. In poker it's maybe not so clear how important "Book" moves might be nor is it so clear how to compute all game theoretic future repercusions of Poker "moves". Still, I can't help but recall the widespread Human bluster and bravado years ago, about how computer programs would NEVER be able to beat the best human chess players. Although Kasparov can still make it a match with the best computer programs, the writing is pretty well on the wall for the human vs. computer chess question.

Yes, an "Expert System" trained by a world class player would seem the natural way to produce a Poker program that might imitate AI. But that may not be the only solution.

In many ways I think Poker is similiar to the Prisoner's Dilema. Two accomplises in a crime are interrogated. If neither rats on the other they both go to jail for say 1 year. If one rats on the other but the other remains mute, the rat gets say 0 time and the chump gets say 4 years. If they both rat they both get say 2 years. You can change the numbers to make it a better game if you want.

A challange was issued by Scientific American years ago for people to submit algorithms to play essentially this game against a population of other algorithms. Two algorithms from the population randomly meet. If they co-operate they both gain a little. If one co-operates and the other betrays, the deceiver wins more while the fooled loses. The alogirthms retain memory of at least the transactions they are involved in. One way of playing this game, rather than just tallying points for each algorithm, is to let the population evolve by rewarding successful algorithms by reproducing them. I believe such a "game" has many of the same dynamics as Poker. For example, there are Non-Transitive aspects to it. A does well against B and B does well against C. But C may badly beat up on A. It was somewhat suprising that out of all the complicated algorithms submitted for the contest, the ones that did the best were the simple "Tit for Tat" programs. You co-operate, I co-operate. You betray, I betray. Algorithms that tried to improve on this had a lot of trouble working well against the while population.

A very suprising later finding was that the Genetic Algorithm could be applied to this problem with considerable success. Strategic variations on the Tit for Tat were encoded into 16 bit strings. A population of such 16-bit Sring algorithms were run using Genetic crossover and mutation to search for imporved versions. The champ was then run against the human alogrithms and did considerably better than the best Tit for Tat human champs.

I suspect something like this could be done for Poker to produce a viable Poker AI program.

Re: Poker AI: An Essay on the Topic

I think many of you are missing several key differences (and potential advantages) of an automated online poker player, or ‘bot’. At the root of this misunderstanding is not realizing that a bot essentially represents a paradigm shift in what constitutes an effective player.

One: The ability to calculate the odds of every situation with perfect accuracy and make other strategic adjustments as ring size and table texture change. Given the huge player turnover in online games, the speed of the games (40-100 games an hour) human tracking of these changes is near impossible for the vast majority of players.

Two: The ability to play 24/7/365 with no fatigue, steaming etc. i.e., without emotion.

Three: Since your win rate requirement can be much lower than when you are playing in person, the bot can play much more patiently than any human player ever could. Since all current poker theory is based on squeezing the maximum out of every situation (i.e., to be as efficient as possible), an approach not driven by this consideration, in effect, allows for the creation of a hitherto unknown style of expert player.

Four: The ‘soft’ skills the expert has built up over many years are irrelevant, since no-one ever sees your face, tics, expressions etc. (I, for one, have played tens of thousands of hands without ever once sitting down at a real table). Five: Anyone who can develop the technology to create a bot of this type could also design an automated opponent tracker. The tracker would sit in on tables gathering information on all the players. Over a period of a few days substantial statistical dossiers on the playing habits of the regulars would be available. This information ,would be updated in real-time during the games and coordinated with table texture stats.

Five, the bot would roam the online HE tables looking for soft games. (Visualize the snoop-and-attack drones in the Matrix!).

Six: Simple random behavior would be built in to vary response times and other electronic tells making the bot virtually indistinguishable from its human competitors. Given the tens of thousands of highly inexpert human online players, a few bots of this type would be able to survive indefinitely.

Now two challenges: One: Tell me what you think of the preceding argument. Two: How would YOU design the bot to take maximize its superhuman capabilities?

Re: Poker AI: An Essay on the Topic

"human tracking of these changes is near impossible for the vast majority of players." And for a bot as well, at least in real time.

" The ability to play 24/7/365 with no fatigue, steaming etc. i.e., without emotion." A losing bot will simply lose more money and quicker.

"Since all current poker theory is based on squeezing the maximum out of every situation (i.e., to be as efficient as possible), an approach not driven by this consideration, in effect, allows for the creation of a hitherto unknown style of expert player. " The reason this is so is to win in the first place, not to win more. Without the ability to "squeeze" those extra bets you go from a winner to a loser.

"The ‘soft’ skills the expert has built up over many years are irrelevant..." So is this statement, think about it.

"Anyone who can develop the technology to create a bot of this type could also design an automated opponent tracker. " Not to mention they would also be able to design a device that spins straw into gold which by the way would be more profitable.

"Over a period of a few days substantial statistical dossiers on the playing habits of the regulars would be available. This information ,would be updated in real-time during the games and coordinated with table texture stats. " Incomplete information, enough said.

"Now two challenges: One: Tell me what you think of the preceding argument." I did.

" Two: How would YOU design the bot to take maximize its superhuman capabilities?" I have no time for this, I am still working on that straw to gold problem. Do you know where I might find a little fella named Rumpelstilzchen?

That being said I believe a winning bot is possible, just not with today's technology.

Some previous threads to check out:

http://www.twoplustwo.com/forums/showthreaded.php?Cat=&Board=genpok&Number= 186902

http://www.twoplustwo.com/forums/showflat.php?Cat=&Board=inet&Number=159004

http://www.twoplustwo.com/forums/showthreaded.php?Cat=&Board=books&Number=1 90061

http://www.twoplustwo.com/forums/showthreaded.php?Cat=&Board=genpok&Number= 197446

Through these links, check out some posts by "Botman" who claimed, plausibly, to have made a successful bot for lowlimit online play.

Also, there was a fascinating article produced by the University of Alberta people that won a prize at an AI conference. The paper was about software they wrote to play "pseudo-optimal" headsup limit holdem. Theoretically using game theory you could play "optimal" headsup holdem (i.e. a mixed strategy against which no other strategy would have positive EV). However, the computing power to obtain the optimal strategy is much too large, so the University of Alberta team reduced it by several orders of magnitude by making some simplifying assumptions. Most of the paper was about why they thought their assumptions would not cause very much deviation from the true optimal strategy.