The BOT issue

[SoBeDude]

I've talked to the customer support of one of the online sites.

They said bots such as winholdem are easily detected and anyone caught using one will have their account blocked/frozen.

This sounds like a load of BS to me although I wish it were true.

Anyone know if this was just a line or do you think there is hope that bots can be blocked from the sites?

No matter how good/bad they play today, its just a matter of time before bots are too tough. So I'd rather not have to play against them.

-Scott

[daryn]

i disagree with you when you say eventually bots will becomee too tough. i just don't see how this could be.. poker is not a game of complete information like chess... i believe this thing has been talked about extensively.

bring on all bots!

[Jim Easton]

[ QUOTE ]
Anyone know if this was just a line or do you think there is hope that bots can be blocked from the sites?


[/ QUOTE ]

I don't know, but it seems to me it could be as simple as "if screen gets scraped, then xxx". We have lots of computer experts here, would it be easy to tell if the screen gets sraped?

[Wake up CALL]

"would it be easy to tell if the screen gets sraped? "

Pretty much impossible......

[OrangeHeat]

They could easily combat this by alternating card shapes and colors randomly throughout the playing sessions so they can't be scraped.

I don't see a way to detect the bot from their end though - but I am not an expert.

orange

[Adde]

I don't see a way to detect the bot from their end though - but I am not an expert.

They could easily have the client send a list of running processes (on your computer) back to the server. If they don't like you using WinHoldem.net, they could stop you from playing.

The WinHoldem guy said something about WinHoldem supporting only Win2000 and up for this reason, but I can't see how they could stop the bot from being detected, but I'm no expert on Windows programming.

[XlgJoe]

Just because poker is a game of incomplete information doesn't mean you can't develop a bot that is an expert player. It may have more info than a person, such as a database of all individuals it has played and knowing each person tendencies.

That’s just today’s technology. Bots that learn with time may become the latest breakthrough. If there is a market for an expert poker bot I wouldn’t bet against it.

On the other hand there would be programs available that would tell you the probability, based on playing style and other techniques, whether you are playing against a bot.

[daryn]

bring on all bots.

[XlgJoe]

go to propoker.com /images/graemlins/grin.gif

[Pistachio]

I have no idea why so many people are afraid of bots. Bots have just as many problems as human players and they are really not that smart (yet) and can be defeated thru observation. I doubt you will see many bots operating in higher limits although I am sure they exist. Whether a bot has a good hand or a human has a good hand I just do not see the difference. Most bots play fairly tight..so are you really going to call a raise from a tight player anyway? Probably not. The cheating winholdem bot I can see getting worked up about but not bots in general.

[ChipWrecked]

poker is not a game of complete information like chess

Considering that chess does involve complete information, and the fact that it took 'bots' decades to be able to beat chess masters, it seems that bots able to beat 'good' poker players would be nothing to fear for quite a while yet.

(Forgive me if this has been mentioned already. I skim the bot threads at best, at my limits I don't worry about bots)

[Alobar]

Wow, I just want to go on record as saying, you are all very very stupid. (heh, I bet that gets me flamed). The fact that you think your puny human intellect could match the infinately vast (in comparison) resources of a computer is mind boggling to me. Add in the fact that the information the computer has is COMPLETELY accurate, vs. what you "think" you remember an opponent did last time. The way you talk about poker is also absurd. "well uh duh, I'll just observe what the bot does, and then I will be able to beat it easily". Gosh, all you have to do is observe an opponent and then "whala!" you take all his money? Gee, I didnt know poker was that easy. What the hell am I playing 2/4 for, I should jump right into 30/60 cuz all I have to do is observe! sweet, fame and fortune here I come!!

[N4CER1]

I called and talked 2x to Party Poker today 2/12/2004 and they just upgraded their software so no one can use Winholdem on free or money games.....for now anyway...

[N4CER1]

It is not so much the Bots play...I think your right....It is that Winholdems bot has a buddy list and lets everyone on that buddy list see each others hole cards all of the time...The colluders is the big thing IMO......

[Pistachio]

Yeah..very friendly you are. All I am saying is that bots in general very predictable as they usually follow a strict set of rules. You can modify the bot to make it more difficult to figure out obviously but then the programming of a bot begins to get complicated. I am sure many bots are operating that are way ahead of mine but I am just telling you what I know from watching it play thousands of hands...and that is there is nothing to be afraid of. Poker is not an easy game and even my lowly bot has trouble. You can play a pretty good game of poker and still lose. All I meant by observation is that when a particular player raises and everytime he has the goods then you know that his raise generally means business.

All I meant about bots being defeated is that I doubt my bot could beat a good player over a long period of time. My bot is so tight post flop that it would not be very profitable to even attempt trying to beat it anyway. There are easier fish in the sea besides a bot.

[zooey]

I might be interested in a counter strike. I went to winholdem's website, and there was no price info. I might be willing to go in with fellow 2p2ers for the express purpose of counter-research:

* Verifying for which sites its currently working

* Building a statiscal description of how it plays. Tight players are relatively scarce, and have fairly unique %Raise-Bet-Fold-Call signatures. Furthermore, these #'s converge to their true values relativley quickly, within 1000 hands certainly. I suspect a winholdem player could be identified and outed rather easily once those values were known.

* habitual mistakes could be identified.

Btw, I side with those who think we don't have a lot to worry about right now. I would bet there is a long way to go, but there is substantially more $ in poker than chess, so who knows until we see it play?

I wouldn't want to waste a lot of money on it, but I might be interested in shouldering a share. I would expect a high ethical standard among co-conspirees, with the agreements of transparency of results (we share what happens) and no personal use.

And i might change my mind about all this tomorrow, but it seems like a good idea at the moment.

Best,

Zooey

[BradleyT]

That's pretty much an invasion of privacy and they would be seeing lawsuits. What I have on or running on my computer is none of their business.

[BradleyT]

Please tell me how this information is more useful to a bot than a player.

For example UTG I only play 99-AA and AKs and AQs and always come in with a raise.

The bot has QQ in late position.

The flop is A97 two suited.

Please tell me how the bot is going to use his huge database of my play to figure out what I have after I bet the flop.

[The Dude]

To block bots, you could use something similar to what TicketMaster does. They give you a word written oddly over crossing lines (virtually impossible for a computer program to read) and ask you to re-type the word.

If they required users to do this once per orbit (say, on the button), this would pretty much eliminate the problem (although it would be a bit annoying if you are trying to play more than one game at a time).

[Pistachio]

Exactly..its not going to help much at all. The other thing is that how often do you really play against the same people? There are so many people playing online poker now that I do not think having this info would really be used anyway. I would like the bot to be able to identify weak players or maniacs..now that info would be useful in shorthanded situations. Bots need to be able to adjust on the fly and that is not easy to code..this is where a human is far superior. Even this information can get you into trouble..just because a player calls garbage now does not mean he will in the future..so again the database may even be a hinderance.

[Adde]

That's pretty much an invasion of privacy and they would be seeing lawsuits. What I have on or running on my computer is none of their business.

I don't care at all about that. Poker sites have all incentives in the world (read: millions of raked dollars a day) to not abuse my privacy.

[fluff]

You obviously know nothing about programming bots.

[mosch]

This is trivial to get around. I've always assumed that if you wanted to make a truly stealthy bot, you'd hide it in kernel space. Writing it as a standard userland program seems like step one of 'how to make an easily detectable bot'. Shove it into the kernel, and suddenly there's no way for a userland program to know whether it's there or not.

These sorts of problems aren't particularly hard for a competent professional programmer. Fortunately, they're sufficiently tedious, and the number of competent programmers is sufficiently low that you won't see too many of them.

[BradleyT]

They scan joes computer and find child porn on it and turn him in to the authorities. Does that stand up in court?

They scan your computer and find MP3's and turn you in to the RIAA. They scan your brothers computer and find divx movies and turn him in to the MPAA.

Where does it stop? It stops before it starts because it's an invasion of privacy and that's why no one does it!

[Gotmilk]

I believe you meant "voila" (I'm too lazy to put the accent in), not "whala". Even if a poker bot as complete information of how someone acted in the past, it won't always know the proper play because every situation is slightly different, plus good players vary their play on patterns the computer wouldn't neccesarily be able to understand. Go get Turbo Hold'em and beat the snot out of it and you'll see...its a good program but good players beat the snot out of it.

[Alobar]

I think you could use "whala" as well. But yeah you're right "viola" works better.

I think everyone needs to make a distinction between a crappy bot and the fact that a bot at some point is going to be programmed that 98% of you cant beat. Winholdem bot may suck and texas turbo holdem bot may suck and blah blah blah, but that doesnt mean that everybot is going to suck. I can beat the sh!t out of turbo chess on my PC, does that mean I can waste deep blue? see where I'm going with this?

[Alobar]

I wasn't directing my negativity at you specifically pistachio, I just meant everyone in general. Didn't mean it to sound like I was calling just you stupid...I meant EVERYONE! /images/graemlins/smile.gif

[Alobar]

Again, like I said, I didn't know poker was that easy. You're talking like there is one specific way to play every hand and once you know how to do that your incapable of losing. If that were the case no one would make money at poker.

I don't like your example for the above reasons, but here's why a bot is going to do better than a normal human per your example. A bot is going to KNOW what hands you've played UTG and what hands you've always came in on with a raise, then its going to be able to calculate the odds of what specific holding you have vs the flop that he can beat and know if he has the odds to bet/raise/call. He does all that in .000004 seconds. You average human makes an ASSUMPTION of what hands you play UTG based on the memory of a very small (in comparison to the bot) sample of hands he can remember you playing. (pray he remembers correctly). See the advantage Mr. Bot has??

[fishhook]

Seems to me that as time passes, the computer models for weather forecasting are becoming extremely accurate. Can poker be much more unpredictable than the weather? Of course no BOT will ever be perfect, but it won't have to be. There are no perfect human poker players, are there?

[BradleyT]

No I don't see the advantage the bot has. I specifically told you the hands I would play UTG and raise with. I then raised with a hand UTG. This means I have 99-AA or AKs or AQs.

Again you offered no proof whatsoever of how a bot could possibly have an advantage over a player in this situation.

So is the bot going to fold because the hands I could be playing (AA, AKs, AQs, KK, 99) that are way ahead outnumber the hands I could be playing where I'm behind (QQ,JJ,TT)?

If so I'll bet EVERY flop after I raise UTG and destroy the bot because I know he'll fold.

This is an extremely tricky hand to play whether you're a bot or a human (the KK hand that is) and you still haven't shown any proof that a bot could outplay a human given the same cards.

[fishhook]

Bradley, you are correct that you will beat the BOT in the situation you are mentioning. But you are assuming that you are playing against a STATIC BOT. But what about a DYNAMIC BOT, one that catches on to your stategy? What about the thousands of other starting hand combo's that exist? What about a BOT that evolves with the help of human guidance just as our weather prediction models have? Are you willing to bet money every day that the weather forcast is totally wrong? Right now there are no poker BOTS that are that good. But someday there will be some that will be able to beat all but the best poker players.

[Terry]

[ QUOTE ]
it seems that bots able to beat 'good' poker players would be nothing to fear for quite a while yet.

[/ QUOTE ]

Yep. This thing doesn’t even present itself as a winning bot – you have to program in your own strategy. All they’ve done botwise is give you the interface, not the play.

What the program does do correctly is send your hole cards to their computer. Keep that in mind if you’re playing around with this thing.

Me, I feel sick about the fact that I downloaded it because now I’m not really 100% positive that I’ve gotten rid of the whole thing, even after searching the Windows registry for pieces of it.

Notice that not only is there no “uninstall” feature, but the program is also installed in an unusual place. You’d better believe that the guy(s) who wrote this knew where it would normally be installed, and, for some reason, chose to put it somewhere else.

The sites will never be able to tell if you’re running a bot, because the bot could be “named” anything ... they wouldn’t know what to search your computer for ... but that’s not a problem, since a winning bot has yet to surface and is probably quite a ways in the future.

What the sites can easily do is buy winHoldem (so they can keep up to date on what site it is using for card sharing) and keep their software updated so it knows when a user is connecting to the “sharing” site. That part is simple, and if the latest updates haven’t included it, the next one will.

This program is bad news, guys ... very bad news indeed. The best possible outcome a user can hope for is to be allowed to cash out after he is banned.

[Alobar]

You play UTG the same way every hand, and you're arguing that you can beat a bot because it will play the same way everytime.....er, ok.

Again, you're giving a specific example and then saying there is one correct way to play to play it, and since you know the correct way the bot has no advantage over you. Again I say, thats not poker. You make money in poker by making better decisions than your opponents. Yeah, in a specific situation a bot may not have any advantage what so ever against human opponents. Bot limps with 22 on the button after 4 limpers, and the flop comes A K J. Being a bot isnt going to help you in that situation. But neither is being Phil Ivey. Arew you going to say you can beat phil ivey now because you know how to play 22 in that situation just like he does? The bottom line is, a bot (a good one) is going to make way more better decisions in an hour than his opponents and have WAAAAAAY more data to base those decisions on. The bot is going to make more money per hour than you, and that money is coming from you and every other human at the table.

We can argue till the end of time wether a bot will ever be good enough to beat a world class human (I say yes), but the fact is 99% of poker players and readers of this forum, arn't world class, and lose to said bot.

[Alobar]

[ QUOTE ]
... but that’s not a problem, since a winning bot has yet to surface and is probably quite a ways in the future.


[/ QUOTE ]

Technically thats not true. There is a guy in one of the other forums who is working on a bot that currently plays on line at 2/4 and he claims its making more than 2BB/hour. Thats a winning bot. There may not be a bot out that can beat 10/20 and up yet. But then if there were such a bot, do you really think you would know about it?

[BradleyT]

Try running Active Ports to see if this thing is sending anything it shouldn't be - it doesn't.

You can also run Process Explorer from Sysinternals to see if it's running any hidden processes - it doesn't.

www.sysinternals.com (http://www.sysinternals.com) for PE
http://download.com.com/3000-2085-10121832.html?tag=lst-0-1 for AP

[BradleyT]

Ok then, please show me a single example where a bot has an advantage over a player. Any example please.

[Pistachio]

There are lots of winning bots..I know of many. A bot can win at any level but it just depends on game selection. I have no problem putting my bot in with good players at higher limits..but I will not make any money so whats the point? Also..a good player may be able to beat my bot over time..but I also know he is not going to win much..if anything in a general ring game...he will give up long before the bot will. That is one nice thing about a bot..it never gets tired, calls out of boredom, sits out to grab a beer, drain the dragon, or answer the phone. It just sits and will not stop until somebody is out of money..and hopefully that somebody is not the bot runner.

[Bubu]

The most dangerous aspect of bots is that they will take money out of the online poker world. By consistently beating the lower limit games, money will be taken out of poker ecosystem. Its like the worst rake you could imagine -only that the money is not used to sponsor freerolls. Or like throwing in a bunch of robotic sharks into the
fish breeding shallows. Bad.

Bots can beat the low limit games. From comments on this forum rudimentary ABC poker can be programmed into winholdem. In the general forum someone discussed his own bot, which could beat the 2-4 games.

Yes, maybe you as a player can beat the bot, but you don't want this bot to kill all your fish - even the microlimits minnows contribute to keeping the higher mid limits soft.


Bubu

[illunious]

[ QUOTE ]
To block bots, you could use something similar to what TicketMaster does. They give you a word written oddly over crossing lines (virtually impossible for a computer program to read) and ask you to re-type the word.

[/ QUOTE ]

I think this is a great idea, I would love to see this as an optional table choice.

[Lori]

But what about a DYNAMIC BOT, one that catches on to your stategy?

This is the Zoo, not SS.

If a player adapts to my strategy, I adapt back whether it be human or bot.

SS'ers seem to have a general feeling for playing a hand one way, and one way only, but personally if somebody catches onto my strategy, I alter it, even if I don't know it is a bot.

Even more importantly if it catches onto someone elses strategy I alter my play against it when the someone else is in the pot.

Poker is not simple and the days of super-strong bots are many years away yet.
If I live long enough to see them, I doubt I'll be too concerned by how they play, I'll just accept my huge old age as compensation.

If I do know it's a bot, it had better watch out.

Lori

[GuyOnTilt]

Since everybody always capitalizes all the letter in "BOT", I want to know what it stands for. Anybody??

GoT

[Lori]

"Beats Other Topics"

Expect BSS to be the new acronym shortly.

Lori

[BradleyT]

robot. BOT stands out more than bot does.

[GuyOnTilt]

Oooooohhh! Ro-BOT! Thank you!

GoT

[CrackerZack]

I'd attempt to enter this argument somewhat seriously but its obvious that you're not listening to anyone anyway. But I'll leave you with a tidbit that you can take anyway you want.

Chess = complete information
Poker = incomplete information

[SoBeDude]

[ QUOTE ]

I think everyone needs to make a distinction between a crappy bot and the fact that a bot at some point is going to be programmed that 98% of you cant beat. Winholdem bot may suck and texas turbo holdem bot may suck and blah blah blah, but that doesnt mean that everybot is going to suck. I can beat the sh!t out of turbo chess on my PC, does that mean I can waste deep blue? see where I'm going with this?

[/ QUOTE ]

Good analogy. But remember that the early computer chess programs did lay the groundwork for Deep Blue. Its an evolution of the program that eventually leads to a incredibly tough opponent.

Also remember that early chess programs were severaly hampered by the speed of the computers at the time. This is no longer the case and won't hamper poker bot development.

You will see a poker bot go from OK to super tough very very quickly if the effort is put in to its development.

One of the best ways to impede its development is to remove the places it can play for real money (the online sites).

Without a justification and financial reward, the development will slow.

-Scott

[SoBeDude]

no one is saying the bot has to win every hand.

but if he can put you on a narrow range, that sure dictates how to play against you well.

with QQ he could revert in to check-call mode, or he could raise to see where you're at and fold to a reraise.

A bot can live quite well on probabilities

this isn't tough to program.

-Scott

[SoBeDude]

bradley. you're naive and confused.

and you clearly know nothing about programming, and only a very little about poker.

[SoBeDude]

scraping a screen is not detectable from the other end.

'screen scraping' is a colorful analogy for simply looking at the incoming data stream.

The Heisenburg uncertainty principle only applies to subatomic particles

-Scott

[Tater10]

Let's take a more simple game than hold'em. 1 card poker, suits count (spades, hearts, clubs, diamonds.) So, As can't lose (the nuts) and 2d can't win. Heads up. It is in 'limit' format - you can check/bet/fold/raise 1 unit back and forth.

You can easily assign probabilites after the deal (example, you get the Ah, and figure you are a 51/52 favorite). Now you, the computer programmer, has to come up with a winning BOT. How would you do that? Could you program the computer to beat yourself in the long run? Even if you could program it to learn your behavior, you could always mix it up by feeding it bad data (call with As, raise with 2d, fold with Ah.)

I would say that any human could adapt to any code and beat it in the long run.

I think on-line bots win based on the fact of bad play by players, mostly found at the lower limit tables.

[TimM]

[ QUOTE ]
That's pretty much an invasion of privacy and they would be seeing lawsuits. What I have on or running on my computer is none of their business.

[/ QUOTE ]

They will get around this by burying some line in the terms and conditions that says you agree to this if you use their site/software.

[daryn]

actually 2 /images/graemlins/diamond.gif beats 2 /images/graemlins/club.gif

[fishhook]

Lori,
I am new to this Zoo, but I don't know what you mean by SS. Maybe Sunday School????????????

I am not a great programmer, but if I was designing a dynamic BOT, I would program in a little randomness, so you would never know exactly what to expect or when to expect the unexpected. Just like when you play against a human. There are so many ways to play any given circumstance that a program could mix up it's play enough too keep you confused.

[daryn]

right, and any human can keep you confused too.. that is the point! no bot will be super-human at this game.

[t_perkin]

Ok I have a little experience in this area and I am certain that I have done more reading on this subject that 99% of people posting here:

First I will address a few points on screen scraping and "bot prevention" :

Screen Scraping is the process of taking data from the screen (i.e. individual pixel RGB values) it is sometimes extended to encompass the act of taking text from an application window - this can often be done without having to reconstruct words from individual pixels.

Screen scraping is a hard, laborious and error prone technique which is very rarely required - the analogy is breaking into a house by drilling through the wall... there is almost always an easier way.

I created a screen scraping application to scrape data from Ladbrokes Poker. It was a very very boring piece of coding. Then they changed their interface and the screen scraper was useless.
In hindsight an easier way to access the data of the poker room would be to "packet sniff" (read the data that is being sent to the poker software). The packets are of course encrypted but you have the key (your password).

A few simple precautions could prevent the poker room from knowing if you are screen scraping or packet sniffing.

Using a "hidden word" system such as used at Yahoo etc. will work, but it requires considerable additional bandwidth (the word must be sent as an image) and using a bot does not necessarily mean that someone cannot be sat there to type the hidden word in.

-------------------

The rest of this post consists of an extract from my Degree dissertation. It is out of date already as it does not mention SparBot and VexBot the two heads up players produced by the GAMES group.

SparBot is professional to world class heads up player, which is capable of beating professional players *despite doing no player modelling whatsoever* (i.e. it does not adapt). It simply plays a near-optimal game. (note this is an optimal game *not* a perfect game)

You can read more about it and play against a slightly weaker version of it at:

http://games.cs.ualberta.ca/webgames/poker/

VexBot I do not know a lot about at the moment, but it is a highly adaptive player which attempts to play by modelling itself on its opponent. (The quote I heard was: "This is a very frustrating bot to play against - you have to ask yourself: How would I beat me?" )

Ok here is the literature review from my dissertation (which was titled: Collection of Real World Data for Poker Utilising a Standard Notation" )


Sorry about the formatting...
----------------------


2 Related work

Research into game playing can broadly be split into two halves; games of perfect information and games of imperfect information. Games of perfect information can be characterised by the simple fact that all players have equal and complete knowledge of the state of play of the game. Games of imperfect information are characterised by the opposite, players in the game do not have perfect information about the state of play in the game, and often this is also extended to each player holding different pieces of information.

2.1 Games of perfect information

At the beginning of AI research almost all effort was concentrated on games of perfect information. Games of perfect information are commonly easier to model, due to the generally smaller search space. For imperfect information games, all possible values of an unknown variable are possible often leading to a very large branching factor.
The most prominent games studied in the perfect information arena are those of Chess, Checkers and Go. Chess and Checkers are now considered “solved” in the sense that an automated player exists that is considered the best player in the world. The well chronicled Deep Blue [ ] series of chess players beat Kasparov in 1997 to become World Champion. The less well known but earlier triumph of Chinook [2], Schaeffer’s checkers player took the World Championship in 1994.
Both of these players are often criticised when labelled as “triumphs” of AI. Both programs rely heavily on domain knowledge and large databases for openings and endgames. Some see their approach as not so much a triumph of AI but a triumph of computer processing power. None however can dispute the awesome ability of these players at the task they perform.
A much “purer” AI solution to the game of checkers is offered by Anaconda a.k.a. Blondie 24 [ ]. This player uses pure AI techniques to compete. With the exception of legal moves, it has no domain knowledge. Anaconda uses a neural network which is evolved through an evolutionary strategy using self-play experiments. The only input is the board position. The output of the neural network is used in a mini-max search. This is a vastly different to the approach of Schaeffer and Campbell, and can be considered a strong indication of the potential of AI techniques in game playing. Anaconda is not yet the best player in the world [ ], it is unable to beat Chinook although it can occasionally win.
Hex [ , ] was invented by P. Hein in 1942, and, independently by John Nash in 1948. Originally called Polygon, it was marketed in 1952 using the name Hex. It was introduced to world through Martin Gardener’s column in Scientific American [ ]. The idea is to take it in turns to place black and white pieces on the board, with the aim of the player using black to connect opposite sides of the board with a chain of black pieces. White has the same objective. The best computer program (HEXY), when using a 10x10 board, cannot yet compete with the best human players but it is coming close.
After Chess the game which has attracted most attention in the AI community is Go. This is one of the sole remaining major hurdles for perfect-information AI research. There are no automated players capable of playing at much more than a good amateur level of play.
The first Go player to beat an absolute beginner was in 1969 written by Zoborist. Since then vast amounts of research time has been devoted to creating a good Go player, no-one has yet succeeded. Probably the primary reason for this is that traditional techniques for solving 2 player games such as chess and checkers are not directly applicable to Go. Evaluating a given board position is expensive. Variations in single stones can cause dramatically different outcomes; human professionals can spot these patterns. Training a computer to do so has so far been elusive. The number of positions to consider is also very large, the branching factor is higher for Go than for chess. The search space alone however cannot justify the difficulty of creating a good player, no solution has been found for 9x9 Go which has a smaller search space than Chess [ ].
The best automated Go players can play in some aspects of their game to a medium standard, however once a player identifies the weaknesses of the automated player they can quickly exploit them to leave the automated player as little more than a beginner. So Go remains a major challenge, and one that appears to be defying the traditional approaches of AI game playing.


2.2 Stochastic games

Stochastic games are games which have a factor which has a probable outcome, for example with the roll of dice, spin of a wheel etc. These games do not have any hidden information, but they do require probabilistic analysis in order to play. A co-evolutionary approach has been applied to Backgammon [ ] where games play against themselves to evolve their own evaluation function. The evaluation function, in a similar way to Fogel for Checkers [5], is represented by a neural network but, instead of using an evolutionary strategy to adapt the weights. Tesauro utilises temporal difference learning [ ], adding some hand-designed features to the neural network and a shallow search, Tesauro reports that “…a level of play is reached which by all indications is beyond current human capabilities.” [8].

2.3 Games of imperfect information

Games of imperfect information are characterised by the fact that players do not have complete information about the game state. This generally requires a very different strategy when constructing an opponent. The branching factor is generally very large due all possible situations having to be considered. In addition most imperfect information games involve some element of misinformation or bluffing, which must also be factored in. For these reasons until recently games of imperfect information have not been studied in as great a depth as perfect information games. Some success has however been achieved.
Brian Sheppard’s MAVEN is now considered to be the best scrabble player in the world [ , ].

I have outright claimed in communication with the cream of humanity that MAVEN should be moved from the ‘championship calibre’ class to the ‘abandon hope’ class, and challenged anyone who disagrees with me to come out and play. No takers so far, but maybe one brave human will yet venture forth.”
Brain Sheppard

Ginsberg’s GIB bridge player [ ] also plays to a very high standard, and will probably be the best player in the world within the decade.

In the realm of Poker though, world class play is still elusive. Competitive world class Poker play would be a major success for the AI community. Poker combines a plethora of aspects of game playing; multi player action, bluffing, statistical analysis, player modelling and searching. It is also the one game at which a direct monetary incentive can be found. A world class poker player can make millions of dollars a year. All this from a game in which a player can only take one of three actions, fold, call or raise.

2.4 Early Work into Poker

Some of the earliest work done in the field of machine learning with poker was carried out by von Neumann in 1944 [ ]. Using a simplified version of poker he applied principles of game theory to study poker, exploring the issue of bluffing while playing poker. Work was also carried out in a similar vein by John Nash [ ]. Following this Findler [ ] made the first concerted effort to study poker, working for 20 years on a simplified version of 5 card draw. This version of poker contained no ante or blinds and was played between only two players. Findler approached the problem from a social sciences background, concentrating on reproducing the thought processes of humans rather than creating a world class player. For example the players he created bet in direct proportion to the strength of their hand, a strategy that would be far too revealing in a proper game of poker. Future work in poker would have to use very different techniques, based on a far more mathematical and dynamic framework in order to create even a competent player.
Korb et al [ ] created a Bayesian network to model the game of Stud-Poker, dismissing Texas Holdem as the use of shared board cards meant that hands were being evaluated relative to an average hand. They concluded that this approach could outperform a relatively simple statistical player and a simple rule based player.
Koller and Pfeffer [ , ] created the Gala system, a way of specifying and solving games of imperfect information, however due to the size of the game trees produced by even their simplified variants of poker they state:
“Full-scale poker is much larger… and it is unlikely that we will be able to solve it”
As such this technique can be considered too abstract for the purposes of this project.

2.5 The Evolutionary Hypercube

While and Barone [ , ] have attempted to evolve a poker player to play a simplified version of Texas Holdem poker. They construct a “hypercube” which is a multidimensional structure for storing candidate solutions for a number of preset situations. The current system only has dimensions covering varying position, risk management (pot odds) and opponent type.
For a given situation a candidate solution is picked. The result (win or lose) is fed directly back to the candidate. Evolution of the population occurs when all candidates have been used a fixed number of times.
An opponent-type dimension has been added more recently [17], they report that this added dimension leads to an improved performance over the generic opponent model. However this approach has probably led to a brittle evolution. While and Barone have evolved their player against Findler’s 4 player types (loose aggressive, loose passive, tight aggressive, tight passive). This may well have resulted in the player evolving 4 particular strategies against the 4 opponent types.
This is probably a limiting factor of the player; its current evolution against only 4 types almost certainly means that it struggles against players who cannot easily be pigeon-holed into these vague categories.
In addition Findler's early static players are by all accounts very weak and predictable; they were never designed to be worthy adversaries and are not a very plausible benchmark. The hyper cube’s evolution is also far too slow to evolve to a player in a single session of maybe only 50-200 hands, ruling out evolving to a more diverse set of human players.
In addition the feedback for evolution is only whether the hand was won or lost, not how much was won or lost on the hand. This is a serious limiting factor, the player will never evolve desirable strategies such as check-raising which are used to maximise winnings rather than to win the hand. “The aim of playing poker is to win as much money as possible, not win as many hands as possible.” [ ]

2.6 Poki

The most sophisticated and successful machine player is undoubtedly Poki [ , ]. Poki plays the full version of Texas Holdem poker, the game used to decide the annual World Series Of Poker (WSOP). Poki is the product of the GAMES group headed by Jonathan Schaeffer, the creator of Chinook the world class checkers player. The GAMES group have moved away from games of perfect information such as Chess and Checkers, concentrating now on imperfect information games, seeing it as the major challenge of the game playing AI community.

Poki is fundamentally based around a probability triple {Pfold, Pcall, Praise}. Poki must generate this triple whenever it is its turn to act [ , ].



Pre-flop Poki effectively uses an expert system to decide whether to fold call or raise. This is based on a table of rates of return for all possible hole cards, the bet size, the number of players who have acted and the position.

Post-flop Poki takes the pocket cards and the board and evaluates the probability that it currently holds the best hand (Hand Strength); this is based on a weight table for each opponent which represents the likelihood of them holding any pocket.
It then calculates the Ppot and Npot for the hand. This is the probability that the hand is currently not the best but will improve to become the best (Ppot) and the probability that the hand is currently the best but will be superseded and lose in a showdown (Npot).
By combining the Hand Strength, Ppot and Npot an Effective Hand Strength is generated which represents Poki’s decision on whether to play or not. There is then a simulation phase [22, ] in which Poki plays out a number of scenarios where it either calls or raises and attempts to predict how the opponents will respond. This is done to establish the decision which will give maximum return.

The opponent model [21, ] consists of two parts. A weight table which represents the probability of an opponent holding a given hand, this is based on the actions of the opponent (if he bets he is more likely to have A-A than 7-2) and is used to calculate the effective hand strength. The second component is an Action Predictor which is generated from opponent’s historical actions and the board cards to make a prediction of whether they will fold, call or raise on their next turn. This is used in the simulation to generate a maximal winning strategy.

This is a simplification of the Poki architecture, but gives an overview of the way in which Poki works (fig1). Poki is a pretty solid player and has gradually improved over the last 5 years, however there are some areas where there are obvious improvements to be made.
Poki does not take into account its position (how many players play before it and after it) when deciding how to act post flop. Poki indirectly takes this into account via the simulation, but the benefits of playing late are large and Poki probably needs to incorporate this much more deeply into the simulation such as by returning the probable accuracy of the simulation result as well as the result itself. Neither does Poki make any use of one key piece of information. This is the revelation of card values at showdown. This is also information that a computer player can probably make more use of than a human player. Card values can be worked back through the previous actions of the hand to note where the predictions of the system were incorrect, and incorporate these in the player model. As the player model is probably the component of Poki where the greatest improvements can be found, this is an obvious opportunity for improvement

In [20] Billings et al comment at length on the issues of developing Poki against various types of opponents. Much of the development of Poki has been in self play experiments, although attempts are made to vary the style of its opposing “self” through varying parameter settings, all the players are essentially “thinking” in the same way. “The major drawback of self play experiments is that they lack the wide variety of styles and game conditions exhibited by real players.” … “The problem is much more acute and limiting [than for chess] for the development of a poker playing system”
Poki also competes extensively on IRC Poker, although these games are played seriously and competitively they are not for real money. Although the objective ability of a few of the IRC players is probably very high, Poki is not experiencing the full range of playing styles that would be encountered in real money games.
Billings also comments on the possibility of playing at real-money online poker rooms considering it a “completely realistic setting”. Billings does note that this is not a controlled environment and as such it would be hard to obtain concrete evidence in performance improvement. As a benchmark however, this would be an ideal venue for testing Poki against real players for real money.

---------------------
and some references
---------------------

[1] V. Anshelevich, A Hierarchial Approach to Computer Hex. In Chips Challenging Champions: Games, Computers and Artificial Intelligence. (Ed. J. Schaeffer and J. van den Herik) pp. 141-160, Elsevier, 2002.
[2] L. Barone and L. While. Evolving Computer Opponents to Play a Game of Simplified Poker. Proceedings of IEEE International Conference on Computational Intelligence (ICEC'98), 108-113. 1998.
[3] L. Barone and L. While. An Adaptive Learning Model for Simplified Poker Using Evolutionary Algorithms. Proceedings of Congress of Evolutionary Computation (CEC'99), 153-160. 1999.
[4] L. Barone and L. While. Adaptive Learning for Poker. Proceedings of enetic and Evolutionary Computation Conference 2000 (GECCO'00), 566-573. 2000.
[5] D. Billings. Computer Poker. 1995. University of Alberta.
[6] D. Billings, A. Davidson, J. Schaeffer and D. Szafron, The Challenge of Poker. Artificial Intelligence 134, 201-240 (2002).
[7] D. Billings, A. Davidson, J. Schaeffer and D. Szafron, The Challenge of Poker. In Chips Challenging Champions: Games, Computers and Artificial Intelligence. (Ed. J. Schaeffer and J. van den Herik) pp. 243-282, Elsevier, 2002.
[8] D. Billings, D. Papp, L. Pena, J. Schaeffer and D. Szafron. Using Selective-sampling simulations in poker. Proceedings AAAI spring symposium on search techniques for problem solving under uncertainity and incomplete information, 13-18. 1999.
[9] D. Billings, D. Papp, J. Schaeffer and D. Szafron. Opponent modeling in poker. Proceedings of AAAI-98, 493-499. 1998.
[10] D. Billings, D. Papp, J. Schaeffer and D. Szafron. Poker as a Testbed for Machine Intelligence Research. Proceedings of AI'98, (Canadian Society for Computational Studies in Intelligence). 1998.
[11] D. Billings, L. Pena, J. Schaeffer and D. Szafron. Using probabalistic knowledge and simulation to play poker. Proceedings of AAAI-99, 697-703. 1999.
[12] B. Bouzy and T. Cazenave, Computer Go: An AI oriented survey. Artificial Intelligence 132, 39-103 (2001).
[13] C. Browne, Hex Strategy: Making the Right Connections, A.K. Peters, Natick, MA, 2000.
[14] M. Buro, The Othello Match of the Year. International Computer-Chess Association Journal 20, 189-193 (1997).
[15] M. Buro, Improving Hurisic Min-max Search by Supervised Learning. In Chips Challenging Champions: Games, Computers and Artificial Intelligence. (Ed. J. Schaeffer and J. van den Herik) pp. 125-139, Elsevier, 2002.
[16] M. Campbell, A. J. Hoane and F. Hsu, Deep Blue. Artificial Intelligence 134, 57-83 (2002).
[17] K. Chellapilla and D. B. Fogel, Evolution, Neural network, Games, and Intelligence. Proceedings of the IEEE 87, 1471-1496 (1999).
[18] K. Chellapilla and D. B. Fogel, Evolving Neural network to Play Checkers without Relying on Expert Knowledge. IEEE Transactions on Neural Networks 10, 1382-1391 (1999).
[19] K. Chellapilla and D. B. Fogel. Anaconda Defeats Hoyle 6-0: A Case Study Competing an Evolved Checkers Program Against Commercially Available Software. Proceedings of the 2000 Congress on Evolutionary Computation, 857-863. 2000. IEEE Press.
[20] F. A. Dahl. A Reinforcment Learning Algorithm Applied to Simplified Two-Player Texas Hold'em Poker. De Raedt, L. and Flach, P. Proceedings of ECML, 85-96. 2001. LNAI.
[21] F. A. Dahl, The Lagging Anchor Algorithm. Reinforcement Learning in Two-player Zer-sum games with Imperfect Information. Machine Learning To appear, (2003).
[22] A. Davidson, D. Billings, J. Schaeffer and D. Szafron. Improved opponent modeling in poker. Proceedings of the International Conference on Artificial Intelligence (IC-AI'2000), 1467-1473. 2000.
[23] N. De Firmian, Modern Chess Openings, Times Books, 1999.
[24] N. Findler, Computer Model of Gambling and Bluffing. IRE Transactions on Electronic Computers EC-10, 5-6 (1961).
[25] N. Findler, Computer Model of Gambling and Bluffing. In Computer Games Vol 1. (Ed. D. Levy) pp. 409-412, Springer-Verlag, 1961.
[26] N. Findler, Some New Approaches to Machine Learning. In Computer Games Vol. 1. (Ed. D. Levy) pp. 304-324, Springer-Verlag, 1969.
[27] N. Findler, Some New Approaches to Machine Learning. IEEE Transactions on Systems Sciences and Cybernetics SSC-5, 173-182 (1969).
[28] N. Findler, Studeis in Machine Cognition Using the Game of Poker. In Computer Games Vol. 1. (Ed. D. Levy) pp. 430-460, Springer-Verlag, 1977.
[29] N. Findler, Studies in Machine Cognition using the Game of Poker. Communications of the ACM 20, 230-245 (1977).
[30] N. Findler. Compter Poker. Scientific American , 112-119. 1978.
[31] N. Findler, Aspects of Computer Learning. Cybernetic Systems 11 1-2, 67-86 (1980).
[32] N. Findler, H. Klein, W. Gould, A. Kowal and J. Menig, Studies on Decision Making Using the Game of Poker. In Computer Games Vol. 1. (Ed. D. Levy) pp. 413-429, Springer-Verlag, 1972.
[33] N. Findler, H. Klein, W. Gould and J. Menig. Studies on Decision Making Using the Game of Poker. Proceedings of IFIP Congres 1971, 1448-1459. 1972.
[34] N. Findler, H. Klein, R. C. Johnson, A. Kowal, Z. Levine and J. Menig. Heuristic Programmers and their Gambling Machines. Proceedings of the ACM National Conference, 28-37. 1974.
[35] N. Findler, H. Klein and Z. Levine. Experiments with Inductive Discovery Processes Leading to Heuristics in a Poker Program. Proceedings of Conferene on Cognitive Systems, 257-266. 1973.
[36] D. B. Fogel, Blondie24: Playing at the Edge of AI, Morgan Kaufmann, 2002.
[37] M. Gardner, The Scientific American Book of Mathematical Puzzles and Diversions, Simon and Schuster, 1959.
[38] M. Gardner, The Second Scientific American Book of Mathematical Puzzles and Diversions, Simon and Schuster, 1961.
[39] M. Ginsberg. GIB: Steps towards an expert-level bridge-playing program. Proceedings of IJCAI-99, 584-589. 1999.
[40] M. Ginsberg, GIB: Imperfect information in a computationally challenging game. Journal of Artificial Intelligence Research 14, 303-358 (2001).
[41] F. Hsu, Behind Deep Blue: Building the Computer That Defeated the World Chess Champion, Princeton University Press, 2002.
[42] D. Koller and A. Pfeffer, Representations and solutions for game-theoretic problems. Artificial Intelligence 94, 167-215 (1997).
[43] K. Korb, A. Nicholson and N. Jitnah. Bayesian poker. Proceedings of the 15th Conference on Uncertainity in Artificial Intelligence, 343-350. 1999.
[44] M. Muller, Computer Go. Artificial Intelligence 134, 145-179 (2002).
[45] J. F. Nash, Non-cooperative games. Ann Math 54, 286-295 (1951).
[46] J. F. Nash and L. S. Shapley, A simple three-person poker game. Contributions to the Theory of Games 1, 105-116 (1950).
[47] D. Papp. Dealing with imperfect information in poker. 1998. The University of Alberta, Edmonton.
[48] L. Pena. Probabilities and simulations in poker. 1999. The University of Alberta, Edmonton.
[49] L. Russ, The Complete Mancala Games Book, Marlowe & Company, New York, 2000.
[50] J. Schaeffer, One Jump Ahead: The Story of Chinook, the World Champion Checkers Computer Program, Springer-Verlag, 1997.
[51] J. Schaeffer, The Games Computers (and People) Play. Advances in Computers 50, 189-266 (2000).
[52] J. Schaeffer, D. Billings, L. Pena and D. Szafron. Learning to Play Strong Poker. Proceedings of the Sixteenth International Conference on Machine Learning. 1999.
[53] J. Schaeffer, D. Billings, L. Pena and D. Szafron, Learning to Play Strong Poker. In Machines that learn to play games. (Ed. J. Furnkrank and M. Kubat) Nova Science Publishers, Huntington, NY, 2002.
[54] J. Schaeffer, R. Lake, P. Lu and M. Bryant, Chinook: The World Man-Machine Checkers Champion. AI Magazine 17, 21-29 (1996).
[55] C. Shannon, Programming a Computer for Playing Chess. Philosophical Magazine 41, 256-275 (1950).
[56] D. Sklansky, The theory of poker, Two Plus Two Publishing, 1992.
[57] D. Sklansky and M. Malmuth, Hold'em Poker For Advanced Players, Two Plus Two Publishing, 1999.
[58] S. Sutton and A. Barto, Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning), MIT Press, 1998.
[59] G. Tesauro, Temporal difference learning and TD-gammon. Communications of the ACM 38, 58-68 (1995).
[60] G. Tesauro, Programming backgammon using self-teaching neural nets. In Chips Challenging Champions: Games, Computers and Artificial Intelligence. (Ed. J. Schaeffer and J. van den Herik) pp. 223-241, Elsevier, 2002.
[61] G. Tesauro, Programming backgammon using self-teaching neural nets. Artificial Intelligence 134, 181-199 (2002).
[62] J. von Neumann and O. Morgenstern, The theory of games, Princetown University Press, NJ, 1947.

[Mike Haven]

"A few simple precautions could prevent the poker room from knowing if you are screen scraping or packet sniffing.

Using a "hidden word" system such as used at Yahoo etc. will work, but it requires considerable additional bandwidth (the word must be sent as an image) and using a bot does not necessarily mean that someone cannot be sat there to type the hidden word in."

Should this read: "A few simple precautions could enable the poker room to know if you are screen scraping or packet sniffing"?

[bigpooch]

Actually, you are giving a game where an optimal strategy
can be computed: it's a heads up finite zero sum game and
therefore there exists at least one optimal strategy. It's
such a simple game that an optimal strategy (or a very good
approximation to one) can be easily determined and a bot
playing in such a manner will beat almost any human player
(any nonoptimal strategy would simply lose in the long run).

On the other hand, if there are more opponents (say it's ten
handed), then I think an expert player (who adjusts well
according to the opposition) will do better simply because
humans aren't very good at coming close to an optimal
strategy and some humans are quite awful. It might even
be true that the bot will be a winner but doubtful that he
would be as big a winner as the expert.

In any reasonably complex game, I do agree that the expert
would tend to do much better than a bot simply because he
makes the proper adjustments based on the mistakes that his
opponents are making.

[Vehn]

Out of curiousity I had a programmer and poker playing friend of mine look into decrypting and packet sniffing one of the big 4's data stream.

He was able to successfully obtain all the needed information (what was dealt, what the board is, what the other player's actions are) within a few days without any tedious screen scraping.

[SinCityGuy]

[ QUOTE ]
Technically thats not true. There is a guy in one of the other forums who is working on a bot that currently plays on line at 2/4 and he claims its making more than 2BB/hour. Thats a winning bot. There may not be a bot out that can beat 10/20 and up yet. But then if there were such a bot, do you really think you would know about it?

[/ QUOTE ]

If his BOT is beating 2/4 for 2 BB's per hour, then he's gold. All he has to do is to get a wireless router at home and set up 3 computers to play 24 hours per day (remember, this thing never gets tired). He could easily make six figures annually be doing this.

[t_perkin]

exactly, screen scraping is not necessary. Packet sniffing is a much easier and more robust technique.

[t_perkin]

nope...

The only thing that an application can readily do to spot a packet sniffer/screen scraper is look for applications that it recognises as packet sniffers and shut itself down if they are running.

To prevent this you can:

create a packet sniffer which gives itself a random process name and renames itself on each run. (then create a statically named batch file which holds the dynamic .exe name)

Sniff the packets on another PC.

Packet sniffers tend to be very invisible...after all their only real use is to listen into somebody elses business.

Tim

[jedi]

Plenty of examples, but none of them game related:

1) A bot doesn't need to eat
2) A bot doesn't need to sleep
3) A bot doesn't need to use the crapper
4) A bot won't care if the wife whines about taking out the trash
5) A bot won't go on tilt

Of course, none of these stand up to the major disadvantage a bot has, the ability to play poker.

[ QUOTE ]
Ok then, please show me a single example where a bot has an advantage over a player. Any example please.

[/ QUOTE ]

[BradleyT]

Ok but how is this outplaying a human who is holding QQ?

[Terry]

t_perkin,

There are occasional claims to have a winning bot, but we only ever have the poster’s word for it. You appear to be very up to date on the state of the art.

In your opinion, is it likely that there are now any winning poker bots playing online?

If so, at what level, and how rapidly would you expect them to progress to higher levels?

[Mac]

I appreciate the research you've done in this area, but everytime the BOT question comes up, you talk about the difficulties in screenscraping and your work as a prof. The simple fact is, this process is not difficult at all, can be done by a solid coder within hours - forget the packet sniffing and all this other crap. I don't doubt your talent, but your approach to the problem is what got you in trouble here - I make the same types of mistakes in my coding all the time.

- MAC

[BradleyT]

[ QUOTE ]

but if he can put you on a narrow range, that sure dictates how to play against you well.


[/ QUOTE ]

How can the bot put me on a tighter range of hands than a player can? They both have to play against me N number of hands before they can make any conclusion as to what I'm holding. Using poker tracker and player notes, the human player can put me on the same range of hands that the bot can.

[ QUOTE ]
or he could raise to see where you're at and fold to a reraise.

[/ QUOTE ]

Major flaw? If I find out that's true then I re-raise every time.

[BradleyT]

ACTools (used for cheating at video games) has a decent "screen scraping" capability built in to it. Basically you set up an object like so:
Position screen offset X,Y
[0,0] colorA
[0,1] colorB
[0,2] colorA
[0,3] colorB
[0,4] colorC
[1,0] colorD
[2,0] colorE
[3,0] colorA
[4,0] colorA

You then tell it to look at offset X,Y and start matching. The pixel at [0,0] needs to match the RGB value of colorA (you also set a "difference threshold" to account for how different video cards displays colors differently). The pixel at [0,1] needs to match colorB and so on and so forth.

So basically you set up a grid N,N and define the color of each pixel within that grid and then do a comparison to see if the colors are a match.

You'd need 13 objects to match cards and 4 objects to match suits and you'd have a fast comparison routine.

ACTools is written using Delphi and can be found at http://www.cameroncole.net/actool.html The Object Mapper is the important thing here as it will define the objects pixel offsets and colors for you when you dump a screenshot into it.

[DcifrThs]

WOAH buddy, ... RELAX lol

the problems bots have (even exceptional ones like the university of alberta POKI bot-whose small bets per hour won chart looks like the uphill marathon against almost all opponents) is that there is not an exact correct weight given to the NEW information relative to the OLD information. THATS where the POKI bot of UofA is at its best...

it somehow has something that allows it to VARY the weight of new info depending on certain factors.

the fact is, an expert can determine that the once tight good player just got his aa cracked and is now steaming so his raises mean MUCH less than they would in a normal setting SOOOO 3betting with aj or aq or 99 or 88, where once unthinkable, becomes correct and good poker.

bots don't know human emotions but i guarantee you after a few showdowns the bot will adjust and start raising with more hands. this will last until the bot sees the raise % and specific hands shown down of that player return to previous levels.

in any case, bots are good players against a mediocre field and there are some bots designed specifically for expert heads up play (see Abdul Jalib's post about a bot that beat HIM heads up- it ended up three betting him with nothing after being checkraised on the turn by a better hand)

just my thoughts...questions? comments?
-Barron

[driller]

From the complaints I read on this forum, the suckouts and non-rational play would put a BOT on tilt.

[thwang99]

It's fine if they warn before they do this. For example in the agreement you click "yes" to before downloading and before installing the software, etc.

It is invasion of privacy, but if it's to prevent bots, then sure, let them do this, I say. - Tony

[t_perkin]

[ QUOTE ]

I appreciate the research you've done in this area, but everytime the BOT question comes up, you talk about the difficulties in screenscraping and your work as a prof. The simple fact is, this process is not difficult at all, can be done by a solid coder within hours - forget the packet sniffing and all this other crap. I don't doubt your talent, but your approach to the problem is what got you in trouble here - I make the same types of mistakes in my coding all the time.

- MAC


[/ QUOTE ]

I don't doubt your abilities as a coder either.

I am not quite sure what you are saying. Are you saying that screen scraping is the wrong approach? that packet sniffing is the wrong approach? or that I am just not making a very good job of it and that that is why I am saying it is hard?

There is no way that you or anyone else can write a screen scraper "within hours". I would be willing to stick my kneck out and say that it would take a very good hacker at least a week or two, full time, to write a screen scraper to be able to interact with a poker room in all its functionality (navigate lobby etc.)

However I am more than willing to e-mail my screen scraper to anyone who wishes to have a look at it. It is now worthless as the interface it scrapes from no longer exists.
It is in no way a beautiful piece of coding - but it is solid and it could function at the room for around 80-100hours before it would encounter something unexpected.

More to the point, if you can knock one up in a couple of hours I will happily pay you to do so at whatever the going rate for expert hacker is.


Tim

[Alobar]

[ QUOTE ]

How can the bot put me on a tighter range of hands than a player can? They both have to play against me N number of hands before they can make any conclusion as to what I'm holding. Using poker tracker and player notes, the human player can put me on the same range of hands that the bot can.



[/ QUOTE ]

So your telling me that you play me N number of hands and you remeber EVERY hand I played against you? Here's the difference between you and a bot. I'm playing you and I raise UTG, you fumble around in poker tracker to try and see what I raised with UTG before, and you use your feeble memory to try and remember what I played UTG for a raise before, and while you're doing that the screen is beeping at you because your time is running out. The BOT remembers EVERY single hand I raised UTG and knows it in like a milli second. Sounds like a BIG advantage to me.

You keep talking about how is a bot going to play QQ better than you? You're missing the entire point. A bot isnt going to win every single hand. All a bot is going to do (programed correctly) is make better decisions than you are. That adds up to more money, and advantage bot.

[Alobar]

[ QUOTE ]
I'd attempt to enter this argument somewhat seriously but its obvious that you're not listening to anyone anyway. But I'll leave you with a tidbit that you can take anyway you want.

Chess = complete information
Poker = incomplete information

[/ QUOTE ]

I'm not listening to anyone? neet, I didn't know that.

my chess vs. poker analogy was to point out that just because there is a poker bot that sucks, doesnt mean they are all going to suck. Not that the same type of coding goes into each one.

Just because poker is a game of incomplete information doesnt mean humans are somehow superior at it. I would argue that a game of incomplete information would give the advantage to a computer. How do you play a game of incomplete information against another player? You use information gained from previous experience and apply it to the present situation. Who has more experience, a computer that can recal every hand its ever been in against a specific opponent, or the human who can't?

[Pistachio]

The screen scaping part of the bot code is easy..it only took about 2 hours. If any of the casinos change the cards its only about an hours worth of work to update it...not a big deal.

[Pistachio]

Tim,

You are way off base here. The interface between the bot and the poker room is BY FAR the easiest part of bot coding. That portion of the code only took a day or two at most. I am sure other bot runners will attest to this fact. How the bot reacts to different players and situations is the really difficult task. I have been working on this part of my bot for the last month and its still not that great.

[t_perkin]

Ok I am now in guess territory - please don't flame me. Someone asked for my opinion so I am giving it, if you disagree I am happy to hear it. But please don't say I am an arrogant know-it-all. This is all just my own opinion I am sure some or all of it is wrong.

the GAMES group is a group of extremely well regarded academics. These guys are the brightest and the best. Just look at the list of games that they hve defeated on their website, these guys are no ameuteurs.
They have 6 or more people working full time on this for 10 years. I will stick my kneck out and say that their offerings Poki, SparBot and VexBot are the best there is.

And how good are they?

This is what they say:
[ QUOTE ]

Q: How good is Poki?

A: The older version of Poki that plays in full 10-player games is better than a typical low-limit casino player, and wins consistently against average opponents; but it is not as good as most expert players. The newer programs being developed for the 2-player game are quite a bit better, and we believe they will eventually surpass all human players, perhaps within a few years, or less.

Q: Does Poki play on any of the online poker servers?

A: No. Poki may participate in online games in the future (including real-money games), but the program will always be clearly identified, so players will always have a choice of whether they want to play in a game with Poki or not.



[/ QUOTE ]

These guys are respected academics, as such I am willing to believe them when they say that they do not play on commercial poker rooms. You may not believe them, that is up to you.

----

So what about anyone else?

I would say that there isn't anyone else with a player capable of winning above 1/2 - and I would be surprised if they could win at that.
I would be very surprised if anyone had a winning NL player.
I would be extremely surprised if anyone had a winning tournament or SnG player, especially NL.

I would be extremely surprised if anyone used a winning heads up player on the internet - although someone may have created one. I say this because the opportunity to exploit a heads up player that you can beat is very high, so if a good HU player spotted the bot they could annihilate them.

Why do I think this? Because of the amount of work that the GAMES group of put in and how far they have got.

----

When will there be winning bots?

Ok pretty much pure speculation:

My guess is within the next 5 years - plus or minus 10 years /images/graemlins/smile.gif


Within 10 years a computer will be the best player in the world

---

So should we all run and hide from the killer bots?

Most will agree that this is the "golden age" of poker. Typified by there being more games and more fish than ever before.
You will hear a lot of "pre golden age" players say that just because you are a winner now does not mean you will be a winner when the bubble bursts. This sounds valid to me.
So whether there are bots (winning or not) is irrelevant, games at every level are easier than they have ever been before and probably than they ever will be again. So if you want to win at poker now is the time to play. Bots or not.

Tim

[t_perkin]

I am not saying that writing the interface is the hardest bit to code! It is not in any way shape or form.

All i said is that it is boring and error prone.

I don't believe that you can write a screen scraper that can sit indefinately and give a complete machine interface fot an entire website in two days.
Writing one that will pick up the values of the cards and bets - yeah sure. One that can navigate the site and keep an accurate representation of the current game state at all times is not so straight forward - although still not a *major* piece of work.

All I said was that I thought that packet sniffing is a more robust way to interface with the poker room.

Tim

[Terry]

Thank you.

[BookOfIcculus]

[ QUOTE ]

My guess is within the next 5 years - plus or minus 10 years /images/graemlins/smile.gif

Within 10 years a computer will be the best player in the world



[/ QUOTE ]
Sounds like this would be good for www.longbets.org. (http://www.longbets.org.) Even if not it makes for an interesting site to look at.
I'd be willing to put 100 on a BOT beatting the best players in the world 75% of the time within 10 years.

[mbpoker]

How do you decrypt the SSL stream?

[t_perkin]

You have the key for the encrpytion - your password.

It is probably not a very trivial process, but niether would it be incredibly difficult, maybe a days work for someone who knows their stuff (not me - I don't know anything above the basic theory of modern cryptography, none of the practicalities)

[PrayingMantis]

[ QUOTE ]
In any reasonably complex game, I do agree that the expert would tend to do much better than a bot simply because he makes the proper adjustments based on the mistakes that his opponents are making.


[/ QUOTE ]

I don't understand why, in the near or far future, a bot woudn't be able to do the same, i.e., adjust to the mistakes of his opponents. It seems very achievable. You only need to express your opponents mistakes in whatever mathematical term, which is absolutely possible, and your bot will play optimal (or close to it) game against them.

Not *one* post, in this thread or others related to this topic, convinced me that a bot will never have the ability to play poker as the best human expert, if not better.

[DarkKnight]

I'm surprised by the people who welcome the bots. The simple fact is that bots do not need to be better than the best players to be dangerous. They only need to be able to play +EV in order to take equity away from you.

Every spot at a table taken up by a +EV player reduces the value of playing at the table. It's like playing at a table where you're the best player but everyone else is only slightly worse - your edge is so small that you need to find a new table.

The bots will eat the fish just like we eat the fish. That's the problem. More competition will destroy our edge.

[Bubu]

This _is_ the crux of the issue. I posted something along this line as well, but nobody seems to recognize this problem. I don't understand why people care so much whether a machine can beat them or not.

Bubu

[BradleyT]

It's because most of them think you'll be able to plop a bot down at a seat and it will be able to play 72 hours non stop.

Playing logic notwithstanding here are some features that have to be considered.

Table gets down to 6 handed. Does botty adjust?
Table gets down to heads up. Does botty adjust?
Table goes empty. Does botty get up and find a new table?
Site/and or Internet connection goes down for 10 minutes which results in you having to manually rejoin any tables. Does botty know how to do this?
Tables at party can fill so fast that sometimes when you double click one, all the tables have shifted and you end up clicking on a different stakes game. Does botty detect this?

There's 100 possible problems with the interface alone that could arise, these are just a few off the top of my head.

[DarkKnight]

These are all relatively trivial

[mbpoker]

Nope. The encryption is very unlikely to use your password.

[t_perkin]

hmmm. Yes thinking about it, you are probably right.

Packet sniffing may not be so straight forward...Someone here claims they have done it...I would be interested to know how.

Tim

[frizzfreeling]

Alobar, you might as well be talking to a brick wall. These people will continue to refuse to see whats coming until it smacks them in the face. It's all about ego.

[heyrocker]

[ QUOTE ]
In hindsight an easier way to access the data of the poker room would be to "packet sniff" (read the data that is being sent to the poker software). The packets are of course encrypted but you have the key (your password).


[/ QUOTE ]

While the data is in fact encrypted the key is not your password. Think about this, how would a compiled executable which you download from the internet know what your password is in order to decrypt the data? The decryption key is coded into the app and is the same for all the apps, and if we're all lucky its a totally random series of characters that nobody will ever figure out (although I can't imagine any site being stupid enough to send data over that line that would help anyone play better, encrypted or not.)

[BigEndian]

From what I understand, the trick to getting at the key is that it's relatively easy to follow the data in memory since the networking calls are written in a windows DLL. When you watch the data, you can watch what is being done to the data.

Also, a good number of encrytion algorithms are publically available.

I don't know what poker rooms are using for their security and I hope they have a key schedule, but it's a classic problem in encryption: when the "enemy" has to have the key resident on their system, staying secure is an uphill battle.

[edit]heyrocker makes an important point: in regards to specific hand information - there is absolutely no need for the site to send this information to all clients until there is a show-down.

The big concern about security should be your login and account information. And even then, the would-be hacker needs to be in a position on the network to intercept your data.

- Jim

[spike]

You're right theencryption key is very unlikely to be your password. But everything else is wrong... how would the client know your password? Hmmm.... you type it in when you log in? The client knows your password, the server knows your password... why wouldn't they use it? Well... because it's way too short.

But neither is the password a hard-coded long string of funny characters. Do some research on public key encryption.

[spike]

At least one large online card room uses the freely available SSLEAY module for encryption. This would be fine if they linked it as a static module, but they call it from a DLL. It's a simple matter to spoof the DLL entry points and capture the data stream before it's encrypted and after it's decrypted.

[heyrocker]

[ QUOTE ]
From what I understand, the trick to getting at the key is that it's relatively easy to follow the data in memory since the networking calls are written in a windows DLL. When you watch the data, you can watch what is being done to the data.

[/ QUOTE ]

This is an interesting angle, I hadn't thought to spoof the entry points.

[ QUOTE ]
heyrocker makes an important point: in regards to specific hand information - there is absolutely no need for the site to send this information to all clients until there is a show-down.

[/ QUOTE ]

I would go so far as to say that if I ever discovered that any site was sending information about an opponent's hands before the showdown I would instantly withdraw from that site and publicly reveal them to be the morons they are. I don't care if your data is encrypted or not, there's no excuse for that kind of idiocy.

[RiverMise]

I think that bots can beat even the most skilled humans... the way a good bot works is not just an algorithm (series of steps) but rather an adaptive, learning bot that changes with every hand and new piece of data. It would be impossible for one to "figure out" the bot and tailor play accordingly because the bot would be able to "detect" this and change in response. Computers are better number crunchers than humans and also better at detecting patterns (in betting etc.)

The bot issue is a real threat and there is little their software can do to catch a decent bot. Many of the bots are graphics-based analyzing the screen whenever an action changes, it is not a hard application to build for an experienced programmer. Thier software looks for trends in betting and ensures that you cannot "plug in" to their interface but any sophisticated bot will get around this.

-RiverMise