Mathematical models

[Aisthesis]

I've been messing around lately trying to develop a very explicit model of NLHE (simply because that's the game I play) as a stochastic process and just wondered if anyone knew of anything already out there along these lines.

The way I develop this "logic of NLHE," I do get to a precise general formula (developed recursively) for value--or at least that's what I'm working on right now. But the formula is pretty cumbersome even with recursive assumptions. So, it would definitely be nice to simplify.

Anyhow, does anyone know of anything like this having already been done? With all of the computer simulations, game theory, popularity of the HE and such, it would kind of surprize me if no one has attempted it, and it's not really THAT hard (although not THAT easy either)...

[sk_man]

Have you tried citeseer yet? I just did a search on "Texas Hold'em" and came up with a lot of game theory and AI papers.

If you wanted to post your model, I'd be interested to read it. If not, I understand. Writing things up takes forever.

-dave

[Aisthesis]

Hey, thx! I'll check those out and see if there's anything similar.

Actually, I'd also be more than happy to post the model, but it's already 35 pages single-spaced in a Word file including LOTS of functions--and, in working on it, I just found another bug.

The short version is simply this: I define a certain partially ordered set D, which represents what I call card-situations. These are conjunctions of cards and betting actions and tells. The set is partially ordered because from one situation, there are various possible successors to it in terms of the rules of the game. So, I set up a bunch of axioms (I have 32 of them) according to which the set must "behave" in terms of various functions--which return values for the rake, for holdings, for the flop, etc., etc.

Anyhow, (and here's the trick to it) I then introduce these functions, which I call psychological state functions or PSFs, which give the probabilities that a player takes a certain course of action in card-situation d. You could also call them "strategies," and they are to some extent, but I feel like the PSF idea is a little more general and suggests also any kinds of uncertainty (as well as randomized strategies), feelings of confidence, feelings of fear, etc. into the probabilities of taking various actions. So, a PSF, while it gives a probability function specifying probabilities for all allowable actions at d, has a lot of different factors going into it.

The first actual substantive result (no idea what becomes "provable" on the basis of this model--maybe only trivialities, maybe some interesting stuff) would then be a general definition of value for all possible card-situations and all possible PSFs. Obviously, a PSF in conjunction with a card-situation only has value relative to the other players' PSFs.

Anyhow, that's the basic idea, but it's not at all easy getting to an explicit (recursive) definition of value, which begins, so to speak, at the end, namely when the pot is distributed either by default (all but one fold) or at showdown.

You might want to check out the University of Alberta games group.

[Leonardo]

To start with, try to build a program that just plays the river well. To do so, you would simply need a set of hands that the other player can have, and a set of actions that the player will take with each hand. Work out the optimal play. Dont worry about past betting. You can add a funcionality later that takes into consideration past betting and updates possible holdings. Leave that till last. Once you have the optimal play for the river you can move backwards to preflop quite easily. Then you have the problem or working out a) What the player has and b) what they will do with each holding. This is terribly difficult. A way of doing that would be to define 10 or 15 profiles and then try to fit each player to a profile from his play. For making the profiles you can either go through and define every play (stupid way and very long and difficult) or you could work out the optimal play that your opponent should make in theory and then slant the play towards certain tendencies, such as being too aggressive or dumb or whatever.

[Aisthesis]

Yeah, that's what kept coming up on that link the first guy gave. It looks like they're the closest, although they're doing it more from an AI standpoint--which has lots of overlaps with what I'm trying, but not entirely the same. But there are definitely some interesting articles along similar lines.
Without having a website of my own, do you know of any place I could get it out on the web once I have gotten at least all of the bugs that I myself see out of it?

I'd like to get some good critique, but the "interest group" is definitely going to be restricted to people who have had at least something like a solid formal logic course in college and are willing to wade through a lot of formalisms for the sake of poker analysis. Keeping the latter under control is really the biggest problem (I think I'm doing moderately well at it, but wish I could simplify some more)...

I've lurked here for a while. As you can see I registered to respond to this post. Anyway, it seems like you have some interesting ideas. My questions are what exactly are you trying to do? When you talk about deriving the values of any given situation from the model, it seems like what you'll want is some kind of game theoretic solution that gives you the optimal course of action for any given situation and then the expected value for the optimal action, but I want to make sure I understand what you're trying to do.

If this is what you're trying to do, then I think you want to do this in a game theory framework. Look for perfect Bayesian equilibria and then (since you'll never be on the equilibrium path) try to use the same logic to figure out what happens off the equilibrium path, and then calculate the expected outcomes of those.

From what you've done so far, I have the following thoughts:
1. 32 axioms is an awful lot. Too many axioms is usually a sign of a bad model. Sacrifice some accuracy for simplicity and clarity. A model that is a bit inaccurate is better than one that is more accurate but too complicated to solve.
2. Use the standard game theory technique. Work out optimal play at the river given any possible beliefs about what your opponents hold. This is difficult, but not THAT difficult relative to the problem. Once you have that, start working out the value of reaching the river in any given situation. After that, start thinking about turn play given what we know about the river. Then you'll have some basis for starting to think about how people form their beliefs on the river. Eventually, you'll start to figure out the relationship between turn play and river beliefs and using the combination of the ability for turn play to influence river beliefs and the river outcomes given beliefs you'll have a model of turn and river play. Now we repeat this process and get back all the way to preflop.

This really is a pretty difficult problem. My other suggestion is to start with limit play because the information is more limited which makes belief formation a lot easier to model. The problem with NL is that bets are essentially continuous which means you'll probably need fairly complicated functions to explain belief formations (or you can reduce the way we look at bets to discrete values (ie underbet, overbet, etc) but then you sacrifice some clarity since where we put the borders are arbitrary). I hope this is some help, if you work further on this I'd be interested in seeing how it develops.

[Aisthesis]

Hmmmm... that's sort of what I'm trying to do, but in a sense less ambitious. Let's say we're sitting at a table with 9 players. Each has a PSF given. Now, given the PSFs of everyone else, you could certainly come up with a set of optimal PSFs for you (there will no doubt be a lot of co-optimal solutions).

So, I guess that will be part of the result... "in principle." The way the model is set up, you can actually calculate the value of your PSF given the other PSFs even before any cards are dealt. Actually defining the set of co-optimal PSFs would presumably be simply impossible given the variety of ways in which the game can go. It will be possible, though, to define the set of co-optimal PSFs at a given card-situation d later on in the hand--i.e., when there's just not as much to figure.

As to the set of axioms, I agree that I don't like having so many, but my real goal here is completeness. I want to provide an accurate description of the whole thing. As a result, a lot of the "axioms" are kind of trivialities corresponding to rules of the game. For example, there's a relationship "Next" between card-situations (this is the relationship which makes the set D of card-situations partially ordered).

A few sample axioms:

If Next (d1, d2), then tau(d1) = tau(d2) where the function tau returns the number of players at the table.

If Next(d1, d2), then beta(i, d1) <= beta(i, d2) for 0 < i < tau(d1) + 1 and where the beta function returns the total amount of money put into the pot by the player in the i-th position in card-situation d.

Anyhow, there are lots of functions and relationships necessary to give a complete picture of all the relevant aspects, and my goal here, for the moment, is completeness.

Also, with this view, one can at least CONSIDER various criteria for "optimal"--e.g., positive but sub-optimal value but lower swings for a given PSF (standard deviation).

Anyhow, your point is well-taken regarding the complexity. It will be too complicated to solve in the general sense. But I think it will allow for some theorems--just not THE optimal PSF given other PSFs. It will be more along the lines that "this PSF is better than another PSF given the other PSFs at the table."

One thing I will say, though: At least the way I'm doing it, I don't think NL is much more complicated than limit. Perhaps in some way easier. The NL aspect is essentially captured in another axiom (there are probably a few, but I'd have to check):

For all d and for all i with 1 <= i <= tau(d), then beta(i, d) <= sigma(i, d) where sigma returns the initial stack-size of the player in the i-th position.

At least on the basis of the model I'm using, I think it would require more axioms to define allowable bets in a limit game. Also the unlimited number of re-raises allowed makes it easier (at least in terms of axioms) rather than more difficult.

So, anyhow, while I think simplifications can definitely lead to greater ease of analysis of quite a lot of situations (where the more complex model can make it very difficult to get anywhere), I guess all I'm really claiming is that a complete model seems to allow proving SOME things and should have SOME value as a tool for analysis--how far it actually gets us is completely unclear to me, though.

[Aisthesis]

Here's a link to a first draft. In the meantime, I've finished my definition of value, and hopefully that will be available on the web soon. This gets most of the machinery needed out of the way, but doesn't have the definition of value yet. The last part is still really in the form of "work in progress" unfortunately.

http://www.bet-the-pot.com/assets/do...TheoryNLHE.doc

Too bad I'm only a 17 year old high school senior, so most of the stuff I read in just the first 3 pages I don't understand yet, mostly because I haven't learned it (will I be seeing any of this in AP Calculus AB? [img]/images/graemlins/wink.gif[/img] ). However, it's all very interesting, as I (try to) take a very mathematically based approach to the game mixed with some good intuition (kind of like those "situations" you described), so naturally I would like to get to know what's going on in your theory. Anyway, it obviously looks like you know what your doing so keep up the good work, maybe I'll be able to use it some day.