Predicting BB/100 based on other stats

[AnyAce]

Great work Nate.

A couple of quick comments on the model. (I also have a lot of undergrad/statistics and econometrics coursework from way back when [img]/images/graemlins/wink.gif[/img])

You may get better results using dummy variables for the VPIP and others because the relationship might be non-linear in a way that your squared term is not catching, such as there is likely a level at which VPIP is too low for maxing BB/100, then a sweet spot, then once it reaches a certain point it's negative again. So we might expect the coefficient on a dummy for VPIP under 15 to be negative, and then coefficient on a dummy for say VPIP15 to 20 to be positive, etc. PM if you want to discuss this more.

Finally, in terms of using solver to find the optimal coefficients, the problem I've run into in the past is that Excel solver doesnt necessarily find a global maximum solution, it might only find a local max. ie. it finds an area where the solutions keep getting better, then stops tweaking the model when it doesnt get any better. It doesn't know if its reached the highest mountain peak or just the top of a little hill (once it reachs the hill top it stops). To get around this my company started using OptQuest by Crystal Ball/Decisioneering. It takes a global approach and will keep searching and does a much better job.

Good luck.
AA

[sammy_g]

Nate,

Interesting stuff.

I suspect if you included avgerage pot size in big bets, you could come up a formula that is very accurate across limits. For instance, a 2/4 player might have a lower W$WSF than a 15/30 player, but a higher win rate since the pots he wins will be bigger (relative to the limit). The formula as it is probably underestimates win rates for lower limits since those games tend to be looser.

Do you see any practical uses for this? Can we use the formula to determine optimal, or at least better, stats? It seems difficult because changing one of the numbers changes the others. For example, trying to raise W$SD will lower W$WSF. So you can't just plug in the numbers that give you the highest win rate.

Sam

[Ryno]

Nate -

Great post. I have tried to do the same thing you are doing, but every time I give up because it is seemingly impossible to isolate luck from the data.

VPIP is the only statistic that is reasonably independent of running well. I tried several methods to remove bias from the data, specifically WSD and W$SD, but could not come up with a satisfactory result.

An avenue you might try is isolating certain stats, given ranges of other stats. So, find a collection of players who have BB/100 rates in a reasonable range (from 1-3BB/100, let's say). Then, do your analysis on those players, with the goal of defining the relationships between the regressors. Like, if your VPIP is 15%, what should your WSD and W$SD be? Or, what is the optimal VPIP for someone who likes to fold a lot (i.e. low WSD), if that happens to be the way you play.

In your formula, if you go to more showdowns, you make more money, because people who went to more showdowns made more money. Maybe tracking the winrate of big pocket pairs would help remove the luck bias, since they are such a determinant of how you are running. Perhaps peter_rus has tried this.

[rory]

Very good ideas.

[Nate tha' Great]

[ QUOTE ]
Did you try running any models with pre-flop raise percentage? This should have some effect on win rate, but at 15/30 there probably isn't as much variance in pre-flop raise percentages than there is at the lower limits.

[/ QUOTE ]

PFR didn't have any marginal predictive impact on win rate once we'd already taken these other three things into account. I think the main reason is that PFR is very highly correlated with W$WSF.

[ QUOTE ]
Another question Nate. What is the R squared for this model?

[/ QUOTE ]

Around .61.

[Nate tha' Great]

[ QUOTE ]
Do you see any practical uses for this? Can we use the formula to determine optimal, or at least better, stats? It seems difficult because changing one of the numbers changes the others. For example, trying to raise W$SD will lower W$WSF. So you can't just plug in the numbers that give you the highest win rate.

[/ QUOTE ]

It *might* help us get a clue as to the "optimal" VPIP, but probably not without a significant margin of error. I don't want to oversell this; it's mostly just a "hey, that's interesting!" kind of thing.

[TommyO]

I haven't read every response yet, so maybe this has already been covered. If you look at the first to terms of your equation:

y = 0.753vpip - 0.0102(vpip^2)

This function has a max at about 35%. We know that optimal vpip =~ 20% so maybe these coefficients need to be tweaked to better fit reality?

[Worrots]

This is impressive work. I have some technical statistics oriented questions:

1) Is this OLS? I'm worred about heteroskedasticity.

Specifically with reference to VPIP and number of hands. I think we'd all agree that optimal VPIP converges to something around 20, but that this convergence occurs way above 1000 hands. I therefore worry that you might have a wider error range of BB/100 for low sample players than among high sample players at a given VPIP. Your VPIP maxes BB/100 at ~36 which seems odd. Have you formally tested this?

Perhaps a difference estimator for VPIP instead (VPIP-20)? I agree that it has to be a non-linear relationship.

(For the non-stats geeks, I'm saying that there may be lots of players with too high VPIP that are running hot over 1000 hands in the data. If unaccounted for, this can bias the results of the regression and exaggerate the explanatory power of the entire model, not just the significance of VPIP.)

2. I'm concerned about multicollinearity.

[ QUOTE ]

The relationship between W$WSF and W$SD themselves is a little bit more ambiguous ... However, for the most part they operate independnetly,

[/ QUOTE ]

This sounds odd to me. For every hand you W$SD, you must also have W$WSF. The one doesn't happen without the other -- there has to be a non-random relationship between the two for the typical player. The universe of hands winning showdown is entirely within the universe of hands winning after seeing flop -- showdown means went all the way to the river and had to show a hand, right, not winning when everyone else folds? Have you formally tested for this as well?

(For the non-stat geeks, the issue is assigning the explanatory power between the two variables, not the overall accuracy of the model. I'm arguing that the two variables are related to an extent that you might not be able tease out the impact of one independent of the other.)

Like I said, this is impressive and cool work. I'm genuinely interested in your methodology, not trying to rip apart your results.

Ah, if I was back in school and had the time... there really is enough data out there these days to do real econometric work on poker. Wonder if there are game theory phds based on partypoker published yet.

[sthief09]

he explained this a couple fo times already. none the stats are independent of one another. if you could sustain a high W$WSF and W@SD then a 35 VPIP could be ideal. on this planet in this game that's not that easy to achieve

[Greg J]

Interesting post. I have advocated doing a regression analysis to determine bb/100 several times, but never actually DID it. Kudos on actually doing it.

That being said, I have a couple of suggestions, comments and constructive criticism. I have some statistical methodological training, so I think my suggestions are valid. I do public opinion research and am currently ABD... so I'm not a dummy at this stuff (though not a statistician either).

1) I have trouble with VPIP being treated as a linear variable. <<EDIT: My mistake -- you control for this by introducing the sqrt function, but since it's not a log type relationship, I still perfer using dummies here.>> Of course the way you operationalized this equation it will be positive by including the WSF and WSD -- it would be pretty much impossible not to if you think about it! I think it would me much more useful to not include WSF and WSD and instead include a series of dummy variable representing a range of vpip scores.

2) Relatedly: including WSF and WSD is not only not theoretically useful, but a bad idea methodologically. You are essentially saying that the more you win the more you win. It's like using a country's gross domestic product to predict that country's tax base. They are intrinsically related. Winning hands make you win money. Perhaps more importantly, this will potentially suck explanatory power away from other potentially significant variables by introducing multicolinearity into the model.

Here are variables I would like to see.
- Dummy vpip <7
- Dummy vpip 7>10
- Dummy vpip 10>12
- Dummy vpip 12>15
- Dummy vpip 15>17
- Dummy vpip 17>18
- Dummy vpip 18>20
- Dummy vpip 20>22
- Dummy vpip 22>25
- Dummy vpip 25>30
- Dummy vpip 30>35
- Dummy vpip 35>40
- Dummy vpip 40>50
- Dummy vpip 50>60
- Dummy vpip 60>70
- Dummy vpip 70>85
- Dummy vpip 85>100
- Dummy pfr 0>1
- Dummy pfr 1>5
- Dummy pfr 5>6
- Dummy pfr 6>8
- Dummy pfr 8>10
- Dummy pfr 10>12
- Dummy pfr 12>15
- Dummy pfr 15>20
- Dummy pfr 20>35
- Dummy pfr 35>50
- Dummy pfr 50>75
- Dummy pfr 75>100
*************
I would also like to see similar variables for aggression broken down, as well as some exploratory analysis into interactions for a proper range of play (the right combination of vpip, pfr and aggression).