How many hands do you need to be comfortable with the data?

[naphand]

[ QUOTE ]
You can do virtually nothing based on 50 hands of pure stats with no specific hand reads.

[/ QUOTE ]

Yes and no.

The reliability of the figures can be mathemtically defined, indeed there was a post to this effect just this week.

A player who plays 90% of his hands over a samples size of 50 is likely very different to a player who has played <10. Certain player types converge into more or less well defined "groups". Some converge faster than others. I posted on this many moons ago but lack of interest killed the project I was undertaking to define these groups (it is akin, of course, to the auto-rate "feature" of PT but is not so obvious as the settings indicate - there are overlaps and sub-categories).

Players diverge into a few general categories first, and these probably can be recognised in as few as 100 hands. These general groups then sub-divide after a significantly larger sample size. I am not sure of what the requirements are but probably between 500-1,000 hands. I believe at least one other poster has requested PokerTracker Pat to incorporate a feature that allows classification based on hand numbers (and what a great feature that would be).

The problem is, unfortunately, rather complex as it is much harder to define player characteristics statistically than by observation. Much more work required in this area, and more comprehensive tools.

[Derek in NYC]

What you are describing is essentially a standard multiple regression analysis. In market research, they do this stuff all the time to define customer segments. I have often thought that sophisticated statistical analyses could yield much richer profiles than the standard PFR, AF, VPIP profiles we currently use.

[marand]

Stats like VPIP and PFR can be trusted rather quickly.
If you have 100 hands of someone and he has a VPIP of 50%, then you can be pretty sure he is playing way too loose and having him in your game is profitable.

If you have statistics of 10000 hands you still really cannot use his winrate to determine if he is good or not.

When it comes to winrate you need to consider your standard deviation / 100 hands and what confidence interval you want.

Lets assume that your std dev is somewhere between 15-20 BB/100 for a 6-handed limit game. Your std dev will depend on several factors like how loose/aggressive you are and how your opponents play. Playing in a loose and crazy game will give you a higher std dev. Playing against rocks (not very common) will give you a fairly low std dev.

In Poker Tracker you can find it at the "Session Notes" tab under "more detail". Mine is only 14.9 BB/100 so I guess that means that I am "very tight" (21% VPIP 15% PFR).

If you want to be 90% sure about your winrate and have a std dev of 15, the following is what you can say:

10k hands, your true winrate is 90% sure to be within +- 2.47 BB/100 from your observed value.
20k, +- 1.74
30k, +- 1.42
40k, +- 1.23
50k, +- 1.10
100k,+- 0.78
200k,+- 0.55
500k,+- 0.35

So you can see that it takes a VERY long time before you can say be pretty sure (90%) about your winrate.


A higher std dev will of course make your winrate even less "sure".
A std dev of 20 with 90% confidence will give you the following:

10k, +- 3.29
20k, +- 2.33
30k, +- 1.90
40k, +- 1.65
50k, +- 1.47
100k,+- 1.04
200k,+- 0.74
500k,+- 0.47


So with my "shot" at 10/20 SH with 13459 hands, winrate of + 0.43 BB/100 and std dev of 14.9 BB/100 I can say:
50% chance that my true winrate is between -0.44 and +1.30
75% chance between -1.05 and +1.91
90% chance between -1.68 and +2.54
95% chance between -2.09 and +2.95
99% chance between -2.88 and +3.74 yes, like I needed statistics to know this [img]/images/graemlins/smile.gif[/img]

So with only 13.5k hands I really can't say much about my winrate.

[naphand]

I am assume you are referring to the graph presented on p.47 "Aggregate Hand Values in EV/BB".

The sample size is for one player over 60,000 hands.

Ed also states that the graph would be different for different players, and the sample size is "not nearly large enough to determine the win rate of any one particular hand".

This analysis works as starting hands have clearly delineated relative values that over-ride player skill, position etc.

Trying to use a similar analysis for a single player or hand is unlikely to work. You are not measuring relative values, but absolute values (or at least attempting to). It would work if you analysed a single hand in each position (as I posted a long time ago after running some TTHE SIMS) as each position has relative value and as such, trends can be expected to appear fairly quickly.

The margins involved in HE are quite small, and WR in BB/100 is swamped by the Standard Deviation. The huge complexity of possible hands and plays contributes to the need for large sample sizes. I am not sure how "directional analysis" can be used for determining WR unless we factor into account such things as WR in the various positions, or with specific hands. Neither of these escapes the need for substantial sample size.

We can, as was said, ascribe a probability that after x hands if player A has a WR of y BB/100 he is more or less likely to be a player of z BB/100 ability. This is just a function of the Standard Error surely? and does not provide any conclusive proof of WR.

Poker Grapher offers to plot "converging WR". Perhaps this is more what people are looking for? Trends can be observed that become progessively more stable after a few as 5-10K hands. The trouble is that although the initial curve of the graph looks very convincing (in that it begins to approach an average), this is more a function of the WR averaging from a few hands to many and does not necessarily reflect an approach to a "true mean" value. This graph becomes more complex after 10K hands and begins to resemble some kind of maggoty path, at least on my stats... [img]/images/graemlins/mad.gif[/img]

[naphand]

We think alike. I just do not have the know-how to do it. I would love to see some really serious analysis but we need an objective that can be applied to a collective database, at least initally so we can then refine player classes etc.

[Derek in NYC]

I dont have the knowledge to do it either. In a different life before my current gig, we used to hire quantitative market research firms like Lieberman Worldwide to do these studies for us. The results were absolutely fascinating, but it cost about 50grand per study.

[flair1239]

[ QUOTE ]
[ QUOTE ]

Watching someone play 5 hands is worth more than 50 hands of stats and probably a close with 200 hands of stats.

To answer the rest of your questions: plausible confidence intervals


[/ QUOTE ]

Definitely not. What if he's dealt 84o five times in a row?

[/ QUOTE ]

MAson said somewhere, that sometimes watching somebody for 20 hands is enough to tell if they are a winning player.

For instance what about if one of those times with 84o he called a UTG raise cold and then called down to the river with a pair of 4s on a AK4TJ board. What assumptions would you make about this player now?

You would probably think he was either advertising or really bad. And probably lean heavily towards really bad.

[ihardlyknowher]

[ QUOTE ]
Stats like VPIP and PFR can be trusted rather quickly.
If you have 100 hands of someone and he has a VPIP of 50%, then you can be pretty sure he is playing way too loose and having him in your game is profitable.

If you have statistics of 10000 hands you still really cannot use his winrate to determine if he is good or not.

When it comes to winrate you need to consider your standard deviation / 100 hands and what confidence interval you want.

Lets assume that your std dev is somewhere between 15-20 BB/100 for a 6-handed limit game. Your std dev will depend on several factors like how loose/aggressive you are and how your opponents play. Playing in a loose and crazy game will give you a higher std dev. Playing against rocks (not very common) will give you a fairly low std dev.

In Poker Tracker you can find it at the "Session Notes" tab under "more detail". Mine is only 14.9 BB/100 so I guess that means that I am "very tight" (21% VPIP 15% PFR).

If you want to be 90% sure about your winrate and have a std dev of 15, the following is what you can say:

10k hands, your true winrate is 90% sure to be within +- 2.47 BB/100 from your observed value.
20k, +- 1.74
30k, +- 1.42
40k, +- 1.23
50k, +- 1.10
100k,+- 0.78
200k,+- 0.55
500k,+- 0.35

So you can see that it takes a VERY long time before you can say be pretty sure (90%) about your winrate.


A higher std dev will of course make your winrate even less "sure".
A std dev of 20 with 90% confidence will give you the following:

10k, +- 3.29
20k, +- 2.33
30k, +- 1.90
40k, +- 1.65
50k, +- 1.47
100k,+- 1.04
200k,+- 0.74
500k,+- 0.47


So with my "shot" at 10/20 SH with 13459 hands, winrate of + 0.43 BB/100 and std dev of 14.9 BB/100 I can say:
50% chance that my true winrate is between -0.44 and +1.30
75% chance between -1.05 and +1.91
90% chance between -1.68 and +2.54
95% chance between -2.09 and +2.95
99% chance between -2.88 and +3.74 yes, like I needed statistics to know this [img]/images/graemlins/smile.gif[/img]

So with only 13.5k hands I really can't say much about my winrate.

[/ QUOTE ]

It should be noted that all of these calculations assume that WR is normally distributed, which it is most likely not.

[Slimmah]

[ QUOTE ]

It should be noted that all of these calculations assume that WR is normally distributed, which it is most likely not.

[/ QUOTE ]

I would bet quite a bit that WR cannot be normally distributed. I'm working on a better model, but it will take more data, time, and possibly smarts than I have.

[ihardlyknowher]

[ QUOTE ]
[ QUOTE ]

It should be noted that all of these calculations assume that WR is normally distributed, which it is most likely not.

[/ QUOTE ]

I would bet quite a bit that WR cannot be normally distributed. I'm working on a better model, but it will take more data, time, and possibly smarts than I have.

[/ QUOTE ]

My gut is that is skewed to the right (especially in games where there tend to be 3+ players to the flop) and that is is leptokurtic (since an experts WR is ~0.03BB/hand and VPIP ~20%, 80% of the hands will have a WR of 0BB; thus, a very large number of hands are concentrated very close to the mean).