Sample size needed to estimate SD?

[randomfish]

Search didn't yield anything, although I'm sure this is a regular topic. Apologies if I'm blind and there's been a lengthy discussion on this already.

Anyway; consensus seems to be you need at least 100k hands to estimate your win rate -- but how many hands are needed for standard deviation? Volatility converges much faster, right?

I'm asking because after 4k hands, my SD playing nano-limit Hold'em is a staggering 25+ BB/100, and has been hovering around that figure ever since I made the switch from NL.

Is SD mostly a function of your own playing style? Or table texture? Something else?

[jason1990]

I have 3K hands in nano-limits and I'm at a SD of 21.3 BB/100. I was surprised. But I think it's accurate (based on some crude mental calculations on the bus). Sometime soon I intend to formally go through the analysis needed to answer your question.

The idea is easy. We use SD to determine how accurate our winrate is. So we need the SD of the SD to determine how accurate our SD is. But how do we get the accuracy of the SD of the SD? Well, we use crude methods such as boundedness arguments. But that should be fine since, as you pointed, volatility should converge much faster.

Now, assuming the SD is accurate, why are ours so high? Is this normal for nanos? I hope someone else will have the answer.

[pfkaok]

FWIW, my SD at both 2-4 and 3-6 stayed pretty constant after 10k hands. Don't know if thats how it normally work though, but it certainly does converge much faster than win rate. I'd like to see what amount of hands it takes to be, say 95% sure that your SD is within a BB, or something like that.

From waht I've read:
SD goes up if you're more aggressive
SD goes up if you're looser
SD goes up if your opponents are more aggressive
SD goes up if your opponents are looser

I'm not exactly sure which factors are more important than others though.

My data shows that my SD at 2-4 is 1.5-2 BB/100 larger than my 3-6 SD.


All I can attempt to conclude from my data, is that the increased looseness in 2-4 players has a greater increase on SD than the increased aggro level at 3-6... I'm curious though if others with experience at both levels have similar differences in SD's.

[pfkaok]

BTW, my stuff was for limit, but for NL i think 20 BB/100 is pretty low for nano's (PT terms meaning BB is 2x big blind), as you have tons of bad players. If you're talking about Big blins/100 then that's increadibly low I think... YOu should be able to make 10BB/100 (PT terms) at least at nano's, and if your WR is half of your SD, or more, thast pretty damn good IMO.

[gaming_mouse]

A good discussion of variance is here:

http://games.cs.ualberta.ca/poker/IRC/mm-h1-var.txt

You can estimate your SD in between 5 and 10K hands. But a baseline average for limit holdem SD is 6 SB/hand, so you should expect yours to be somewhere in that range.

HTH,
gm

[pfkaok]

SOme good info on there, but their WR's are not usual at all... .5 SB per hand?!? Some quick math shows that would be 25 BB/100!!!
I think we all know that's a little crazy.

Their SD's seem really high to me too... Mine is about 14 BB/100 , which would be a little under 3 SB per hand. I think that the reason that theirs is so high though is b/c their WR is so unsustainable, that they're obviously having a lot of fluctuations from it from hand to hand. I doubt they're earning a "steady" .5 SB per hand

[pfkaok]

also, FWIW, my SD from NL 50 at party was 25 BB ($50) / 100 for about 30k hands that I have on my PT database. It was right about 3x my WR, so I guess at nano's you could probably have it 2x or less if you're playing well.

[Sarge85]

[ QUOTE ]

But a baseline average for limit holdem SD is 6 SB/hand, so you should expect yours to be somewhere in that range.
HTH,
gm

[/ QUOTE ]

This seems WAY WAY to low. But I'm not a math guy, so I can't refute it, other than it just seems low compared to what I've read.

Sarge/images/graemlins/diamond.gif

[uuDevil]

[ QUOTE ]
[ QUOTE ]

But a baseline average for limit holdem SD is 6 SB/hand, so you should expect yours to be somewhere in that range.
HTH,
gm

[/ QUOTE ]

This seems WAY WAY to low. But I'm not a math guy, so I can't refute it, other than it just seems low compared to what I've read.

Sarge/images/graemlins/diamond.gif

[/ QUOTE ]

I agree w/ pfkaok that it's way too HIGH. Note that the number he has given is "per hand", not "per 100 hands." I think 6 SB/hand would be 30 BB/100. A more typical number for online limit HE is probably ~15 BB/100. 18 BB/100 is probably common. 20 BB/100 seems high.

Of course, this still doesn't answer the original question....

[jason1990]

I've played about 3000 hands of .02/.04. My SD is 21.3BB/100. I often wonder if this is accurate. It seems high, but then again, it's .02/.04!

The original question can be answered easily if you make certain assumptions; namely, if you assume that the net result after each block of 100 hands is Gaussian and the SD is computed using the net results of 100 hand blocks. But, intuitively, one would expect that if you had per hand data, then you could do much better by not using 100 hand blocks. But if you do that, the Gaussian assumption is way off.

One could get some sort of estimate without the Gaussian assumption by using Chebychev's inequality, provided one has a reasonably accurate estimate of the fourth moment. But it seems very difficult to get such an estimate. One possible way to get an estimate (that I haven't tried yet) is to break up the data into pieces. Only about 1/8 of my hands result in a net win or loss of more than 2BB. So the result of a single hand could be viewed as follows: flip a biased coin with probability of heads 1/8. If it's heads, generate the "large" random variable. If it's tails, generate the "small" one. Presumably a good fourth moment estimate of the "small" one is possible; and a bad fourth moment estimate on the "large" one will not be as damaging since the "large" one only occurs 1/8 of the time.

Also, we could change the 2BB cut-off to anything we want. The probability that the coin comes up heads would be unknown parameter p. We could estimate this value p over 3000 hands quite accurately, provide p is not too small and not too large. (Which would constrian the cut-off values we could use.)

Anyway, this is just an idea I intend to work on in the next few weeks. If anyone has any comments or suggestions, I'd love to hear them.

[uuDevil]

[ QUOTE ]
...if you had per hand data, then you could do much better by not using 100 hand blocks.

[/ QUOTE ]
If you have PokerTracker and Access, this is not hard to get. For instance:

Single Hand Distribution (http://forumserver.twoplustwo.com/showflat.php?Cat=&Board=probability&Number=1274385 &Forum=f11&Words=&Searchpage=0&Limit=25&Main=12743 85&Search=true&where=bodysub&Name=3962&daterange=1 &newerval=1&newertype=m&olderval=&oldertype=&bodyp rev=#Post1274385)

Unfortunately I've been too lazy to put this to any use yet....

I wouldn't be surprised if nanolimits players have much larger SD's than higher limit players because of the generally increased looseness and wild play. (One of my 2 sessions of 1c/2c resulted in a 67 BB win. I've never come close to that at higher limits.) SD should be maximum for play money players.

[jason1990]

I saw your histogram earlier. I have an Excel macro that extracts per hand results from hand histories, so I haven't bothered with PT. This is the data I plan to use in the coming weeks. But I also have been too lazy/busy to do anything with it. If we ever get off our butts, we should collaborate.

[uuDevil]

[ QUOTE ]
...we should collaborate.

[/ QUOTE ]
I doubt I could do more than kibitz a little bit. I'm just goofing around and don't have a strong background for this kind of thing. But I would be interested if you post any intermediate results.

[Emmitt2222]

OK, this math stuff=confusing suckiness. I have played 27.5 hours which is a little over 1,000 hands playing one table at Pacific and have made almost exactly $50 at .5/1. Obviously the sample size is WAY too small but thats not my concern here. My BB/hour is 1.8ish and my SD came out to be about 11. All I want to know is what the heck does this freakin number mean in english. This may have already even been said in this thread but I am really having a hard time coming to grips with what this SD number means in real life in terms of my winrate and my small sample size. Any help would be great because I am completely lost.

[uuDevil]

[ QUOTE ]
....my SD came out to be about 11. All I want to know is what the heck does this freakin number mean in english.

[/ QUOTE ]
In English, Mason Malmuth likes to say that SD is a measure of short term luck. Does that do it for you?

Sometimes you win. Sometimes you lose. Sometimes a little, sometimes a lot. SD is a measure of the range of your results. A big SD might mean you have big winning sessions but also big losing sessions. A small SD might mean you win consistently win small, without big swings.

Your numbers seem typical for a good, winning player. But with the small sample size you can't be too (statistically) confident.

[Emmitt2222]

Thanks Devil, these big numbers just freak me out and that was simple enough to understand, now when I have an actual good sample size I will know what to make of that crazy SD number

[randomfish]

[ QUOTE ]
(One of my 2 sessions of 1c/2c resulted in a 67 BB win. I've never come close to that at higher limits.)

[/ QUOTE ]

I recently won 115BB in a single hour's worth of play at 5c/10c. Kinda ridiculous.