View Single Post
  #5  
Old 08-21-2005, 10:19 AM
AaronBrown AaronBrown is offline
Senior Member
 
Join Date: May 2005
Location: New York
Posts: 505
Default Re: Standard Deviation questions.

I think you are heading down a bad path, a guy named William Gosset figured out the right way about a hundred years ago.

You measure something by an average (like poker winrate) and you want to know how much you can trust it. So you measure standard deviation to get a confidence interval. But then you wonder how much you can trust standard deviation, so you measure standard deviation of standard deviation. Then SD of SD of SD. Of course, you never get satisfied.

Gosset proved that under certain assumptions, you can collapse all the uncertainties into a single distribution, called the Student-t (he published under the name "Student" because his employer, Guinness Beer, didn't want the brewery business associated with anything as disreputable as statistics.

With more than 30 observations, unless you're going way out into the tail (like wanting a 99.999% confidence interval for your win rate), the Student t is quite close to the Normal. With a Normal you go 1.96 standard deviations to either side of the mean for a 95% confidence interval, with a Student t with 30 observations you go 2.05 standard deviations. Since you need at least 1,000 observations to get the standard deviation small enough for useful inference, the standard deviation of standard deviation is not a problem. There are other problems like the fact that your win rate is not constant nor independent from hand to hand that are much more serious.

If you want the formula, the variance of the variance is equal to the variance squared times [2/(n-1) + kurtosis/n], where n is the number of observations. That means, loosely speaking, the error in your standard deviation, expressed as a fraction of standard deviation, is on the order of 2/(n-1) + kurtosis/n. With 1,000 observations, the first term is 2/999, or a 0.2% error. The kurtosis is a measure of how "fat" the tails of the distribution are, it's 0 for a Normal and negative for a uniform distribution. Unless you play a crazy no-limit game in which one hand a night determines the entire outcome, your kurtosis is unlikely to be big enough to affect your standard deviation much after 1,000 hands.
Reply With Quote