Calculations vs. Simulations

[SheetWise]

After reading (and responding) to the "3 door problem" (revisited yet again), I was reminded of a poker problem I encountered several years ago. I was retained to design a slot machine which was based on poker, but included (by nature) replacement and also included wild cards.

I wrote a program to calculate the frequencies, and soon after realized the simulation would take 2 months to run. I built a K7 dual 3G processor box and knocked it off in 2 days -- but I still wanted to see the calculations which matched the simulation. Being a commercial endeavor, I sent the question to Google answers to save time and money. You can read the thread at http://answers.google.com/answers/threadview?id=147900

I was astounded by both my errors and the errors made by very skilled mathematicians. Poker problems employ a different calculus at every level, I had made errors and so had they.

In the end I confirmed results both by equations and simulations. This post is offered because I find that many of the calculations posted on this site disagree with my simulations. Please don't ask for specifics. I am interested in your approach to problems, and whether you use simulations to confirm calculations.

SheetWise

[AaronBrown]

For the kind of problem you mention, both calculation and simulation should give excellent answers. Although your original simulation took 2 months to run, for any practical purpose a simulation of, say, 10 billion slot machine pulls would suffice. Even with a $100 million top prize (an unreasonable value), the error in expected value would be in the pennies, far less than the error from imperfections in the mechanism of the machine or other factors.

The nice thing about having both calculation and simulation is you tend to make different kinds of errors doing each one. In calculation, you often make an unwarranted assumption, or forget a case. In simulation, it's easy to make a programming mistake, or to have the right program but not run it long enough, or have a flaw in your random number generator.

I usually try to compute things first, because I enjoy doing that and it gives an exact answer if it's right. If it's important, I check it with a simulation. Actually, I also enjoy doing that.

Therefore, if you can get the same answer from both, you have far more confidence than in either one individually.

[SheetWise]

You describe where I inevitably land -- the area between certainty and inevitability.

While I trust the simulation much more than the calculation, I tend to make decisions based (for lack of a better lanyard) on a combination of empirical data and gut instinct.

Some Darwinian force makes me believe there must be more ...

SheetWise