Pokernomics

[jim leach]

[ QUOTE ]
I guess my objection is that Levitt seems to think he can just go in to things, learn some basics and make conclusions based on data. Unfortunately, sometimes more knowledge is needed in order to understand what the data is trying to tell you. Levitt should really hire Ed Miller or King Yao, who would both probably be interested and flattered.

[/ QUOTE ]

First, this thread about how Levitt's unqualified because he knows nothing about poker is based entirely on speculation. You do not know whether Levitt is or is not prepared, and the degree to which he has the knowledge needed to undertake this kind of study. This "theory" that he's unqualified is based entirely on speculation. The best anyone has managed to show as evidence is King's points about the NFL study, but even there, all I can sense is that he is disagreeing - not with methods or data - but with his specific interpretations about the volatility of the market. I'm not trying to be whiny here, but this point is welltaken, but is irrelevant until there is something substantive to which to respond. It seems like this is a cause for some skepticism, at best, but nothing more. Have an open mind and give him your data. The best way to see if he is qualified is to provide him with your data so he can do the BEST job he with the best, most untainted data, he can get his hands on. Ultimately, the study can do no harm because you know it'll get dissected with considerable care, if not the peer review process, then at least here on 2+2 and elsewhere.

As to whether he needs a consultant like Miller - how do you know that he doesn't already have one? No one knows anything about what he is and is not doing; there's just all this unfounded worry about Levitt. First, poker is very popular, and at a school like Chicago, I seriously suspect there will be no shortage of men and women with considerable knowledge of the game, at the faculty level and/or the student level. But secondly, Levitt's careful and I won't be surprised if many of the very concerns many are having will be adequately addressed. Also, FWIW, as someone who is working on a project right now, I am CONSTANTLY being given advice by people who really know nothing about what I'm doing. People are constantly telling me (like my father-in-law or pastor), have you talked with these people? You need to talk this person. You can't know anything about this because you're not black (my favorite). It gets old. Just let the guy do the study and respond when there's something to respond to. No offense intended.

Arnold Kling wrote a negative review of his book, as did James Q. Wilson. There was a mix of both positive and negative reviews, in fact. And as you can imagine, Levitt's work has invited a lot of controversy. I do know that he has responded to some of the published criticisms of some of his more controversy work, though. For instance, the abortion study (the most notorious study) prompted both Ted Joyce (someone who's studied abortion and fertility their entire career, and also an economist) and John Lott (someone who I personally think is more than just a little jealous of the attention Levitt's been getting, but that's just my opinion) both have written competing empirical studies in response. Levitt has responded in print to the Joyce study (I think it is called exactly that "A Response to Ted Joyce," and co-authored by Donahue the original co-author) and he responds brutally to Lott on his blog, which if you're interested, I can dig up and post.

My point here is that Kling's "review" is part of a larger conversation on Levitt's work. I would strongly encourage that before you hang your hat on any of these guys' criticisms, that you read very carefully the Levitt study in question, read the criticism again, and then look for Levitt's response. Of course the response may be legitimate and Levitt wrong. But here is what I see mainly - people who only read Levitt with suspicion, and not very carefully, and then find someone who makes good points against him. Then they just say, "Levitt's an idiot; see Lott's paper on why the abortion study sucks" or "see Kling's review."

When I said that the guy is sharp, I meant that he is more qualified to do this type of study (no offense) than anyone on here. He's not a green novice empiricist who has never had to deal with the kinds of problems people on here are describing - like whether he is even qualified in the first place to study the thing in question. Economics, as a science, has been moving towards developing a more general theory of human behavior, if not dating back to Becker's pioneering work on discrimination, crime, family, education, etc., then at least crystalizing around that point forty years ago. And as it has attempted to "imperialize" the other social sciences, it has gotten a ton of flack from sociologists, political scientists, anthropologists, you name it. A lot of what I'm hearing here sounds identical to those same old arguments - how can this economist come in and study this thing? I bet he doesn't know a tenth about it like me/Ed Miller/whoever. But, I claim that Levitt is actually more qualified to do this TYPE of study than anyone on 2+2. That is NOT to say that he is a better poker player, that he knows more about poker, or that he is a better theorist of poker. But he is a well-trained and skilled applied econometrician, and the questions he's asking will be answered using sound empirical methods. The only concern that I would have, personally, is the selection bias inherent in this kind of study. But so what? What are you going to do? Not to be a crude pragmatist, but this is the problem economists are always faced with - I know theoretically what to look for, but now I have to get the data, and the data is ALWAYS less than ideal. We're not biologists, who can use very simple methods to control for whatever phenomenon we're seeking to analyze. But econometrics as an empirical science has evolved specifically to address the problems inherent in social science research, and Levitt's colleague James Heckman is at Chicago, is a Nobel Prize winner, and has written extensively on selection bias. The problem of it, and the methods for dealing with it, are no secret nor are they arcane. Levitt will have to deal with it, and I'm curious to see if he does, and if not, what he says about the study's reliability.

But thinking about the data, using Poker Tracker, he'll be collecting a large database probably in the tens of millions of observations on hands. Unless you're an online casino, what else could only possibly do to try and get the sort of data one would need for this kind of study? He's soliciting people to submit their Poker Tracker hand histories, and is compensating them with what I am willing to say is well worth the trouble anyone will incur for emailing him.

Also, I do sense some tension towards Levitt in Ed Miller's earlier post in which he claims that Levitt is tacitly "poo-poo'ing" on theorists work. I can only speak to this debate as someone training in econometrics and economic theory, but there's long been tension between the more deductive types and the more inductive types in economics. Ultimately, there should be no tension between theory and empirics, but there has always been tension between the two (at least in economics). Austrian economics, for instance, eschews econometrics almost entirely, partly because methodologically, it sees deductive reasoning as sufficient for arriving at economic truths.

As I think about it, this is really a great contribution to the growing literature on poker theory. There does not exist, at present, ANY empirical literature on the game. There is merely (a) Monte Carlo simulations and (b) anecdotes from players about certain hands. But nothing solidly statistical. Even the Poker Tracker software only reports summary statistics for your own play, but not for how you compare with others. Even if you disagree with many parts of the study's findings, it will be incredibly valuable for all theorists of poker to have a study like this. And, it's entirely plausible to believe that this is just going to be the beginning, not the end, of more sound empirically-oriented studies on poker. I know that Levitt was trying to get data from casinos before he started this Pokernomics.com project, but he never was able to. For instance, I know that he was meeting with executives at the Rio back in April to get some data, but I think he was unable to do so. One of Levitt's chief strengths is in getting good data, and he's leveraging the popularity of his book to help him in his research agenda.

Anyway, off my soapbox. Don't flame me.