Pokernomics

[jim leach]

[ QUOTE ]
I guess my objection is that Levitt seems to think he can just go in to things, learn some basics and make conclusions based on data. Unfortunately, sometimes more knowledge is needed in order to understand what the data is trying to tell you. Levitt should really hire Ed Miller or King Yao, who would both probably be interested and flattered.

[/ QUOTE ]

First, this thread about how Levitt's unqualified because he knows nothing about poker is based entirely on speculation. You do not know whether Levitt is or is not prepared, and the degree to which he has the knowledge needed to undertake this kind of study. This "theory" that he's unqualified is based entirely on speculation. The best anyone has managed to show as evidence is King's points about the NFL study, but even there, all I can sense is that he is disagreeing - not with methods or data - but with his specific interpretations about the volatility of the market. I'm not trying to be whiny here, but this point is welltaken, but is irrelevant until there is something substantive to which to respond. It seems like this is a cause for some skepticism, at best, but nothing more. Have an open mind and give him your data. The best way to see if he is qualified is to provide him with your data so he can do the BEST job he with the best, most untainted data, he can get his hands on. Ultimately, the study can do no harm because you know it'll get dissected with considerable care, if not the peer review process, then at least here on 2+2 and elsewhere.

As to whether he needs a consultant like Miller - how do you know that he doesn't already have one? No one knows anything about what he is and is not doing; there's just all this unfounded worry about Levitt. First, poker is very popular, and at a school like Chicago, I seriously suspect there will be no shortage of men and women with considerable knowledge of the game, at the faculty level and/or the student level. But secondly, Levitt's careful and I won't be surprised if many of the very concerns many are having will be adequately addressed. Also, FWIW, as someone who is working on a project right now, I am CONSTANTLY being given advice by people who really know nothing about what I'm doing. People are constantly telling me (like my father-in-law or pastor), have you talked with these people? You need to talk this person. You can't know anything about this because you're not black (my favorite). It gets old. Just let the guy do the study and respond when there's something to respond to. No offense intended.

Arnold Kling wrote a negative review of his book, as did James Q. Wilson. There was a mix of both positive and negative reviews, in fact. And as you can imagine, Levitt's work has invited a lot of controversy. I do know that he has responded to some of the published criticisms of some of his more controversy work, though. For instance, the abortion study (the most notorious study) prompted both Ted Joyce (someone who's studied abortion and fertility their entire career, and also an economist) and John Lott (someone who I personally think is more than just a little jealous of the attention Levitt's been getting, but that's just my opinion) both have written competing empirical studies in response. Levitt has responded in print to the Joyce study (I think it is called exactly that "A Response to Ted Joyce," and co-authored by Donahue the original co-author) and he responds brutally to Lott on his blog, which if you're interested, I can dig up and post.

My point here is that Kling's "review" is part of a larger conversation on Levitt's work. I would strongly encourage that before you hang your hat on any of these guys' criticisms, that you read very carefully the Levitt study in question, read the criticism again, and then look for Levitt's response. Of course the response may be legitimate and Levitt wrong. But here is what I see mainly - people who only read Levitt with suspicion, and not very carefully, and then find someone who makes good points against him. Then they just say, "Levitt's an idiot; see Lott's paper on why the abortion study sucks" or "see Kling's review."

When I said that the guy is sharp, I meant that he is more qualified to do this type of study (no offense) than anyone on here. He's not a green novice empiricist who has never had to deal with the kinds of problems people on here are describing - like whether he is even qualified in the first place to study the thing in question. Economics, as a science, has been moving towards developing a more general theory of human behavior, if not dating back to Becker's pioneering work on discrimination, crime, family, education, etc., then at least crystalizing around that point forty years ago. And as it has attempted to "imperialize" the other social sciences, it has gotten a ton of flack from sociologists, political scientists, anthropologists, you name it. A lot of what I'm hearing here sounds identical to those same old arguments - how can this economist come in and study this thing? I bet he doesn't know a tenth about it like me/Ed Miller/whoever. But, I claim that Levitt is actually more qualified to do this TYPE of study than anyone on 2+2. That is NOT to say that he is a better poker player, that he knows more about poker, or that he is a better theorist of poker. But he is a well-trained and skilled applied econometrician, and the questions he's asking will be answered using sound empirical methods. The only concern that I would have, personally, is the selection bias inherent in this kind of study. But so what? What are you going to do? Not to be a crude pragmatist, but this is the problem economists are always faced with - I know theoretically what to look for, but now I have to get the data, and the data is ALWAYS less than ideal. We're not biologists, who can use very simple methods to control for whatever phenomenon we're seeking to analyze. But econometrics as an empirical science has evolved specifically to address the problems inherent in social science research, and Levitt's colleague James Heckman is at Chicago, is a Nobel Prize winner, and has written extensively on selection bias. The problem of it, and the methods for dealing with it, are no secret nor are they arcane. Levitt will have to deal with it, and I'm curious to see if he does, and if not, what he says about the study's reliability.

But thinking about the data, using Poker Tracker, he'll be collecting a large database probably in the tens of millions of observations on hands. Unless you're an online casino, what else could only possibly do to try and get the sort of data one would need for this kind of study? He's soliciting people to submit their Poker Tracker hand histories, and is compensating them with what I am willing to say is well worth the trouble anyone will incur for emailing him.

Also, I do sense some tension towards Levitt in Ed Miller's earlier post in which he claims that Levitt is tacitly "poo-poo'ing" on theorists work. I can only speak to this debate as someone training in econometrics and economic theory, but there's long been tension between the more deductive types and the more inductive types in economics. Ultimately, there should be no tension between theory and empirics, but there has always been tension between the two (at least in economics). Austrian economics, for instance, eschews econometrics almost entirely, partly because methodologically, it sees deductive reasoning as sufficient for arriving at economic truths.

As I think about it, this is really a great contribution to the growing literature on poker theory. There does not exist, at present, ANY empirical literature on the game. There is merely (a) Monte Carlo simulations and (b) anecdotes from players about certain hands. But nothing solidly statistical. Even the Poker Tracker software only reports summary statistics for your own play, but not for how you compare with others. Even if you disagree with many parts of the study's findings, it will be incredibly valuable for all theorists of poker to have a study like this. And, it's entirely plausible to believe that this is just going to be the beginning, not the end, of more sound empirically-oriented studies on poker. I know that Levitt was trying to get data from casinos before he started this Pokernomics.com project, but he never was able to. For instance, I know that he was meeting with executives at the Rio back in April to get some data, but I think he was unable to do so. One of Levitt's chief strengths is in getting good data, and he's leveraging the popularity of his book to help him in his research agenda.

Anyway, off my soapbox. Don't flame me.

[King Yao]

[ QUOTE ]
It seems like this is a cause for some skepticism, at best, but nothing more. Have an open mind

[/ QUOTE ]

I agree with this. I think he made an error in his NFL paper, but that doesn't mean he would make errors in his poker study. What it does show me is that even he, an acclaimed economist, can make errors. Maybe I'm naive, but I was surprised by this. Even based on what I have said, I do have an open mind, I am looking forward to any results and conclusions he comes up with. I'll attack those results only after I see them and disagree.

[ QUOTE ]
For instance, I know that he was meeting with executives at the Rio back in April to get some data

[/ QUOTE ]

I don't know exactly what you mean, and I don't know if Levitt actually approached the Rio and asked for data...but if he did, this is laughable. The Rio doesn't have the data, and they have no way to get the data. To even think about approaching a brick and mortar about getting poker hand data shows little understanding about the field. All that one can hope is that he is just beginning to sort things out, but if he doesn't learn more about the field, he is going to have problems.

But again, open mind, yes. I agree with that. He hasn't come up with anything for us to say he's wrong or right. All we can do is wait.

[jim leach]

[ QUOTE ]
I don't know exactly what you mean, and I don't know if Levitt actually approached the Rio and asked for data...but if he did, this is laughable. The Rio doesn't have the data, and they have no way to get the data. To even think about approaching a brick and mortar about getting poker hand data shows little understanding about the field.

[/ QUOTE ]

I have no idea what the subject of their conversation was, but obviously it wasn't to get poker hands. I mentioned that anecdote mainly to say that Levitt has been attempting to get data from casinos (not necessarily for this one project) in my comments about the quality of his data. I wasn't implying that Levitt was trying to get starting hands, but I did meant to imply that he was studying gambling, and that he had been talking with agencies that have quality data.

Casinos do keep considerable databases on players, I imagine, and I interpreted his comment to me that he was trying to get those databases to study casino cheating. But that was speculation on my part. After his study on sumo wrestling, from what I understand, he has been solicited to study cheating and corruption in a variety of settings (for instance, he has been approached by a former Tour de France winner to study doping in the race). He may have been meeting with them at their request, for all I know, and maybe something will come of it. I wasn't intending to imply, though, that he was trying to get them to give him data on starting hands which even someone who knows nothing about cards knows impossible. But my comment was clear, and I was digressing without realizing it, so I apologize for the confusion.

[Robk]

[ QUOTE ]
his "theory" that he's unqualified is based entirely on speculation. The best anyone has managed to show as evidence is King's points about the NFL study

[/ QUOTE ]

people who spend hundreds of hours on 2+2 reading things that strangers have written about poker develop an innate sense for a writers' poker knowledge that is strikingly accurate. its based on the phrases a writer uses, the questions he asks, and the manner in which he asks them. i think that much of the skepticism you are receiving on here is due to the fact that (to me at least) the pokernomics page reads like it was written by someone who knows little to nothing about poker. (i can point out the specific passages that make me think this if you want.)

i dont think this means the study will be bad. for all i know levitt didnt even write the page and is a poker expert. or will acquire more knowledge before/during the study, or will do a great job in spite of a lack of poker expertise. im just trying to explain why some may not have total confidence. since the page was meant to appeal (or adverise, even) to hardcore poker players (those who play many hands and have saved datatbases of them) i do feel it was a mistake to put up such underdeveloped thoughts without consulting one of those members of the u of c family who know more about the game.

also you state that we have scant evidence, yet have not responded to the most damaging comment imo, namely Ed Millers point that advice generated from 10,000 hands will be misleading and perhaps dangerously so. also king yaos point about the flawed nfl study is much more damaging than i think you are giving it credit for, although perhaps thats only because you havent read the study.

in any event im optimistic about the study and will be sending in 50k hands in a few days. good luck to you.

[Ed Miller]

[ QUOTE ]
also you state that we have scant evidence, yet have not responded to the most damaging comment imo, namely Ed Millers point that advice generated from 10,000 hands will be misleading and perhaps dangerously so. also king yaos point about the flawed nfl study is much more damaging than i think you are giving it credit for, although perhaps thats only because you havent read the study.

[/ QUOTE ]

I went back and forth once with Dr. Levitt, and I'm still not convinced the feedback he will give will be at all valuable for the large majority of his donors. He said that the feedback will be limited in scope and specificity. But it still could easily be "bad" feedback.. and even if the feedback is technically correct, it could send players to the wrong parts of their game looking for things to change.

[vulturesrow]

Ed,

I will submit the same hand history to you that I did to the pokernomics site and then compare the analysis. I wont even charge you. [img]/images/graemlins/smile.gif[/img] Seriously though, I wasnt expecting much out of the analysis, I really just wanted the free stuff and I was pretty unconcerned about someone looking me up at the Party Poker low limit tables to exploit my hand histories. But it will be interesting to see what the results are and how it compares with what I see as my weak areas.

[djack]

What if all those that might submit our databases to Levitt gave them to someone like Ed?

I don't know if mr. majorkong would be interested, of course. It'd have to be someone that the majority of posters here trust.

Ed just seems to be like the perfect guy because he doesn't play much online (at least I don't think so), he's respected by people here, and he's a better theorist than Levitt. heh.

Would anyone actually be interested in doing this?

[vulturesrow]

[ QUOTE ]
What if all those that might submit our databases to Levitt gave them to someone like Ed?

I don't know if mr. majorkong would be interested, of course. It'd have to be someone that the majority of posters here trust.

Ed just seems to be like the perfect guy because he doesn't play much online (at least I don't think so), he's respected by people here, and he's a better theorist than Levitt. heh.

Would anyone actually be interested in doing this?

[/ QUOTE ]

Id be willing to but I imagine Ed is pretty busy these days and this would be a fairly large undertaking I would think. But if another 2+2er wants to step up to the plate, Im game. However it would be somewhat of an invalid comparison if our analysis set wasnt the exact same one that the Pokernomics team is using.

[Ed Miller]

[ QUOTE ]
Id be willing to but I imagine Ed is pretty busy these days and this would be a fairly large undertaking I would think. But if another 2+2er wants to step up to the plate, Im game. However it would be somewhat of an invalid comparison if our analysis set wasnt the exact same one that the Pokernomics team is using.

[/ QUOTE ]

Ya, I definitely don't have time for the next six plus months to work on a project like this. And, frankly, I'm curious what Levitt will come up with on his own without "help" from us know-it-alls. [img]/images/graemlins/smile.gif[/img]

After he publishes a paper or two, if his results are impressive, I'd be interested to see if maybe we could direct future research to answer specific questions we want answered.