Two Plus Two Older Archives

Two Plus Two Older Archives (http://archives2.twoplustwo.com/index.php)

- Books and Publications (http://archives2.twoplustwo.com/forumdisplay.php?f=31)

- - Pokernomics (http://archives2.twoplustwo.com/showthread.php?t=294213)

http://www.pokernomics.com

Levitt is attacking poker with the brute force approach. It will be interesting to see what he comes up with, and if you haven't read Frekonomics it's a nice chance to get his book signed and free. (You can learn the effect of Roe v Wade on armed crime at the turn of the millenium!)

Too bad he is writing this book, I'd been nourishing the same idea myself. Who wouldn't want to be the next Ed Miller after all? Oh well, now it's actually going to get done instead of dreamt about. [img]/images/graemlins/smile.gif[/img]

Sounds very interesting. He can probably pull it off because people are less afraid of their stats being used against them or for profit.

Have you tried it? I'm wondering if they give quality, accurate and useful feedback.

[ QUOTE ]
Have you tried it? I'm wondering if they give quality, accurate and useful feedback.

[/ QUOTE ]

Sent in a database of some 35k hands today, so haven't had a chance to get the analysis yet. I'll get back on it.

Levitt's paper on the NFL gambling markets was not good. He used poor data to come to incorrect conclusions. So I am not convinced he can do a good job analyzing poker data.

[ QUOTE ]
Levitt's paper on the NFL gambling markets was not good. He used poor data to come to incorrect conclusions. So I am not convinced he can do a good job analyzing poker data.

[/ QUOTE ]

I'm even less than "not convinced." Part of my evidence for this conclusion is that he asks for only 10k or more hands. Frankly, I don't think he has a clue about what he's getting into.

[ QUOTE ]
[ QUOTE ]
Levitt's paper on the NFL gambling markets was not good. He used poor data to come to incorrect conclusions. So I am not convinced he can do a good job analyzing poker data.

[/ QUOTE ]

I'm even less than "not convinced." Part of my evidence for this conclusion is that he asks for only 10k or more hands. Frankly, I don't think he has a clue about what he's getting into.

[/ QUOTE ]

wouldn't this only apply to the "free analysis" you get emailed and not the main point of the project? For example, I have fewer than 10k hands logged at pokerroom, yet I don't consider their excellent EV stats page to be inaccurate because of it.

[ QUOTE ]
wouldn't this only apply to the "free analysis" you get emailed and not the main point of the project? For example, I have fewer than 10k hands logged at pokerroom, yet I don't consider their excellent EV stats page to be inaccurate because of it.

[/ QUOTE ]

Sure, but the fact that he claims to be able to give you a meaningful analysis about "which hands you play best" after only 10k hands tells me either:

1. He's unwilling to pay people for data, so he's going to offer something that sounds good, but that will, in reality, have very little value instead, or

2. He doesn't know what he's getting into.

[ QUOTE ]

Sure, but the fact that he claims to be able to give you a meaningful analysis about "which hands you play best" after only 10k hands tells me either:

1. He's unwilling to pay people for data, so he's going to offer something that sounds good, but that will, in reality, have very little value instead, or

2. He doesn't know what he's getting into.

[/ QUOTE ]

But...but....free t-shirt! [img]/images/graemlins/smile.gif[/img]

For instance, think about how many hands you would need to figure out LOOKING FROM A STATISTICAL PERSPECTIVE ONLY (i.e., making zero judgements about which plays are good or not based on "human" skills like hand reading) if you pay off too often on the river. It's a very large number. 10k doesn't even begin to scratch the surface, yet that is quite a funadmental thing someone might want to know about their play.

[ QUOTE ]
For instance, think about how many hands you would need to figure out LOOKING FROM A STATISTICAL PERSPECTIVE ONLY (i.e., making zero judgements about which plays are good or not based on "human" skills like hand reading) if you pay off too often on the river. It's a very large number. 10k doesn't even begin to scratch the surface, yet that is quite a funadmental thing someone might want to know about their play.

[/ QUOTE ]

Upon rereading the FAQ, apparently the "personal analysis" bit isn't slated as just a gimmick, but part of the overall project. From the FAQ:

"5. How is your free analysis going to differ from statistics given by Poker Tracker or other software?

The advantage of a pokernomics analysis over statistics reported by Poker Tracker is that our analysis incorporates data about how other people play. A naive analysis would look at a Poker Tracker statistic that says "I win 57 % of the time after seeing the flop with hand Y" and conclude that playing more of those hands will be successful. A Pokernomics analysis will test how other players with a similar style to yours who had that same hand but played it greater percentage of time (controlling for relevant factors) actually did. By comparing the play of many different people over the course of hundreds of thousands of hands our analysis is far richer.

Using advanced statistics, the pokernomics team will also be able to control for table conditions, position, and other relevant factors in order to generate more meaningful statistics. "

Although I don't know if that would even make it valuable. To exaggerate for the purpose of clarity, say I were able to send this guy just one hand history of where I played AA in the cutoff in a NL game and only stole the blinds - what could his analysis possibly tell me about my own game, even if it were based on a gazillion hands where other people got AA in the cutoff?

Nothing more than the pokerroom EV-by-hand and position thingamabob could tell me, I'd imagine.

[ QUOTE ]
[ QUOTE ]
Levitt's paper on the NFL gambling markets was not good. He used poor data to come to incorrect conclusions. So I am not convinced he can do a good job analyzing poker data.

[/ QUOTE ]

I'm even less than "not convinced." Part of my evidence for this conclusion is that he asks for only 10k or more hands. Frankly, I don't think he has a clue about what he's getting into.

[/ QUOTE ]

Which means that someone with a clue can still write the real version. 2+2 and PokerRoom could probably accomplish some fairly interesting datamining... *nudge nudge wink wink*

[ QUOTE ]
Which means that someone with a clue can still write the real version. 2+2 and PokerRoom could probably accomplish some fairly interesting datamining... *nudge nudge wink wink*

[/ QUOTE ]

I'd love to see something like this done right. And perhaps Mr. Pokernomics will do a fine job, and I'll eat my words.

To be honest, though, I think a "real" implementation of this sort of analysis requires a melding of both "human" understanding and purely statistical. That is, the statistical stuff is most useful to determine the right play in situations that people can't easily derive theoretically. For instance, what to do with A9s UTG in a typical low-limit online game. [img]/images/graemlins/smile.gif[/img]

But the tone of Mr. Pokernomics's little spiel portends another flaw I see with his approach. He seems to poopoo the work that those quaint little theoreticians have done. It's almost as if he's saying, "For years we've had to make stuff up about poker theory because we didn't have real data. But now that the data is available, let's let the big boys step in and do the job right."

Unless he hires some sharp theoretical minds to tell him where his models are going haywire, I don't think he's got a chance. He's gonna come up with some very silly results in many places. Of course his book will probably sell well regardless, so all is not lost. [img]/images/graemlins/tongue.gif[/img]

So yes, I think many people at 2+2 could probably do a better job of this sort of thing than this guy. Anyone want to step up to the plate? [img]/images/graemlins/smile.gif[/img]

[ QUOTE ]
[ QUOTE ]
Have you tried it? I'm wondering if they give quality, accurate and useful feedback.

[/ QUOTE ]

Sent in a database of some 35k hands today, so haven't had a chance to get the analysis yet. I'll get back on it.

[/ QUOTE ]

Maybe I'll play a few hands at $1-$2 and intentionally make some modest, systematic errors. Then I'll submit them and see what they have to say.

[ QUOTE ]
For instance, what to do with A9s UTG in a typical low-limit online game. [img]/images/graemlins/smile.gif[/img]

[/ QUOTE ]

Going off topic here, but I think it honestly doesn't matter if you raise or call. The difference is going to be so fine that it will be darn near impossible to see a difference even in a humongous sample.

[ QUOTE ]

Maybe I'll play a few hands at $1-$2 and intentionally make some modest, systematic errors. Then I'll submit them and see what they have to say.

[/ QUOTE ]

Just out of curiosity, but how would you go about challenging any claims this guy will make at the end of his project which you deem incorrect? I don't mean to offend; I'm genuinely curious, and can only see this project as a Good Thing overall - if he actually comes up with something useful and it helps people to understand the game better, great! If he just comes up with a bunch of absurd conclusions that help people to understand the game better by understanding why they are absurd, that's great too. But it just seems to me that there would be an awfully great temptation for a theoretician to hear Levitt say something like "based on this and that data, I conclude that you should come in for a raise with A9o UTG" and simply reply "hogwash!" without ever properly addressing the argument, you know? (in fact, upon consulting gocee, that is a conceivable argument that a solely statistics-minded person might make, since A9o will win 1.9% more than its fair share of pots against 9 random hands)

Judging from his last book, this work will probably try to appeal to the masses including/especially non-poker players. Even if all his analysis and data-collecting methods are 100% sound, there is unlikely going to be anything useful for a typical 2+2 poster. Hopefully, I am wrong.

[ QUOTE ]
Judging from his last book, this work will probably try to appeal to the masses including/especially non-poker players. Even if all his analysis and data-collecting methods are 100% sound, there is unlikely going to be anything useful for a typical 2+2 poster. Hopefully, I am wrong.

[/ QUOTE ]

Definitely agreed. I figure it's a way for me to pick up a free book and tshirt.

[ QUOTE ]
Judging from his last book, this work will probably try to appeal to the masses including/especially non-poker players.

[/ QUOTE ]

If you read the post about in in Poker Theory it seems clear it will first be published as a research article. Hence pretty unpenetrable to laymen. It might develop into a book though, but that doesn't seem to be the primary goal.

Been thinking about this project a lot for the past 24 hours

My thoughts have crystalized somewhat:

1. I'd love to see someone take a shot at this.

2. I think drawing reliable conclusions about the complex stuff that we'd like to see addressed will be really hard, much harder than it sounds. I liken the complexity of the problem to that of speech recognition or natural language. When I worked at Microsoft, I initially worked on the search engine, MSN Search. About six months in, I was moved to a new project that was supposed to develop a natural language search solution for the next version of Windows. Ultimately I left to play poker, and the project failed. (In that order, of course) The crux of the problem is that natural language is a lot tougher problem that it sounds like it should be. It seems like if we just gather enough training data from enough people about the way they use language and what they mean by it, it just becomes a classification problem to make sense of everything. But in practice it doesn't quite work like that. And people have been trying for the better part of 30 years to do decent natural language and speech recognition with still limited successes.

So don't have blind faith in large datasets. There's a lot more to solving these problems than merely gathering lots of data. Brilliant minds have been pouring over natural language data for decades and still haven't really come close to "solving" the problem.

3. I'm really uncomfortable with the promises Levitt is making regarding giving hand history donors insights into their play. Frankly, and perhaps I'm jumping to conclusions here, but I feel I'm probably not, he's got no idea at this point whether he's going to draw meaningful conclusions or mere garbage.

People play poker for money, and bad advice is expensive. It behooves anyone who gives people advice on their games generally to know what they are talking about. Now that's obviously not to say that many advice-peddlers presently active aren't failing in exactly that way. But it is to say that Dr. Levitt, unlike some other poker types, clearly aspires to be an upstanding citizen, as it were. His interests are probably academic in nature. So he should seek to "do no harm."

I think he should dump the "I'll give you advice on your game" promise and just go with the T-shirts. If he wants to keep that aspect of the incentives, I think he should hire someone who really knows how to play poker to review the recommendations for reasonability before they go out.

Sure, the whole point of the project is to uncover things that people like me have missed in our books. But when you cross over from mere observer to advice-giver, you take on a LOT more responsibility.

4. My initial reactions sound disturbingly (to me) like how the baseball old guard reacted to Billy Beane in Moneyball. (Though I think the analogy is very incomplete.)

5. I think this is a cool project, and I hope it's successful.

Re: Been thinking about this project a lot for the past 24 hours

Maybe it will be like just after Phil Helmuth's "Play Poker like the Pros". A bunch of fools faithfully following terrible advice and Steamboatin gets to run good until they run out of money.

Immediately after this book came out the play at Caesar's IN changed dramactically for the worse. Much more aggression with terrible hands and capping preflop with any pocket pair, etc.

I never read the book and most likely never will but for a few months after it's debut, low limit poker was a whole lot of fun so I drew the conclusion from listening to other player's discussions and posts from 2+2 that it was most likely to blame for the sudden change in Low Limit poker.

That is why, Phil Helmuth is my favorite author for other people to read and I anxiously await his next masterpiece.

Re: Been thinking about this project a lot for the past 24 hours

I agree completely that his advice giving is a real risky business for him.

And like you I think the rest of the project is very interesting. You should probably email them a fully fleshed out criticism if you can spare the time. support@pokernomics.com would probably be glad to hear from you.

Re: Been thinking about this project a lot for the past 24 hours

[ QUOTE ]
I agree completely that his advice giving is a real risky business for him.

And like you I think the rest of the project is very interesting. You should probably email them a fully fleshed out criticism if you can spare the time. support@pokernomics.com would probably be glad to hear from you.

[/ QUOTE ]

I just emailed them.

Re: Been thinking about this project a lot for the past 24 hours

I definitely want to hear their response.

I sent him about 200K HHs, I don't really care about what he tells me, becasue a) most of these are older HHs (I've been playing mostly SNGs for the past 6 mos) b) I just wanted a free book.

[ QUOTE ]
I sent him about 200K HHs, I don't really care about what he tells me, becasue a) most of these are older HHs (I've been playing mostly SNGs for the past 6 mos) b) I just wanted a free book.

[/ QUOTE ]

Same. A free book would be sweet.

He won't write a book. The book, FREAKONOMICS, was nothing more than he and Stephen Dubner taking several of Levitt's most important academic papers, and then making them appropriate for the masses. But Levitt's a serious economist, and his work is sophisticated. Pick up any of his articles published in peer reviewed journals, and/or his NBER working papers, if you're skeptical of his abilities. The guy's sharp. This study will be good.

[ QUOTE ]
b) I just wanted a free book.

[/ QUOTE ]

[ QUOTE ]
The guy's sharp. This study will be good.

[/ QUOTE ]

Just because he's smart and insightful doesn't mean that his study will be good. And yes, I've been familiar with him for awhile.

[ QUOTE ]
Pick up any of his articles published in peer reviewed journals, and/or his NBER working papers, if you're skeptical of his abilities.

[/ QUOTE ]

I don't know much about peer reviewed journals or exactly how the academic world works, so correct me if my assumption is wrong.

It seems to me that when a study is peer-reviewed it is not as useful if it isn't reviewed by people in the know in that particular industry. The peer reviewers are coming from basically the same background as the originator of the paper and they may not understand the nitty gritty issues that may make the study inaccurate. So the peer reviewers would miss the same issues that the originator may miss also.

For example, in his NFL gambling paper, it is clear that he is not a bettor himself, so he doesn't know the nitty gritty of how the market moves. He states that the market barely moves based on his limited research, but if he was betting and hunting for the best lines (as most pro gamblers do), I think he would start to see that the market moves more than he thinks. I don't expect his peers - other economists that probably aren't heavy sports gamblers - to understand these issues either, so they can't correct him or point to that issue as a possible problem. So in the peer review, all they can comment on is how he converts data into conclusions, but his peers may not know if the data is accurate or useful.

I hope Ed Miller gets involved in this project. It would bring in someone who knows the nitty gritty and thus eliminate this possible problem.

We could hope that one of the peer-reviewers is a 2+2er.

[ QUOTE ]
[ QUOTE ]
Pick up any of his articles published in peer reviewed journals, and/or his NBER working papers, if you're skeptical of his abilities.

[/ QUOTE ]

I don't know much about peer reviewed journals or exactly how the academic world works, so correct me if my assumption is wrong.

It seems to me that when a study is peer-reviewed it is not as useful if it isn't reviewed by people in the know in that particular industry. The peer reviewers are coming from basically the same background as the originator of the paper and they may not understand the nitty gritty issues that may make the study inaccurate. So the peer reviewers would miss the same issues that the originator may miss also.

For example, in his NFL gambling paper, it is clear that he is not a bettor himself, so he doesn't know the nitty gritty of how the market moves. He states that the market barely moves based on his limited research, but if he was betting and hunting for the best lines (as most pro gamblers do), I think he would start to see that the market moves more than he thinks. I don't expect his peers - other economists that probably aren't heavy sports gamblers - to understand these issues either, so they can't correct him or point to that issue as a possible problem. So in the peer review, all they can comment on is how he converts data into conclusions, but his peers may not know if the data is accurate or useful.

I hope Ed Miller gets involved in this project. It would bring in someone who knows the nitty gritty and thus eliminate this possible problem.

[/ QUOTE ]

King has a point. For example, one section of Levitt's Freakonomics book has to do with predicting what names will become popular in the future. While I have not read Leavitt's book (I have only read about it), it seems that he made a bit of a fool of himself with the name predictions--he in fact predicted that certain names would become popular in the future, when they were already at the top of the charts at the time he was writing. Apparently he didn't know anything about babynaming practices, and never bothered to look at the current status of the names on his list.

For a critique of Leavitt's methodology concerning his babyname popularity predictions see http://www.babynamewizard.com/blog/2...e-onomics.html

So the question arises, is he likely to know anything more about poker than he does about baby names? And if he doesn't, is he likely to consult/collaborate with someone who does?

LOL

[ QUOTE ]
It seems to me that when a study is peer-reviewed it is not as useful if it isn't reviewed by people in the know in that particular industry. The peer reviewers are coming from basically the same background as the originator of the paper and they may not understand the nitty gritty issues that may make the study inaccurate. So the peer reviewers would miss the same issues that the originator may miss also.

[/ QUOTE ]

I'm a fourth year doctoral student in economics who has never published nor peer reviewed anything, so take this with a grain of salt. But from what I understand, when someone is publishing in an academic journal, the editor of the journal sends the submission to a number of people whom the editor believes can speak to the study. If it's an empirical study, for instance, then the paper may be sent to general econometricians, as well as scholars who have published in that specific area. Levitt publishes in the top general journals, which means that the standards are even higher, and the review process even more rigorous.

There are other phases in the publication process that invite fairly rigorous criticism. Levitt's a research scholar at the NBER, and his work is usually paraded first as an NBER working paper. That means it gets read widely by a lot of people, but there is a lot of self-selection in who reads NBER working papers (unlike peer review, where this is not necessarily the case, or at least, is mitigated since the editor assigns the responsibility, and its an understood thing that people, if they hope to get published in that journal, will comply to an editor's wishes). That is, if you see Levitt's name and he's written something on abortion and crime, you're going to read it (a) if you like Levitt generally, (b) you are interested in fertility and/or crime, or (c) there's buzz about it and you're curious. A lot of problems are caught at this stage, since working papers are shown as a way of soliciting feedback prior to publication. It is not uncommon for a study to stay in working paper form for many years before publication. I think his abortion and crime study sat in working paper form for several years, as it was hotly contested and disputed by attorneys, criminologists, economists and sociologists. Many people, as you can imagine, hated the hypothesis and the conclusion of that study (ie, that legalized abortion was primarily responsible for the fall in violent crime rates in the early 1990s), but the methodology was sound, his question important, and the data fine. Still, Ted Joyce (another economist whose specialty is in abortion and fertility) has published recently in the Journal of Human Resources in which he finds no evidence for this "Roe hypothesis". So who is right? Joyce or Levitt or neither? Social science is kind of aggravating in part because, unlike the natural sciences, we cannot perform controlled experiments. We must, instead, rely on statistics and naturally occuring experiments to help us answer these questions. The question about the plausible link between abortion and crime more or less began with Levitt and Donahue's published article, not ended. It may be that the problems you note with the NFL study are similar in this sense - the question may not be settled, at all. Maybe because Levitt's asking the wrong question, making the wrong inference, using the wrong data - I don't knot. But I digress.

So, there's a lot of ways to get a diverse response rate. The peer review phase is the last phase, and I *think* editors send it to people whom s/he believes can help ferret out any remaining problems that may exist. It's not a perfect process by any means, but it is thorough.

[ QUOTE ]
For example, in his NFL gambling paper, it is clear that he is not a bettor himself, so he doesn't know the nitty gritty of how the market moves.

[/ QUOTE ]

I don't think this is true. From what I have read of him, he does gamble some on horse racing. He apparently grew up around horseracing (I have a grandfather in mind, though I may be just imagining this), and has a fondness for it. I know that he also still bets on horses (read his blog around the time of the Kentucky Derby, as he notes he picked the longshot horse who won).

I seriously doubt he is a gambler, though, in the way that some on 2+2 may mean the term (as in, he is not a professional gambler who earns a substantial portion of his income gambling). But I do think he is solidly familiar with it, for whatever that's worth.

He obviously may still be incredibly wrong in this NFL paper you're talking about; I've never read that paper, and probably won't. It's not a subject I'm much interested in, so never bothered. But isn't this also a working paper, still? Or did it get published? I remember seeing it on NBER a few years ago, or so I thought, so it may not be finished. But again, I may be wrong this. I would strongly recommend that you email the email address on the front page of the article (if it has not been published) with any and all criticisms of the study. I am sure that if there are errors of the sort you've described, that Levitt would like to know of it.

[ QUOTE ]
He states that the market barely moves based on his limited research, but if he was betting and hunting for the best lines (as most pro gamblers do), I think he would start to see that the market moves more than he thinks. I don't expect his peers - other economists that probably aren't heavy sports gamblers - to understand these issues either, so they can't correct him or point to that issue as a possible problem. So in the peer review, all they can comment on is how he converts data into conclusions, but his peers may not know if the data is accurate or useful.

[/ QUOTE ]

First, know that whatever conclusion he comes to is data-driven. So if he is wrong in his conclusion about how this market moves, it may be because the data, as you note, is inaccurate or non-representative in some form, or because the methodology he uses is flawed, or both. Just out of curiousity, do you think that the data is flawed or problematic in some way? Is your criticism, in other words, because the conclusion is counterintuitive or because something empirically seems wrong, or both? Again, I have no idea what this paper says or does, so I'm not defending it.

One last thing, though. As for the possible problems he may run into here, I think the potential problem that he will have to address is one of selection-bias. To what degree are the people who submit their pokertracker stats to him representative of all poker players? My dissertation is on the economics of sex and the family, and so there is one particular study that I keep reading about which reminds me of Levitt's study. The pioneer in sex research, Albert Kinsey, published two books on male and female sexuality in the late 1940/early 1950s. These studies (now referred jointly simply as the Kinsey Reports) are believed to have suffered from selection-bias which skewed the results profoundly. Many of his more controversial findings surrounding the prevalence of homosexuality and infidelity were a result of the non-representative nature of his data. For instance, he polled the individuals who attended his lectures (which were not mandatory) to get their sexual histories - people who probably were not representative of the population. He also sought sexual histories from homosexual friendship networks, prisons, and various detention centers. The agreement among later studies is that he found too much of certain types of behavior (like, for example, extramarital infidelity) because the urn from which he drew his sample was not representative of the population at large.

Since Levitt is asking poker players to select into the project, I'm not sure how he is going to address the potential problem, but it's one that is immediately mentioned whenever I tell anyone that he is working on this study. There are some techniques out there for addressing selection bias, I think, but I'm only casually familiar with them. James Heckman (one of Levitt's colleagues at Chicago) won the Nobel Prize for many things, but one of those things was for his work on selection bias. I think there is a technique that he fashioned, but I know so little about it that I won't comment. But it is called "heckit" if I'm not wrong.

All of this to say - I think that Levitt is a very careful and thorough researcher. The kinds of problems people are worried about on here (ie, that he isn't a gambler, and therefore cannot appreciate the nuances of poker play, and thus will come to incorrect conclusions) are really probably not going to be problems in the end. I suspect that he knows far more about gambling and the game that you know. I'm not saying that his infallible, or incapable of mistakes, but he is transparent in his research, and this will be workshopped considerably, as well as available in working paper form online well before it ever reaches the publication stage. This means, it will be possible to review the study's results, and be a part of the debate as to its merits, well in advance. My saying that "he is sharp and smart" was merely to say that he's not going to make amateurish errors like the ones that I see being worried about on here.

But again, take this all with a grain of salt. What do I know? I just learned I failed my econometrics field exam and have to take it again in three weeks, so I'm not the best source for addressing methodological errors in empirical studies (yet).

[ QUOTE ]
He states that the market barely moves based on his limited research, but if he was betting and hunting for the best lines (as most pro gamblers do), I think he would start to see that the market moves more than he thinks.

[/ QUOTE ]

Okay, I'm going to read this. You've got my curiousity piqued. I can't download it from home, since I don't have an NBER subscription and thus cannot get it off of SSRN. But in the abstract, it says that "in sports betting, bookmakers announce a price, after which adjustments are small and infrequent." But above, you said, " the market barely moves" and that most gamblers search for the best lines. But that's not the same thing, no? I mean, isn't he just saying that individual adjustment by a bookie is slow and infrequent? Not that there is uniform lines across the market.

Re: Been thinking about this project a lot for the past 24 hours

[ QUOTE ]
4. My initial reactions sound disturbingly (to me) like how the baseball old guard reacted to Billy Beane in Moneyball. (Though I think the analogy is very incomplete.)

[/ QUOTE ]

If you're defending Moneyball, then you may have one more reason to doubt Levitt's ability to carry out this study. He appears to think Moneyball is overrated; maybe even incorrect (though I don't know if he goes that far; I never read these entries in his blog, so I'm just passing it along).

This is the first entry:

http://www.freakonomics.com/2005/04/...-of-billy.html

Or go here to see several entries related to Moneyball.

http://www.technorati.com/search/bea...s.com/blog.php

[ QUOTE ]
The peer review phase is the last phase, and I *think* editors send it to people whom s/he believes can help ferret out any remaining problems that may exist. It's not a perfect process by any means, but it is thorough.

[/ QUOTE ]
It certainly is not perfect. Even the most prestigious journals make embarrassing mistakes. To give an extreme example, consider the case of Hendrik Shoen, whose fraudulent papers on molecular electronics were published and later retracted by the journal Nature.

Self-selection will be a problem. Losing players are far less likely than winning players to have collected tens of thousands of handhistories.

Despite the potential problems, I look forward to seeing the results of this study. I'm also eagerly anticipating my free book. [img]/images/graemlins/smile.gif[/img]

Re: Been thinking about this project a lot for the past 24 hours

I don't know what he knows or doesn't know about baseball, but to say that the A's and Beane are doing a bad job because the expected wins is 83 ... is not on target. 83 wins is only slightly above winning 50% of their games. This looks like an average team, right? But who should we compare the A's against - the Yankees, the Red Sox and the Dodgers? or the Royals, the Devil Rays and the Reds? Clearly the A's success rate should be measured against the latter group because they have similar payrolls as those teams. Whereas the former group can afford to spend 2-5 times as much. The expected records of the latter group were probably around 71 games.

When Levitt doesn't take that into account, it shows that he doesn't fully understand baseball. His conclusion from the data that he presents is fine, but his data is not fully accurate in context.

Re: Been thinking about this project a lot for the past 24 hours

[ QUOTE ]

If you're defending Moneyball, then you may have one more reason to doubt Levitt's ability to carry out this study.

[/ QUOTE ]

Yup. His attacks on Billy Beane do give me pause.

Scott,

Thank you for your response and explaining a bit about how peer review works. The NBER paper I read was a "NBER Working Paper Series" from December 2002. I have no idea if it was published or not...and frankly from my point of view, it matters not. Apologies in advance for my long and possibly rambling post below.

[ QUOTE ]
...in the abstract, it says that "in sports betting, bookmakers announce a price, after which adjustments are small and infrequent." But above, you said, " the market barely moves" and that most gamblers search for the best lines. But that's not the same thing, no? I mean, isn't he just saying that individual adjustment by a bookie is slow and infrequent? Not that there is uniform lines across the market.

[/ QUOTE ]

On page 1, he writes: "With sports wager, however, market makers (casinos and bookmakers) simply announce a "price" (which takes the form of a point spread, e.g., the home team to win a football game by at least 3.5 points), after which adjustments are typically small and infrequent."

He adds a footnote saying: "In the five days preceding a game, the posted price changes an average of 1.4 times per game. When the price does change, in 85 percent of the cases the line moves by the minimum increment of one-half of a point. Thus the posted spread on Tuesday is within one point of the posted spread at kickoff on Sunday in 90 percent of all games....."

Levitt doesn’t understand how huge a move of a half point or a whole point is in the NFL. It is a gigantic, volatile move. To write: “Thus the posted spread on Tuesday is within one point of the posted spread at kickoff on Sunday in 90 percent of all games....." is like saying the Dow Jones isn’t volatile because 90% of the trading days it moves less than 1,000 points.

A little background on football markets first in case you are not aware:

A pointspread is basically a handicap. A team that is a 3 point favorite is expected, on average, to win the game by 3 points. So subtract 3 points from the favorite (or add 3 points to the underdog) and then you find who won or lost the game as far as the pointspread is concerned. The Patriots were a 7 point favorite against the Eagles in this past Super Bowl. They only won by 3, so that means the winning side of the pointspread wager is the Eagles +7.

Lines have a juice (vig) attached to them, and it is usually -110 either way. This means the bettor has to risk $110 in order to win $100, or any fraction of that ratio ($11 to win $10, $22million to win $20million, etc.). If one were to look at this in terms of percentages, that would be roughly a market of 47.6% bid, and an offer of 52.4%. As a former derivatives trader, I equate this roughly to a market of $47.60 / $52.40 on a stock. As you can see, this market is incredibly wide. If the true price was $51, the market would still be $47.60 / $52.40 as the bookies would just keep the conventional -110 vig on the game. The same market as when the price is $49. There are reasons, which I won’t get into right now, why it is useful for the bookmaker to keep a -110 / -110 line instead of moving that line all over the place based on “minute” changes in the market.

There are some key numbers in the NFL. For example, if a line is the home team laying 3 points, there is around a 10% chance the home team will actually win by exactly 3 points. So now consider what this means when a bookie moves a line from 3 to 3.5. Using the mid-market, if 3 is the true fair price, being able to bet +3.5 on the underdog at no juice means that you will win 55% of the time and lose 45% of the time. That’s a great bet. When a line moves from 3 to 3.5 that’s equivalent to a stock moving from $50 to $55. How often do $50 stocks move $5 up or down in one day? The answer is not often. So a half point or full point move is a very large move. Even when it doesn’t seem like the line has moved, it could have moved but is not visible due to the line he is watching. In recent years, some low-vig online sportsbooks have become successful. At these books, it is possible to see the market move from +100 / -108 to +105 / -113 to -105/-103 in a few minutes based on the action that the book took. Yet, at the books with less competitive lines (almost all the places in Vegas), the customer would never know anything changed. They would have simply looked up at the board and seen -110 / -110, and it would look like nothing was going on. Why is there a difference in these markets? Reasons include: accessibility (you have to deposit cash at the online sportsbooks way in advance before you can make a bet, whereas in Vegas, you can just plunk down $50 from the check you cashed a few minutes ago), legality and the degenerate gambling attitude (for years, Nasdaq market makers would routinely buy or sell shares outside of the real market against their own customers. This is because the customers were not able or did not understand there was a better number for them. This may have changed since I’ve been out of the stock market for almost a decade now.)

It is simple stuff like this that he doesn’t present in the paper, and it is clear to me he doesn’t understand the nitty gritty of the NFL betting industry. The market can be volatile and can move without the layman knowing, even if he’s watching the betting screen. That was the main issue I had with it. There are also other problems with his paper, mainly due to his data that he is working with (he does acknowledge that is a possible problem). His data came from a low dollar season long handicapping contest where the entry fee is $250. Few serious true professional gamblers would use up their valuable time to enter such a contest. So therefore, I find his conclusions irrelevant. I mean, his conclusions may actually be true, but based on the data, he can’t draw some of the conclusions that he did.

But don't get me wrong - I think he does come up with some interesting aspects to football wagering in that paper. Using his data, he is able to figure out some issues that pro gamblers already know and figured out after years of betting - and that is to be commended. But it is the fact that he missed these basic things that are crucial – that’s what makes me wonder if he will make the same mistakes.

In another post, you stated something along the lines of his experience in horse racing. To say that he knows horse racing, and equating that to saying he understands sports betting or poker, is like me saying that a Phd candidate in Biology can be expected to write a legitimate paper in Physics (ok, maybe that’s a bit of an over-exaggeration [img]/images/graemlins/cool.gif[/img] ) . Even among the individual sports, there are incredible differences where one person could be an expert in one and have zero expectancy in betting another sport (I’m thinking specifically of the NFL versus Major League Baseball).

It may seem like I am attacking Levitt, but I’m not specifically. Instead, I’m attacking academics in general. I applaud Levitt for taking on these issues because they are interesting to me, I don’t think Milton Friedman or John Maynard Keynes would bother with NFL gambling markets. I read this paper only because that’s what I’m interested in. I would bet that many other economists would make the same mistakes if not more mistakes. That makes me sound like an arrogant prick, right? It sounds that way to me when I re-read what I typed. This coming from me, who only has a B.S. in Economics and who has barely remembered any of it. Oh well, I believe I am correct, and I have stated my reasons.

Re: Been thinking about this project a lot for the past 24 hours

[ QUOTE ]
[ QUOTE ]

If you're defending Moneyball, then you may have one more reason to doubt Levitt's ability to carry out this study.

[/ QUOTE ]

Yup. His attacks on Billy Beane do give me pause.

[/ QUOTE ]

Lol. I was mostly joking in writing that, but I think those posts of his on his blog did get a ton of similar responses to yours. At least, based on the subseqent posts along those lines, that seems the case.

Re: Been thinking about this project a lot for the past 24 hours

It's not just the moneyball/billy beane attack -- where I think he was offbase, because the point was market inefficiences in player valuation -- but other things too. There's also the real estate study, which Arnold Kling critiqued.

I guess my objection is that Levitt seems to think he can just go in to things, learn some basics and make conclusions based on data. Unfortunately, sometimes more knowledge is needed in order to understand what the data is trying to tell you. Levitt should really hire Ed Miller or King Yao, who would both probably be interested and flattered.

(and am I the only one who really wishes King Yao were teaching me to bet sports?)