When Do Prediction Markets Beat the Common Wisdom?:
Over at the Chicago Faculty Blog, Cass Sunstein has an interesting post on prediction markets. Commenting on the broad coverage of the fact that prediction markets didn't predict that Hillary would win the New Hampshire primary, Cass notes the possibility that skepticism about prediction markets may result from the availability heuristic. That is, when a prediction market predicts a low probability of something and that event occurs, observers may see that as the prediction market not "working" rather than it being an instance in which a low probability event did indeed occur. He concludes:
The problem with prediction markets in elections is that the media and the public are already obsessed with predicting the outcome of elections. Everyone loves the horse race. Those who have particularly deep insights into who will win elections -- assuming those people exist -- already are eager to share them with the world, and the media is eager to broadcast their views.
If the market for experts is competitive, the experts who are the best predictors will end up having the most impact on the common wisdom about the election. In that setting, there's no particular reason to think prediction markets will beat the common wisdom instead of just reflecting it.
To be sure, we are continuing to obtain information about how prediction markets perform and when they do well and poorly. Perhaps they will turn out to be less reliable than they seem -- and in all likelihood, we will obtain a better understanding of when they work. And of course no one has a crystal ball. But . . . if you want to have a sense of the probabilities, you'd probably do best to consult Intrade.Perhaps that's right. At the same time, I would think prediction markets will tend to beat the common wisdom only when there are people actively participating in the market who have better knowledge than a reasonably informed member of the public. And for the most part that usually won't be the case when the future event is a subject of intense media speculation like an election.
The problem with prediction markets in elections is that the media and the public are already obsessed with predicting the outcome of elections. Everyone loves the horse race. Those who have particularly deep insights into who will win elections -- assuming those people exist -- already are eager to share them with the world, and the media is eager to broadcast their views.
If the market for experts is competitive, the experts who are the best predictors will end up having the most impact on the common wisdom about the election. In that setting, there's no particular reason to think prediction markets will beat the common wisdom instead of just reflecting it.
In any case I agree that they have no useful insight about political races. The people making the bets base them on the polls and their own gut feelings.
Sunstein has also pointed out that, especially in these days of media fragmentation, the "common wisdom" of one particular social community can diverge radically from the "common wisdom" of another one. An example we are currently witnessing: Beltway people have no clue what is going on in the minds of Ron Paul supporters, and vice versa. Markets and polls allow us to aggregate these widely divergent and mutually ignorant views into estimates of overall "common wisdom".
:-> ), which argues convincingly that most experts are wrong in their predictions most of the time.
I always thought the difference between "experts", whom you can see on major and cable news channels all year pontificating, and prediction markets is that the experts on TV are not at any financial risk if they're wrong. Why, they even get continually invited back; that is, they don't even lose "TV expert" income for being wrong. While, of course, those investing in prediction markets are confident enough of their own opinion, for whatever reason, that they're willing to bet their own money on the outcome. This distinction still holds even when all the points made by Orin are true, does it not?
Seriously though, Clinton and Obama tied in delegates, so the market wasn't totally wrong in predicting he would win.
Relatedly, I recall reading that when it comes to predicting the value of stocks, buying a diversified sample of the market did better than following the most-quoted experts. That's old news, of course, but the interesting twist was that those experts who did better at prediction systematically did worse at getting quoted by the media because they hedged their predictions with lots of qualifications rather than brashly predicting something spectacular.
Better, would be information aggregation market.
In computer science, there is a well-known acronym. GIGO. Garbage-In, Garbage-Out. Basically, the idea is that there are no processes that are going to take garbage input, and somehow output useful information.
Now, I would not go so far as to say the inputs into a prediction market is so bad as to constitute garbage. And it is in fact sometimes true that combining the results of multiple artificial intelligence algorithms can result in more accurate predictions. But, when you have a lot of bets on prediction markets that are not any better informed than the conventional wisdom, you are not going to expect the aggregation of those bets to be better than the conventional wisdom.
Of course, there is the idea that if the conventional wisdom is wrong, there is a hugely profitable arbitrage opportunity. The question is, why wasn't their some genius arbitrager who knew that Clinton was going to win before everyone else, who then made money on that superior information?
Basically, a prediction market is never going to be more accurate than the best informed arbitrager who is able and willing to put his own money at risk on this beliefs about the future.
Basically, sometimes that best informed arbitrager who is able and willing to put his money at risk based on beliefs on the future just isn't going to know much more than the conventional wisdom.
That is what explains the failure of prediction markets with respect to the New Hampshire primary.
I think a huge reason the prediction markets end up working better than other forms of information aggregation is the cost of irrationality. As the "myth of the rational voter" convincingly argues people often prefer to believe certain irrational things but will give up those beliefs when they cost too much. I suspect we see this in political pundantry (even if the beliefs aren't partisan favoritism) and the prediction markets minimize the irrationality in prediction.
Of course they still can't do better than the data they receive but assuming the conventional predictions embody closely held irrationalities then the prediction markets can do better than that.
Second, the 'political markets' are quite different beasts from real financial markets. Political markets are toys (leisure) whereas real financial markets implicitly weight people's influence over the price by their historic success (how much money they have and thus how much volume they produce). One man one vote creatures they are not.
...and the views of a bunch of people who have no better information than anyone else, but are skilled at making it appear that they do.
You surmise that the prediction market will not outperform "conmon wisdom unless" individuals with "better knowledge" (i.e., inside traders) are participating in the market.
This conclusion reckognizes one of the advantages of a prediction market, which is the inclusion of "insider" knowledge.
But it ignores another, more significant, advantage of prediction markets, which is the power of collective thought. There is a theory that prediction markets work because the consensus views of a large group are more reliable than the individual views of experts. I am not in a position to explain the logic behind the theory, but it needs to be acknowledged when considering whether prediction markets work.
Removing the participants with "better knowledge" from the prediction market does not remove the "collective thought" advantage, so a prediction market should still outperform "common wisdom."
Is there any evidence that they do work better? They seem to have a worse record at political prediction than the conventional polls.
That brings up a slightly off-topic question I've been wondering about - to what extent do normal polls skew the results of the political process? If everyone did not know that candidate X was polling better than candidate Y, how often would candidate Y win? There seems to be a large degree of measurement induced change in the results. It's a topic for some bright sociology researchers to look into.
I'm not sure if there is. Perhaps I should have been more careful and said one way they might work better.
That's an interesting point about the cost-effectiveness of buying the top spot in the intrade markets. I'm not sure if these markets attract quite enough interest yet for that to be a good deal but if they start receiving more media attention that certainly is an interesting question.
But even if you are a partial cause of the result doesn't mean you didn't predict it.
http://www.unc.edu/~cigar/papers/ManipHIT_Jan2007.pdf
I don't think anyone argues that "prediction markets" exist to "beat the common wisdom." They simply reduce to a number the collective wisdom of those participating in the particular market. If there are an insufficient number of participants or if there is an insufficiently diverse group of participants or if a host of other conditions aren't met, then the resulting price won't be very helpful.
That is, when a prediction market predicts a low probability of something and that event occurs, observers may see that as the prediction market not "working" rather than it being an instance in which a low probability event did indeed occur.
Or, the observers criticizing the prediction market could be quite right and the market miscalculated the probability of the "low probability" event occurring in the first place. Perhaps the predicted probability for Clinton should have been much higher. Financial markets routinely miscalculate the probability of "low probability" events occurring.
When calculating a potential winner, you have to use the information available to you. In my case, being in the South, I have the news media, the polls, and my acquaintances to gather information from. I do not have access to the same data as people voting in a New England primary.
I assume that the price of a candidate is determined by the number of people who believe, based on their individual sources of information, that the candidate will win.
It makes sense that the markets should echo the information available to the people who wager.
In the case of New Hampshire, it would have been interesting to have had a market that was open only to eligible New Hampshire voters. I believe it would have been more predictive of the outcome because those are the people with the best information.
I think the markets have an advantage over the polls because they are confidential. I can wager without having to publicly air my view. I am also making a decision on who I think will win, as opposed to who I am voting for.
I think polls are invalid to some extent because there are, as far as I know, no built in validity scales of the type found in psychological tests such as the MMPI. These types of questions are designed to catch random answering and lying. It would be interesting if something could be developed for pollsters to help them determine if their results are valid. If these exist, I'd certainly appreciate some education. The research I've done shows psychometricians discussing validity scales with regards to polling for at least 50 years.
My belief is that the prediction markets are better than experts because they aggregate more information, and I think they are better than polls because there is no reason to lie, and decisions are based on most probable winner(from the gambler's perspective) instead of most favored candidate.
But the key is the quantity and quality of information available. If I were betting on say, the South Carolina primary, I would want to know who the people in South Carolina are betting on. I think they probably have the best information. (If I were a Democrat, I'd want to know who Diebold was betting on. ;))
This is silly. A liquid market will reflect and aggregate information from many such arbitragers with many different sources of information.