pageok
pageok
pageok
[Michael Abramowicz, guest-blogging, February 1, 2008 at 11:47am] Trackbacks
Normative Prediction Markets:

Suppose that you are a member of a large group that has a large number of decisions to make. It might seem that you have two basic choices.

First, allow everyone to vote on every decision. This approach produces high representativeness (at least if everyone votes), but the votes will be based on little information. Second, allow a subset of the group to make each decision. This approach reduces representativeness, but allows for more informed decision makers.

Democratic institutions combine these two basic approaches in elaborate ways to overcome the trade-off between unrepresentative and uninformed decision making. All enfranchised citizens select a few citizens to serve as legislators, for example, and legislators divide into committees. For different types of decisions, we accept different trade-offs. Three-judge panels are unrepresentative but informed, so in theory we allow them to resolve legal questions but not to change national policy.

None of these solutions is perfect, and we face the usual perils of republican decision making: ignorant voters, special interests, legislative inertia, activist judges, and executive policies highly sensitive to the quadrennial preferences of a small number of voters in places like Florida and Ohio. But we may well structure voting regimes reasonably efficiently given the fundamental trade-off.

There is, however, a way of overcoming this basic trade-off using prediction markets rather than votes. We can commit to selecting someone at random from our group, or from a subset of it, to say what the decision should be. We will require this person to listen to detailed arguments and to produce a detailed explanation. But this will not be our decision. Instead, our decision will be based on the forecast generated by a "normative prediction market" predicting what this person will conclude is best.

Moreover, we don't even need to have someone conduct this evaluation for every decision. We can use a pseudo-random number generator to pick only, say, one-tenth of the decisions for ex post evaluation. Before we make the random selection, we run a conditional normative market, where the condition is that the decision is selected for ex post evaluation. But every time, it is the market's prediction that we will use as the decision.

A summary of the steps: (1) Subsidized conditional market predicts decision. (2) This prediction determines the group's actual decision. (3) Random number from 0 to 1 is drawn; if it's greater than 0.10, all money from market is returned. (4) Person is picked at random from group, and must eventually announce what he or she would have decided. (5) This evaluation is used to determine payouts in the conditional market.

This is a radically new way of making decisions, and I emphasize in the book that there are strong reasons not to transform radically our democratic institutions. I use dramatic examples (e.g., prediction market legislatures, trial by market) to illustrate the approach colorfully, but I don't believe we should rewrite the Constitution. Normative markets could serve as useful inputs into more traditional decision making (change step 2 above to "This prediction provides a recommended decision."), or be used in private settings.

All I want to show here is that this approach, which could also be used in private settings, helps overcome the trade-off between unrepresentative and uninformed decision making.

If the prediction market is sufficiently subsidized, then the prediction can be highly informed. Since we only have a few decisions that need ex post evaluation, and only one decision maker per decision, we can demand a lot of the ex post decision maker, who will then become informed too. Picking a random citizen might not be the best strategy, since an informed dolt is still a dolt, so we might have the ex post evaluator randomly drawn from a body akin to a judiciary (experts selected by indirectly elected representatives).

Meanwhile, the system provides a virtual representativeness. Traders don't know who the actual ex post decision maker would be, so they will average the anticipated decisions of a broad ideological range of potential decision makers. We may be able to increase representativeness still further by delaying decisions a decade or so, so it won't matter if we happen to have an unbalanced set of ex post decision makers at any one time.

Critically, it doesn't matter if the actual ex post decision maker makes a foolish or unrepresentative decision. What matters is the average expected decision, because it is only the prediction of the ex post evaluation that determines policy.

Of course, my claims here depend on my earlier claims that prediction markets will be sufficiently accurate and deliberative.

EvilDave (mail):
So you are suggesting ... jury duty for decision making?
2.1.2008 11:56am
Gilbert (mail):
Your market won't be representative. People with more money have less to lose by wagering more and will therefore be over-represented.
2.1.2008 12:03pm
Michael Abramowicz (mail):
EvilDave -- Not necessarily. Remember, we can use a subset of the group (e.g., assign an administrative agency official 10 years from now to do the ex post evaluation).

Gilbert -- I deal with this issue at some length in the book, but the short answer is that the demographics of the trading group don't necessarily matter, because traders just want to make money by predicting accurately. The Iowa Electronic Markets were pretty accurate even when it consisted almost exclusively of white male Republicans in Iowa. It is important to have the random selection of the ex post decision maker be of a sufficiently diverse group.
2.1.2008 12:22pm
The Ace:
In order for this to work, the person announcing what her decision would have been must not know that she is the lucky "decider" prior to bets being made in the market. If not, the "decider" possesses inside information would could assist her friends (if she herself was prohibited from trading). In this respect, it is important to keep the number of people who could be the decider as large as possible. So are you arguing the decider should be a totally random "man on the street" as opposed to at least a minimally qualified person educated on the topic at issue?
2.1.2008 12:55pm
drewsil (mail):
If the only source of reward is through the market then your are correct that "the demographics of the trading group don't necessarily matter." However, I can easily imagine a regulatory, land management, or other decision, where groups will stand to lose or gain alot of money by the result of the decision, rather than through the market. These situations would lead to market manipulation on a large scale.

Sometimes the effects of the manipulation will cancel out, ie. in the case of two large companies fighting over a contract. In the case where a large number of people stand to lose a little money while a small concentrated group stand to gain a windfall, the prediction market would seem to fail. Of course this is precisely the place where our current legislative system also fails most egregiously. So in the end the prediction market may provide equivalent results.
2.1.2008 1:32pm
Robin Hanson (mail) (www):
Drew, we have data and theory suggesting manipulation makes these markets more accurate on average.

Clearly this approach can lower average costs of deciding, but it is less clear how much it can improve the quality of decisions. Take a random person and put him in a room with a pile of papers related to some decision - will you really get a much better evaluation of that decision if you make him sit in that room for a month as opposed to an hour? Will his decision really be much better if the decision happened ten years ago versus last year? If you give him no incentive to make a good choice, I suspect rapidly diminishing returns to increased decision time. Perhaps if you pay him to predict another evaluation, which has a 50% chance of being made, and so on to infinity.
2.1.2008 2:17pm
Guest 1L:
I don't understand how the person picked at random has any incentive whatsoever to make a reasonable decision. Why expend the effort if the choice has no effect on the real decision. What really rides on the choice are the winners and losers in the market. It seems odd to suppose that the random "decision maker" would be willing and able to pretend that his choice had an effect that it didn't. If I were involved in a market like that, that's not an assumption I'd make.

I'd assume 1) that everyone would make their own policy decision based on very little info, 2) any information they'd seek because of the market would be information about their fellow decision makers, and 3) the randomly selected person would make the decision based on her original reaction, possibly expending some effort to rationalize the decision.

If all the participants assumed this, then at best, you'd get the result of majority vote. At worst, the market, for no good reason, would fail to predict a majority vote.
2.1.2008 2:27pm
Michael Abramowicz (mail):
The Ace -- We shouldn't know who the decider will be until after the betting. Even then, it would probably be prudent to prohibit those who bet from deciding. I think this decision-making approach can have more populist or more technocratic flavors, but my own preference would be to have a very educated person, but not necessarily informed in advance on the immediate issue.

Drewsil -- I addressed the issue of manipulation in my previous post. My fallback position is also relevant here: We might limit trading to a limited number of people (who might even be salaried and be trading for "points").

Robin -- I think a month would be better than an hour. But again, this "random person" might be a "random person from a subset of the large group chosen to perform these ex post evaluations based on factors like education."

Guest 1L -- This is an excellent counterargument. But appellate courts are quite careful in their decision making, and I would imagine a similar body of ex post decision makers. I think that the care appellate judges put into their decisions has more to do with cultural factors and reputation than with the fact that their opinions have direct effects on the litigants before them.
2.1.2008 2:35pm
randal (mail):
I agree with The Ace - the most flawed part of this scheme is guaranteeing that the chosen decider doesn't know what the market predicted. Even knowing the prediction would unfairly influence the decision. But of course, the prediction has to be made before the decision. So you end up sequestering the decision maker for some time while the market churns, then have them announce their decision around the same time that the market closes. Is that practical?
2.1.2008 2:40pm
Guest 1L:
I think I understand the scheme better now. (But I'm not sure.) I was picturing a moderately sized decision-making body who participate in the market and then are all potential ex post decision makers. Now I'm picturing an open market and a fairly small pool of possible ex post decision makers. For SCOTUS, say, instead of a vote that involves nine expenditures of sorting out the issue, you could have an open market that essentially predicts a SCOTUS majority at the cost only of occasionally having one justice sort out the issue. Very cool idea.

The smaller the pool of ex post decision makers, though, the fewer savings in effort. Also, with a small pool of ex post decision makers, information gathering efforts in the market would largely focus on trying to get into the ex post decision makers' heads--an effort that the ex post decision makers presumably don't have to make themselves. With a larger pool, it seems less likely to me that you could expect them to pretend to make the decision in good faith. Also with a large pool, it seems more likely that the ex post decision maker would take the short cut of deciding with the market rather than going out on a limb. That kind of feedback would make the market interesting but complicated (as I tried to suggest with my multiple eq'a example in a comment to your last post).

More generally in terms of cultural factors that would pressure ex post decision makers to make a good faith decision, I can imagine a world in which they would exist, but I'm not sure we live in it. People act out of duty rather than self interest all the time, but it's maybe hard to create that sense of duty out of whole cloth. Very interesting stuff. I look forward to getting ahold of the book.
2.1.2008 3:30pm
Michael Abramowicz (mail):
Randal -- It's true that the decider might be expected to shift his/her evaluation in the direction of the predicted evaluation (and thus the actual decision). I'm not sure that this is so problematic, in part because this status quo bias should affect all possible decisions roughly equally. But if you're really concerned about it, you could do something like keep the decision maker away from the markets (much as we tell jurors not to read the news), or even have do the ex post evaluation before the market, keeping the identity of the evaluator secret, so that the "ex post" part would really just be the revelation of what the decision was.
2.1.2008 3:35pm
Michael Abramowicz (mail):
Guest 1L: It could still be a pretty decent sized pool (say, akin to our federal appellate judiciary), or you could take a very small pool, but delay the selection of that pool (the agency head 10 years from now), so the effective potential pool is very large. If you're really skeptical about incentives to reach a decision that will only affect the ex post evaluator's reputation, we could pay the ex post decision maker based on the quality of the analysis. There would be a variety of ways of doing this in ways that wouldn't prejudice the conclusion, with or without additional prediction markets to help.
2.1.2008 3:39pm
Elliot123 (mail):
I acknowledge I haven't read the book, and I acknowledge the answers to my question may be there. But humor me. If you send me back to the book again, I will go quietly.

How do you determine what the market has decided? At what point in time do you consider the market has delivered its decision? Exactly what will tell you what the market has decided? Exactly when will it deliver that information?

If you are looking at some defined expiration point, why will anyone remain in the market at expiration? We have hundreds of markets, and 150 years of experience that tell us people are almost all out of the market prior to expiration.

When I do get the book, what I will be most interested in is whether these proposed markets will be attractive to traders. Do they reflect the real world behavior rather than academic theory? Or more simply put, do they account for emotion.
2.1.2008 4:15pm
Guest 1L:
I'm still a little skeptical of the ex post decision maker's incentives, but I like the idea of the ex post privately happening before the market. Since I'm surprised at being as close as I am to being convinced (given my initial reaction), I figure I might as well mention another reservation I have.

A lot of people have pointed out that the market participants might have an interest in the outcome of the decision, and I realize this was addressed in your previous manipulation post. In the comments there, though I didn't see the point made that there's a discontinuity in the decision process that might make non-market incentives effectively very strong. If the market is thick enough, I see that any major long-term manipulation would be extremely costly. (There might be a timing issue: if the decision time is known to market participants, manipulation might be less costly than otherwise, but that's not my worry.) What worries me is that a price of 0.49 gets you decision A, and a price of 0.51 gets you decision B. Even if incentives to affect the decision are generally swamped by profit incentives in the market, given that there are any non-market incentives at all, they have to overcome market incentives for some region around 0.5. This would mean that for close decisions, the side most willing and able to bear cost and risk would win. If you think the probability is 0.49, but you want the market to be at 0.51 because of the ultimate decision, the expected cost of buying a million ($1 payoff) shares at 0.51 would only be $20,000. It wouldn't take George Soros or Bill Gates to pull something like that off, but, of course, if there are investors interested in the other decision, they'll fight back, and you potentially have an a huge tug of war around 0.5, with the deepest pockets winning.
2.1.2008 4:18pm
Robin Hanson (mail) (www):
Do we have any data or theory to indicate quantitatively how decision quality varies with decision time, even for any identifiable demographic groups, when the only incentive is reputation? Without some sort of basis it is pretty speculative to claim that giving them more time will give us better decisions.
2.1.2008 6:23pm
Michael Abramowicz (mail):
Elliot123 -- I left this somewhat ambiguous because there are a variety of ways to implement normative markets. One specific proposal that I have is for predictive cost-benefit analysis, which is just what it sounds like (but you can read just that one section on the predictocracy.org web site if you want the details). I agree that most people will be out of the market prior to expiration, and if you are worried that people won't stay in the market until expiration, you can structure the market as a deliberative market, so, for example, your return will always be determined in a month or a year.

Guest 1L -- It's a very interesting point that manipulation might be particularly powerful around 0.5 (if that's the cutoff). No one can really tell the difference between 0.49 and 0.51, anyway. On the other hand, if that's true, that will provide an easy signal to reputable third parties that manipulation may well be occurring, and that this may be a good opportunity to make money from potential manipulators. But you're right that manipulation is more likely to make a difference in close cases. Of course, because these cases are close, the expected costs of error are also lower.

Robin -- I think we have a better sense now that we did a few years ago about whether the war in Iraq was a good decision. Of course, there are some issues on which policy enactment might not give us much more information. And perhaps a view for which information might become stale, and assessments worse, but given the possibilities of digital preservation, I think this is rare.
2.2.2008 9:57am