“…written by James Surowiecki about the aggregation of information in groups, resulting in decisions that, he argues, are often better than could have been made by any single member of the group.

Types of crowd wisdom
Surowiecki breaks down the advantages he sees in disorganized decisions into three main types, which he classifies as:

Cognition   –   Market judgment, which he argues can be much faster, more reliable, and less subject to political forces than the deliberations of experts, or expert committees

Coordination   –   Coordination of behavior includes optimizing the
utilization of a popular restaurant and not colliding in moving traffic flows. The book is replete with examples from experimental economics, but this section relies more on naturally occurring experiments such as pedestrians optimizing the pavement flow, or the extent of crowding in popular restaurants. He examines how common understanding within a culture allows remarkably accurate judgments about specific reactions of other members of the culture.

Cooperation   –   How groups of people can form networks of trust
without a central system controlling their behavior or directly enforcing their compliance. This section is especially pro-free market.

Four elements required to form a wise crowd
Not all crowds (groups) are wise. Consider, for example, mobs or crazed investors in a stock market bubble. Refer to Failures of crowd intelligence (below) for more examples of unwise crowds. According to Surowiecki, these key criteria separate wise crowds from irrational ones:

Diversity of opinion   –   Each person should have private information even if it’s just an eccentric interpretation of the known facts.

Independence   –   People’s opinions aren’t determined by the opinions of those around them.

Decentralization   –   People are able to specialize and draw on local knowledge.

Aggregation   –   Some mechanism exists for turning private judgments into a collective decision.

Failures of crowd intelligence
Surowiecki studies situations (such as rational bubbles) in which the crowd produces very bad judgment, and argues that in these types of situations their cognition or cooperation failed because (in one way or another) the members of the crowd were too conscious of the opinions of others and began to emulate each other and conform rather than think differently. Although he gives experimental details of crowds collectively swayed by a persuasive speaker, he says that the main reason that groups of people intellectually conform is that the system for making decisions has a systematic flaw.

Surowiecki asserts that what happens when the decision making environment is not set up to accept the crowd, is that the benefits of individual judgments and private information are lost, and that the crowd can only do as well as its smartest member, rather than perform better (as he shows is otherwise possible). Detailed case histories of such failures include:

Too homogenous   –   Surowiecki stresses the need for diversity within a crowd to ensure enough variance in approach, thought process, and private information.

Too centralized   –   The Columbia shuttle disaster, which he blames on a hierarchical NASA management bureaucracy that was totally closed to the wisdom of low-level engineers.

Too divided   –   The U.S. Intelligence community failed to prevent the September 11, 2001 attacks partly because information held by one subdivision was not accessible by another. Surowiecki’s argument is that crowds (of intelligence analysts in this case) work best when they choose for themselves what to work on and what information they need. (He cites the SARS-virus isolation as an example in which the free flow of data enabled laboratories around the world to coordinate research without a central point of control.) Recent reports indicate that the CIA is now planning a Wikipedia style information sharing network that will help the free flow of information to prevent such failures again.

Too imitative   –   Where choices are visible and made in sequence, an “information cascade” can form in which only the first few decision makers gain anything by contemplating the choices available: once this has happened it is more efficient for everyone else to simply copy those around them.

Too emotional   –   Emotional factors, such as a feeling of belonging, can lead to peer pressure, herd instinct, and in extreme cases collective hysteria.


“We analyze the extent to which simple markets can be used to aggregate disperse information into efficient forecasts of uncertain future events. Drawing together data from a range of prediction contexts, we show that market-generated forecasts are typically fairly accurate, and that they outperform most moderately sophisticated benchmarks. Carefully designed contracts can yield insight into the market’s expectations about probabilities, means and medians, and also uncertainty about these parameters. Moreover, conditional markets can effectively reveal the market’s beliefs about regression coefficients, although we still have the usual problem of disentangling correlation from causation. We discuss a number of market design issues and highlight domains in which prediction markets are most likely to be useful.”




“In 2000 Michael Foster, who ran (and still runs) the National Science Foundation quantum computing research program, convinced DARPA (Defense Advanced Research Projects Agency, the blue-sky research arm of the U.S. Defense Department) to fund research on prediction markets starting in 2001. Prediction markets are speculative markets created for the purpose of aggregating information on topics of interest. Previous field studies had found that such markets out-predict co-existing institutions regarding the weather, printer sales, movie sales, elections, and much more.

This research program was eventually named “FutureMAP”, but the first DARPA call for proposals went out under the name “Electronic Market-Based Decision Support.” This call basically said “We’ve heard this works elsewhere; show us it works for problems we care about.” The call went out in May 2001, for proposals due in August, and by December two firms had won SBIR (Small business independent research) grants. The winners were Neoteric Technologies, subcontracting to Martek and professors at the University of Iowa, and Net Exchange, founded by a Caltech professor (John Ledyard) and subcontracting to professors at George Mason University (myself and David Porter), and later to the Economist Intelligence Unit. The Net Exchange project came to be called the “Policy Analysis Market” (PAM).

The plan was for two firms to get $100K for a six month Phase I, and after which one of them would be awarded $750,000 to continue Phase II over two more years. There was also the possibility of applying to get $100,000 of funding for the six months between these phases. More money became available than initially planned, so in fall 2002 both firms were funded to continue to Phase II, and Net Exchange applied for and won interim funding. Also during 2002, the infamous John Poindexter (who I have never met) became a DARPA executive, and Foster’s FutureMAP program was placed within Poindexter’s organization, the Information Awareness Office (IAO). In December 2002, DARPA called for proposals for related research, at this point using the name FutureMAP. In summer 2003 a half dozen teams, at Penn State, Metron, ICT, GMU (including me), Sparta, and BBN, were awarded $100,000 each.

Neotek sponsored an end of phase I conference in June 2002, and showed a few demonstration markets, using their pre-existing software, on SARS and the color security threat level. When FutureMAP was cancelled, Neotek had still not identified their market topics, and had surely spent less than half of their Phase II funding. Net Exchange spent about two thirds of their Phase II funding, and the new small projects had spent little of their funding. Michael Foster had asked for, but not received, $8,000,000 more in FutureMAP funding over the next few years.

From the very start, the Net Exchange team began laboratory experiments to study the issue of price manipulatoin, as this was a widely expressed concern. Also from the start, we planned to focus on forecasting military and political instability around the world, how US policies would effect such instability, and how such instability would impact US and global aggregates of interest. The Net Exchange president, Charles Polk, named this the Policy Analysis Market (PAM). We later had to narrow our focus to a smaller region, the Mideast, because the Economist Intelligence Unit charged a high price to judge after the fact what instability had actually occurred in each nation.

We planned to cover eight nations. For each nation in each quarter of a year, we planned to have traders predict its military activity, political instability, economic growth, US military activity, and US financial involvement. In addition traders would predict US GDP, world trade, US military casualties, and western terrorist casualties, and a few to-be-determined miscellaneous items. This would require a hundred or so base markets. Most important, we wanted to let our traders predict combinations of these, such has how moving US troops out of Saudi Arabia would effect political stability there, how that would effect stability in neighboring nations, and how all that might change oil prices.

For many years before PAM, Net Exchange had specialized in combinatorial markets, where buyers and sellers can exchange complex packages of items. So from the start of PAM, we planned to see how far we could go in developing combinatorial prediction markets. In Phase I Net Exchange put together a combinatorial market similar to their previous markets, and at the end of Phase I we ran a complex simulation where a dozen students traded over a few days for real money. Unfortunately, only about a dozen trades occurred, a serious failure.

In the interim phase, the Net Exchange team prepared for and ran lab experiments comparing two new combinatorial trading mechanisms with traditional mechanism. These experiments, where six traders set 255 independent prices in five minutes, found that a combinatorial market maker was the most accurate. Phase II was mostly being spent implementing a scaleable production version of this market maker. It requires a net subsidy to traders, and so because we had budgeted $50,000 for this subsidy, individual bets were limited to a few tens of dollars.

We were concerned that we might not attract enough traders to achieve a meaningful test. While we had considered running markets within government agencies, we choose public markets due to legal problems with conditional transfers of money between agencies and the absence of a single agency strongly interested in collaborating. On May 20, 200, DARPA reported to congress on the IAO, and described FutureMAP in terms of predicting a bioweapons attack against Israel. In June 2003 we began to tell people about our webpage, and to give talks drum up interest. Charles Polk created the PAM website, wherein in the faint background sample screen, he included as colorful examples of miscellaneous items the assassination of Arafat, and a missile attack from North Korea.

In the summer of 2003, the Senate but not the House had cancelled IAO funding, which included all FutureMAP funding, because of privacy concerns with another IAO project, “Total Information Awareness.” Due to this funding uncertainty, when the media storm hit our plans were to start on September 1 with one hundred testers to which we had each given $100. Registration to be one of those testers was to open August 1, with public trading to being January 1, 2004.

The media storm hit on July 28, 2003, when two senators complained that we were planning to let people bet on terrorist attacks. The next morning the secretary of defense announced that FutureMAP was cancelled. In the intervening day, no one from Congress asked us if the accusations were correct, or if the more offending aspects could be cut from the project. DARPA said nothing. The next day, John Poindexter resigned, and two months later all IAO research was cancelled.”


Futarchy: Vote Values, But Bet Beliefs
by Robin Hanson, August 2000

This short “manifesto” describes a new form of government. In “futarchy,” we would vote on values, but bet on beliefs. Elected representatives would formally define and manage an after-the-fact measurement of national welfare, while market speculators would say which policies they expect to raise national welfare.

Democracy seems better than autocracy (i.e., kings and dictators), but it still has problems. There are today vast differences in wealth among nations, and we can not attribute most of these differences to either natural resources or human abilities. Instead, much of the difference seems to be that the poor nations (many of which are
democracies) are those that more often adopted dumb policies, policies which hurt most everyone in the nation. And even rich nations frequently adopt such policies.

These policies are not just dumb in retrospect; typically there were people who understood a lot about such policies and who had good reasons to disapprove of them beforehand. It seems hard to imagine such policies being adopted nearly as often if everyone knew what such “experts” knew about their consequences. Thus familiar forms of government seem to frequently fail by ignoring the advice of relevant experts (i.e., people who know relevant things).

Would some other form of government more consistently listen to relevant experts? Even if we could identify the current experts, we could not just put them in charge. They might then do what is good for them rather than what is good for the rest of us, and soon after they came to power they would no longer be the relevant experts. Similar problems result from giving them an official advisory role.

“Futarchy” is an as yet untried form of government intended to address such problems. In futarchy, democracy would continue to say what we want, but betting markets would now say how to get it. That is, elected representatives would formally define and manage an after-the-fact measurement of national welfare, while market speculators would say which policies they expect to raise national welfare. The basic rule of government would be:

When a betting market clearly estimates that a proposed policy would increase expected national welfare, that proposal becomes law.

Futarchy is intended to be ideologically neutral; it could result in anything from an extreme socialism to an extreme minarchy, depending on what voters say they want, and on what speculators think would get it for them.

Futarchy seems promising if we accept the following three assumptions:

* Democracies fail largely by not aggregating available information.
* It is not that hard to tell rich happy nations from poor miserable ones.
* Betting markets are our best known institution for aggregating information.

GDP is today the most common measure of national wealth. It seems hard for frequent travelers to escape the impression that people in high GDP nations tend to be richer and better off than those in low GDP nations. Economists thus tend to be willing to recommend policies that macroeconomic data suggest are causally related to increasing GDP. It seems that it is not that hard to, after the fact, tell rich satisfied nations from poor miserable ones. GDP may be good enough, and with the full attention of our elected representatives, we should be able to do even better, such as by including happiness, inequality, health, leisure, and environment measures.

If we can measure how rich nations are, we can use such measurements to settle bets. This is good because betting markets, and speculative markets more generally, seem to do very well at aggregating information. To have a say in a speculative market, you have to “put your money where your mouth is.” Those who know they are not relevant experts shut up, and those who do not know this eventually lose their money, and then shut up. Speculative markets in essence offer to pay anyone who sees a bias in current market prices to come and correct that bias.

Speculative market estimates are not perfect. There seems to be a long-shot bias when there are high transaction costs, and perhaps also excess volatility in long term aggregate price movements. But such markets seem to do very well when compared to other institutions. For example, racetrack market odds improve on the predictions of racetrack experts, Florida orange juice commodity futures improve on government weather forecasts, betting markets beat opinion polls at predicting U.S. election results, and betting markets consistently beat Hewlett Packard official forecasts at predicting Hewlett Packard printer sales. In general, it is hard to find information that is not embodied in market prices.

A betting market can estimate whether a proposed policy would increase national welfare by comparing two conditional estimates: national welfare conditional on adopting the proposed policy, and national welfare conditional on not adopting the proposed policy. Betting markets can produce conditional estimates several ways, such as via “called-off bets,” i.e., bets that are called off if a condition is not met.

Egyptian election workers count votes at a polling station on June 17, in Cairo.

Leave a Reply