FEATURES:
Making Development Work

By Robert W. Hahn and Paul C. Tetlock

Using markets to improve performance

Our dream is a World Free of Poverty” reads the sign at the entrance to the World Bank headquarters. That’s quite a lofty goal. So how do we achieve it? The short answer is that no one is certain. The long answer is that there is a way to substantially improve on the basic model for economic development — using a new kind of market and paying for performance.

Not too long ago, foreign aid was viewed as a path to economic growth for the developing world. In some quarters, most notably the development banks and the United Nations, it still is. But there is dissension in the ranks. Scholars have been chipping away at the aid-buys-growth paradigm for over 30 years — with some going so far as to suggest that state aid could actually hurt the poorest of the poor. Over the past decade, revisionists have asserted that foreign aid can be helpful, but only if countries pursue good policies. So, if a country has good domestic economic policies and open trade, aid can help; but aid can do little in the presence of poor policies. Some scholars have questioned even this view, noting that measuring the impact of foreign aid depends on the definitions of terms like “aid,” “policies,” and “growth.”

Foreign aid, consisting of labor and capital that flow to particular countries, will tend to be good if those assets are spent wisely and bad if they are not. The real question is how to spend those assets wisely. At one level, this is a difficult question because it involves trying to get governments that may be near-sighted or corrupt to take a longer view. It asks them to think about investing in areas such as education, health, and roads instead of squandering resources on wasteful activities. Solving this problem is difficult. One can point to several success stories in getting developing countries to clean up their acts, but there are numerous failures.

The potential perverse incentives of aid are well known. Recipient-country governments that use aid productively may not receive any more. Aid bureaucracies that solve problems effectively could put themselves out of a job. These perverse incentives prevent policymakers from spending aid wisely. Furthermore, like many government programs that give out money, aid programs rarely evaluate how well the aid is actually spent. The Meltzer Commission notes, for example, that three to ten years after final disbursement, the World Bank reviews the broad policy impact of just 5 percent of its programs.

To some extent, these problems can be overcome by setting up rules for giving out aid. One such rule, currently in vogue, is that aid should be given to really poor countries that promote good policies in general. Another rule is to make sure that aid actually does what it is intended to do by paying the project implementers based on actual results. Both of these rules may make sense, but both have problems. Just giving aid to well-behaved poor countries may mean that donor countries have to write off a large part of the developing world. Paying for performance sounds great in theory, but it may be difficult to do in practice.

While problems with foreign aid are legion, the problems in developing countries are too important to be ignored. In 2000, an estimated 2.7 billion people were living on less than $2 per day. These people could potentially benefit from aid from rich countries and international institutions. The question is how to make the best use of that aid.¹

A new development model

Aid agencies want to spend their limited resources wisely, but they frequently fall short. To allocate resources to their most highly valued uses and get the maximum bang for each aid dollar spent, an agency needs to do two things. The first is to get reasonable information on the likely costs and benefits of different projects. The second is to implement projects effectively.

Assume that we can solve the information problem. We will shortly explain how to do so by making use of a new kind of market, an “information market.” For now, consider an example that illustrates one way to implement projects effectively: Suppose an aid agency is interested in getting children vaccinated to prevent the outbreak of a disease in Malawi, and suppose that the recipient government or the aid agency decides it is worth $5 for each child vaccinated. The agency can then auction off the right to administer the vaccines to the highest bidder. That bidder receives $5 for each child vaccinated where the number of children vaccinated would be measured by a third-party auditor.

This is an example of paying for performance. The government gives out the performance contract and waits to see whether the winning bidder will deliver. The winning bidder gets paid on the basis of what she delivers.

While there is much to be said for paying for performance, there is even more to be said for paying for performance when all parties have a good sense of what they are likely to get before the project gets started. In particular, such information can help an aid agency allocate its limited resources to their most highly valued uses — no small feat if it can be accomplished.

This is where information markets can help. They can provide information on the expected benefits of a project before things get underway. Here is how these markets work. Suppose a stock exchange facilitates trading in a contract that pays $0.01 for each child who will be vaccinated with the vaccination program, and suppose that the current price of that contract is $1. This price implies that market participants expect 100 children will be vaccinated if the program goes into effect (100 times $0.01 equals $1). The ultimate value of the contract is determined by the actual number of children vaccinated at the end of the program. So, if 110 children get vaccinated, then the final contract value is $1.10. This is a simple example of an information market. These markets allow informed parties to trade contracts that yield payments based on the outcome of an uncertain future event, such as the number of children who would be vaccinated if the vaccination program were auctioned off to the highest bidder.

Now, suppose the same exchange offers another contract, which pays $0.01 for each child vaccinated if there is no vaccination program in place. Further, suppose the price of that contract is $0.10, which means the market expects that 10 children will be vaccinated if the program does not go into effect. Using this market pricing information, we can estimate the benefits of the vaccination program. The market estimates that 100 children get vaccinated with the program and only 10 without it, so the program is expected to vaccinate an additional 90 children. Valued at $5 per child, the expected monetary benefits are $450 in this example.

Note that the information markets can provide a way of distinguishing total vaccinations of children from those that would likely have taken place anyway. This is a valuable feature of these markets that was not available up to this point except through reliance on so-called experts. These markets permit the aid agency and/or the host government to assess the incremental impact of a program by determining how many additional children the program is likely to vaccinate.

Implicitly, we are assuming that the market prices of the information market contracts are not affected by the government’s reliance on these prices in implementing its decisions. If traders anticipate the government’s use of the market prices, they will recognize that only programs with higher benefits will be implemented. This anticipation will alter their willingness to buy information market contracts based on policy benefits, potentially biasing the resulting market prices and the government’s decision based on these prices. Elsewhere, we propose a mechanism that deals with this concern by separating the information collection and decision tasks. Specifically, if the information market contracts do not depend on the government’s decision rule, then the market prices will be unbiased measures of benefits that can be used by the government. The idea is to have the market depend on a random decision but to use this random decision rule only infrequently. Most of the time, the government can simply implement its preferred policy, using the unbiased market prices as a guide.

We now have an estimate of the benefits of the program, which could be useful for both the host country and the aid agency — but only if the numbers tell us something meaningful.

What can we say about the quality of estimates that come from information markets? The short answer is that information markets appear to do better than experts in a number of settings. For example, Las Vegas odds and point spreads predict the outcomes of sporting events better than sports experts. The prices in Iowa political markets are more accurate than the polls in forecasting elections 451 out of 596 times. Information markets at Hewlett-Packard Labs beat official forecasts of printer sales 15 out of 16 times. Even Hollywood play-money markets perform better than four out of five columnists in predicting the Oscars.

These markets work for several reasons: First, almost anyone can participate; second, they allow a person to profit from trading contracts that accurately forecast the future — buying when the vaccination forecast is low and selling when the forecast is high can result in profits; third, the profit motive encourages people, including speculators, to look for better information all the time. So the market price reflects a lot of information from diverse sources, resulting in what James Surowiecki calls “The Wisdom of Crowds.”

Besides getting data on expected benefits, the aid agency would like to know the expected net benefits of the vaccination project. A measure of expected net benefits could be obtained by conducting a pay-for-performance auction in which contractors were invited to bid for the right to implement a program to increase vaccinations and to receive $5 for each additional child who was vaccinated. The baseline vaccination rate could be determined by the level predicted in the information market with no change in policy — ten children in the above example.

Note that there would be no bids unless at least one bidder expected to be able to increase vaccinations at a cost of less than $5 per child. The auction price should represent the difference between what the winning firm gets from producing results, in this case social benefits, and the costs of producing those results. In other words, the auction price is an estimate of social net benefits.

Suppose the auction price is $300 and the agency has decided to award the contract to the highest bidder, so long as the bid suggests there are net benefits. We now have a measure of expected benefits and expected net benefits. With these measures, we can also estimate the cost to the development agency and/or state footing the bill. The payout, based on expected benefits, is $450. The expected net benefits based on the auction is $300. Taking the difference yields the expected cost of the project to the agency — $150.

Now we have three pieces of information that we did not have before: an estimate of benefits from the vaccine program, an estimate of net benefits, and an estimate of costs to the agency. This information is critical for the agency in making a decision on whether to fund the project. Even if these estimates turn out not to be perfect, they are generally better than experts’ estimates.

Furthermore, the information on the vaccine program is not just available to the agency or the host government. It is available to everyone. That means the government can use it to choose wisely, potential bidders can base their bids on better information, and others can use it to assess whether the government’s proposed policy is likely to do what it claims. The information from these markets is likely to promote greater openness and accountability.

Information markets have another advantage in the context of development: They can help the winning firm with project financing, thereby encouraging competition in an area where ventures are often very risky. If the winning bidder for the vaccine project sells some information contracts to raise money, it can both reduce its risk and cover some of the costs of the project.

The vaccine example was based on a classic model of aid that comes from a state-sponsored institution, like the World Bank, the United Nations, or the U.S. Agency for International Development (usaid). In some cases, the cost of the project may be split between the recipient country and the aid donor. Note, however, that the example could just as easily be applied to the private sector or foundations.

For example, suppose that the Gates Foundation were considering offering a performance contract that would give $1,000 per reduction in hiv infections in sub-Saharan Africa before 2010. The foundation could go through exactly the same exercise as we did for vaccines. This would yield information on the likely benefits from the project in terms of reduced infection rates, the cost to the foundation of paying for results, and the cost to the firm or non-governmental organization of implementing the project. It could then decide whether this project is worth doing in comparison with other worthy social projects.

We have just addressed the two big problems that confront all decision makers who want to give out aid. Information markets can provide the aid agency and the host government with information about the likely effects from decision alternatives. And performance-based contracts can ensure that the contractor is paid for what she actually delivers. Now, let us consider how these ideas can be applied to a broad range of development problems.

Priorities: the Copenhagen Consensus

To illustrate the power of performance-based policy (pbp) in setting priorities, consider the recently completed “Copenhagen Consensus.” This was a high-powered attempt at prioritizing solutions to the world’s most pressing problems. In May 2004, a group of eight distinguished economists, including three Nobel laureates, assembled in Copenhagen to see whether they could achieve consensus on the best ways to meet the biggest challenges. To make the problem interesting, they assumed that governments had an additional $50 billion to spend.

The group was able to rank 17 social investments in four categories ranging from bad to very good. The very-good category included investments in controlling hiv/aids and malaria, reducing malnutrition, and promoting free trade. The bad category included investments in slowing climate change and employment in guest-worker programs.

The experts based their rankings on their estimates of economic and social net benefits from different projects. To develop these estimates, they relied on their collective wisdom, papers done by other experts, and criticisms of those papers done by yet other experts. There is nothing wrong with that approach. It may even be the best approach if one is forced to rely on experts.

But there are two critical problems with the expert model adopted by the Copenhagen Consensus, both of which could be addressed by properly designed information markets. First, the experts have access only to a subset of the information available to the potential traders in information markets. The likelihood of success for many of the proposed policy interventions depends on information privately held by consumers, businesses, nongovernmental organizations, and other interested parties. It is virtually impossible for a group of experts to replicate the information aggregation abilities of market prices — a point that Hayek made over half a century ago in his critique of central planning.

Second, the Copenhagen Consensus experts had no financial incentive to make accurate estimates. Although we do not dispute the motives of these experts, we caution against relying on experts for advice when it is costless for them to speak from their hearts and not from their heads. This may not even be a conscious, malicious act. There is considerable psychological evidence showing that experts’ predictions are subject to cognitive biases and perform poorly relative to simple statistical models. By contrast, information markets offer powerful financial incentives to overcome these biases or at least repress them. Experts and others bold enough to bet on their personal beliefs or preferences would incur large costs from inaccuracy.

We examined all of the social policies ranked by the Copenhagen experts and found that information market contracts could help provide guidance on each one. For example, one policy proposes an intervention that could slow the spread of hiv. One could use information markets to estimate the effect of the policy intervention and then decide whether it was worth paying a certain amount for each infection reduced below the business-as-usual scenario. If the performance contract for reducing the spread of hiv were implemented, it would be important to use statistics from reliable sources to verify performance. So, for example, in the case of the spread of aids, one could use statistics from the World Health Organization.

This brief analysis of the Copenhagen Consensus suggests there is a great deal of knowledge to be gained from using information markets to help experts reach decisions. We suggest the following policy experiment: Let the heads of state of leading developed countries commit a modest amount of resources to making contingent payments for one or two problems that would be designated by a select group. One possible role for experts would be to help monetize the benefits and design the contracts for the information markets. The experts could also consider policy suggestions from the public at large, because information markets provide a low-cost method for evaluating different policy alternatives. Following our suggested pbp framework, governments could implement the information markets and auction the rights to the benefits from specific policies.

The experiment could be evaluated by assessing how the prices of the information market contracts change over time as the policy proposals are implemented. If the markets are functioning well, prices will follow an unpredictable path, implying that the original estimates of benefits were reasonable. It will also be important to assess whether the firms implementing the projects realize excessive profits. Finally, if the information markets forecast realized benefits and net benefits well, governments should consider running the experiment on a larger scale.

Rethinking existing development efforts

In september 2000, the un issued the Millennium Declaration, which contained the Millennium Development Goals (mdg). The declaration identifies eight broad social goals, including eradicating extreme poverty and hunger, reducing child mortality, promoting gender equality and empowering women, combating hiv and other diseases, and ensuring environmental sustainability.

The goals have received widespread support. All 191 current un member states have agreed to try to achieve the goals by the year 2015, using 1990 as a reference year. The World Bank has signed on and displays all of the goals prominently in the lobby of its Washington, D.C., headquarters.

Several organizations are allocating considerable resources to achieving the mdg. In 2003, the World Bank spent $18.5 billion and worked in more than 100 developing countries. In late 2002, usaid announced that it would begin “monitoring and tracking all of its development assessment through the lens of the Millennium Development Goals.” In 2003, usaid provided $14.2 billion in assistance.

One problem with the goals, recognized by the un, is that they lack specificity. What does it mean, for example, to eradicate extreme poverty? To make the goals operational, the un published targets associated with each one and indicators associated with each target. There are 18 targets that provide verifiable measures of achievement. There are also 48 indicators that measure progress toward the targets. Under the goal of eradicating poverty, for example, there is a target of halving the fraction of people who earn less than $1 per day between 1990 and 2015. One of the indicators for this target is the proportion of the population earning below $1 per day, using an exchange rate based on purchasing power parity. The United Nations keeps data on how well individual countries are progressing on these indicators.

While the approach taken by the un is ambitious, it has two big problems. First, very little attention has been given to setting feasible goals that could maximize net benefits. Second, very little attention has been given to implementing policies in the most effective manner.

It should come as no surprise, then, that even the agencies charged with helping to achieve the goals, such as the World Bank, suggest that the mdg may not be achieved unless considerably more resources are devoted to the task. A 2002 World Bank study estimated that the world would require an additional $40 billion to $70 billion of development assistance per year to meet the mdg by 2015. And while progress has been made in some parts of the world, such as East Asia, sub-Saharan Africa is lagging far behind.

At this point, the mdg represent little more than a wish list specifying what some well-intentioned practitioners would like to see happen. The goal setters do not appear to have paid significant attention to the benefits and costs of different options before setting goals; nor does it appear that the goal setters paid sufficient attention to real budget constraints so that they could provide a realistic assessment of the feasibility of meeting the goals. It also does not appear that the goal setters have given much serious thought to putting proper incentives in place to assure that maximum benefits will be achieved for a given level of expenditures. Instead, hundreds of countries and organizations have signed on to support the goals without any clear rewards if they are reached or penalties if they are not.

It is almost a certainty that the un or some other agency will go through a similar goal-setting exercise in the near future. Thus, it is worth asking how the process could be improved and, in particular, how performance-based policy could help. The answer is that pbp could help both in setting broad goals and in implementing them, provided that there is some concrete way to measure progress.

At the top level, performance-based policy could help with establishing priorities in the same way that we suggested for the Copenhagen Consensus. Combining pbp with information markets could provide information on the costs and benefits of different alternatives. Specifically, markets could provide estimates of the costs, benefits, and expected results of different development projects. In the context of the vaccine example considered above, the agency may want to know the difference in the number of children vaccinated if it pays $3 per vaccine instead of $5. The agency may also want to compare the likely results of different plans, such as one that pays $100 per reduction in infant mortality versus a vaccine program that pays $5 per vaccination.

Armed with such information, an agency like the un could make reasonable decisions about allocating limited resources to their most highly valued uses. It would do so by comparing the effectiveness of different programs, based on the market’s assessment of the expected impact of specific programs. The un would get well-deserved credit (or blame) for actions that directly result from its interventions. In this way, pbp encourages accountability. It also encourages openness, because the information gained in evaluating the effectiveness of projects and paying for results could be made public.

The same kind of approach could be applied to domestic foreign aid programs. The U.S. is currently engaged in an exercise that could be tailor-made for performance-based policy. In January 2004, the Congress created the Millennium Challenge Account (mca) as a vehicle to provide more targeted aid to developing countries. The aim of the mca is to help developing countries that satisfy certain criteria meet specific goals. Congress appropriated $1 billion for the mca for 2004 and has requested $2.5 billion for 2005.

Paul Applegarth, the ceo of the U.S. Millenium Challenge Corporation, said the U.S. “will enter into a compact with mca countries that defines responsibilities. Each compact will include clearly defined objectives, outcomes and intermediate benchmarks. Monitoring and evaluation will be built in from the start and be ongoing throughout the program.”

While the mca’s focus on performance is laudable, we are concerned that this effort could get bogged down in unnecessary paperwork and bureaucracy. Countries wishing to receive aid must submit detailed project proposals that explain the financing required and the mechanisms for evaluation. Unfortunately, a proposal may give policymakers little information about a project’s true costs and benefits. If projects cost more or yield fewer benefits than countries expect, the mca will have wasted money.

One clear alternative to the mca is to pay for results using pbp without introducing all of the complexities of the country eligibility requirements. For example, a country’s eligibility for aid is based on whether it rules justly, invests in people, and encourages economic freedom. However, the countries that fail to meet the requirements may need aid the most.

Suppose a pbp framework indicates that the net benefits of vaccinating 500,000 children in Ecuador are much greater than those from an irrigation project in Yugoslavia. In this case, the U.S. may want to give a contract to the vaccination company rather than the irrigation company, irrespective of which government meets the basic qualifications. Because performance-based policy is somewhat insulated from government corruption and misuse, the mca’s complex qualification process would be unnecessary. pbp would allow the U.S. to aid countries with “bad policies” and still get good results. The key point is that performance-based policy allows the donor to target aid to its highest-valued uses without imposing conditionality.

Some experts in the development field argue that performance-based policies are not likely to be helpful because policymakers already know which projects are most valuable to society. These critics view the problem in terms of government corruption and political instability. Our view is that this is an empirical question that can be answered only by experimenting with the performance-based policy mechanism. In any case, because the performance-based policy framework increases accountability and transparency, it may prove to be part of the solution to the corruption problem as well.

It is unclear in many situations which policies are best for developing nations. Thus, investing in mechanisms that provide better information upfront could pay handsome dividends.

Transition to performance

To move to a pbp paradigm for development, the government should reduce regulatory barriers to the use of information markets. Interested parties should build prototypes to determine what really works. Finally, more attention needs to be paid to how this new approach will affect various interest groups.

There is already a lot of activity in the area of information markets. Professors at the University of Iowa pioneered the use of these markets to help forecast elections in the late 1980s. A Web site called Tradesports.com has information markets for sporting events, financial indices, political events, and legal outcomes. And Goldman Sachs supports an exchange that hosts auctions for derivatives based on the value of economic indices. Furthermore, firms are approaching regulators in Washington to find out whether they can set up other markets. Hurdles have arisen because information markets are regulated under “Internet gambling” laws.

To encourage the use of information markets for improving policy, we strongly recommend that regulators distinguish between markets for gambling — like on-line poker games — and information markets aimed at improving, say, economic development. While there are clearly gray areas, regulators could use a number of criteria for deciding whether contracts should be allowed, including whether the contract provides useful information on a policy objective and whether it would allow interested parties to spread risk more efficiently. Thus, a market used to predict the number of vaccinations that would result from a vaccine program should be permitted without question.

The next step is to develop prototypes to learn where the approach works best. This could be done by foundations, developing governments, or places like the World Bank. There is no magic formula for ushering in a new paradigm, but there may be some useful rules of thumb. We believe it is sensible to start on a small scale to refine the pbp model before deciding to ramp up. If pbp is shown to work well, then the government should focus on projects that will have a substantial impact.

The limitations of information markets also need to be acknowledged. To work well, the pbp process needs to be relatively free of corruption. While we recognize that corruption is a serious problem in some developing countries, we think that its potentially adverse impacts can be managed through a judicious choice of project selection and pbp design. If, for example, the host country limits parties that are eligible to implement the performance contract, this will raise the costs of implementing the project. pbp could still work in this case, but it would be more expensive than if the project were bid on competitively.

A second problem is that these markets require a reasonable number of motivated buyers and sellers. Liquidity in these markets cannot be assumed. The government may want to explore ways of subsidizing liquidity if it is interested in addressing a particular problem.

Performance-based policy also cannot work if the results of a development project cannot be defined or measured. While some projects with unquantifiable benefits and costs may be worthwhile, we think it is important for agencies charged with development to work on finding better performance measures before they embark on large development initiatives using taxpayer dollars.

Finally, moving to a performance-based policy paradigm for development is likely to create winners and losers. This system is designed to produce results by paying for those results. There may be some parts of the development community that are more comfortable with the status quo, precisely because they benefit from the current system and know how it works. To the extent that these groups can block change, they will need to be compensated in some way. Furthermore, it is important to educate the public and interest groups on why this approach can lead to better decisions.

We have argued that combining information markets with paying for performance has the potential to improve how aid is delivered. In addition to introducing an approach that yields real economic benefits, lawmakers will be able to hold bureaucrats more accountable for results. Ultimately, voters will be able to hold their elected officials more accountable for expenditures on economic development.

¹ The authors would like to thank Roger Bate, Ed Campos, Nick Eberstadt, Adam Lerrick, Robert Litan, Allan Meltzer, Roger Noll, Mary Shirley, and Scott Wallsten for helpful comments, and Katrina Kosec for excellent research assistance.