Every now and then an idea comes along that is so breathtakingly sensible, one wonders why no one ever thought of it before.
The National Results Council, a new nonprofit organization based in St. Paul, Minnesota, plans to give users of human services the same kind of comparative performance measures available to buyers of cars and CD players.
It will begin by measuring the effectiveness of vocational rehabilitation programs that try to find jobs for people with disabilities. With some 15,000 programs nationwide serving 500,000 people, this is a $3-4 billion industry, funded mostly by federal and state taxpayers. Despite all this money, roughly two-thirds of people with disabilities remain unemployed. Yet, astonishingly, there are no industry-wide measures for judging how well individual programs are doing.
"For the same amount of money, we could help 25 percent more people get jobs." |
"When you're choosing between different mutual funds, you can compare their performance in Morningstar," says Terry Etling, the CEO of the National Results Council (NRC). "But if you have a disability and want a job, you don't know which rehab programs have the best track record. And if you're a legislator trying to get the most bang from the taxpayers' buck, you don't know which programs to expand or scale down." Etling ought to know. As a former top official at the Ohio Rehabilitation Services Commission, he used to hand out money to service providers.
To fill this void, the NRC will compare every vocational rehabilitation program in the country according to two measures of performance:
- Four months after graduation, what percentage of participants have been employed for at least 30 days earning at least $180 per week? (For some kinds of programs, the earnings threshold will be lower.)
- How much does the program cost per beneficiary?
The NRC will then rank the programs according to the benefits delivered and the cost per beneficiary. Rank order will be adjusted according to labor market conditions and barriers to employment, so that programs are rewarded for serving consumers with the greatest difficulty finding jobs.
The NRC will test its methodology in a pilot study of 90 rehab programs at Goodwill Industries International. Kenneth Shaw, Goodwill's research director, says his organization has conducted internal studies of its programs, "but we think an independent third-party evaluation will have more credibility."
Shaw thinks the NRC study will help Goodwill do its job better. "We want to know if some of our programs aren't achieving industry standards, and then figure out why," he says. "We also want to identify which of our programs work best, so that we can learn from our own success, and transfer those lessons to our other operations."
That's exactly what the NRC hopes will happen throughout the rehab industry. Robert Walker, the organization's president, is a program-evaluation specialist who used to run a Minneapolis rehab center. He estimates that the industry could improve its cost-effectiveness by 20 to 25 percent, simply by bringing low performers up to the standards of the best in the business. "For the same amount of money, we could help 25 percent more people get jobs."
The big question is how consumer information will transform the industry. "Simply pointing to models of excellence isn't enough," says Chris Olander, an NRC board member. "We have to change incentives." Olander is the executive director of New York City's JM Foundation, which conducted a seven-year search in the 1980s to identify the nation's most effective vocational rehab programs. "Guess what? The winners were exemplary but nobody cared. Government and private funders didn't reward the best performers, or punish the worst ones. That's because we don't pay for outcomes in this industry, we pay for quantity of services. We have to start paying for results."
After vocational rehab, the NRC's next target for evaluation will be the larger world of employment and training--a $20 billion industry. It is also considering programs for alcohol and drug treatment, mental health, and teen-pregnancy prevention. One issue the NRC may be able to resolve is whether faith-based alcohol and drug addiction programs have much higher success rates than medical therapies or secular counseling programs.
The NRC hopes it can reach five audiences with its information: consumers who can make more informed decisions; service providers who can compare their results with similar programs; government officials who can better direct tax dollars for maximum benefit; researchers and trainers who can help improve programs; and private funders who can invest their charitable contributions for greater impact.
"Our mission," says president Robert Walker, "is to help human services achieve higher success rates while reducing costs and serving more people."
National Results Council, 2677 North Innsbruck Dr., Suite E, St. Paul, Minn. 55112. Tel.: 612-604-0366, fax: 612-604-0367.
