Could the popular wisdom about the FDA’s sloth and risk-aversion be wrong? A recent, well publicized study of the approval process for oncology drugs in Europe and the United States purportedly found that the FDA performed significantly better than its counterpart across the pond, the European Medicines Agency (EMA). Can this be true? Well, you can’t conclude that from this study: Methodological problems and possible sources of author bias provide reasons to doubt its conclusions.
The study, published in the journal Health Affairs, was done by the Friends of Cancer Research, a self-described think tank and advocacy organization. It found that between 2003 and 2010, the FDA approved 32 new cancer drugs versus 26 by the EMA. The FDA supposedly both approved more cancer drugs and did so more quickly: The FDA approval process averaged 182 days while the EMA’s averaged 350 days. According to Ellen Sigal, the chairman and founder of Friends of Cancer Research, the FDA’s regulatory delays and intransigence toward industry are nothing more than an urban legend.
Illustration by Barbara Kelley
I beg to differ. The only legends in evidence are the conclusions of the study itself. Several things are disturbing about the methodology and the possible sources of bias or conflict of interest on the part of the authors. These were not mentioned in either a disclaimer in the Health Affairs article or in the media’s coverage of it.
First, the timing and location (for example, Europe vs. the United States) of drug approvals depends in large part on where and when drug companies decide to submit their applications for approval, how aggressive they are, and whether the drug has been previously approved elsewhere. The quest for approvals is not a race from the same starting gate. Of the 25 drugs in the study that were approved in both the U.S. and Europe, just two were submitted to European regulators first—one by a mere 11 days, the other by a single day. By contrast, most applications were submitted to the FDA several months or in some cases two or three years earlier.
Second, not a single media report mentioned that the FDA often uses various tricks to "stop the clock" or even delay the start of the clock by "refusing to file" an application for marketing approval that it has received. (The FDA's user fee authority requires the agency to demonstrate that it meets strictly defined approval timelines, creating a potent incentive to "game" the official review times.) Thus, the agency's "review times" often have little correlation to the calendar. The Health Affairs article itself cites as one of its methodological limitations that it accessed only "official review times." The results would have been more meaningful if the authors had reported the actual number of calendar days from the date of the regulators' initial receipt of the application until the date of approval.
Risk-averse and capricious, the FDA makes drug development extraordinarily difficult.
Third, when deciding on drug approvals, European regulators may take into consideration cost and cost-effectiveness (including comparisons to other available therapies), whereas the FDA cannot. In other words, European regulators sometimes use the review that is ostensibly for a drug's safety and efficacy as a first screen for cost-effectiveness, and sometimes they delay or deny approval on those grounds.
Fourth, neither the extensive coverage of the Health Affairs study nor the article itself disclosed that two of the authors of the study, Ellen Sigal and Samantha Roberts, have close ties to the FDA. According to her online biography, Sigal is "co-chairman of the board of directors of the Reagan-Udall Foundation, a partnership designed to modernize medical product development, accelerate innovation and enhance product safety in collaboration with the U.S. Food and Drug Administration." Prior to joining Friends of Cancer Research, Roberts was a post-doctoral fellow in the FDA's Center for Biologics Evaluation and Research. The website of the Friends of Cancer Research—the professional home of all three of the authors of the study—lists one of its activities as "FDA Advocacy." Under that heading is the claim that the FDA is "responsible for advancing the public health by helping speed innovations that make medicines and foods more effective, safer and more affordable."
Few scholars who study the FDA would agree that the agency has promoted innovation. Excessively risk-averse, capricious, and at times even hostile, FDA regulation has made drug development extraordinarily difficult in recent years. Bringing a new drug to market now requires on average 12 to 15 years and costs more than $1.4 billion—in no small part because the average length of a clinical trial increased 70 percent between 1999 and 2006. During the four-year period between 1996 and 1999, the FDA approved 176 new medicines; during the most recent four-year period from 2007 to 2010, the number fell to 88, a decline of 50 percent. That trend is destined to continue because in 2010 the number of applications for approval of new drugs was the lowest in decades. But perhaps the most ominous statistic of all is that drug manufacturers recoup their Research and Development costs for only one in five approved drugs. This fraction is likely to decrease further as the FDA demands ever-larger, ever-more expensive clinical trials to accumulate data on the safety of drugs in development.
Policy research is one thing. Advocacy is another.
Expressing the industry's frustration at the FDA's capriciousness and intransigence, Fred Hassan, the former CEO of drug company Schering-Plough, said of the regulatory climate: "What will it take to get new drugs approved? The point is, we don't know." Kenneth Kaitin, director of the Tufts Center for the Study of Drug Development, described the obstructionist culture at the FDA as having caused it to become viewed as "an agency that is supposed to keep unsafe drugs off the market, not to speed access to life-saving drugs."
The abstract of the Friends of Cancer Research study reveals what may be the real agenda behind the report: "Our findings reinforce the need for strong financial and public support of the Food and Drug Administration..." That certainly fulfills the Friends’ mandate for “FDA advocacy”: Undoubtedly we will see this study used to urge Congress to increase FDA’s appropriations and to support renewal of the agency’s user fee legislation when it comes up for reauthorization next year.
In fact, many observers of the FDA feel that regulators could do far more with far less and that the agency has become an obstacle to pharmaceutical and medical device innovation. Instead of concentrating on getting more drugs through the pipeline to patients, the FDA has focused on the kinds of compliance actions that punish drug companies and their executives but please activists and congressional critics while offering little if any benefit to patients. The agency has been particularly hard on oncology drugs. In May of last year, the FDA rejected pixantrone for treating non-Hodgkin's lymphoma, a blood cancer that kills almost 12,000 Americans a year. The rationale for the rejection was not that the drug was not effective—in clinical trials it was a qualified success—but that the trial was not "flawlessly executed," an absurd standard.
The example of a medical device called MelaFind is another instance of the FDA’s unhelpfulness to the field of oncology. It is intended to help identify potentially deadly skin tumors called melanoma early (when they are curable with surgical removal) and in a systematic way. A sophisticated camera captures an image of a lesion, and a computer then analyzes it using an algorithm that has been developed on thousands of patients. This would be a marked improvement over the current method of a dermatologist subjectively considering factors such as lesion symmetry versus asymmetry, border irregularity, color, and diameter.
After squabbling with the company for six years, last year the FDA denied approval to MelaFind, saying that the company needed to conduct an additional large study because the one published in October 2010 didn't adequately compare the device’s abilities with those of dermatologists. But after an external advisory recommended approval and Congress held a hearing about the device, the FDA reversed course and in September, finally judged it “approvable,” which means that approval for marketing should follow.
This dithering will have delayed for more than a year a device that can lower the death rate from a cancer that kills about 8,800 Americans annually.
Regulators seem to be inexplicably cavalier about the need for timely approval of critical new drugs. During a February 2011 hearing, the FDA made it plain to drug companies that it considers the "accelerated approval" route for drugs to be too lenient. Introduced almost two decades ago, accelerated approval permits the FDA to issue what amounts to a limited, or conditional, approval of a new drug that is intended for a "serious or life-threatening disease" and for which there is an "unmet medical need." It has worked well and has saved countless lives. Ironically, a study by Tufts economist Joseph DiMasi and Duke University professor Henry Grabowski found that between 1990 and 2005, 47 percent of oncology drugs received accelerated approval compared with 13 percent for all other drug classes.
According to the Health Affairs website, the journal sends out only about 40 percent of manuscripts for external review; if the Friends of Cancer Research article was one, the journal needs to find new reviewers. If not, it needs new editors.
Policy research is one thing; advocacy is another, and the Friends of Cancer Research has blurred the line. Neither the Health Affairs editors nor the media that reported the study detected that.
