Monday, September 27, 2010

Establishing Credibility via Acknowledging Uncertainties and Ignorance

On Saturday, the NYT had an interesting analysis of a new approach to presenting uncertainties to the public:
During the Bush administration, the Food and Drug Administration was mostly a place of black-and-white decisions. Drugs were approved for sale or they were not, and the agency’s staff was expected to publicly support those decisions.

But as Thursday’s landmark decision on the controversial diabetes medicine Avandia makes clear, things have changed under the Obama administration. Certainty, staff unanimity and even the approval status of big-selling medicines are no longer so black and white.

Presented with what seemed to be a choice between keeping Avandia on the market or withdrawing it, the Obama administration decided on an unusual middle path — allowing sales, but with tight restrictions. Even more unusually, the agency admitted that many of its top scientists disagreed, some passionately. Competing memorandums were posted immediately on the agency’s Web site.

And the agency’s three top officials co-wrote a highly unusual explanation of their action in The New England Journal of Medicine.

Some of these changes have been in the works for years, but they have accelerated under the Obama administration, driven by increasingly sophisticated measures of drug safety and growing skepticism about whether the F.D.A. is making the right decisions and making them appropriately.

“I think that F.D.A.’s credibility really depends on being able to explain its decisions well,” said Dr. Joshua Sharfstein, F.D.A.’s principal deputy commissioner. “We can’t expect people to think that F.D.A. has decided, therefore it’s the right answer.”
Being open about uncertainties, the contested views of experts and how these factors play into decision making can help to establish the legitimacy of FDA decision making.  Ironically enough, more knowledge can actually mean greater uncertainties:
“In the past, we would approve the drug after a couple of efficacy trials and that was it,” Dr. Janet Woodcock, chief of the F.D.A.’s drug center, said in an interview. “We didn’t know too much more about the drug. It was simpler.”

Now, sophisticated analyses present the F.D.A. with a complex picture. “It’s good for public health that we’re learning more, but it creates a more complex environment in which to regulate,” Dr. Woodcock said.

It is an environment in which top agency officials are in some ways at sea. The agency has no systems or standards to follow in deciding which studies deserve their attention or should lead to changes in a drug’s status. And since new tests are being created constantly, creating such a standard would be an ever-evolving process.

Dr. Lynn Goldman, dean of the School of Public Health and Health Services at George Washington University, said the F.D.A. was being forced to become more comfortable with studies done in academic rather than regulatory settings. “They have to get used to a less controlled environment,” Dr. Goldman said.
Effective science arbitration is a complex process that goes well beyond "just the facts" or even establishing a consensus view.