Robert E. Gropp
The academic community recognizes that peer review is essential for evaluating research. Indeed, in recent years the scientific community has vigorously defended the integrity of the peer review process and championed its use for evaluating data that underpin government actions. If rhetoric and policy proposals are any indication, federal officials also appear to have embraced the concept. The devil is in the details, however, and the details of some recent peer review proposals have many scientific organizations concerned.
In September 2003, the White House’s Office of Management and Budget (OMB) issued for public comment a draft "Information Quality Bulletin on Peer Review." The bulletin came on the heels of reports that the Bush administration misrepresented scientific data and manipulated advisory committee membership to obtain recommendations that supported administration policy (www.aibs.org/washington-watch/washington_watch_2004_06.html). The intent of the plan was to provide common peer review guidelines for federal agencies responsible for promulgating "significant" regulations. The proposal would not have affected the way in which peer review was conducted by grant programs administered by agencies such as the National Science Foundation and the National Institutes of Health. The bulletin quickly attracted attention and a clamorous response from the public—179 individuals and organizations, from citrus growers and mining companies to environmental groups and scientists, commented on the proposal.
Representative Henry Waxman (D–CA), the ranking member on the House Government Reform Committee, described the bulletin as a "wolf in sheep’s clothing," arguing that it advances a "far-reaching policy that will impede efforts to protect health and the environment." Waxman and six of his fellow members of Congress wrote to OMB detailing areas of concern with the bulletin, namely, that it was too broad—a one-size-fits-all policy—and that it inappropriately addressed conflict of interest and failed to demonstrate the need for a new federal policy. On the basis of these criticisms, the representatives asked OMB to retract or significantly revise the bulletin.
While scientists recognize the potential for peer review to ensure that government actions are based on the best available data, some, including National Academy of Sciences (NAS) president Bruce Alberts, echoed many of Waxman’s concerns about the bulletin’s prescriptive approach to peer review. The method for safeguarding against conflict of interest between a review panel member and the agency convening the panel was a primary worry for many other scientists. Individuals who received grants or contracts from the agency convening a review panel would have been excluded from serving on a government panel reviewing data that could ultimately be used to formulate regulations. Given the specialized nature of some issues, this requirement could conceivably prevent the federal government from obtaining expert scientific analysis from the most qualified researchers in a specialized area of study.
Not all in Congress share Waxman’s trepidation. Representatives James Gibbons (R–NV) and Chris Cannon (R–UT), vice chairman of the House Resources Committee and cochair of the House Sound Science Caucus, respectively, think the bulletin is a good first step but does not go far enough. For instance, Gibbons and Cannon asked OMB to ensure that scientific or technical studies containing influential information are "published or are otherwise confirmed by double-blinded research." Noting that publication of results in scientific literature is generally a good indication of quality, the representatives contend that "where studies cannot be published because of fashionable objections," agencies must obtain independent confirmation before issuing a regulation. Critics of the plan consider this proposal a recipe for "paralysis through analysis."
To the relief of scientific organizations, OMB heeded the recommendation of scientists that the bulletin be reconsidered and revised in a collaborative but transparent process with the scientific community. OMB worked with NAS to convene public meetings on the topic of peer review in the public policy process. Following these meetings, OMB revised the guidelines to reflect the peer review procedures used by NAS. Though the final version of the bulletin has yet to be released, OMB acknowledged that the conflict-of-interest provision initially proposed was unnecessarily restrictive; OMB also indicated a willingness to provide agencies with more latitude to select the most qualified scientists.
OMB is reviewing comments received on the second draft of the bulletin. It is uncertain when a final version will be released. Certainly, with keen public and business interest in how federal regulations are set, the issue of peer review in the federal decisionmaking process will remain center stage as long as issues such as stem cell research or conservation of biodiversity retain public interest. Thus, on 15 July 2004, the Ecological Society of America and 12 other scientific societies, including AIBS, issued a statement of principles on peer review in the public policy process (www.aibs.org/position-statements/040715_scientific_peer_.html). These societies hope that this document will assist policymakers in evaluating future peer review policy initiatives.
Robert E. Gropp (email: ) is AIBS’s senior public policy representative.