"What news from the sea?"

The fish replied: "I have a lot to say, but my mouth is full of water." - Armenian proverb

The San Diego, California shoreline. Credit: Frank McKenna

A small semi-transparent triangle for visual interest
Science Marches On

News & Events

Explore the most recent news about AIBS's initiatives, programs, resources, and events.

Bullet peer-review · Aug 24, 2018

Validity of Peer Review Evaluated by AIBS

The American Institute of Biological Sciences (AIBS) has just published a literature review summarizing results of empirical tests on the validity of peer review decisions using impact measures of investigator output.

These results were published as part of a research topic for the journal Frontiers in Research Metrics and Analytics. Of all of the results, less than half were US based and the majority focused on bibliometric measurements of applicant/project success. Only 25% used more than one type of metric.

Nevertheless, the vast majority of studies provided evidence for at least some level of predictive validity of review decisions, although many detected sizable type I and II errors. Moreover, many of the observed effects were small and several studies suggest a coarse power to discriminate poor proposals from better ones, but not amongst the top tier proposals or applicants.

The article can be accessed here.