July 14, 2011
Dr. Ray M. Bowen, Chairman
National Science Board
4201 Wilson Boulevard
Arlington, VA 22230
Dr. Subra Suresh, Director
National Science Foundation
4201 Wilson Boulevard
Arlington, VA 22230
Dear Dr. Bowen and Dr. Suresh,
Thank you for the opportunity to provide input on the proposed merit review criteria. The American Institute of Biological Sciences (AIBS) commends your leadership in working to clarify and improve the merit review process.
AIBS is a nonprofit scientific association dedicated to advancing biological research and education for the welfare of society. Founded in 1947 as a part of the National Academy of Sciences, AIBS became an independent, member-governed organization in the 1950s. AIBS is sustained by a robust membership of individual biologists and nearly 200 professional societies and scientific organizations with a combined individual membership exceeding 250,000.
AIBS strongly supports the National Science Board's decision to retain and revise the existing merit review criteria. The National Science Foundation (NSF) has been well served by considering both the intellectual merit and broader impacts of the research that it funds.
The addition of overarching principles to ground the merit review criteria is a welcome addition. This should provide a solid framework upon which review panel members may evaluate proposals. The addition of national goals to be addressed by NSF-supported projects is admirable, and should help to elucidate the agency's important role in the nation with respect to science, education, and economic growth. We encourage you to add several additional national goals: enhancing environmental quality, protecting biological and cultural diversity, improving the effectiveness of social service systems, and enhancing international cooperation in research and education. Moreover, we encourage NSF to provide clarification about the types of activities that would fulfill the proposed national goals of increased economic competitiveness and increased national security.
We respectfully suggest refinement of the fourth merit review principle, which requires ongoing assessment of the review criteria. Although NSF should periodically re-evaluate the merit review criteria, it could be burdensome to potential grantees and to the agency if the criteria change too frequently. A moving target would make it more difficult for applicants to prepare grant proposals that appropriately meet the review criteria. This principle should be clarified to make clear the timeframe on which evaluation will occur.
The proposed broader impacts review criterion is an improvement over the existing criterion. We appreciate that the revised criterion makes it clear that each grantee is not expected to address all aspects of broader impacts (e.g. education, training, broadening participation, economic competitiveness, etc.). Instead, applicants are encouraged to focus their efforts on one or more national goals. This narrower set of expectations should guide applicants to choose activities that are appropriate in scope, given the applicant's experience and available resources. The revised questions should also help reviewers to better assess the ability of applicants to meet the goals stated in the proposal.
The broader impacts review criterion is an important aspect of NSF's efforts to improve science, technology, engineering, and mathematics (STEM) education; broaden participation by women, persons with disabilities, and underrepresented minorities; and increase public scientific literacy. In order to ensure that the activity is achieving its goals, we encourage NSF to formulate and provide best practices to grantees regarding methods to evaluate the success of broader impact activities. Additionally, NSF should work with professional scientific societies, grantees, and research institutions to facilitate communication between the scientists and the education researchers that it funds. This would aid in the dissemination of the outcomes of education research supported by NSF to researchers in other disciplines who are planning broader impact activities. The resulting dialogues should help to improve the quality of activities conducted under the auspices of the broader impacts criterion.
Thank you for your thoughtful consideration of these comments. If AIBS may be of further assistance to you on this or any other matter, please contact Dr. Robert Gropp, AIBS Director of Public Policy at 202-628-1500.
Richard O'Grady, Ph.D.