Common Fund Evaluation and Assessment
Assessment and Evaluation Philosophy
The NIH Common Fund believes assessment and evaluation are essential to planning, improving, and maintaining the quality of our science programs. They can ensure accountability of programs and build a case for future science investments. These webpages provide access to formal evaluations, bibliometric analyses, and ongoing programmatic assessment and describe our assessment approach and methodologies.
To obtain a 360-degree view of complex systems, we use multiple sources of information, employ multi-platform, mixed-method designs. This includes triangulation of data from multiple sources of data gathered during program planning, implementation, and close-out. Data sources include input from science experts, NIH leadership and staff, tool and resource users, publication analysis, portfolio review, landscape analysis, site visits, program documents, and others.
Evaluation Report Library
We have included publicly available evaluations and assessments of Common Fund programs. The following list is ordered by the Common Fund program name or topic for cross-program evaluations.
Common Fund Program Evaluations
Bioinformatics and Computational Biology Program (which supported the National Centers for Biomedical Computing)
National Centers for Biomedical Computing Mid-Course Program Review Report (July 13, 2007)
An external panel assessed the status and progress of the National Centers for Biomedical Computing Initiative three years after the program started. This report presents the panel’s findings and provides guidance for the future course of the program.
Bridging Interventional Development Gaps (BrIDGs) (now housed in the National Center for Advancing Translational Sciences (NCATS))
NIH-RAID Pilot Mid-Course Review Meeting Summary (March 7, 2008) -part of Bridging Interventional Development Gaps (BrIDGs)
This review of the NIH Rapid Access to Interventional Development (RAID), an initiative of the BrIDGs program, was conducted by an external panel. The report provides the findings and recommendations that include how to improve the approach to this area of science and the management of the program.
High-Risk, High-Reward Research Program
(Four unique NIH Director’s awards are available for exceptionally creative scientists who propose highly innovative approaches with high-impact potential to major challenges in biomedical research.)
NIH Director’s New Innovator Award
Feasibility Study of an Outcome Evaluation of the National Institutes of Health’s New Innovator Award Program – Final Report (May 2011)
To determine whether an evaluation was warranted and feasible, staff interviews were conducted, literature on how innovative research has been defined and operationalized was reviewed, and a comparison group was identified. This resulted in evaluation questions, an evaluation design, a logic model, and collection of pilot data to guide a future outcomes evaluation.
Process Evaluation of the National Institutes of Health Director’s New Innovator Award Program: FY 2007–2009 – Final Report (May 6, 2011)
After reviewing the origins of the program and changes over the first three years, examining the characteristics and perceptions of applicants and external reviewers, and analyzing the scoring of the applications, the findings indicated that the program had been implemented without significant challenges. Six recommendations were made to improve the program.
National Institutes of Health Director’s New Innovator Award Outcome Evaluation Fiscal Years 2007-2009 (2016)
This outcomes evaluation examined the initial awardees of the 5-year New Innovator Award (NI) program. Two key questions were addressed: 1) Is NI research significantly more innovative, high risk, or impactful than traditionally funded NIH research; and 2) What are the impacts of NI awards on the careers of awardees compared to the career impacts of a comparable traditional NIH award?
NIH Director’s Pioneer Award
FY 2004 – 2006 NIH Director’s Pioneer Award Process Evaluation – Comprehensive Report – Final Report (January 2008)
As a pilot, the NIH Director's Pioneer Award (NDPA) continued to evolve and change structurally and conceptually. Using administrative data, interviews, and surveys, this report summarized NDPA’s design, implementation, and participation. The findings included recommendations to continue to clarify program criteria and operations; increase consistency, communication, and transparency of the selection process; and attract a diverse pool of investigators to apply to the program.
FY 2004 – 2008 NIH Director’s Pioneer Award Process Evaluation – Comprehensive Report Final Report (January 2010)
This report summarizes the process evaluations of the first 5 years of the program, highlighting changes in the program’s design and implementation and describing program participants’ perceptions. Also included are overall assessments of the program and key recommendations.
Outcome Evaluation of the National Institutes of Health (NIH) Director’s Pioneer Award (NDPA), FY2004 2005 – Final Report (July 22, 2011)
The 5-year NDPA awards represent a novel approach for supporting biomedical and behavioral research. This evaluation examined the outcomes of the first two cohorts of the NDPA. Two over-arching questions drove this evaluation: (1) Did the awardees conduct pioneering research with the funds? (2) What are the spillover effects of the program?
An Outcome Evaluation of the National Institutes of Health (NIH) Director’s Pioneer Award (NDPA) Program, FY 2004–2006 (August 2012)
Expanding on the previous outcome evaluation of the NDPA, this evaluation focused on scientific publications to answer questions: (1) To what extent does this research produce unusually high impact, (2) To what extent are the research approaches used highly innovative, (3) To what extent is this research interdisciplinary, and (4) To what extent are the Pioneers awardees collaborative?
NIH Director’s Transformative Research Award
FY 2009-2010 NIH Director’s Transformative Research Award Process Evaluation (August 2010)
For this evaluation, Transformative Research Award applicants and application reviewers were surveyed. Based on the findings, recommendations about the application and review process were made.
Interdisciplinary Research Program
Facilitating and Experiencing Interdisciplinarity in Biomedical Research – Mid-course Evaluation of the Interdisciplinary Research Consortium Program: an NIH Common Fund Program (Sept. 2011)
This mid-course program review identified facilitators and inhibitors of interdisciplinary at the project level.
Molecular Libraries and Imaging Program
NIH Roadmap Molecular Libraries and Imaging Program Mid-Course Review Meeting, December 20–21, 2006 – Executive Summary
This progress review made the recommendations to (1) focus the program on difficult or unique problems as an organizing theme to drive innovation and differentiation from drug discovery screening efforts in industry, (2) manage the program as a diversified portfolio of initiatives, and (3) reassess the program and chart the overall direction at the 5-year point.
NIH Roadmap Molecular Libraries and Imaging Program Needs Assessment Report – Final Report (January 2010)
This needs assessment examined whether the program accomplished its goals during the initial pilot phase, and gathered feedback from network users and potential users on their level of satisfaction with Molecular Libraries Program services.
Nanomedicine Program
Evaluating the Selection Processes for the NIH Roadmap Nanomedicine Initiative Nanomedicine Development Centers (2005-2006)
This evaluation examined the processes for selecting the Nanomedicine Development Centers. This included identifying which aspects of the selection process facilitated the solicitation and identification of applications best suited to meet the objectives of the program. Also, the use of Flexible Research Authority for selecting Centers was assessed.
Post-Award Management of the NIH Nanomedicine Development Centers (2008)
As an extension of the 2005-2006 evaluation of the Nanomedicine Development Centers, this evaluation focused on the effectiveness and efficiency of management and operations at the program level.
Midcourse Review of the NIH Nanomedicine Roadmap Initiative – (2009)
In 2009, an external panel of scientists was convened to examine questions related to the structure, management, and direction of the program. The panel identified successes and challenges of the NIH Nanomedicine Program and made recommendations to improve the program.
Patient-Reported Outcomes Measurement Information System (PROMIS) Program
The PROMIS Initiative and Mid-Course Review – 2007
This process evaluation was conducted by an external panel of scientists to examine if the goals were being achieved and if the program continued to be relevant and significant in relation to re-engineering the clinical research enterprise. The panel made recommendations to support the initiative’s continued relevance.
Common Fund Evaluation
NIH Council of Councils Common Fund Evaluation (2014)
The NIH Common Fund (also known as the NIH Roadmap) was established in 2004. This 2014 process evaluation examined the planning and management of the Common Fund. Two major questions were answered: (1) Are planning processes optimal for identifying program areas that meet the Common Fund criteria? (2) Are management and oversight processes optimal for achieving program goals?
Common Fund Patent Analysis
Common Fund Patent Report (2017)
The Common Fund is a unique funding entity at NIH, functioning as a “venture capital” space, where high-risk and innovative research can be supported. This led to the question, does research supported by the Common Fund generate more patents compared to the entire NIH research portfolio. Read the full patent report to see patent numbers from Common Fund programs compared to the NIH and to learn about select Common Fund patents.
