Rigor & Reproducibility in NIH grant proposals (1 of N, where is likely to be large and unpleasant)

May 10 2016 Published by under Uncategorized

NIH is changing has changed proposal guidelines rules again. Here is a link to what they have to say about the new parameters: Rigor  & Reproducibility.

How do I know I'm getting old? I look at this page and I sigh deeply and wonder how much longer I want to do this. Then I start thinking about ideas for projects and I get charged up again. So, I'm gonna dive into this one, and see what makes sense to me. I've not read anything else about this yet, so this is mostly my own take on it.

First off, it would be very easy to do a cynical parody page of this introduction:

Scientific rigor and transparency in conducting biomedical research is key to the successful application of knowledge toward improving health outcomes. The information provided on this website is designed to assist the extramural community in addressing rigor and transparency in NIH grant applications and progress reports.  

I mean, for pete's sake, who of us doesn't want to be rigorous and transparent? But there are goals, blah blah blah, NIH has the highest standards, blah blah blah. That's not what we need. We need: what is changed in the proposals? What kind of wording needs to be added? Of course, its wording, because of course, we have always been rigorous and transparent. We now just need to make sure that the reviewers perceive our inherent rigor and transparency. This. Is. Not. A. Parody. I really do want to figure it out and write something meaningful for you, (and learn it for myself). Right now this applies to research grants (R's)and mentored career development (most Ks)awards. Institutional Training Grants (T32), Institutional Career Development (K12/KL2), and Individual Fellowships (F-awards) will come "no sooner than 2017".

There are four main areas or foci:

  1. The scientific premise of the proposed research
  2. Rigorous experimental design for robust and unbiased results
  3. Consideration of relevant biological variables
  4. Authentication of key biological and/or chemical resources

Here's the pretty picture: (and a link to the better resolution pdf):

Infographic courtesy of Ms. Nichole Swan, Dr. Shana Spindler, and Dr. Yvette Pittman of the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD).

The thing I'm left with, tonight, as I write this post is that NIH wanted to add these concepts explicitly. But, unlike the changes in page length, or adding Innovation and Significance sections, these are not straightforward in terms of the writing of the proposal. The exception is the 4th, authentication. This is explicitly included in an extra attachment of no more than one page. From the FAQ:

Authentication of key resources will be addressed in a separate attachment and will not be scored.

Not being scored is important, as it means do not put extra science in this attachment. Doing so will only irritate the reviewer. Remember, the primary meta-rule of grantsmanship is: make the reviewer your ally and champion.

Reading through this, I found dealing with the other foci less transparent. To start, one of the questions in the FAQ is where to address these focus areas:

Where in grant applications should applicants address the four focus areas of the NIH policy on rigor and transparency?

Scientific premise, scientific rigor, and relevant biological variables such as sex should be addressed within the Research Strategy of research applications, as these elements are integral to the research plan. Since scientific premise will be reviewed and scored as part of the Significance review criterion for research grant applications, applicants should address premise as part of their corresponding Significance section in the Research Strategy. Scientific rigor and relevant biological variables will be reviewed and scored as part of the Approach review criterion.

For mentored career development award applications, all three areas (scientific premise, scientific rigor, and relevant biological variables such as sex) should be addressed in the Research Strategy and all three areas will be reviewed as part of the Research Plan.

This seems a bit vague, but basically you've got to include it everywhere.

My second favorite question is always this. My perception of "how am I supposed to do this in the page limits" is that it is a newbie question. I stand by my earlier advice about page limits.

Are the current page limits sufficient to describe rigor and transparency?

The NIH expects that rigor and transparency can be included within the existing page limits for the Research Strategy. ... [more language on if you screw with page limits, we will not review you, and you will not get funded and you will die a thousand little deaths].

Ok, it's late, I'm tired, and tomorrow is going to be a long day. I'll come back and think about the foci, and try & write about how to cope. For Sex as a Biological Variable (SABV) go see the DM.

2 responses so far

Leave a Reply