Rigor & Reproducibility: Scientific Premise (2/N)

May 11 2016 Published by under Uncategorized

We're back thinking about Rigor & Reproducibility, as perceived by the NIH. Firstly, there are lots of online resources from the NIH,  including the reviewer guide (a pdf download) for these issues. It is always worth taking a look at what NIH tells its reviewers. Much is a repeat of the information that they have on the various web pages, but some of it is not. When writing a proposal, taking time to try and think in terms of a critical reviewer can make a difference.

So here's new concern number 1: The scientific premise of the proposed research. Premise, in logic, is the stuff that comes before, the basis of what comes next. The next, in this case, are the hypotheses one proposes to test, or the questions one proposes to answer. It is the basis of the argument, of the undertaking. Here's the NIH wording on this:

The scientific premise for an application is the research that is used to form the basis for the proposed research question(s). NIH expects applicants to describe the general strengths and weaknesses of the prior research being cited by the applicant as crucial to support the application. It is expected that this consideration of general strengths and weaknesses could include attention to the rigor of the previous experimental designs, as well as the incorporation of relevant biological variables and authentication of key resources. See related FAQs, blog post

In the reviewer guidance it starts with this:

Scientific Premise: The key data introduced by the applicant to justify the project.

This is useful because it is a little bit different from the definition on the main page. To reviewers, premise is explicitly defined as "the key data".  Basically, what in the past (I'm trying to avoid using 'why' here) gives you justification for doing what you propose to do?  The FAQ has more insight for us:

Scientific premise concerns the quality and strength of the research used to form the basis for the proposed research question. NIH expects applicants to describe the general strengths and weaknesses of the prior research being cited by the applicant as crucial to support the application.

Now, where should this go in a proposal? When skimming all of the information (and there is a lot of it), it seemed easy to get confused about what goes where. In the web page explication of premise, Mike Lauer explicitly says:

...as a part of the Significance section of the Research Strategy, ...

Just as a reminder:

 SignificanceR01, R03, R21, R34. Does the project address an important problem or a critical barrier to progress in the field? [more... ]

Significance is demonstrating the "importance" of the project. So, how do Significance and Premise differ and interact, beyond being in the same place in the proposal? Significance explicitly asks: how will the field change if this research is done, how will the world in general, and a number of specific things, including knowledge, etc, be improved as a result of the proposed work. The FAQ says [my emphasis]:

Scientific premise includes a retrospective consideration of the foundation for the application, rather than a prospective analysis should the aims be achieved.

This is one of the main clues for us. Premise includes: what in the past (the literature, other results) suggests that this is important work to do? General significance is: what will happen in the future if this project is carried out.

For example, the significance of a study of age of onset of  type 2 diabetes in children, comparing males and females, urban and suburban, socioeconomic class, could be that type 2 pediatric diabetes is a precursor of other pediatric concerns, as well as a predictor of adult obesity. In asking for the premise, NIH wants an evaluation of the data that links pediatric type 2 diabetes to (1) other pediatric concerns and (2) adult obesity. The use of the word "precursor" as I did here, suggests a need for data that demonstrates a causal link. "Predictor" implies correlation and less likely causation. You may wish to argue about the specific implications of these words. But, the words linking the underlying background to the proposed work should chosen with care, as they can be replete with implications.  I chose these with those specific implications in mind.  The larger significance in this example would be determining if the premise holds for both sexes/genders, and equally for other parameters. This might include the idea that finding any differences would be a suggestion of where to look next for the underlying causes of the relationships in the premise.

As a closing note, the reviewer guidance includes the following bon mots addressing the reviewer, "you" in the quote [my emphasis, again]

You should factor a weak premise or the failure to address scientific premise adequately, into your criterion score and overall impact score. The page limit is not an acceptable excuse for an applicant to not address scientific premise.


One response so far

Leave a Reply