I really don't got to study sections as much as I used to. I only ad hoc, that is, I'm not a sitting member but they call me when there's something tor a bunch of somethings with which I can help. It's important, but, (1) I've done it a lot, and the returns to me are diminishing, balanced with I'm busy with other stuff and (2) someone younger should be doing this (I suggest every time).
When I do go, I just open a file and take notes during the meeting. There is a fine line here. I don't, I can't talk about any particular proposal. But I think that sharing impressions in general is very helpful. Here, then are some thoughts I've had for the most recent go-round.
Keep in mind this study section is for a specific early career mechanism. I totally support separating these. It's not so much that it makes for competition among these young guys, but that it removes any comparison to the Big Dogs. In my experience, even when mechanisms are reviewed separately, for example, all the ESI proposals together, then all the "grown-up" R01s, reviewers cannot help but see different levels of grant-competence and scientific chops between categories. When I review, I don't say "OK, I'm going to do the ESI ones first, finish those, and then start on the others". This section was ONLY NI/ESI/ECR/fellowships.
There were a number of consistent concerns raised across multiple proposals. (oh how easily ones slips into reviewer-speak). Several times the section discussed the fine line between ambitious (good) and over-ambitious (bad). Did the applicant not understand what is feasible? Do they think they can do 5 or 6 or 7 years work in three? This problem was correlated with the concern of where to draw the line between problems of grantsmanship or poorly written proposals with insufficient explanation and problems of research design. In some sense, this is trying to read the mind of the person submitting the grant: did they just poorly state what they want to do, or do they not know what they want to do? Are they missing a critical part of the design (a variable, an experiment, a control)? Or, is it implicit in the writing if you read it a certain way.
When I have been helping young people, when I see this problem in a proposal, I often hear various versions of "the problem is the page length". In fact, I heard it the other day at a thesis proposal that was required to be in the format of an NIH proposal. Remember the page length is your friend: it is telling you how much information the reviewers want to/expect to/need to read. If you are having trouble putting everything into the proposal, consider whether you have 1) included unnecessary detail or 2) are trying to put too much into the proposal.
Back to study section, as someone said: if they had just reread the proposal one more time, or had someone outside read the proposal one more time and see if it makes logical sense in the way it flows, they might have caught the perceived problems.
There are several pieces of advice that flow from this:
- Writing does matter.
- Your perception that your logic is tight is not necessarily true
- Get someone else to read your proposal. Someone who will tell you what is wrong, both in willing to critique it and willing to put the time in to see what is wrong.
- The best proposals are "tight", they are focused and they are feasible.