Resilience and funding and the NIH

Jun 16 2017 Published by under Uncategorized

@thenewPI has a new post up titled: Is resilience the name of the game in academia?

Go read it. I'll wait...

Ok..

She talks about @doc_becca, who is one of my alltime favorite people on the intertubz. Heck, we've even met in IRL, and doc Becca is twice as impressive in person as she is on the web (which is not true of all of us). I don't want to dredge up problems, etc, but she been done wrong. Many people who are Good and Working Hard, and as Doc_Becca sez " I have done EVERYTHING I was supposed to...".

But we live in a harsh funding climate, are being pushed and shoved out of academia. We live in a climate that is particularly harsh for the young, for URM, for women, for people who tick off more than one box. And these people are being denied tenure by zealous administrators who think about the bottom line more than the content.

As I, and many we all respect (lookin' at you, DM and datahound), have said over and over, one of the issues, if not THE ISSUE, is too many mouths at the trough.  See here. and here. and here. (These are all good reads, and if you don't know them, they are also worth a minute or ten of your time).

Applications for NIH funding are rising faster than the money for those projects. There are lots of suggestions about how to diivy up the existing funds, limits on the oldies, bumps for the young. But these, in my view are not just rearranging the deck chairs. They are worse, because they distract from the real problem and they divert energy from the solutions that really need to happen. See also this set of tweets from Michael Hendricks.

But one of the points I want to get back to is something that NewPI does a good job of talking about: the problem is really not so much that NIH peer review is broken. Lots and lots of chatter on the Tweets and various other places that talk about how horrible peer review is. From NewPI:

Taking a look on the inside of NIH peer review earlier this year gave me some prospective. I don't necessarily think that peer review itself is broken. I enjoyed participating and found that everyone was fair, but I realized that the 10-15% pay lines introduce an element of pure luck which has nothing to do with your worth as a scientist.

DM has also said this: when you get to 5-10% paylines (my IC is at 9% for established investigators), you are looking at lots of things other than just how good the science is. The difference in the proposals at 9% and 11% maybe trivial in quality. And this is where NIH staff comes in, and there are massive issues there, too.

But back to peer review: some of these issues are random, wrt to you, but not in respect to other factors:  Are you the last proposal before lunch, the first proposal of the day, following a bruising discussion about another proposal? Is one of your reviewers "saving it up for another proposal" and thinking that they can't go all out and advocate for two?

These are not the hallmarks of a broken system, although it could be perceived that way. They are the hallmarks of a human enterprise, where human beings are making decisions, lots of decisions, and giving scores and trying their best. Me? I get tired at study section. I do my damnest to stay alert, to read every proposal's specific aims, every proposal's full reviews. For the proposals I've reviewed: I read the other reviews, I take notes. In short, I prepare for study section. And yet, I am sure I make mistakes. Despite myself.

So what is a young person to do? Read TheNewPI's advice here about working with the study section. Read DM's advice and also here and here (and much more). Grantsmanship means looking at the system and doing what you can to come out on top. Read the damn instructions to reviewers and know what they are looking for and looking at when they read your proposal.

To those of you starting out: it's not an easy road. And yeah, resilience is gonna be important. But remember there are people out there who do want to help. There are people who will be on your side. Find them.

Resilience. Cleverness. Hard work. Desire.

 

33 responses so far

  • A Salty Scientist says:

    Apologies in advance for the following rant, but I'm fucking livid over the arguments by some senior people on Twitter. Claims that you *need* >3 R01s to run a mouse lab. No, you *want* >3 R01s to run your ideal operation (size, scope, etc.). Less, but not zero, science would get done in your lab with 1 R01. That is the reality for the vast majority of investigators. But the really fucking insulting part is insinuating that the science that would get funded instead is shit. In this funding environment, damn few shitty grants are funded, and that would be true if paylines doubled. For those senior assholes making this argument, what was the worst percentile you received that was still funded? Thirty percent? Forty? Then why is are 15th percentile grants suddenly shit. Some fucking people.

    • potnia theron says:

      Rant away, my friend.

      People get a view of what they *must* have. These are also some of the worst offenders in reproducing those mouths at the trough.

      But I suspect you have a choir here. Folks like that seldom come to play in our sandbox.

  • I'll go with option B: it isn't too many mouths, it's not enough support (but definitely too many mouths for the support we have).

    Support for basic research has been rising at or slower then inflation for physical science for a while now. Biomedical research has had even crazier ups and downs (the doubling and undoubling). Research is an expensive and long term project. With fewer companies in the game, the government should be spending more not less. If the US won't, others eventually will (I hope). My friends and I often lament that in 20ish years, the iProducts may be fewer and further between, given that we are living on the benefit of basic research done 15-20 years ago, but not doing what is necessary to give future entrepreneurs and inventors the same gift.

    My experience on review panels matches yours--getting to the top 1/3 to 1/4 is a reasonable task, and most panelist agree on which proposals are in the top 30% or so. Cutting down from there is mostly a matter of taste, meaning luck is super important in the end. I worry so much that I will run out of funds, as does every other PI I talk to. The pressue does not abate after tenure!

    It is not a shock that a vanishingly small percentage of the students in my department want to stay in academia long term, seeing what we go through with the proposal shuffle. I am grateful to my more experienced colleagues, who have helped me refine my ideas, gave me copies of their successful proposals so I can see a style that has worked, and volunteered to read my crucial proposals for both content and style. So yes, those people are out there--newbies need to find them as soon as possible!

  • Anon says:

    @Prodigal & Potnia: I think your are missing the point. If it is indeed the case that the payline levels introduce an element of pure luck into the process, how does advice from well-meaning senior folks help?

    • A Salty Scientist says:

      Umm, because you still want to be competitive for funding. While luck is involved, funding is not decided by random lottery. Sure, there is quite a bit of luck to where you place within say the top 30%. However, panels can reasonably bin grants into the top, middle, and bottom thirds. Your grant needs to be competitive enough for luck to work.

    • potnia theron says:

      Because getting funded is a multi-variable, and non-linear function. Luck is one element. Improving grantsmanship is another. Knowing what is impt to get funded is another. Understanding the review criteria is another. If you want to be funded, you still need to get a fundable score. Some people *do* get funded. I do not pretend to have all the answers. But I've reviewed grants and sit on study section, and have watched who does and does not get funded. If my (well-meaning, but from an ancient person) advice is irrelevant to you, that is fine. If things I say *are* useful, then I've done something worthwhile.

    • Anon says:

      Excuse me, but we are talking about folks line newPI and Doc Becca who have already gotten all the well-meaning advice they should have and likely more. And that's gotten them what, exactly? There are a lot of others out there who have also done this -- umm, thanks for giving me the benefit of the doubt! -- and remain unfunded.

      The point of the post you are responding to is that none of this helps. It is not about writing better grants. It is about accepting that even if you do everything right, even if you are resilient as hell, it may still not be enough.

      So the answer to the question asked in the OP, is resilience the name of the game in Academia, is no. Luck is the name of the game. And while luck is always an element in everything, it has never been this huge or decisive before.

      • I am two years from my tenure decision, so perhaps not as senior as you think. I am deep in the struggle for funding myself, though I am not a biomedical researcher.

        I wouldn't say none of it helps. It is your job to write a fundable proposal. After that, you need luck. If the advice you get is not useful to you in writing a fundable proposal, feel free to ignore it. As you say, there has always been luck involved. There still is, and in every field (not just academia). Without resilience luck does you no good. I am sorry that you have been unsuccessful in getting funded. You have the same options now you have always had (the same ones I have myself--its not like funding pressure ends at tenure): quit trying and never get funded, or keep going and try again. When I am trying again, I often find the advice of others helpful, but you may not. Your choice.

      • AcademicLurker says:

        Having seen things from the applicant side and the study section side, my take is that funding is basically a lottery, but you need to do everything right just to buy a lottery ticket in the first place.

        So the grantsmanship advice people are giving is good, but without a large slice of luck, it's still not enough. Everything sucks, in other words.

  • Paleogould says:

    And yet the fundamental problem of the case you pose remains. If you tried hard, were good, did everything right, showed resilience and still failed (ie were denied tenure) that is a small but real tragedy (in fact, in the technical sense: as in a conflict between two unyielding sets of contradictory rules, NIH funding and tenure decisions).
    But when that situation becomes common enough, we can begin to ask if it is worth it. And, perhaps, if we should not do something about it.

  • Ola says:

    I know this is gonna sound harsh/unsympathetic, but instead of vacillating about the boxwood, how about some real numbers to facilitate a more honest appraisal of claims such as "I've done everything I should".

    - Instead of "I published in good journals!" How many papers, as first/mid/senior author, and in what journals? As a benchmark, at least 1 (preferably more) senior author papers in solid journals (IF>5) per year, plus some middle-authorships or reviews here and there. Maybe a 1 year gap upon lab startup (with a review or 2 during that time to keep name current in the literature).

    - Instead of "I gave talks", state if these were internal/external? Invited? Plenaries at big conferences? Did the talks win any awards? How many people were in the room? How did the questions go afterward?

    - Instead of "I mentored students", did you actually graduate anyone with a PhD/MSc from your lab yet? How are your teaching feedback reviews? Can you point to specific success stories in your former mentees?

    - Can you share the pink-sheets? Those of us with experience interpreting StockReviewsTM might have some insights into what actually drove the score? If the Approach score was poor and everything else (Signif/Invest/Innov/Envir) was 1s and 2s, then sorry but they don't like your science. If there were problems in the other categories, fix that shit because it's low hanging fruit.

    - Think about what's on your biosketch? Is your "contributions" section all stuff from your post-doc? 1 or 2 good vignettes about what's really yours = better than 5 where you're middle author from a decade ago. What's your h-index compared to your peers? What are your 5 most cited papers (if they're all reviews you're doing it wrong)?

    - I note a total lack of discussion about how good was the prelim' data in the grant? Remember, you won the lottery, you got a TT position with a start-up package. There are no excuses for not having prelim' data. Ask yourself honestly, "what have I spent my startup money on?" Think about it from the University perspective - "what do we have to show for these $?"

    - Responsiveness to previous reviews? If it's an A1 and the score got worse, you have to be prepared for the fact that they didn't "buy" your responses.

    - Despite claims to be a good "University Citizen", is your message on campus fundamentally at odds with the wishes of the old white bearded doods in charge? Is your area of science even mentioned in the strategic plan? Are you regarded by some as a trouble-maker/rabble-rouser? Do you STFU in faculty meetings? Save the controversial stuff until after tenure.

    Hate on the messenger if you must, but if you look at each of the above points and can honestly say that you have done EVERYTHING by the book, then you might have a case for sympathy. But, having seen many such cases before (and read about plenty here on Potnia's blog), there is often a "blind spot" on the part of the person involved. I hate being that person, but someone has to cut through the "I'm so sorry" crap and figure out WTF happened here? Maybe you genuinely did get screwed, but I think the more likely scenario was a blind spot in regards to some institutional requirement or expectation.

    • Luminiferous aether says:

      I don't want to speculate about Doc Becca's situation, but these are excellent points to ponder for any junior investigator (I am one myself) as they progress in time/career. Thanks, Ola!

    • Cab says:

      1. In the specific situation you cite, many people with specific knowledge, including faculty at the institution in question, have determined the tenure decision to be suspect.

      2. It's always tough to assess the specific contributions of the candidate without seeing the other side of the curtain: a decision process that is subjective, nonuniform, and opaque. Addressing these factors, as well as alleviating implicit bias and unrealistic expectations, will improve the institution of tenure.

    • drugmonkey says:

      Ola, should we really have a system where one has to be absolutely *perfect* on a list such as yours to remain employed? Does this make sense?

      • Ola says:

        I apologize for the harsh tone. There's no polite way to ask "have you missed something?" If someone claims to have identified ALL of the factors that drive these decisions (funding, tenure), and addressed everything, I think it's reasonable to point out they may have missed soething. There's ALWAYS something you don't see - a blind spot. I'm sorry if some of my suggestions for where the blind spot might lie were off-the-mark. But to deny it exists at all is problematic.

        • Dr Becca says:

          My question is, why is your instinct that I am the one who's missed something? Why does the benefit of the doubt go to the administration, rather than me?

          While we're all doing some introspection here, I recommend you do some of your own.

        • cab says:

          Department opinion was solid. Nothing is really 'missing' other than admin agenda, which is certainly focused on cash flow. One question is whether it is appropriate for this institution to predicate tenure on a particular level of outside funding. A second question is whether this requirement is applied in an unbiased and uniform manner. I suspect the answer to both questions is no.

    • DJMH says:

      She said her dept. voted for her to get tenure. So...despite your hopeful assertion that she's a trouble-maker, that's not likely to be the problem here.

    • Susan says:

      This list is detailed and all, but I am with Becca: exactly what gives you the idea that someone who, after all, had the brainz and skillz and diligence to win a TT position .... doesn't know this in detail and by heart? To whom do you think this is news? Why would you suspect that the "deep honesty" you've listed here is something we've skimmed over?

      Sure, it gets skimmed over in a 140-character format. But you'd better bet your arse that I knew all this, and more, and every point (and then some) is addressed in my tenure portfolio, and I am willing to put money down that it was in Becca's as well.

      • Susan says:

        We've spent years and years pondering exactly what it will take to get tenure. I'm just dumbfounded that your immediate response is: well, since you never asked, I'll tell you how it goes ...

        Mansplaining for the win.

  • xykademiqz says:

    My former postdoc is in the second half of tenure track and might be turning in his tenure package soon. Great papers, good teaching evaluations, graduated some MS and soon his first PhD or two. Has collaborative money, but still no money as PI. I sort of see what the issues are with his grants versus the agencies he's trying to target and I have given him plenty of feedback on his grants... At the end of the day, he wants to do what he wants to do in the way that he wants to do it and that's what he proposes to do in his grants. Also, a big issue is that the work he does does not have a natural home in the NSF, as NSF in the physical sciences is fairly partitioned in terms of whom (which types of departments) each program will support. Perhaps I need to be super blunt about my comments to him, but on the other hand what do I know, it's not like I can claim to have cracked the code of fundability within NSF [other than if you are a (female?) theorist/computational scientist, you are much more palatable to many programs in the NSF when playing second fiddle to an experimentalist] -- I feel I have been able to luck out about 10-20% of the time after throwing many different kinds of proposals at the NSF wall to see what sticks (hint: sometimes it's the half-baked borderline stupid stuff).

  • Dave says:

    Can you share the pink-sheets? Those of us with experience interpreting StockReviewsTM might have some insights into what actually drove the score? If the Approach score was poor and everything else (Signif/Invest/Innov/Envir) was 1s and 2s, then sorry but they don't like your science. If there were problems in the other categories, fix that shit because it's low hanging fruit

    You haven't been paying attention if you think these issues are related to grantsmanship. This kind of nonsense helps BSDs and the oldz sleep at night, but nothing more. I would submit that the vast majority of apps coming for ESI/NI in junior positions are very nicely presented with strong preliminary data. More often than not, BSD apps look like shit to me and read terribly, but they get very, very good scores. Grantsmanship is not the one. Move on.

    Maybe a 1 year gap upon lab startup (with a review or 2 during that time to keep name current in the literature).

    A 1 year gap is totally unreasonable if the lab is doing experimental work and starting from scratch. You have to know that. You HAVE to know that. Right?

    There are no excuses for not having prelim' data.

    Again, do you honestly believe that a TT investigator would put in an R01 without preliminary data?

    Is your area of science even mentioned in the strategic plan?

    What has this got to do with landing an R01? Do you check the strategic plan of each applicants institution before awarding a score? Do you often find yourself believing what is written in a strategic plan? I thought I'd go my whole career without finding that unicorn faculty member who paid any attention to admin-driven strategic plans, but now I've found you.

    • potnia theron says:

      The strategic plan issue is not quite so clear cut. I've had my subdiscipline move from one IC to another about 20 years ago. I remember crying on the phone with the PO (I was much younger, then, I'm older than that now). Recently, IC2 cut that area/disease entity from their strategic plan. On purpose. They didn't want to fund it any more, for several reasons. Staff has used that strategic plan to justify rejecting mechanisms that are in-house reviewed (Ts, Ks, some mid-Rs). In this case, ignoring *what* the IC funds can cost you a full cycle of review.

    • drugmonkey says:

      Ignore what is in the stated interests of a given I or C at your peril, Dave.

    • A Salty Scientist says:

      I read Ola and Dave's comment as regarding the strategic plan of the research institution and not the IC.

  • Twitter Tired says:

    Ola's comments are extremely refreshing, considering the Twitter circle-jerk over Doc Becca.

  • potnia theron says:

    One more (tired) time: my post was not about Doc Becca, and I dont know if Ola's was or was not. I do not perceive this as a "circle jerk".

  • What I find odd is that Ola's (and some others') comments assume this process is entirely deterministic. With really low funding rates, even if you submit 10-12 proposals, a fraction of 'decent' grant writers will not receive a grant. Unless someone is repeatedly being an idiot (and I don't think most TT's are), some people just have a bad run through no (or little) fault of their own.

Leave a Reply