I am sorry, but I am actually doing science! And my science becomes a 24 hr thing for about a week, and then drops to about 12-14 hr/day for the next two. I love you all dearly, and will return when I have three brain cells to synapse together.
I do not remember that I ever had to remove a comment. But I will remind us all of the guiding principle for at least this blog: respect for individuals even when we disagree with their positions. If someone's position is so egregious that you need to critique them as opposed to what they say, then its probably time to leave this corner of the blogosphere.
I want to say a few words about DocBecca and some of the comments directed her way, posted here.
I do not think the sympathy to her situation, of which there has been much, especially on the tweets, is inappropriate. She has been a leading figure, here, and elsewhere in our e-community. She has done a great deal for others (her TT aggregator page, for example, was of incredible value to many people). As I have often said, this kind of resource was not available to me when I was young, and it would have made a difference. The kind of support she is receiving was not available before social media.
Further, knowing DocBecca IRL, and knowing her research which is in the large tent of neuroscience, as is some of my work, I know her work is good. I've read some of her papers, and I'd be very glad of having her as a colleague. I am guessing that some of the support for her, and outrage at her situation comes from people who like me, know enough to see what she does. But flaunting that information has not been part of the discussions, as is appropriate.
One of the tools to keep people "in their place" is isolation. It's historically true, and is still true. Doing what we can to help others is a fundamental task, in my world, and in the world of people who are in this community. So one step we take in the comments/posts/tweets that say to DB: what happened to you is hurtful, and I am sorry.
It is another matter to dissect what happened to her. We don't know beyond the details she provided (as she has said, I put this out there): her department voted unanimously to support her, the Dean turned her down, the next level (Provost, I believe) said "get funding and I will reverse". But, none of us (that I know of) are in her dept, are the Dean, the Provost, part of the SS that evaluated her proposal, or the NIH PO that did not help her. And speculating what happened there might be satisfying, but I think not useful.
On the other hand, the posts that I (and others) have put up are about the system and the process in general. They were catalyzed by DocBecca, but they are not about her. The goal of these is less to help one person in particular, but top provide insight that might be useful to a larger group of people. These posts attract comments. Some have been comments that I interpreted as being general, but other people interpreted differently, as being about DB in specific.
There are difficult to draw lines between attacking someone, and criticizing them. The internet is having some problems with this issue. I do not want this blog and the comments to posts to become part of those problems; my goals are much more modest.
The ocean in which we swim is large, and cold and dangerous. Part of my goal is to help minnows navigate these waters. I invite you to do the same.
I object to the across the board, frequently negative, characterizations of "millennials" for a bunch of reasons, but they fall largely into three baskets.
In no particular order:
Firstly, I object to the whole damn generation thing on statistical grounds. See here.
Secondly, It's easy to find people who have negative traits or embody things you dislike or distain. You can probably find those characteristics in people of any age, and people with those characteristics at any point in historical time. Is it about the person or about the group? Do this trait appear more frequently in a given group, defined by some other group characteristic?
And thirdly, which is really an extension of secondly, when someone does or says something you don't like, it's easy to paint with a broad brush and attribute it to their group membership. It's ok to do that with age groups (both young and old) these days, but we've sort of come to our senses about doing this when the group is race, religions, gender, orientation, and maybe a few more. (Note, this is different from Political Party Membership, which is much more of an active choice, but still, one needs to stop and think).
Now, maybe the thing they said or did or didn't do or didn't say that was objectionable to you was because of their group membership. But maybe they're just an asshole. When you start making group generalizations, you run the risk of characterizing people who you might like, who you might find as a friend or an ally, who might help catalyze your growth, as being just like the asshole.
So why write this now? There was a comment on this blog, and it was considered rude and wrong and horrible by many people on the tweets. I'm not going to censor the comment, nor unfollow because of some very strong tweets. The discussion has had many good points that have made me think, and made me take some actions to support and defend young people, where I have the power to do so. But I'm not interested in that broad brush that says "discount all the oldies". Discount me, because I'm ignorant or selfish or dress inappropriately. Unfollow me, because I'm a jerk, or insensitive or I like coffee too much. But because I'm old? It won't matter to me, but it might matter to you.
My mother, the gerontologist, was a life long democrat. She worked for Adlai Stevenson (ok, go read the link, I've made it easy for you) in the 50s and Civil Rights in the 60s. When Ronald Regan ran for office, there was a lot of talk about his not being able, because he was so old. She was furiously opposed to this line of thinking. "Criticize him because he's wrong. Because his policies are selfish. Because he's not too smart. But leave his age out of it. What if the guy you really liked for policy reasons was that old?".
In the post on "genius" and publication rate, someone remarked on the "the incremental drivel that populates 99.9% of journals today".
Here's a better model than genius and drivel: explorers/pioneers/settlers. I won't guess %'s because I suspect it is a constantly evolving thing, with %settlers increasing over time.
Explorers sometimes find things and sometimes don't. Its hard to be a full time explorer today. Even for older, funded people, because its tough to get money to support exploration. Pioneers follow where the explorers found a hint of something, a suggestion of something. Pioneers can get money, sometimes. Settlers come after the Pioneers have done some land clearing and make a living there.
Now, one thing that can happen is that Explorers find a New and Exciting! method. Pioneers often find an application for the method, but the progress in science doesn't happen till the Settlers get there. Sometimes. Sometimes, you need a lot of Settlers, collecting a lot of data, till a larger pattern emerges, with New! Ideas! We can all think of examples. PCR? Xrays?
Now as for incremental drivel? We can do basic science/evolution/ecology. We can do medical research. Lots of that incremental stuff is important. It may not be as exciting to the BSD's of this world. It may not get the headlines. But it's absolutely necessary.
Let's think about evolution for a moment. Or even Ecology. Finding fossils, describing distributions of plants, may fall in a social category of exploring, but its often damn incremental work in terms of the science. It is the basic data of what tests theories and drives new ones. Heck, even doing population genetics can require a lot of tedious, incremental bench work. Let's not forgot that Mendel raised peas for years before he got to genetics. Darwin studied worms and corals as well as finches.
The tedious work of documenting ecological networks is often done on a set of species by set of species projects. It can be years in the making, and young folks, heck old folks, publish a bit of it each year. What if you have a set of 10 species in a genus? They exist in sympatric (living in the same place) groups of 2-4 across a very large region (say, Thailand through Malaysia, Borneo, Indonesia and Papua New Guinea). How did closely related species evolve in the same place? How did they get to be different? How did evolution work in this case? One may be able to visit a country or two a year. One must first show they are distinct species: cross breeding experiments? pollen distinction? different pollinators? differences in flower morphology (anyone who has taught or taken multivariate stats knows Fisher's famous Iris flower dataset)? I'm sure documenting the differences in 3 species that are found in Sumatra, Java and Kalimantan would count as "incremental". The person doing the research did. But after 2 or 3 or 4 years of data collection, and having reviewed, published, (validated?) the differences, to find out how distinct species can be right next to each other, tells us something about the ecology and the evolution of this (possibly obscure) set of tropical rainforest trees. They don't have much commercial value (they are small, and the flowers not spectacular). But we've learned something about how evolution works. Someone else does a similar project with small mammals, and small lizards. And one day, either the researchers get together and put it out together, or someone else sweeps in and organizes it, but A Big Picture Emerges! And we learn how this ecosystem works, together, with its diversity. And maybe some poor soul has been laboring to do the same thing in high latitudes, and they put it together, but their together is different from lowland tropical rainforest. That may be a Big Deal. But dammit, it couldn't be done with lots of incremental stuff.
The same thing can happen in medicine, the incremental improvement in chemotherapy drugs. Those increments can mean a lot to you, if it is you, or your mother, or your sister, or your daughter, who is dying. My friend, my beloved friend, who had Stage IV breast cancer is now, 18 months later, free of cancer. Yes, she has many sequelae and will live with a range of health issues. But she is alive. Five, ten years ago, she would be dead. I thank those unknown-to-me settlers who improved the drugs so that my friend is here.
I suspect we'd all like to be explorers, or pioneers. Maybe that's the dream we had when we were 10 or 15. And maybe we can hang onto that dream as we slog through the reality of grad school and being a post-doc cog in someone else's dream machine.
But to call that "incremental" work drivel is to truly miss the point. There is good work there. You may not know it. You may not recognize it. But there is work there that moves science forward. There is work that saves lives. How dare you denigrate that.
A tweet from @doc_becca set things off last night:
All I am asking for is that there is a mechanism in place to keep scores from getting worse. Such a colossal waste of time.
— Dr Becca, PhD (@doc_becca) June 21, 2017
This is going to have to be a quick post. I've got too many urgent things piling up around me. This post isn't urgent, but it is important. There will, without question, be mistakes. I trust my loyal readers to point out, preferably with glee and schadenfreude where I'm wrong.
Another caveat: This is not about Doc Becca. I count her as a friend. I am upset about her situation and where she is and the stupidity of her university. I've been following her for years, since she was stumbling towards the tenure track. I love her writing, and from what I know of her career IRL, she deserves tenure, based on her science and productivity.
Again, this is not about her. This is about all the things that got said about study section last night. Some of this is my opinion, but lots of it is about how study sections (SS) work.
So on to content, and not in a necessarily good order:
Firstly, remember reviews are two steps: Study Section and Council. Study section reviews proposals for scientific merit and assigns a score. This score, later on, by NIH staff, will be normalized into a percentage based on all the scores from the current and the last two iterations of this section. At SS, the proposal may be discussed (if it s in roughly top 50%) or not (triaged, in bottom 50%). Reviewers, any reviewer on the SS, may ask for a triaged proposal to be discussed, before SS, or even at SS. I have done this. It is not common, but not rare, either.
SS does not decide funding. Council does, with much input from NIH staff. The person who runs SS (a Scientific Review Officer) is not part of the team that decides funding or ranking of grants to be funded. SRO's usually work for OER (office extramural research), whereas staff PO's (Program Officers) work for individual IC's. IC's decide funding. There is variation in the score that is funded, and there is variation in who, at what score, gets funded. IC's do this because they have programmatic priorities. If Bunny Hopping is in this year, a worse percentile may get picked up. Something that is outside the "main mission" of an IC may not get funded.
Secondly, (and we are now on our third espresso of the morning): the way that SSs work is not a big mystery. Junior people can do a round on SS to learn how. I've got some posts on that, and will try to dig them up for you. NIH has a program for this and it is very valuable. The most grantsmanship I learned was when I sat on SS.. So now, stuff about how SS work:
Who decides who reviews which proposals? The SRO does. They have a miserable job here, and no time to worry about screwing you. Truly. They have a list of reviewers, which may not include sufficient expertise. They have to go begging for reviewers. Outside reviewers. I've been to reviews where I've been an outside person and there have been more outside people then standing members, because of the range of proposals that come in.
Sometimes outside reviewers don't go to the meeting, and call in. They haven't heard the other reviews, and they are not "calibrated" to the section. Most outside people are aware of this and defer to the ones who are there. But not always. This is another source of variation. But not everyone can drop everything for 2-3 days and go to Washington. You can say "no phone reviews" but that may mean worse reviewers. What's worse? Someone more removed from the area of the proposal, who has no appreciation of either the premise or the design.
People serve on SS for 3 or 4 years, and technically are supposed to be at all of them. Some SS have a program where you do an extra year and come 2 meetings of 3 each year. I was told that option was so over-subscribed at the one I'm joining, that its not an option for now. But, people join, people leave. Maybe the proposal had an ad hoc to start with, and that ad hoc can't do it again. You may not get the same reviewers, you may have 2 of the same, or 1 of the same, or none. There is not a set rule here. BUT! when a reviewer gets a new to them proposal that has been reviewed before, they get the entire summary sheets that the PI received. IME, reviewers read these and consider them. (but more on re-review below).
(BTW: how does your proposal go to a particular section? Thats another post, but in short: you can request, there are key words that help determine, and people who do this).
SRO's have to get expertise, but they also cannot give any reviewer significantly more than others. What is more? Varies from SS to SS. For some, its 4 or 5. But, I've been on ones where I've got 8 or 9 and thats standard for the sitting or standing members. (I sit, because I'm old, and standing hurts my back. This is a joke, do not read anything into it). It's a lot of work for everyone, and you get paid squat for doing it, and it's one of those things you do. I don't think anyone relishes the power involved. Ok, maybe there are a few antediluvian bigdogs who do. I don't. I just try to do the best damn job I can.
IN MY EXPERIENCE: SS members care. They work hard. They are obsessively concerned with being fair, and just and right. They are sensitive to the PI, and take "Investigator" criteria seriously. No one is out to screw you. But, of course, they are human, and have biases and have a lot to do. Sometimes they think proposals are bad. Sometimes they get irritated with a proposal partway through (Make the reviewer your ally, your advocate). But for the most part, the reviews that I've seen, even the ones that don't get discussed, reviewers are capable of partitioning their perspectives. They can find both good and bad in a proposal. They try to balance those things, and realistically evaluate their relative importance.
Finally, a bit about some of things that got said last night.
Scores going up and down? As much as I want Doc Becca to get funded, I do not see how one can be protected against a dropping score. The reviewers have a different proposal. Maybe the proposal is worse, to one of the ones reading. I've received conflicting advice on a first submission before: add human subjects, do not add human subjects. Take out Aim2, expand Aim2. And, yes, conflicting advice is horrible to deal with, and you can't know if the person who gave that advice is going to review again. NIH has tried to circumscribe that kind of advice with "review the proposal in front of you, do not write a new version for the PI". But even just saying "this is good" or "this is weak" can show up in the same review.
If I was told that my review had to be limited by the previous reviews, it would make reviewing very hard for me. I read each proposal. I try to give each proposal my very best thoughts. If I see something glaringly bad, that got missed (as far as I can tell) in the previous review, I am not going to give it a pass because some other reviewer didn't see the problem. But, if I think something is very very good, significant, innovative, and the previous review said "meh", I am also going to point that out, and advocate for it.
People last night said that this is a problem with reviewing, if this happens all the time. I don't know if it happens all the time. I dont have statistics. It has certainly happened to me, more than once, and I've been putting in proposals for a very long time.
You may agree, or not with the idea that scores can drop. But, if you believe that having external reviewers, peer reviewers, reviewers from the larger community assess proposals, is a good thing, and that if proposal can get worse. that you must admit that a score may drop.
If you want to limit scores, then the system will need to change. Maybe that would be a good thing. I tend to think not, as things that would limit, in general, reviewers will not improve the system. In the end, I suspect that those limits would be co-opted by those in power, those with the most grants, and the most time and resources to submit.
Is the system broken? Once again, I say no. It is not perfect. There are problems, and individuals who get lost or hurt or destroyed in the grinding of the gears. But, the alternatives to people like me reviewing grants is letting the PO's at NIH make all the decisions. Even if they could, which they can't, physically, they just don't have time, this would not be a good thing. Right now there is some flexibility there, and as is true of everything else, those people are human beings with all the attendant flaws of human beings.
I've not edited this way I usually do, because its late and I want to get it out. There's a lot more to be said, so likely another post on this.
Mike Lauer did an analysis, and said >75% of proposals improved in prctile. Median improvement was ~10%. https://t.co/GthwImV8N3
— Alex Kwan (@kwanalexc) June 21, 2017
@thenewPI has a new post up titled: Is resilience the name of the game in academia?
Go read it. I'll wait...
She talks about @doc_becca, who is one of my alltime favorite people on the intertubz. Heck, we've even met in IRL, and doc Becca is twice as impressive in person as she is on the web (which is not true of all of us). I don't want to dredge up problems, etc, but she been done wrong. Many people who are Good and Working Hard, and as Doc_Becca sez " I have done EVERYTHING I was supposed to...".
But we live in a harsh funding climate, are being pushed and shoved out of academia. We live in a climate that is particularly harsh for the young, for URM, for women, for people who tick off more than one box. And these people are being denied tenure by zealous administrators who think about the bottom line more than the content.
As I, and many we all respect (lookin' at you, DM and datahound), have said over and over, one of the issues, if not THE ISSUE, is too many mouths at the trough. See here. and here. and here. (These are all good reads, and if you don't know them, they are also worth a minute or ten of your time).
Applications for NIH funding are rising faster than the money for those projects. There are lots of suggestions about how to diivy up the existing funds, limits on the oldies, bumps for the young. But these, in my view are not just rearranging the deck chairs. They are worse, because they distract from the real problem and they divert energy from the solutions that really need to happen. See also this set of tweets from Michael Hendricks.
But one of the points I want to get back to is something that NewPI does a good job of talking about: the problem is really not so much that NIH peer review is broken. Lots and lots of chatter on the Tweets and various other places that talk about how horrible peer review is. From NewPI:
Taking a look on the inside of NIH peer review earlier this year gave me some prospective. I don't necessarily think that peer review itself is broken. I enjoyed participating and found that everyone was fair, but I realized that the 10-15% pay lines introduce an element of pure luck which has nothing to do with your worth as a scientist.
DM has also said this: when you get to 5-10% paylines (my IC is at 9% for established investigators), you are looking at lots of things other than just how good the science is. The difference in the proposals at 9% and 11% maybe trivial in quality. And this is where NIH staff comes in, and there are massive issues there, too.
But back to peer review: some of these issues are random, wrt to you, but not in respect to other factors: Are you the last proposal before lunch, the first proposal of the day, following a bruising discussion about another proposal? Is one of your reviewers "saving it up for another proposal" and thinking that they can't go all out and advocate for two?
These are not the hallmarks of a broken system, although it could be perceived that way. They are the hallmarks of a human enterprise, where human beings are making decisions, lots of decisions, and giving scores and trying their best. Me? I get tired at study section. I do my damnest to stay alert, to read every proposal's specific aims, every proposal's full reviews. For the proposals I've reviewed: I read the other reviews, I take notes. In short, I prepare for study section. And yet, I am sure I make mistakes. Despite myself.
So what is a young person to do? Read TheNewPI's advice here about working with the study section. Read DM's advice and also here and here (and much more). Grantsmanship means looking at the system and doing what you can to come out on top. Read the damn instructions to reviewers and know what they are looking for and looking at when they read your proposal.
To those of you starting out: it's not an easy road. And yeah, resilience is gonna be important. But remember there are people out there who do want to help. There are people who will be on your side. Find them.
Resilience. Cleverness. Hard work. Desire.
In Scienc (in an article about GREs as a less than useful tool for predicting success) we have:
can objective measures such as numbers of publications do any better at spotting true intellectual promise among faculty candidates? Not according to physicist Peter Higgs, whose work on subatomic particles in the 1960s inspired the long but ultimately successful hunt for the eponymous Higgs boson. As he told The Guardian in 2013, while traveling to Stockholm to receive the Nobel Prize in Physics, for years he had been “an embarrassment to [his] department when they did research assessment exercises.” With fewer than 10 papers published since this 1964 breakthrough, he often responded to departmental requests for lists of recent publications with a simple reply: “None.” Given today’s requirement to publish frequently, he added, “It's difficult to imagine how I would ever have enough peace and quiet in the present sort of climate to do what I did in 1964. … Today I wouldn't get an academic job. It's as simple as that. I don't think I would be regarded as productive enough.”
Then there’s mathematician Yitang “Tom” Zhang, who was completely unknown—as in zero peer-reviewed publications and an adjunct teaching job—when, in 2013, at the age of 57 and 12 years out from receiving his Ph.D., he submitted a paper that astounded the mathematical world by solving a long-standing problem in number theory. Now hailed as a “genius” and a “celebrity,” he has since that triumph received numerous major prizes and appointments to two professorships, first at the University of New Hampshire and then UC Santa Barbara.
So we've got two guys who didn't publish a lot, and were eventually considered "geniuses". Does this mean that all people who don't publish a lot are geniuses? Of course not. This only falsifies the statement that "if you don't publish, you are not (or cannot be) good". Nor does it have anything to say about people who do publish a lot. We are back to type I and type II errors, and what do departments and search committees and tenure committees want to guard against.
If, of low-publishing people, 1 in 100 is a Nobel laureate material, and the other 99 are pikers, is it worth hiring or tenuring one person who hasn't who doesn't publish in the hopes that they solve The Problem? In medical terminology, know the existence of one zebra when you hear hoof beats, does that mean that you should expect the next 10 or so to be zebras, or just plain horses? What is the risk in making the wrong decision? Is the 1 in 10000 chance of losing the genius worth the 99.99% chance that you are tenuring something who will not carry their weight in the department?
I remember when I was younger, back in the mid-Oligocene, that we had lots of "dead weight" in the department. These were people who fit the early/external pattern of Prof Higgs. They were tenured, they had had one grant in the Jurassic, and came in to teach one class a term, whether they had to or not. They hadn't written a grant proposal in years, and maybe churned out one paper every other year. I was furious because they sat in judgment of the junior faculty, assessing the quality ("not quite there") of research they couldn't be bothered to do. I remember what it was like to have people who thought they might, someday, become Dr. Zhang, but just couldn't be bothered right now.
I go back and read the quotes from Prof Higgs and I am struck by the sheer arrogance of his position. The entitlement that permits him to think that a department with standards, standards to which he might be held, would somehow inhibit his creativity.
I won't defend the current pace, nor the current obsession with funding. I've seen too many good people, working people, people who are Good Scientists (is the Nobel how we want to judge science, anyway?), chewed up by the system and spat out.
But as always, there is something between an endless, highspeed treadmill and waiting 20 years to publish. There needs to be room, in our science departments, our research communities for all kinds of people. But those people also need to understand that tenure isn't a ticket to endless coffee breaks, either.
Note: I am an olde farte, and am not sure what "live video" means, but here is a link for more info on the whole program: Can’t attend live? Register to watch on-demand. Email email@example.com with any questions.
Affirmative Attention: Advancing Science Through Diversity
Meet the Moderator: Michelle D. Jones-London, PhD As part of our ongoing commitment to promoting the essential contributions of scientists from diverse backgrounds, the Society for Neuroscience (SfN) will host Affirmative Attention: Advancing Science Through Diversity, a live video discussion, on July 17 from 11 a.m. to 12 p.m. EDT.
The moderator for this discussion, Michelle D. Jones-London, serves as Chief in the Office of Programs to Enhance Neuroscience Workforce Diversity (OPEN) at the National Institute of Neurological Disorders and Stroke (NINDS), NIH. In her role there, she chairs the NINDS Diversity Working Group and plays a lead role in the establishment of networks and partnerships to increase neuroscience workforce diversity.
Among many other initiatives, Jones-London has spearheaded an annual conference of all Diversity Research Education Grant programs. The workshop she led this past April, “Activating a Neural Network: Admission Strategies to Increase Diverse Neuroscience Trainees,” offered a venue for program PIs to collaborate, share best practices, and gain knowledge of the tools, resources, and data available for advancing their programs.
Visit Neuronline to learn more about Jones-London and our other speakers. Registration for this event is free and open to the public. Don’t miss the opportunity to engage in this timely discussion.
Questions? Contact firstname.lastname@example.org.
I get the updates from the Foundation for Biomedical Research, a group that supports animal research in biomedicine. They sent a link to a recent post they had, which talked about "Still Alice", the book and movie. There is a quote from the movie (I believe) that just tore at me:
“They’ve been doing all these tests, and I’m really scared,” she said. “I know what I’m feeling. It feels like my brain is dying and everything I’ve worked for in my entire life is going. It’s all going.”
And that is the fear. The fear my mother had, the fear that I now have. We are scientists, and the thought of our brain dying is about the most scary thing, for us, that we can think of.
For my mother, and her working class origins, her brain and her hard work were her ticket out of poverty and illiteracy and the endless soul destroying labor of her parents. She could never admit her illness, never admit she was losing her brain.
Just those words! "losing your mind". They are loaded and painful and mean so much more than just Alzheimer's disease. I do not want to lose my mind. But now, it is something that is there, lurking in a corner of that mind I want to keep.