Response to Disappointment and Discouragement in Research

Sep 11 2015 Published by under Uncategorized

I must say that I very much appreciate Jean-François Gariépy's politeness in disagreeing with me. Props to you, Jean-Francois. Further, he posted a long response, so I wanted to put it in a post, so I could address specific points in his response. As always, I welcome his further responses, as well as other people's comments. I've put the whole thing here, again, so that I am not cherry-picking his words.

Hi Potty,

I do not know what to think about your post, on the one hand it seems like you are confirming most of my complaints, but on the other hand you seem to not want to admit it. First, thank you for recognizing the possibility that all of what I say is true (sic).

I agree that being interested in one's survival is not a defect in and of itself.

Not so much my point as that everyone is interested in survival. And that the big wide world is full of jackasses. Leaving academic research is no guarantee that you won't end up facing the same mindset in other areas. In fact, I suspect you will.  You were giving reasons for leaving; I am saying that you won't get beyond this by leaving.

In the process of doing good law defenses, a lawyer may be interested in his own survival. In the process of saving a life, a doctor may be interested in his own economic well-being. The problem would come when this desire to survive impedes on the expected mission of the professional, and the point of my post is not so much that caring for one's survival is a problem in and of itself, but that it has now impeded upon our search for truth in many ways, including fraudulent data (and one only needs to look at the numbers to show this, my own perception and experience is barely relevant in face of the statistics on scientific misconduct).

More on fraudulent a little bit further down, but you mention statistics vs. perception. This is important. I don't think the statistics I've seen (retraction watch, etc) suggest that fraud is that prevelant.

Concerning #2, your argument seems to be that it's "the state of the world." This does not make it immune to critique. You also write that it causes anxiety and stress, so somehow you agree with me that it can be a problem (although stress and anxiety was not my focus, I was mostly worried about the amount of time lost on written products that are not part of the contribution of a researcher to society).

Yes, its  a problem. But its a problem for just about every and any professional. Again, this was one of your reasons for leaving. Again, I do not think that leaving will get you away from this problem. And, none of my non-academic friends feel that they spend less time on this than do the academic ones.

I do not claim that #1 and #3 are universal truths, but they are common enough that my little Facebook post has gathered an unprecedented (unprecedented for my Facebook) wave of support and comments indicating that many others have experienced these problems.

I am glad you are getting support. We live in a hard world, with little support for lots of things. BUT... you're getting lots of support and other people seeing the similar issues  is not at all the same as it being widespread. In the end science is not about who gets to vote, but about what is true.

Your comments on fraudulent data, with all respect, made me genuinely laugh:

Always glad to provide amusement and entertainment.

"I have seen almost nothing in my field that makes me think fraud."

Of course no one ever sees anything about fraud, but there is a lot of self-deception in a lot of fields. The main question here is, when you are a reviewer, are you viewing and requesting the entirety of the data set in a format that could not have possibly be altered or not? If not, then the fact that you have not seen anything is expected and irrelevant.

But you're claiming that you've seen lots of fraud? Why is your experience more valid than mine? I'm not arguing from age (though I have been in the game for longer than you), but from being a person who reviews, with their eyes open. I've seen enough folks go to toe to toe over variation in results, and done enough confirmation/replication to perceive the world differently from you. No, I don't think that there is widespread, frequent and pervasive fraud. Its hard to prove a negative, but when things can be replicated or subsequent results confirm the earlier... then yes, I think that there is honest reporting of results.

But... an important part of my point is that I know one corner of the world. I am not in the sexy, headline-making, BSD-multi-grant world. Maybe in that world the pressure is such that everyone cheats. Maybe your experiences are a statement about that corner of the science-sphere. But you can't possibly know enough of the entire scientific world to make sweeping statements about fraud everywhere.

The other problems of misconduct you point out may not be fraud, but they have the same effect in the end: they bring false data and put them under the limelight. I don't really care if fraud is intentional or if it operates through sub-conscious self-deception, the problem from outside is pretty much the same; perhaps with slightly different solutions.

Again, I disagree. My experience is different. I think attempts at replication would answer some of this, but one thing is true of the BSD/Glam world: there is no time for replication. And again, in my world this does happen, and it does get published. Thinking about this is important. Working on it is important. But your larger statements just don't hold water with my part of the world.

I am sorry to hear that you were insulted by the chicken without a head comment.

Apology accepted.

In my comment is intended as "if the hat fits, put it on." Of course I haven't said that the entire library of research ever performed in the history of mankind is useless. I commented simply on an attitude that I perceive, in some individuals I interact with, which leads to my intellectual dissatisfaction. I certainly do not include my PI in this comment, to the contrary, I have the greatest respect for him and I would include him among the exceptions.

Your caveats here are important. Your perceptions. Your interactions. They are entirely valid. But I read in your post  much more sweeping generalizations. How big was the lab you were in? How much did you interact with your PI? Is it interesting that the mentor you know the best is excluded? What if you knew others better? Your post implied nearly all. I don't think you know nearly all.

One of the things  I remember from when I was a grad  student was we all (and it was a fairly large program) thought: all the grad students are by and large good, or at least OK people; about half of the postdocs were, there were 2-3 junior faculty that wouldn't stab you in the back and all the senior faculty, with one exception were political animals who would gladly eat students for lunch if offered the opportunity. True from my perspective then? Yes. In retrospect, perhaps not. None of us is in a fixed perspective in our observations of the world. Its very tough to take that step back and try to see how who we are influences what we perceive.

I get to multiple of the most important (in size and in specialty) meetings in my field, and in fact in a broad array of fields. I pride myself with reading every single title of every single poster at any scientific meeting I attend,

You say you are a neuroscientist. I've gone to SfN  a couple of times (my work is on the boundary btwn neuro & physio), but I don't think you came up to me or read my poster ( I would have remembered). Do you really read every poster at SfN? I do know folks who try to "read every poster", but one can't, one simply can't attend every talk and see every poster. One has to pick and choose and how that picking and choosing is done is part of the problem.

I have read tens of thousands, and I have stopped to discuss with the researcher at hundreds of them throughout the years. I've been doing so for about 9 years. I am acutely aware of the research that is undergoing in pretty much every area of neuroscience.

Really, really, really? What about evolution of CPGs in mammals? What about motor leaning in cerebellar recovery from pediatric TBI? Those are not my areas, but because I have history or friends in them, I follow them. But I am always amazed at what I don't know about addiction research, neuropeptides in birds and all sorts of other things. What about the physiology of digestion? Or developmental control of growth in spiders?

I do not think I need to be sold on the wideness of the world.

Maybe not sold on, but if you think cheating/falsification is widespread and rampant, maybe you need to dig into those fields where its not. My experience is that glam sub-fields, the stuff that is Hot! Right! Now! gets a lot of attention, and has a much higher level of pressure than other fields. People who have no desire to or expectation of publishing in Science/Nature/Cell (again, IME) have a much different training experience than those who do.

It is totally valid to say "I am quitting this field because of my experiences in this field". What I object to is the sweeping generalizations about Science and  Biomedical Research and Knowledge. Come work in my lab for a while. I can certainly show you something different.

 

9 responses so far

  • drugmonkey says:

    You don't want this guy or anyone remotely like him in your lab. It would be a disaster.

  • tom says:

    is it me, or is it all people in that little subfield that think their subfield defines neuroscience as a whole?

  • The New PI says:

    I mostly agree with you, Potnia, but this discussion hit a trigger point for me...so here is my post in response 🙂
    http://thenewpi.blogspot.com/2015/09/to-glam-or-not-to-glam-this-is-question.htm

  • LincolnX says:

    Conclusions: some people cheat, some people fool themselves, and some people reap rewards they don't deserve.

    Welcome to Earth.

  • LincolnX says:

    Postdocs are a bright people who are entitled to their own perceptions and to set the course of action for their own lives. JFG has done that.

    That said, I do wonder what JFG's postdoc advisor thinks of all of this, and I can't help but personalize this a bit. I wonder whether the railing for the requirement for productivity and the conditions for success in our discipline are quite fair - JFG seems to have a enjoyed a rather enviable postdoctoral experience where in addition to doing good science he was given the time to build a scicomm brand for himself, did a kickstarter for his venture into NeuroTV, blogging for SfN, and is now writing a book. It's great he does these things.

    However, one wonders whether this level of engagement with "service" was helpful at his stage of development as a scientist in this competitive funding environment and whether this might have at all contributed to his own frustration with his progress, or the way he progressed. My thesis here is that JFG is a data point. I think he might agree with this. As a data point is his experience close to the average? That's where we might disagree.

    I absolutely think that postdocs should stretch their wings and explore multiple opportunities, even to the point of other career options: I support this with my own postdocs. But I try to calibrate their expectations about how the system rewards that dirty word "productivity". That calibration has multiple decision trees: academic vs. non-academic? Top tier research universities vs. teaching colleges? JFG seemed to be preparing himself for a job at a nice liberal arts research college that values teaching and outreach. Those are rewarding jobs. He prepared himself for a career in scicomm. I do not perceive that he was preparing himself correctly for an Ivy league or academic medical research institution, which requires an incredible level of independent and grantsmanship training, multiple pubs per year and a strong narrative.

    Yes the system is difficult, and is broken in many ways. But it is not uniquely broken in ways that other walks of life are not also broken, except idiosyncratic things like the overall practice of academic center to leverage NIH dollars to overhire.

    I wish JFG luck.

  • "Not so much my point as that everyone is interested in survival. And that the big wide world is full of jackasses. Leaving academic research is no guarantee that you won't end up facing the same mindset in other areas. In fact, I suspect you will. You were giving reasons for leaving; I am saying that you won't get beyond this by leaving."

    Well you are making a lot of assumptions. First, you are making the assumptions that I am looking for some place else where there is a different mindset. I have not stated "I'm leaving academia because there's something better out there in terms of job." I said: "I'm leaving academia because it does not satisfy my scientific curiosity anymore to be in that job." That being said, I do know that there are a lot of ways in which one can avoid having much colleagues, and so if that's what I would have been looking for, there would have been many options. Many of the work I do as an entrepreneur is deeply solitary - ironically, sometimes the most public stuff is actually done in complete isolation. So I do think there are many options for someone wanting solitude, if that's what I would have been looking for.

    "More on fraudulent a little bit further down, but you mention statistics vs. perception. This is important. I don't think the statistics I've seen (retraction watch, etc) suggest that fraud is that prevelant."

    These sites are very good but they recognize themselves that the picture they present must be an underestimation and it is not known by how much it misses the actual size of the problem. This can depend a lot on the field and the type of data processing that is expected of authors in these specific fields, but there are some fields such as single-unit recordings where there is an on-going race to have more and more neurons in a single paper, up to unrealistic numbers that everyone knows in the field must be very poorly-recorded neurons, if not pure noise, since acquiring quality single units take a lot of time and the stable recording of these faces a lot of challenges. Some of the most pessimistic people in that field would tell you that there must be more than 90% of papers that did not follow proper single-unit analysis and recording.

    "Your perceptions. Your interactions. They are entirely valid. But I read in your post much more sweeping generalizations."

    I genuinely think you have misread my post. I understand how it may look when there's thousands of shares and likes, but keep in mind that I have written this as my Sunday update on Facebook, which usually gets 10 likes. I really simply wanted to explain my decision, not contest the system or anything like that. My post explains why, personally, I leave, and it was first and foremost targeted at my friends and family.

    "Do you really read every poster at SfN? I do know folks who try to "read every poster", but one can't, one simply can't attend every talk and see every poster. One has to pick and choose and how that picking and choosing is done is part of the problem."

    I did not say I read every poster, I said I read every poster title. Then, yes, I do pick and choose, and maybe I went to your poster, maybe not, this is irrelevant.

    "Really, really, really? What about evolution of CPGs in mammals?"

    I have done my thesis on locomotor and respiratory CPGs and I have a doctoral level course in computational neuroscience on that subject. I have worked with one of the world experts in the evolution of CPGs from fish to mammals.

    "What about motor leaning in cerebellar recovery from pediatric TBI?"

    I follow this research from outside, but how are these specific questions relevant? Would my point be more valid if I had worked in a lab related to motor learning and TBI?

    "What about the physiology of digestion?"

    I have worked on the neural control of eating behaviors during my thesis. I may be one of the rare persons (less than 5) in this world who has visualized single motoneurons involved in tongue control on living snake's brains.

    "Or developmental control of growth in spiders?"

    I follow this research from outside and haven't worked on spiders. Again, I don't know who you think you are talking to. I'm a very opened intellectual with a vast, vast, vast repertoire of fields I am interested in following, which range from anthropology, philosophy and politics to molecular neuroscience and computer science.

    "What I object to is the sweeping generalizations about Science and Biomedical Research and Knowledge."

    There is no particular reason to limit my statement to a specific field, since there are many people outside of neuroscience who have already complained about these things, and there is no sign that neuroscience is a particularly problematic field concerning the problems I have spoken about. The Internet abounds with scientists who have stated the same thing that I stated, so much that it is actually surprising that my Facebook post got _any_ attention, since it has all been said already.

    I wouldn't state that I'm quitting "academic neuroscience" because this is not what I'm quitting. I'm quitting publicly-funded research academia. Before considering myself a neuroscientist, I consider myself a scientist and these are the words I want to use.

  • DrivingBy says:

    Again, I don't know who you think you are talking to. I'm a very opened intellectual with a vast, vast, vast repertoire of fields I am interested in following, which range from anthropology, philosophy and politics to molecular neuroscience and computer science.

    lol
    You forgot to inform us how unpretentious you are.

  • […] follows on the heels of Jean-François Gariépy's post on leaving science (my responses here and here), where he talked about the "highly-competitive environment... [with] scientists to be more […]

Leave a Reply