Archive for: September, 2015

Response to Disappointment and Discouragement in Research

Sep 11 2015 Published by under Uncategorized

I must say that I very much appreciate Jean-François Gariépy's politeness in disagreeing with me. Props to you, Jean-Francois. Further, he posted a long response, so I wanted to put it in a post, so I could address specific points in his response. As always, I welcome his further responses, as well as other people's comments. I've put the whole thing here, again, so that I am not cherry-picking his words.

Hi Potty,

I do not know what to think about your post, on the one hand it seems like you are confirming most of my complaints, but on the other hand you seem to not want to admit it. First, thank you for recognizing the possibility that all of what I say is true (sic).

I agree that being interested in one's survival is not a defect in and of itself.

Not so much my point as that everyone is interested in survival. And that the big wide world is full of jackasses. Leaving academic research is no guarantee that you won't end up facing the same mindset in other areas. In fact, I suspect you will.  You were giving reasons for leaving; I am saying that you won't get beyond this by leaving.

In the process of doing good law defenses, a lawyer may be interested in his own survival. In the process of saving a life, a doctor may be interested in his own economic well-being. The problem would come when this desire to survive impedes on the expected mission of the professional, and the point of my post is not so much that caring for one's survival is a problem in and of itself, but that it has now impeded upon our search for truth in many ways, including fraudulent data (and one only needs to look at the numbers to show this, my own perception and experience is barely relevant in face of the statistics on scientific misconduct).

More on fraudulent a little bit further down, but you mention statistics vs. perception. This is important. I don't think the statistics I've seen (retraction watch, etc) suggest that fraud is that prevelant.

Concerning #2, your argument seems to be that it's "the state of the world." This does not make it immune to critique. You also write that it causes anxiety and stress, so somehow you agree with me that it can be a problem (although stress and anxiety was not my focus, I was mostly worried about the amount of time lost on written products that are not part of the contribution of a researcher to society).

Yes, its  a problem. But its a problem for just about every and any professional. Again, this was one of your reasons for leaving. Again, I do not think that leaving will get you away from this problem. And, none of my non-academic friends feel that they spend less time on this than do the academic ones.

I do not claim that #1 and #3 are universal truths, but they are common enough that my little Facebook post has gathered an unprecedented (unprecedented for my Facebook) wave of support and comments indicating that many others have experienced these problems.

I am glad you are getting support. We live in a hard world, with little support for lots of things. BUT... you're getting lots of support and other people seeing the similar issues  is not at all the same as it being widespread. In the end science is not about who gets to vote, but about what is true.

Your comments on fraudulent data, with all respect, made me genuinely laugh:

Always glad to provide amusement and entertainment.

"I have seen almost nothing in my field that makes me think fraud."

Of course no one ever sees anything about fraud, but there is a lot of self-deception in a lot of fields. The main question here is, when you are a reviewer, are you viewing and requesting the entirety of the data set in a format that could not have possibly be altered or not? If not, then the fact that you have not seen anything is expected and irrelevant.

But you're claiming that you've seen lots of fraud? Why is your experience more valid than mine? I'm not arguing from age (though I have been in the game for longer than you), but from being a person who reviews, with their eyes open. I've seen enough folks go to toe to toe over variation in results, and done enough confirmation/replication to perceive the world differently from you. No, I don't think that there is widespread, frequent and pervasive fraud. Its hard to prove a negative, but when things can be replicated or subsequent results confirm the earlier... then yes, I think that there is honest reporting of results.

But... an important part of my point is that I know one corner of the world. I am not in the sexy, headline-making, BSD-multi-grant world. Maybe in that world the pressure is such that everyone cheats. Maybe your experiences are a statement about that corner of the science-sphere. But you can't possibly know enough of the entire scientific world to make sweeping statements about fraud everywhere.

The other problems of misconduct you point out may not be fraud, but they have the same effect in the end: they bring false data and put them under the limelight. I don't really care if fraud is intentional or if it operates through sub-conscious self-deception, the problem from outside is pretty much the same; perhaps with slightly different solutions.

Again, I disagree. My experience is different. I think attempts at replication would answer some of this, but one thing is true of the BSD/Glam world: there is no time for replication. And again, in my world this does happen, and it does get published. Thinking about this is important. Working on it is important. But your larger statements just don't hold water with my part of the world.

I am sorry to hear that you were insulted by the chicken without a head comment.

Apology accepted.

In my comment is intended as "if the hat fits, put it on." Of course I haven't said that the entire library of research ever performed in the history of mankind is useless. I commented simply on an attitude that I perceive, in some individuals I interact with, which leads to my intellectual dissatisfaction. I certainly do not include my PI in this comment, to the contrary, I have the greatest respect for him and I would include him among the exceptions.

Your caveats here are important. Your perceptions. Your interactions. They are entirely valid. But I read in your post  much more sweeping generalizations. How big was the lab you were in? How much did you interact with your PI? Is it interesting that the mentor you know the best is excluded? What if you knew others better? Your post implied nearly all. I don't think you know nearly all.

One of the things  I remember from when I was a grad  student was we all (and it was a fairly large program) thought: all the grad students are by and large good, or at least OK people; about half of the postdocs were, there were 2-3 junior faculty that wouldn't stab you in the back and all the senior faculty, with one exception were political animals who would gladly eat students for lunch if offered the opportunity. True from my perspective then? Yes. In retrospect, perhaps not. None of us is in a fixed perspective in our observations of the world. Its very tough to take that step back and try to see how who we are influences what we perceive.

I get to multiple of the most important (in size and in specialty) meetings in my field, and in fact in a broad array of fields. I pride myself with reading every single title of every single poster at any scientific meeting I attend,

You say you are a neuroscientist. I've gone to SfN  a couple of times (my work is on the boundary btwn neuro & physio), but I don't think you came up to me or read my poster ( I would have remembered). Do you really read every poster at SfN? I do know folks who try to "read every poster", but one can't, one simply can't attend every talk and see every poster. One has to pick and choose and how that picking and choosing is done is part of the problem.

I have read tens of thousands, and I have stopped to discuss with the researcher at hundreds of them throughout the years. I've been doing so for about 9 years. I am acutely aware of the research that is undergoing in pretty much every area of neuroscience.

Really, really, really? What about evolution of CPGs in mammals? What about motor leaning in cerebellar recovery from pediatric TBI? Those are not my areas, but because I have history or friends in them, I follow them. But I am always amazed at what I don't know about addiction research, neuropeptides in birds and all sorts of other things. What about the physiology of digestion? Or developmental control of growth in spiders?

I do not think I need to be sold on the wideness of the world.

Maybe not sold on, but if you think cheating/falsification is widespread and rampant, maybe you need to dig into those fields where its not. My experience is that glam sub-fields, the stuff that is Hot! Right! Now! gets a lot of attention, and has a much higher level of pressure than other fields. People who have no desire to or expectation of publishing in Science/Nature/Cell (again, IME) have a much different training experience than those who do.

It is totally valid to say "I am quitting this field because of my experiences in this field". What I object to is the sweeping generalizations about Science and  Biomedical Research and Knowledge. Come work in my lab for a while. I can certainly show you something different.

 

9 responses so far

Disappointment and Discouragement in Research

Sep 07 2015 Published by under Uncategorized

Jean-François Gariépy has written a facebook post about leaving his research position at Duke University. The main reason, he says, is that he is going to be a father, and wishes to devote more time to his child. I do wish Dr. Gariepy well. He has made a choice that is hard to make, and seems to be doing so with clear eyes.

He has a set of secondary reasons:

Other reasons have to do with research academia itself. ... I found scientists to be more preoccupied by their own survival in a very competitive research environment than by the development of a true understanding of the world.

and

By creating a highly-competitive environment that relies on the selection of researchers based on their "scientific productivity," as it is referred to, we have populated the scientific community with what I like to call "chickens with no head," that is, researchers who can produce multiple scientific articles per year, none of which having any significant impact on our understanding of the world.

Some of the problems he cites are:

[1] a high number of scientific articles with fraudulent data, due to the pressures of the "publish or perish" system, makes it impossible to know if a recent discovery is true or not; [2] a large portion of the time of a scientist is spent just writing grants so that they can be submitted to 5-10 agencies in the hope that one of them will accept; and [3] our scientific publication system has become so corrupted that it is almost impossible to get a scientific article published in an important journal without talking one-on-one with the editor before submitting the article.

These are long excerpts, but I wanted to make sure I was not inadvertently slanting his words.

I do not doubt that all of this is true, or seems true, to him. But this is not everyone's experience, although the discouragement is too frequent amongst human beings of all stripes and professions.

Everyone is preoccupied with their survival. Sorry, is this is any different from trying to be an ethical, honorable lawyer or a doctor (yes, they do exist)? Or being a part-time single-parent worker in the service industry with three children in danger of being laid off? The world of scientific research is competitive, but most of the western world is, too.

The list of three problems is certainly a function of the lab one chooses. That #2 plagues nearly everyone in my current and past departments, tenured and untenured, is the state of the world. But not everyone feels this way. I have a friend, untenured, but near. She just got a 5 year r01. She has a grad student, a postdoc and a good collaborator. She is not chasing funding now. In 3.5 years, will she be back in the same position? Probably, but for now she can do her research and work at what she loves. People do get funded. Some people do not run big labs and one grant is enough to make their operation go. I do not deny that its hard. I do not deny that it provokes anxiety and unhappiness and stress. But my friends who are in business for themselves, as graphic designers or pub owners, look at this day in and day out.  They still have to put food on the table for their kids, but say its the price for independence.

#1 (fraudulent research)  and #3 (needing to know the editor) are not universal truths. In fact, I have seen almost nothing in my field that makes me think fraud. I do not think that I am naive. I review (although I try to avoid it) for the major journals in my subdiscipline. I've had a number of collaborators over the years. There is a lot of schlock that gets published. I know who rushes to get something off, and hasn't quite done the right analyses, or pushes the results further than they warrant. Or doesn't bother to find the right context (ie my work) for their results. But its not fraud. As for knowing the editor, I publish regularly, and the only editor I know is an old friend in another field in a journal quite distant. Perhaps the difference is that I don't publish in SNC.

Which brings us to the chicken without a head syndrome. Don't insult me that way. Your PI may be chicken-heading it, but that's not the world I live in, nor many of my colleagues. I know people doing clinical work that may not be as flash as gene-therapy, but it is making a difference for how people with a stroke learn to use their hands again.  I know ecologists who are understanding the interactions among bacteria and health in mammals living in the field. It may be bunny-hopping, but understanding how rabbit locomotor skills, not just speed, but agility, relate to survival across a boreal winter is adding something important to our knowledge. How fucking dare you say that this work being done't doesn't add to knowledge.

Do you only read the science in your field? Do you only see the posters/talks at meetings that are in your corner of the field? If you are a neuroscientist, go to ExBio or SICB or Ecology/Evolution meetings. Yeah, they are expensive, but get a bit of your work, and think of its bigger implications and submit it elsewhere. Go talk to someone else. The world is wide, minnow, and there are oceans you haven't swum in yet.

23 responses so far

quote of the day

Sep 01 2015 Published by under Uncategorized

Truth is tough. It will not break, like a bubble, at a touch; nay, you may kick it about all day, like a football, and it will be round and full at evening. -- Mark Twain

It just sometimes take a while to get to the evening.

No responses yet

Early days?

Sep 01 2015 Published by under Uncategorized

Wes Craven, the horror film director has died:

His name became synonymous with horror, thanks to films like A Nightmare on Elm Street, Last House on the Left, and the Scream series—a most unexpected career for a man who didn’t even see his first movie until he was in college (emphasis mine).

Do you think it would be possible today, to jump into a career and start making movies? The article goes on to quote various interviews with him, and how, almost by chance, he started making movies. I don't know enough about the film business to know if such a trajectory is possible today. I've got the sense it might be possible in some areas of music, some areas of writing.

Which brings us to "breaking into science". As someone once remarked to me (and I have forgotten who), it was a lot easier for Albert Einstein to be a fair to good amateur violinist than for Yeduhi Menuhnin to be a good amateur physicist.

Is it easier in the beginning of a field to break in, to be the person who sets the world on its ear, with only minimal or non-professional training? Or is it a function of the technology, and access to (at the time) expensive infrastructure that determines who can do what?

No responses yet

« Newer posts