5 ms·

The premise of this article is better than its execution. The author even points in the direction of open and honest sharing, saying "others could surely have l

The premise of this article is better than its execution. The author even points in the direction of open and honest sharing, saying "others could surely have learned from the mistakes I made". But the only mistake pointed to is "I stopped reviewing his original data". Is that the root cause and only way to prevent these problems? If so, drive the point home more firmly.

flobosg 3y ago
The analysis in the article is very shallow and raises more questions than answers. What motivated the postdoc to fabricate the data? What kind of project were they working on? A postdoctoral position can be a perilous journey, especially if the project involved is a high-risk one (which would be my guess, given the author’s urge to get recognized in her area). If they don’t generate enough data and publications within those few years, a postdoc can leave the lab effectively empty-handed. There is a tremendous pressure for productivity, at all costs.
- shusaku 3y ago
  It was so shallow that to me it read like an editor took an axe to a shaky draft.
adr1an 3y ago
While I'd love to see bullet points on takeaways, I wouldn't put the 'load' for the lack of them on the author. Even if he expressed the intention to do such thing and then couldn't deliver... I mean, seems like revictimization in a way. He was a victim (even compares the situation to a theft). In these cases, maybe there's not much you can really do to be safe. It's really up to the other person, who should not be dishonest. That being said, on crucial factor on such behavior is paperianism (a.k.a. publish or perish) and the lack of interest of the biggest journals to publish negative results at all (it could be anything, from short communications to just a database)..
- light_hue_1 3y ago
  In my neck of science/AI/ML we've been talking about negative results being important as a community for a long time. But it never really happens. It's so much harder to judge negative results than positive results. And it's so incredibly hard to attribute blame. Why was this result negative? Did you screw something up? Something very trivial you don't normally even report on? It's totally possible and that makes negative papers hard to swallow. Anyone can produce an unlimited number of negative results by being incompetent.
  - throwup238 3y ago
    > Anyone can produce an unlimited number of negative results by being incompetent. This is well said, I'm surprised I haven't heard it expressed before. Assuming competence is rarer than incompetence, rewarding negative results will probably just drive the incentives even more to the extremes. Instead of outright fraud, everyone will get to hide behind "oops."
  - renonce 3y ago
    I can see the reason for not rewarding individual negative results, but not for not publishing them at all. Any single positive result is almost certainly the cumulation of lots of work, most of them being negative. You can’t pretend that you arrived at this positive result with zero failed attempts. Why can’t we ask for past negative results along with every positive result, just for the sake of completeness? The authors could have just released a bunch of Jupyter notebooks or whatever original format the negative experiment results are in - why everything in academia has to be written in a certain format where every word is carefully scrutinized to be considered useful?
- nextos 3y ago
  Another crucial factor is the principal investigator (PI) culture which has, at least in most sciences, turned professors into rent seekers. Nowadays, professors tend to play a middlemen role. Apply for grants, advertise results, and claim credit. Nothing else. Most of the time, they do not come up with ideas, nor care about them or do any of the hard work. Places like the Arc Institute have been born to cut PI middlemen out and get research out of this tar pit.
renonce 3y ago
It’s already very nice of the author to reflect and share his past mistakes aloud, as this would be a good start for others to raise the same question towards the science community in general. Pressing the author for detail into this specific mistake that happened years ago is nothing compared to lots of existing labs whose integrity has not been questioned at all.

flobosg 3y ago

The analysis in the article is very shallow and raises more questions than answers. What motivated the postdoc to fabricate the data? What kind of project were they working on? A postdoctoral position can be a perilous journey, especially if the project involved is a high-risk one (which would be my guess, given the author’s urge to get recognized in her area). If they don’t generate enough data and publications within those few years, a postdoc can leave the lab effectively empty-handed. There is a tremendous pressure for productivity, at all costs.

shusaku 3y ago
It was so shallow that to me it read like an editor took an axe to a shaky draft.

adr1an 3y ago

While I'd love to see bullet points on takeaways, I wouldn't put the 'load' for the lack of them on the author. Even if he expressed the intention to do such thing and then couldn't deliver... I mean, seems like revictimization in a way. He was a victim (even compares the situation to a theft). In these cases, maybe there's not much you can really do to be safe. It's really up to the other person, who should not be dishonest. That being said, on crucial factor on such behavior is paperianism (a.k.a. publish or perish) and the lack of interest of the biggest journals to publish negative results at all (it could be anything, from short communications to just a database)..

light_hue_1 3y ago
In my neck of science/AI/ML we've been talking about negative results being important as a community for a long time. But it never really happens. It's so much harder to judge negative results than positive results. And it's so incredibly hard to attribute blame. Why was this result negative? Did you screw something up? Something very trivial you don't normally even report on? It's totally possible and that makes negative papers hard to swallow. Anyone can produce an unlimited number of negative results by being incompetent.
- throwup238 3y ago
  > Anyone can produce an unlimited number of negative results by being incompetent. This is well said, I'm surprised I haven't heard it expressed before. Assuming competence is rarer than incompetence, rewarding negative results will probably just drive the incentives even more to the extremes. Instead of outright fraud, everyone will get to hide behind "oops."
- renonce 3y ago
  I can see the reason for not rewarding individual negative results, but not for not publishing them at all. Any single positive result is almost certainly the cumulation of lots of work, most of them being negative. You can’t pretend that you arrived at this positive result with zero failed attempts. Why can’t we ask for past negative results along with every positive result, just for the sake of completeness? The authors could have just released a bunch of Jupyter notebooks or whatever original format the negative experiment results are in - why everything in academia has to be written in a certain format where every word is carefully scrutinized to be considered useful?
nextos 3y ago
Another crucial factor is the principal investigator (PI) culture which has, at least in most sciences, turned professors into rent seekers. Nowadays, professors tend to play a middlemen role. Apply for grants, advertise results, and claim credit. Nothing else. Most of the time, they do not come up with ideas, nor care about them or do any of the hard work. Places like the Arc Institute have been born to cut PI middlemen out and get research out of this tar pit.

renonce 3y ago

It’s already very nice of the author to reflect and share his past mistakes aloud, as this would be a good start for others to raise the same question towards the science community in general. Pressing the author for detail into this specific mistake that happened years ago is nothing compared to lots of existing labs whose integrity has not been questioned at all.