6 ms·

If I were a billionaire, I'd fund a research institute whose only purpose would be to detect and call out fraud and shoddy work in academia. I'd hire the very b

by javert 4y ago

If I were a billionaire, I'd fund a research institute whose only purpose would be to detect and call out fraud and shoddy work in academia. I'd hire the very best people I could in each field and put them onto this task. I imagine it would be easy to find excellent but disgruntled scientists in every niche. You'd need to hire the best people with niche expertise. I know from my years in academia, nobody can understand and scrutinize your bullshit unless they work in the same sub-sub-sub-field.

frou_dh 4y ago
Perhaps you'd still have to rotate your own people because of the concept of Regulatory Capture.
Qem 4y ago
Just create a glassdoor-like site, but for ex-students tô tell their experiences with research groups and advisors.
- photochemsyn 4y ago
  That's not going to work very well, you'll get a mixed cohort of people reporting actual fraud and deceptive practices (the desired group) and people who were just pissed off because they didn't get the support they needed (a legitimate complaint, but not as relevant to the issue of fraud) or had negative personal interactions (not such a legitimate complaint, as they might have been the problematic person).
  - Qem 4y ago
    I think the same issues affect Glassdoor, but it didn't stop the site being a hit. Just let people speak, and the audience judge by themselves and filter the outlier report from the problematic person.
hoseja 4y ago
You'd probably suffer a fatal accident with chinese characteristics very soon.
- javert 4y ago
  That made me laugh. Much of the fraudulent work isn't coming from Chinese researchers, but much of it is.
mike_hearn 4y ago
But who would listen? There are already people who do this work for free, no billionaires needed, but we're all told not to listen to anything they say on pain of being classed as an anti-science "denier" / conspiracy theorist / etc. Fundamentally, 97% of scientists (in this study) refused to share their data because the people paying them don't care. The people paying them also wouldn't care about whatever your billionaire foundation would say, and in fact many would class it as a right wing misinformation operation. Then you'd be blocked from Twitter, downranked on Google, blocked from YouTube and you'd be unable to hire the "best" people because they'd be socially ostracized for joining. The problems here are really very deep, and they're social in nature. You can't fix it by simply pointing out the problems because everyone who looks already knows what the problems are. They're obvious and numerous. Fixing them requires fixing the incentives but politicians, the media and large chunks of the electorate refuse to acknowledge that research fraud can even exist at all (the "believe science" brigade).
- curuinor 4y ago
  that just means that you need to hire half disgruntled scientists and half disgruntled PR folks, journos, attack dogs, muckrakers, etc, doesn't it, then?
- javert 4y ago
  Valid points. It's an uphill battle, but still winnable over the long run. A big part of it is maintaining an extremely high level of scientific rigor and professionalism. Take the NSF for example. (Pardon a US-centric example.) If you have a record showing that a particular PI has published 10 provably false/fraudulent papers, is the NSF really going to keep funding that person? At some point, the desire to avoid embarrassment and save their own skin may drive the NSF to de-risk by not funding the flagrant frauds. Imagine if this were a world-class research institute, supported morally and with funding by people like Bill Gates and Elon Musk. Imagine if those people recognize that we can advance science and global progress much farther and faster if we tackle the epidemic of garbage research in academia (which I'd say is over 60%). Imagine running TV ads that say: "No, we are not right-wing nutjobs. There really IS a plague of garbage research." (In fact, right-wing sponsors should be totally ruled out. As well as clearly left-wing sponsors. We want basically nonpolitical people who are basically trusted, like, again, Bill and/or Melinda Gates, Elon Musk, and so on. So, no Peter Thiel, which is unfortunate, because it would be right up his alley.) But then you're publishing papers that try to replicate studies and specifically calling out specific frauds and getting their funding taken away, and more broadly arguing for funding agencies and academic departments to stop tolerating the culture of fraud and shoddy work. And you're calling out the worst offenders (in terms of academic departments and funding agencies) and calling on their political sponsors (like Congressmen) to cut their funding if they don't clean it up. There may be areas of science where the foundation doesn't go, like climate change (maybe race?), because they are too heated. That's OK. It doesn't have to cover everything. Part of the reason academia is really broken is, people aren't rewarded for proving that results can't be replicated. We need to find a way to incentivize that. The scientific process does not work without it. Also, academia doesn't allow people to just say, "This idea is wrong." Academia is composed of a million little old-boys-club silos. You don't get into any of these clubs if you are not going to play ball. De facto, you cannot criticize your peers, if you are in academia. For instance, the NSF has grant-award panels that are composed of researchers in the niche. So if you criticize your peers you literally cannot get NSF funding. Because your funding is literally gated by your "peers" as in "peer review." But if you were gonna be that way, you wouldn't have gotten a tenure-track job in the first place, because tenured professors only want people in the club to become their "peers" and sit on those committees. I'm concerned I may be kind of overstating my case, but it's hard for me to know exactly how widespread the worst-case version of the problem is. Certainly what I'm describing can and does happen.

by javert 4y ago

frou_dh 4y ago

Perhaps you'd still have to rotate your own people because of the concept of Regulatory Capture.

Qem 4y ago

Just create a glassdoor-like site, but for ex-students tô tell their experiences with research groups and advisors.

photochemsyn 4y ago
That's not going to work very well, you'll get a mixed cohort of people reporting actual fraud and deceptive practices (the desired group) and people who were just pissed off because they didn't get the support they needed (a legitimate complaint, but not as relevant to the issue of fraud) or had negative personal interactions (not such a legitimate complaint, as they might have been the problematic person).
- Qem 4y ago
  I think the same issues affect Glassdoor, but it didn't stop the site being a hit. Just let people speak, and the audience judge by themselves and filter the outlier report from the problematic person.

hoseja 4y ago

You'd probably suffer a fatal accident with chinese characteristics very soon.

javert 4y ago
That made me laugh. Much of the fraudulent work isn't coming from Chinese researchers, but much of it is.

mike_hearn 4y ago

But who would listen? There are already people who do this work for free, no billionaires needed, but we're all told not to listen to anything they say on pain of being classed as an anti-science "denier" / conspiracy theorist / etc. Fundamentally, 97% of scientists (in this study) refused to share their data because the people paying them don't care. The people paying them also wouldn't care about whatever your billionaire foundation would say, and in fact many would class it as a right wing misinformation operation. Then you'd be blocked from Twitter, downranked on Google, blocked from YouTube and you'd be unable to hire the "best" people because they'd be socially ostracized for joining. The problems here are really very deep, and they're social in nature. You can't fix it by simply pointing out the problems because everyone who looks already knows what the problems are. They're obvious and numerous. Fixing them requires fixing the incentives but politicians, the media and large chunks of the electorate refuse to acknowledge that research fraud can even exist at all (the "believe science" brigade).

curuinor 4y ago
that just means that you need to hire half disgruntled scientists and half disgruntled PR folks, journos, attack dogs, muckrakers, etc, doesn't it, then?
javert 4y ago
Valid points. It's an uphill battle, but still winnable over the long run. A big part of it is maintaining an extremely high level of scientific rigor and professionalism. Take the NSF for example. (Pardon a US-centric example.) If you have a record showing that a particular PI has published 10 provably false/fraudulent papers, is the NSF really going to keep funding that person? At some point, the desire to avoid embarrassment and save their own skin may drive the NSF to de-risk by not funding the flagrant frauds. Imagine if this were a world-class research institute, supported morally and with funding by people like Bill Gates and Elon Musk. Imagine if those people recognize that we can advance science and global progress much farther and faster if we tackle the epidemic of garbage research in academia (which I'd say is over 60%). Imagine running TV ads that say: "No, we are not right-wing nutjobs. There really IS a plague of garbage research." (In fact, right-wing sponsors should be totally ruled out. As well as clearly left-wing sponsors. We want basically nonpolitical people who are basically trusted, like, again, Bill and/or Melinda Gates, Elon Musk, and so on. So, no Peter Thiel, which is unfortunate, because it would be right up his alley.) But then you're publishing papers that try to replicate studies and specifically calling out specific frauds and getting their funding taken away, and more broadly arguing for funding agencies and academic departments to stop tolerating the culture of fraud and shoddy work. And you're calling out the worst offenders (in terms of academic departments and funding agencies) and calling on their political sponsors (like Congressmen) to cut their funding if they don't clean it up. There may be areas of science where the foundation doesn't go, like climate change (maybe race?), because they are too heated. That's OK. It doesn't have to cover everything. Part of the reason academia is really broken is, people aren't rewarded for proving that results can't be replicated. We need to find a way to incentivize that. The scientific process does not work without it. Also, academia doesn't allow people to just say, "This idea is wrong." Academia is composed of a million little old-boys-club silos. You don't get into any of these clubs if you are not going to play ball. De facto, you cannot criticize your peers, if you are in academia. For instance, the NSF has grant-award panels that are composed of researchers in the niche. So if you criticize your peers you literally cannot get NSF funding. Because your funding is literally gated by your "peers" as in "peer review." But if you were gonna be that way, you wouldn't have gotten a tenure-track job in the first place, because tenured professors only want people in the club to become their "peers" and sit on those committees. I'm concerned I may be kind of overstating my case, but it's hard for me to know exactly how widespread the worst-case version of the problem is. Certainly what I'm describing can and does happen.