5 ms·

Reddit, Google+, LinkedIn as Science Peer Review Systems

2 points by IgorCarron 14y ago

lutusp 14y ago
> Science is only as robust as its peer review system. This exaggerates the influence of peer review on modern science. Modern peer review is frequently (not always) a rubber-stamp way to catch perfect rubbish before it gets into print, but it cannot detect intentional fraud or sloppy work: Source: http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124 http://www.plosmedicine.org/article/info:doi/10.1371/journal... Title: "Why Most Published Research Findings Are False" In fact, science is only as robust as its practitioners and their motivations. At the moment, science is in the midst of a credibility crisis because of sometimes overwhelming pressure to produce "results" when there are none to be had: Source: http://www.reuters.com/article/2012/03/28/us-science-cancer-idUSBRE82R12P20120328 http://www.reuters.com/article/2012/03/28/us-science-cancer-... Title: "In cancer science, many 'discoveries' don't hold up" Quote: "During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 'landmark' publications -- papers in top journals, from reputable labs -- for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development. Result: 47 of the 53 could not be replicated." This is deplorable and justifies the "crisis" label. But changes in peer review won't make any difference.
- deleted 14y ago
  [deleted]
- IgorCarron 14y ago
  You are mistaken because you are, like many others, confusing peer review with "pre- publication peer review", the act of your peers judging a manuscript before publication as is currently implemented. The peer review that is allowed by the system I mention actually enables post publication peer review. The review of papers that are already public (either in some preprint or in some published form). Right now the only mechanism that is allowed in terms of feedback in the pre-publication peer review model, is writing to the journal for a corrigendum, or even a retraction. As witnessed in retractionwatch.com blog, few are enthusiastic about publishing corrections to papers that have gone through their inefficient pre-publication peer review process. Not only it is inefficient but it also directly yield examples such as the ones you, rightfully, point to.
  - deleted 14y ago
    [deleted]
  - lutusp 14y ago
    > You are mistaken because you are, like many others, confusing peer review with "pre- publication peer review" I invite you mimic the behavior of a scientist and uncover evidence that I took that position anywhere. Here's what I said: "Modern peer review is frequently (not always) a rubber-stamp way to catch perfect rubbish before it gets into print, but it cannot detect intentional fraud or sloppy work ..." I am well aware of post-publication peer review, but it's less likely to solve problems that pre-publication peer review haven't solved. > The peer review that is allowed by the system I mention actually enables post publication peer review. Yes, and post-publication peer review still cannot prevent the kinds of fraud and abuse that have led to the present credibility crisis, issues I outlined in my original post.
    - IgorCarron 14y ago
      The position you take on the rubber stamping part of peer review is simply not serious at best in many areas of science and engineering. Ask anybody around you that has published in good journals or conferences. Hence I did not address this statement because it simply is not true. Try sending papers to ICML, Siggraph, SODA, Nature, etc, I doubt the peer reviews are performing any sorts of rubber stamping. But once they have gone through that process and they are published it is simply very difficult to root out bad work given new data. You might be aware of post publication peer review, yet, this model is not in use except through accidental replication exercises that sometimes uncover problems that eventually yield corrigendae or retraction. Currently it is very adhoc. If you understand how science work beyond press releases, you'll know that the fraud and abuses will always be in the system. Post publication peer review is the only way to rooting out bad work that has been published or is in the preprint stage (I consder that if your preprint is out on arxiv or some other medium, it is in effect published). Right now, for published papers, it is left as an exercise for journals to acknowledge they let some dubious work through. There is simply no economic incentive for a speedy process. If you read retractionwath.com often you'll notice that the current system is simply not regulating itself. Open post publication peer review is a way to perform that function. Looking back it is formalizing the process by which people used to trust or not older work. It is also blurring the lines between preprints and published work since they are now under the same scrutiny. I agree it is also less convenient for the press or the science press to be comfortable with this situation (post publication peer review) but Science becomes robust when it is clearly capable of rooting out bad work through processes like this one.

2 points by IgorCarron 14y ago

> Science is only as robust as its peer review system. This exaggerates the influence of peer review on modern science. Modern peer review is frequently (not always) a rubber-stamp way to catch perfect rubbish before it gets into print, but it cannot detect intentional fraud or sloppy work: Source: http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124 http://www.plosmedicine.org/article/info:doi/10.1371/journal... Title: "Why Most Published Research Findings Are False" In fact, science is only as robust as its practitioners and their motivations. At the moment, science is in the midst of a credibility crisis because of sometimes overwhelming pressure to produce "results" when there are none to be had: Source: http://www.reuters.com/article/2012/03/28/us-science-cancer-idUSBRE82R12P20120328 http://www.reuters.com/article/2012/03/28/us-science-cancer-... Title: "In cancer science, many 'discoveries' don't hold up" Quote: "During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 'landmark' publications -- papers in top journals, from reputable labs -- for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development. Result: 47 of the 53 could not be replicated." This is deplorable and justifies the "crisis" label. But changes in peer review won't make any difference.

deleted 14y ago
[deleted]
IgorCarron 14y ago
You are mistaken because you are, like many others, confusing peer review with "pre- publication peer review", the act of your peers judging a manuscript before publication as is currently implemented. The peer review that is allowed by the system I mention actually enables post publication peer review. The review of papers that are already public (either in some preprint or in some published form). Right now the only mechanism that is allowed in terms of feedback in the pre-publication peer review model, is writing to the journal for a corrigendum, or even a retraction. As witnessed in retractionwatch.com blog, few are enthusiastic about publishing corrections to papers that have gone through their inefficient pre-publication peer review process. Not only it is inefficient but it also directly yield examples such as the ones you, rightfully, point to.
- deleted 14y ago
  [deleted]
- lutusp 14y ago
  > You are mistaken because you are, like many others, confusing peer review with "pre- publication peer review" I invite you mimic the behavior of a scientist and uncover evidence that I took that position anywhere. Here's what I said: "Modern peer review is frequently (not always) a rubber-stamp way to catch perfect rubbish before it gets into print, but it cannot detect intentional fraud or sloppy work ..." I am well aware of post-publication peer review, but it's less likely to solve problems that pre-publication peer review haven't solved. > The peer review that is allowed by the system I mention actually enables post publication peer review. Yes, and post-publication peer review still cannot prevent the kinds of fraud and abuse that have led to the present credibility crisis, issues I outlined in my original post.
  - IgorCarron 14y ago
    The position you take on the rubber stamping part of peer review is simply not serious at best in many areas of science and engineering. Ask anybody around you that has published in good journals or conferences. Hence I did not address this statement because it simply is not true. Try sending papers to ICML, Siggraph, SODA, Nature, etc, I doubt the peer reviews are performing any sorts of rubber stamping. But once they have gone through that process and they are published it is simply very difficult to root out bad work given new data. You might be aware of post publication peer review, yet, this model is not in use except through accidental replication exercises that sometimes uncover problems that eventually yield corrigendae or retraction. Currently it is very adhoc. If you understand how science work beyond press releases, you'll know that the fraud and abuses will always be in the system. Post publication peer review is the only way to rooting out bad work that has been published or is in the preprint stage (I consder that if your preprint is out on arxiv or some other medium, it is in effect published). Right now, for published papers, it is left as an exercise for journals to acknowledge they let some dubious work through. There is simply no economic incentive for a speedy process. If you read retractionwath.com often you'll notice that the current system is simply not regulating itself. Open post publication peer review is a way to perform that function. Looking back it is formalizing the process by which people used to trust or not older work. It is also blurring the lines between preprints and published work since they are now under the same scrutiny. I agree it is also less convenient for the press or the science press to be comfortable with this situation (post publication peer review) but Science becomes robust when it is clearly capable of rooting out bad work through processes like this one.