6 ms·

The current academic peer review system is an example of a process that worked fine when number of participants was small but fails at scale. I'd like to see so

The current academic peer review system is an example of a process that worked fine when number of participants was small but fails at scale. I'd like to see something like hacker news or reddit's social version of vetting papers in addition to any formal review process, perhaps with additional moderation from known experts. There is currently no site for community review of articles or papers and I think it would be a great resource for scientists of all kinds.

privong 12y ago
For physics and astronomy, there is the arXiv. Essentially all the papers (at least for astronomy) are posted there. The next step would be to add a community voting/vetting scheme on top of that. Statistics could be displayed alongside the article, and access to more detailed comments could be facilitated. Version tracking would enable people to update their papers to address criticism and suggestions. (arXiv already does allow people to replace their papers with new versions, and they retain the old ones, but there isn't really an easy way to "diff" two versions of the paper).
untilHellbanned 12y ago
Onarbor, https://onarbor.com https://onarbor.com, is trying to tackle the scaling issue you mention. Regarding the reviews and reputation building its very much in the model of HN, Reddit, Stackoverflow. There is also a Kickstarter-like funding component. Worth a look.
IndianAstronaut 12y ago
More than just peer review, what is crucial is replication of experiments by many. So often experiments are found too difficult to replicate rendering the results unusable.
pcrh 12y ago
Check out pubpeer. It is supposedly post-publication peer review, but in practice is usually limited to people making accusations of data manipulation. The conversations can be a bit more chaotic than is ideal for peer review. https://pubpeer.com/ https://pubpeer.com/
scientist 12y ago
Epistemio allows rating publications as well as publishing post-publication peer reviews. Even though ratings and reviews may be anonymous, reviewers are authenticated, in order to prevent the kind of issues mentioned in the target article. http://www.epistemio.com/ http://www.epistemio.com/

privong 12y ago

For physics and astronomy, there is the arXiv. Essentially all the papers (at least for astronomy) are posted there. The next step would be to add a community voting/vetting scheme on top of that. Statistics could be displayed alongside the article, and access to more detailed comments could be facilitated. Version tracking would enable people to update their papers to address criticism and suggestions. (arXiv already does allow people to replace their papers with new versions, and they retain the old ones, but there isn't really an easy way to "diff" two versions of the paper).

untilHellbanned 12y ago

Onarbor, https://onarbor.com https://onarbor.com, is trying to tackle the scaling issue you mention. Regarding the reviews and reputation building its very much in the model of HN, Reddit, Stackoverflow. There is also a Kickstarter-like funding component. Worth a look.

IndianAstronaut 12y ago

More than just peer review, what is crucial is replication of experiments by many. So often experiments are found too difficult to replicate rendering the results unusable.

pcrh 12y ago

Check out pubpeer. It is supposedly post-publication peer review, but in practice is usually limited to people making accusations of data manipulation. The conversations can be a bit more chaotic than is ideal for peer review. https://pubpeer.com/ https://pubpeer.com/

scientist 12y ago

Epistemio allows rating publications as well as publishing post-publication peer reviews. Even though ratings and reviews may be anonymous, reviewers are authenticated, in order to prevent the kind of issues mentioned in the target article. http://www.epistemio.com/ http://www.epistemio.com/