6 ms·

Or this[2] 2007 Science paper on ultra high energy cosmic ray source candidates ("anisotropy") that we had to retract because significance started dropping almo

Or this[2] 2007 Science paper on ultra high energy cosmic ray source candidates ("anisotropy") that we had to retract because significance started dropping almost the day the paper was approved. It was a fascinating experience as a junior member to follow the collaboration internal conversation and investigation on this, because a lot of extremely principled scientists were clearly deeply worried about losing their hard earned reputation. In the end, I am convinced that we were simply unlucky. [2] https://arxiv.org/pdf/0712.2843 https://arxiv.org/pdf/0712.2843

alexpotato 15d ago
> we had to retract because significance started dropping almost the day the paper was approved. It's stories like this that raise my p(we are in a simulation).
WarmWash 15d ago
Reminds me of the FTL neutrinos too, where the scientist where pretty much "hey, something is wrong, can you help us figure it out?" and the general public were the ones screaming "OMG! Physics is dead!" Then when it comes out as measurement error, the public is all "Damn these scientists are all hype machine clowns..."
- dd8601fn 14d ago
  I’m fine with that. Put it at the feet of pop science blogging. I’m less fine with the time and resources spent on mouse models. They already know you’d get the same utility from a magic 8 ball, but they do it anyway.
  - gus_massa 14d ago
    Mouse models are useful to discard very bad ideas. There was a recent experiment to use bacteria to kill cancer https://news.ycombinator.com/item?id=46306894 https://news.ycombinator.com/item?id=46306894 They tried like 40 bacterias in vitro, then like 9 in mice, and only 1 was useful in mice and they will continue only with that, perhaps in humans. Anyway, as you suggest, there is a high chance it will fail. Also, you can do nasty stuff to mice that would never be allowed with humans. In that experiment they injected cancer cells in mice with a bad inmune system, so they could get like 90 mice with cancer and run the experiment in a short time. No ethical committee would approve that in humans.
  - IAmBroom 14d ago
    Your claim seems to be that testing medicines in animals is useless, because "everyone knows it's not going to work". Congratulations. You've just reduced all of medical science to the Tuskegee STD experiment.
- pfdietz 14d ago
  That was a fiendish thing to debug; if I recall correctly it was a slightly and intermittently defective connector.
- dguest 14d ago
  Unfortunately it wasn't just the public: it caused so much uproar within the experiment that two of the highest ranking members resigned their posts [1]. I was a bit dismayed at the reaction within the physics community. Experiments absolutely do need to follow procedures like blinding and careful internal review (especially before the data unblinding), but you can only spend so long designing the analysis before you unblind, and there are opportunity costs to cross checking everything. In an optimized community experiments will inevitably make mistakes. And once you unblind, it does no one any good to sit on an anomalous result forever. [1]: https://www.nature.com/articles/nature.2012.10371 https://www.nature.com/articles/nature.2012.10371
logdahl 14d ago
Wow haha, that is a lot of authors! Never seen this before!
- thiagotomei 14d ago
  The ATLAS-CMS joint Higgs boson mass measurement paper has close to 6000 authors: https://doi.org/10.1103/PhysRevLett.114.191803 https://doi.org/10.1103/PhysRevLett.114.191803 I think the final COVID consortium report has something like 30k authors.
- kens 14d ago
  Most horrifying is a 2014 Science paper on Ebola with 58 authors, 5 of whom died of Ebola before publication. https://www.science.org/content/article/ebolas-heavy-toll-study-authors https://www.science.org/content/article/ebolas-heavy-toll-st...
- smueller1234 13d ago
  As another comment points out: that's nothing compared to high energy/particle physics! The Pierre Auger Observatory certainly is a large collaboration for the astroparticle physics domain though. It's a big international collaboration. Quick anecdote: my name (S Mueller) is not on that paper's author list because we had a rule that you had to be in the collaboration for a year before getting authorship. You stayed on for a year after leaving. Very reasonable! At the time I was nonetheless a bit bummed about missing out on the big Science paper. I guess I'm on the retraction though ;)

alexpotato 15d ago

> we had to retract because significance started dropping almost the day the paper was approved. It's stories like this that raise my p(we are in a simulation).

WarmWash 15d ago

Reminds me of the FTL neutrinos too, where the scientist where pretty much "hey, something is wrong, can you help us figure it out?" and the general public were the ones screaming "OMG! Physics is dead!" Then when it comes out as measurement error, the public is all "Damn these scientists are all hype machine clowns..."

dd8601fn 14d ago
I’m fine with that. Put it at the feet of pop science blogging. I’m less fine with the time and resources spent on mouse models. They already know you’d get the same utility from a magic 8 ball, but they do it anyway.
- gus_massa 14d ago
  Mouse models are useful to discard very bad ideas. There was a recent experiment to use bacteria to kill cancer https://news.ycombinator.com/item?id=46306894 https://news.ycombinator.com/item?id=46306894 They tried like 40 bacterias in vitro, then like 9 in mice, and only 1 was useful in mice and they will continue only with that, perhaps in humans. Anyway, as you suggest, there is a high chance it will fail. Also, you can do nasty stuff to mice that would never be allowed with humans. In that experiment they injected cancer cells in mice with a bad inmune system, so they could get like 90 mice with cancer and run the experiment in a short time. No ethical committee would approve that in humans.
- IAmBroom 14d ago
  Your claim seems to be that testing medicines in animals is useless, because "everyone knows it's not going to work". Congratulations. You've just reduced all of medical science to the Tuskegee STD experiment.
pfdietz 14d ago
That was a fiendish thing to debug; if I recall correctly it was a slightly and intermittently defective connector.
dguest 14d ago
Unfortunately it wasn't just the public: it caused so much uproar within the experiment that two of the highest ranking members resigned their posts [1]. I was a bit dismayed at the reaction within the physics community. Experiments absolutely do need to follow procedures like blinding and careful internal review (especially before the data unblinding), but you can only spend so long designing the analysis before you unblind, and there are opportunity costs to cross checking everything. In an optimized community experiments will inevitably make mistakes. And once you unblind, it does no one any good to sit on an anomalous result forever. [1]: https://www.nature.com/articles/nature.2012.10371 https://www.nature.com/articles/nature.2012.10371

logdahl 14d ago

Wow haha, that is a lot of authors! Never seen this before!

thiagotomei 14d ago
The ATLAS-CMS joint Higgs boson mass measurement paper has close to 6000 authors: https://doi.org/10.1103/PhysRevLett.114.191803 https://doi.org/10.1103/PhysRevLett.114.191803 I think the final COVID consortium report has something like 30k authors.
kens 14d ago
Most horrifying is a 2014 Science paper on Ebola with 58 authors, 5 of whom died of Ebola before publication. https://www.science.org/content/article/ebolas-heavy-toll-study-authors https://www.science.org/content/article/ebolas-heavy-toll-st...
smueller1234 13d ago
As another comment points out: that's nothing compared to high energy/particle physics! The Pierre Auger Observatory certainly is a large collaboration for the astroparticle physics domain though. It's a big international collaboration. Quick anecdote: my name (S Mueller) is not on that paper's author list because we had a rule that you had to be in the collaboration for a year before getting authorship. You stayed on for a year after leaving. Very reasonable! At the time I was nonetheless a bit bummed about missing out on the big Science paper. I guess I'm on the retraction though ;)