7 ms·
FiveThirtyEight articles on the Internet Archive
- ChocMontePy 4mo agoGithub link: https://github.com/palewire/fivethirtyeightindex.com https://github.com/palewire/fivethirtyeightindex.com
- nl 4mo agoThis is because whoever owns Fivethirtyeight now (ABC?) deleted the whole archive of articles on the site.
- Avicebron 4mo ago[flagged]
- tantalor 4mo agoPlease, say that again in comprehensible English.
- Avicebron 4mo agoThe ownership relationship was always load-bearing? The journalism in this case was a tenant, I highly recommend that people promote forms of independent journalism? EDIT: dude have you heard of the s in https, http://johntantalo.com http://johntantalo.com gets flagged.
- bombcar 4mo agoDon't we need more than an index of Archive.org because whomever controls the domain could robots.txt these out of existence if they wanted to?
- ycombinete 4mo agoArchive.org mostly ignores robots.txt https://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/ https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...
- Jiro 4mo agoPeople always use that link as reference to say that Internet Archive ignores robots.txt but it only actually says they are ignoring it for government sites. It suggests that they might do it for other sites in the future (of 2017), but does not actually say that that they have done it. https://blog.archive.org/2018/04/24/addressing-recent-claims-of-manipulated-blog-posts-in-the-wayback-machine/ https://blog.archive.org/2018/04/24/addressing-recent-claims... which is a year later mentions that they have an automated process which is still following robots.txt for displaying old pages where the robots.txt was added later. https://help.archive.org/help/using-the-wayback-machine/ https://help.archive.org/help/using-the-wayback-machine/ does say they follow it for scraping, but this is phrased in such a way that would still be true for past sites whether or not they changed the policy. There is a page https://www.sysjolt.com/2021/archive-org-no-longer-honors-robot-txt/ https://www.sysjolt.com/2021/archive-org-no-longer-honors-ro... which claims they don't follow it, but the site owner misspelled "robots" as "robot".
- bombcar 4mo agoThat first link is confusing; it seems to say they ended up removing the pages not because of a legal threat but because of robots.txt “automated”. If archive.org can be manipulated to remove content either via legal threats or simple robots.txt it loses a significant portion of its societal value.
- zzo38computer 4mo agoThe robots.txt file should be used to restrict (and, in some cases, slow down) crawling at the time it is being crawled, not for SEO or for restricting access to mirrors or for any other purpose. It should never apply retroactively. (Unfortunately it is sometimes used badly despite this.)
- tardedmeme 4mo agoIt's not about robots.txt but yes, the owners of 538 can just send a cease and desist letter to get them all immediately removed. Many sites that don't want to preserve history have done this already.
- nomilk 4mo agoCouldn't figure out why archiving FTE aricles matters, but a quick search yields: > Thousands of FiveThirtyEight articles seemingly vanish from the internet https://www.editorandpublisher.com/stories/thousands-of-fivethirtyeight-articles-seemingly-vanish-from-the-internet,261686 https://www.editorandpublisher.com/stories/thousands-of-five... And discussions here on hn: ABC News has taken all FiveThirtyEight articles offline https://news.ycombinator.com/item?id=48152553 https://news.ycombinator.com/item?id=48152553 Disney erased FiveThirtyEight (article by Nate himself) https://news.ycombinator.com/item?id=48197703 https://news.ycombinator.com/item?id=48197703
- ricardobeat 4mo agoI don’t really know much about it, but remember it as being _fantastic_ journalism every time I encountered one of their articles. As a bonus, great infographics and interactive data visualizations.
- defrost 4mo agoFor any, like myself, wondering "Who is Ben Welsh" ? Hello. My name is Ben Welsh. I'm an Iowan living in New York City. I am a reporter, an editor and a computer programmer. My job is to use those skills, together, to find and tell stories. I work at Reuters, the world's largest multimedia news provider, where I founded the organization's News Applications Desk. In that role, I lead the development of dashboards, databases and automated systems that benefit clients, inform readers, empower reporters and serve the public interest. [...] ~ https://palewi.re/who-is-ben-welsh/ https://palewi.re/who-is-ben-welsh/
- yogorenapan 4mo agoCan't believe Ben Welsh is not Welsh, and FiveThirtyEight has nothing to do with Wales
- simonw 4mo agoBen is one of my favorite people in the world of data journalism. He's the author of many excellent training courses in the field, including: - https://github.com/palewire/first-python-notebook https://github.com/palewire/first-python-notebook - https://github.com/palewire/first-web-scraper https://github.com/palewire/first-web-scraper - https://github.com/palewire/first-graphics-app https://github.com/palewire/first-graphics-app
- dang 4mo ago(Submitted title was "Ben Welsh made an index of all FiveThirtyEight articles on the Internet Archive" - we've since changed it)
- ChrisArchitect 4mo agoLove Ben but title can simply be: Index of FiveThirtyEight articles preserved by the Internet Archive
- buildsjets 4mo agoBut that would be a false attribution. The Internet Archive did not create the index, Ben did. And the Internet Archive is not hosting the index, Ben is.
- ChrisArchitect 4mo agoAh, yes, could be worded better, fairplay. Point is the Ben attribution isn't needed in that place to avoid unnecessary confusion about who that is etc.
- palewire 4mo agoPlease give archive.org all the credit
- 3eb7988a1663 4mo agoIf I wanted to get the complete WARC archive of 538 - how do you do this in a friendly way? No interest in history tracking, just want the last available version from Internet Archive.
- palewire 4mo agoThe site includes a CSV of everything I found. You could hand that to Claude Code and ask it to pull down the WARCs
- internet2000 4mo agoI'm seeing a lot about this. What makes this situation different than any other website going offline?
- deleted 4mo ago[deleted]
- patcon 4mo agoI think it's the fivethirtyeight of of historical significance, and Disney is one of the largest and wealthiest companies on the planet. So it's just kinda like "whoa, this is stratospheric negligence" or "whoa, what is the reason for this... assuming they are not idiots?"
- materielle 4mo agoAlso, they don’t any plans for the IP, and Nate would’ve paid above-market rate just to take over and preserve the content for posterity. He estimates that they deleted 200,000 hours of human labor. This is just some Disney suits being extraordinarily petty.
- dreijs 4mo agoYes, just to add to this: in the article by Nate [0] he says that he tried to buy the IP but Disney refused because they were unhappy with some of his prior comments. "I did approach Disney a year or two ago, through my agent, about acquiring the remaining IP. ... We were told to basically get lost: ABC was annoyed with my critical public comments about their management of FiveThirtyEight. It apparently wasn’t a long conversation, so I don’t have a lot more color to report than that." [0] https://news.ycombinator.com/item?id=48197703 https://news.ycombinator.com/item?id=48197703
- opsnooperfax 4mo agoPride comes before the fall. Sorry, Nate.
- 4mo ago
- arlattimore 4mo agoI'm not a soccer guy, but I still think the piece on Lionel Messi was awesome https://web.archive.org/web/20140701122958/http://fivethirtyeight.com/features/lionel-messi-is-impossible/ https://web.archive.org/web/20140701122958/http://fivethirty...
- stinkbeetle 4mo agoThose 2015-16 ones sure aged poorly, I'm reminded of this https://i.imgur.com/6Z9QQj3.jpeg https://i.imgur.com/6Z9QQj3.jpeg This is why people don't really buy the "but he had Trump at 30%, you just don't understand statistics" apologist line. Sure he hedged in the dying days of the campaign (a cynic might think to try to protect his credibility), but the tone overall was of a person who comprehensively failed to understand the mood of the country from beginning to end. Which is a problem because these election predictions are not just pure "mathematical models" and "data driven" like 538 would have had you believe. What mathematical model should be used? What data should and should not be used? At some point those things are based on the modeller's understanding of reality.
- f1ay 4mo agoI think Nate did a phenomenal job calling out pollsters in that time. Since 538 was predominately a poll aggregator that did tricky stats to rank the reliability of each poll. I remember specifically an interview with him griping about some of the unusual data he was seeing from pollsters that made it look like, and I quote, 'Someone has their finger on the scales'
- stinkbeetle 4mo agoPerhaps critiquing statistical methods used by polling was something he was good at. I have no real opinion of his work there, which I didn't pay attention to. But predicting an election requires a lot more than polling datasets and statistics textbooks. That's the problem that he made himself out to be an election prediction wizard, but really that was off the back of his good prediction in quite a bland and conventional election. When things got slightly more spicy and reality diverged from his vaunted "models", his "data science" predictably fell in a heap. The worst thing is almost not even that he got it wrong, it's that he seemed incapable of recognizing that present reality was quite significantly different from the past data he had used to build his models. Even after being wrong in so many of these predictions. He just kept churning out these pieces about how Trump was probably finished this time.
- 4mo ago
- culi 4mo agoUnfortunately most of the most important visualizations are broken in the archived version. Including the gun deaths visualization and I think the P-hacking interactive https://web.archive.org/web/20230205124354/https://fivethirtyeight.com/features/science-isnt-broken/#part1 https://web.archive.org/web/20230205124354/https://fivethirt... It's kinda sad to know no one else will get to experience those interactive visualizations. Though its nice to see the approval comparison page still works https://web.archive.org/web/20241031232233/https://projects.fivethirtyeight.com/biden-approval-rating/ https://web.archive.org/web/20241031232233/https://projects....
- nomilk 4mo agoCurious why they're broken, as the wayback machine seems to be able to run javascript. Do the visualisations rely on a server (or some other assets not included in wayback machine's crawl)?
- palewire 4mo agoI fixed that: https://fivethirtyeightindex.com/graphics/ https://fivethirtyeightindex.com/graphics/ https://fivethirtyeightindex.com/illustrations/ https://fivethirtyeightindex.com/illustrations/
- Helloworldboy 4mo ago[dead]
- isaisabella 4mo ago[dead]
- plazmatic 4mo ago[dead]
- 7777777phil 4mo agoRelated: Disney erased FiveThirtyEight https://news.ycombinator.com/item?id=48197703 https://news.ycombinator.com/item?id=48197703
- alberto-m 4mo agoI am certainly missing a lot of nuance here, but it seems to me Nate Silver managed to have his cake and eat it too. He surely got good money for selling FiveThirtyEight, and now that the buyer has erased the product, Nate can get back a huge chunk of its readers since he offers very similar analyses on his personal site. Sure, natesilver.net has less brand recognition than fivethirtyeight.com, but it's still decently well-known and can only go up from here.
- its-summertime 4mo agoI think the nuance is that it is notable historical articles about predictions and discussions of political elections, during a time when politics is quite at the fore-front of many people's minds
- xattt 4mo agoIt’s possible NS may have signed a contract saying that he cannot engage in elections prediction for X YZ months to same extent that he did with FiveThirtyEight.
- JumpCrisscross 4mo ago> Nate can get back a huge chunk of its readers The downside is this furthers the divide between folks who pay for subscriptions and masses who get shoveled ad-powered slop.
- jnovek 4mo ago
- armada1122 4mo ago[flagged]
- ninjalanternshk 4mo ago[flagged]
- JumpCrisscross 4mo agoBecause of broad statistical illiteracy regarding early forecasts?
- deleted 4mo ago[deleted]
- choult 4mo ago... so an event that had a (simulated) 47% percent chance of happening happens, and an event that had a 53% chance of happening didn't. You must _hate_ it when you need <= 4 and you roll a 5...
- amalcon 4mo ago538: "We have no idea who will win. It seems like basically a coin toss." Reality: someone wins. Internet: "How did 538 mess this up so badly?"
- noirscape 4mo agoIt's a bit more loaded than that. 538 post-Nate Silver had a model setup that was apparently kind of a mess. 538 was apparently sending strange messages to Republican leaning polling agencies, demanding they gave far more detailed audit information than usual (with the implication obviously being that they were fraudulent pollsters), and the guy running the site had fairly openly tuned his model on the assumption people cared about certain talking points. 538 was predicting Biden victories even when the polls were so overwhelmingly against him that not even the most Democratic leaning polling agencies had trust in him; even if you aren't running difficult math, that means something has gone wrong with the model. (Something which got worse after Harris was picked, although every polling aggregator went barmy - there's suspicions that a lot of polling agencies aggressively normalized their data to avoid being seen as biased, leading to an almost 50/50 split.)
- petros 4mo agoThis is a great service for everyone who appreciates thoughtful analyses about politics. Losing FiveThirtyEight was a big loss, but this archive helps. Bravo!
- aeternum 4mo agoDid FiveThirtyEight really get that much right other than the 2016 election? I remember thinking they were the best data journalists out there, and they had some nice visualizations but did their other predictions actually hold up?
- draw_down 4mo ago[dead]
- akio 4mo agoYes, they forecast thousands of U.S. elections and thousands of sports games, and their forecasts had excellent calibration—e.g., events they said would happen 30% of the time actually happened about 30% of the time. https://archive.is/sId82 https://archive.is/sId82 https://github.com/fivethirtyeight/checking-our-work-data https://github.com/fivethirtyeight/checking-our-work-data
- palewire 4mo agoFWIW, the site now indexes 13,000 graphics and 3,000 illustrations, which I clawed out of the Wayback Machine's HTML and preserved separately in an archive.org collection. https://fivethirtyeightindex.com/graphics/ https://fivethirtyeightindex.com/graphics/ https://fivethirtyeightindex.com/illustrations/ https://fivethirtyeightindex.com/illustrations/ The archive.org backend is at https://archive.org/details/fivethirtyeight-collection https://archive.org/details/fivethirtyeight-collection and will almost certainly outlive my site.