7 ms·

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore

You mention the "Dead Internet Theory" (not heard that phrase before!). I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just legacy content and spam. If you think it's bad for you, imagine what it is like for Google Search! Their entire business is indexing a medium which no longer has any relevancy. People complain that Google no longer delivers good results. But what can Google do? The "good content" is no longer available for them to index. Want to become rich? Make a search engine which indexes the fresh relevant data from the big siloed websites, and ignores the general dead Internet.

marginalia_nu 4y ago
I built my search engine in part to explore whether this was actually true, and I don't think it actually is. There's still a lot of organic human-made content still out there, possibly more than ever, it's just not able to compete with the SEO industry that completely displaces it from Google and social media.
- fifticon 4y ago
  I second that independent sites exist - I maintain my own website on a personally run server. There are dozens of us! to quote a quaint phrase.
- api 4y ago
  All open systems are destroyed by spam once they become popular enough to be profitable targets. This will eventually happen to the Fediverse too. If there is money to be made pissing all over the commons, the commons will be pissed all over. It even happens to proprietary silos if they are too open. Look at how many bots and spammers infest social media. Propaganda and disinformation can also be considered a form of spam. I realize this sounds cynical but don’t shoot the messenger. It’s just something I’ve learned watching the Internet evolve since the middle 1990s. Spam eats everything it can. IMHO the future is enclaves and invite only communities. The Internet is a dark forest.
  - pwdisswordfish9 4y ago
    > This will eventually happen to the Fediverse too. Oh, don’t worry, the Fediverse will never catch on.
    - ffhhj 4y ago
      Why? Serious question.
  - marginalia_nu 4y ago
    As old open systems are destroyed, new ones are created to replace them. The Internet exists in a constant state of rebirth and transformation. You really can't step into the same river twice.
    - nonrandomstring 4y ago
      > You really can't step into the same river twice. I love the maxim and philosophy of eternal refreshment. Seems like the problem is more akin to having nuclear waste dumped into our rivers though.
  - pixl97 4y ago
    It's not cynical, is how every system in nature works. Everything alive must develop an immune system or it is attacked and eaten.
  - NoGravitas 4y ago
    You are probably right about the future; not necessarily because of spam, though that's a part of it, but just because of the toxicity of global, open to the world, mostly public social media. The Fediverse has mostly coasted by so far on obscurity, but it's not great, and it's bound to get worse. All of my online socializing these days is either through short-lived pseuds on topic-oriented fora, or invite-only Matrix rooms.
- indigochill 4y ago
  How do you surface organic human content? I happen to linger around the fediverse/tildeverse sphere where I see organic content from people I personally have a direct (digital) connection to (and I started self-hosting my music after Epic bought Bandcamp), but I'm not clear on how I'd go about digging that kind of stuff up in the more general case.
  - pixl97 4y ago
    Welcome to the billion dollar question. Any place that is authentic will face the zombie horde attempting to fake authenticity in order to capture attention.
    - tomxor 4y ago
      I think your almost right, but it's not necessarily authenticity... I think it's just money. Large "authentic" search engines can exist to serve the rest of the web, those personal blogs and other small communities. Those sites have a natural tendency to not be trying to turn everything into a revenue stream, so if that was the prerequisite for an engine, it would be a perfect match and naturally dissuade marketing types.
      - pixl97 4y ago
        Authenticity is worth money. When you have a 'real' community you're talking about real people with real salaries and desires, add in that you tend to develop a real trust between members. Think of this as fertilized soil. You can grow crops in it, but weed seeds will eventually land and try to take over it. HackerNews is a good example of this, it takes a healthy amount of moderation to keep things on topic where things like politics get peared pretty ruthlessly. If for a minute Dang gave in found ways to additionally monetize the forums, something that would be profitable for a while at least, things would start down a bad path.
  - marginalia_nu 4y ago
    I do a traditional web crawl and exclude anything that looks too much like it wants a high google ranking. Nothing to it.
- alxlaz 4y ago
  This matches my findings 100%. The WWW is active and bubbling, but virtually all the cool websites I've found in the last 10 years or so came through friends, small IRC channels, or more recently through marginalia.nu :-). Google and friends are facilitators for the SEO and tracking industries, so of course they have zero interest to prioritize these things over content spam -- their whole business runs on content spam. But the WWW is as alive as it gets.
- sdoering 4y ago
  I can only agree with my sister comment. I find this industrialized web more and more shallow and taxing to use. While professionally I need to help (smaller, local) clients to reach their audiences I become more and more weary. It is like walking through a supermarket with industrialized fast convenience food shouting in bright colors and advertising while ultimately not nourishing me like slow, real food could. I am still looking for this digital slow food movement.
  - nonrandomstring 4y ago
    > I am still looking for this digital slow food movement. https://digitalvegan.net https://digitalvegan.net Please read it, and if you enjoy it please suggest it to friends.
    - lovskogen 4y ago
      Read the intro. So you find vegans annoying (because they 'are the future'), and your not a vegan yourself – and you write that digital veganism is more important than actual veganism. Now that's a way to start off well!
- ColinHayhurst 4y ago
  Agreed. > If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just legacy content and spam. The "general" Internet is not dead. Though if you just want to participate in just Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord you might well think that. Users of marginalia (author above), Mojeek (disclosure: CEO) and others [0] are well aware that there are riches of organic human-made content; from years back and new. Yes, a lot of noise too, which Google has a bigger (SEO) struggle to compete against. But still there is good and different content available. To find good content, using search, you need to use "search" engines which enable discovery, as Google used to do so. I stress the "search" as the emphasis of Google, Bing and thus their syndicates is increasingly on being "answer" engines. [0] https://seirdy.one/2021/03/10/search-engines-with-own-indexes.html https://seirdy.one/2021/03/10/search-engines-with-own-indexe...
  - mc32 4y ago
    Sounds like we’re back to AskJeeves and a number of failed answer engines from a couple of decades ago!
    - ColinHayhurst 4y ago
      AskBERT but now MUM knows best.
  - tmaly 4y ago
    Everyone is trying to game the Google algorithm. The net result is all this long form content and cooking recipes that are 10 pages long. There seems to be a big disconnect with a typical users attention span and the length of a post.
    - ajmurmann 4y ago
      I thought the recipe thing was to be able to copyright them
- pmontra 4y ago
  I take myself as an example. People that know me and don't meet me regularly might know the URL of my web site and might care to look at it once per year and check if there is something new. Usually pictures and tales from holidays. Covid made those holidays less memorable so I didn't make any update since fall 2019. People that meet me regularly don't need that website, I'm telling them the tales first hand and showing them the pictures without being obnoxious. I guess that this website is a target for your search engine except it's not in English and your search engine seems to want English search phrases. I don't have anything of value to share on a public chat like Twitter and I don't have an ego to pretend I do. I also don't use Facebook anymore. I go there once per year to like the messages that wish me happy birthday. I think it's polite to do so. All my media production is on WhatsApp or Telegram in group chats with people I know in real life. If I really cared about producing content for the world I'd probably be using Twitter, Medium or the fad of the year and they'd take care of my SEO (do they?) or I'd be trying to score points on StackOverflow. To recap: I never intended to compete on SEO. I'm really OK that my website is only for friends and spreads by word of mouth. It probably never did, I bet it's been on a flatline since I created it 20+ years ago.
- dylan604 4y ago
  And who uses your search? I had never heard of "you" until just now. And there is the problem with "new" search engines. Unless you can come up with what would have to be one of the greatest ad campaigns the world has ever seen, no significant number of users will know you exist. Where does the money to pay for that ad campaign come from? How will a search engine generate money to stay relevant? Once people see you becoming relevant, they will figure out how to game your system. It's just the nature of the beast. I don't think I'm being overly cynical about this either.
  - marginalia_nu 4y ago
    Why would I need to generate money to stay relevant?
    - dylan604 4y ago
      <edit>The first </edit>relevant was the wrong word. sustainable would be more appropriate. on the assumption that hosting the search engine isn't free, and unless it is supported by a generous benefactor it will need to have a way of generating money to keep the servers running.
      - marginalia_nu 4y ago
        I'm self hosting so my operational cost is like $50/mo.
  - throwaway14356 4y ago
    then he must be relevant
- kodah 4y ago
  Agreed, the general internet is not dead, but the majority of internet users are on Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. From my perspective, we onboarded a lot (if not most) people to the internet after 2007 (the explosion of social media). People sticking to big sites really speaks to an inability to explore the larger internet and a lack of knowing why you would even want to.
  - kthejoker2 4y ago
    I think the answer is in the name: "social" media. Most (99%) people use the Internet most (99%) of the time to see or hear what other people are up to. The big sites are where all the other people are. QED. (This comment falls into that space)
- Vladimof 4y ago
  I added it to my list of search engines on Firefox... your favicon is really small, that's on purpose?
lkxijlewlf 4y ago
> If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. Interesting. When I search for something topical I search those sites using Google because al(most) (I don't use some like FB and insta) all those sites have really shitty search.
hn_throwaway_99 4y ago
Doesn't this site, and all of the content it links to, pretty much disprove your theory? Yes, sure, I often do go to the "top sites" when searching for content, but I still usually start at Google. And, despite all the SEO spam, Google still does a fairly decent of landing me on, for example, the appropriate Wikipedia page, Stackoverflow post, travel site, etc.
mywaifuismeta 4y ago
I no longer see Google as a neutral "search engine" the way it used to be. Now it's just another company that owns and promotes certain types of content, no different from reddit. For some things Google has the best content, for some things Twitter or Reddit have the best content.
- deleted 4y ago
  [deleted]
- maxwelldone 4y ago
  Back in 2000s Google used to be the place for any type of search (IIRC). Now, I've been conditioned to use it only for specific use cases, mostly for convenience. Some examples include: 1. Anything programming related (searching for man pages, error codes etc) is straightforward. (I do have some UBO filters to exclude SO copycats) 2. Utility stuff like currency conversion, finding time in another city, weather etc. Where Google has really fallen behind is in multimedia search. Not sure if it's due to copyright issues or not but Bing and Yandex provide way better service in this regard. Not to mentions the "reddit" suffix I need to add to any search that even remotely calls for public opinion. In many cases, Google is just a shortcut to take me to the relevant subreddit.
  - LegitShady 4y ago
    another thing seems to be prioritizing current news over past news which makes searching for old.articles youve read quite difficult.
  - ufmace 4y ago
    Programming-related stuff seems to have gotten a lot worse in the last couple of years. Now most terms, at least for common things, return a ton of blogspam, when the official docs or SO are usually the best source.
- photochemsyn 4y ago
  I find one of the best ways to find interesting content on specific subjects using Google is now to start blocking all their top returns (a lot of SEO spam). This is somewhat tedious (lots of -site:seospam.com) and Google doesn't like automated queries. However, a few rounds of this often turns up interesting content down low in the search results. Just don't take what's on offer on page one of search results, basically. Where it's gotten really bad is on news searches as Google either now has some kind of shitlist of independent news sites that it won't allow to show op on, for example, site:youtube.com searches - or, it's filtered through a guest list. It's hard to tell which strategy they're using, but news is definitely being heavily filtered based on very dubious propaganda-smelling agendas.
Gigachad 4y ago
Interesting thought. I just went though my browser history and realised that almost every time I use google search, I already know what website I want, I just don’t know the exact link/page. I’ll use google because the search on stack overflow or reddit sucks but I know I’m looking for a page on one particular site.
- Pelam 4y ago
  I realized this too. I disabled search from address bar and started bookmarking everything even remotely sane I see. I often add a few personal keywords to the bookmark bar. It is starting to pay dividends. Instead of weird stuff thrown up by google when I type in something, I get the "oh yeah, that was the page" from a short list of bookmarks shown to match the words.
- npilk 4y ago
  I had the same realization and ended up setting up a simple Cloudflare script to automatically do an “I’m Feeling Lucky” style search to return the first result: https://notes.npilk.com/custom-search https://notes.npilk.com/custom-search
matheusmoreira 4y ago
The internet itself is probably gonna die soon anyway. Every country wants to impose its own laws on it. I think it'll eventually fragment into multiple segregated continental networks, if not national ones, all with heavy filtering at the borders. I'm happy to have experienced the free internet. Truly a jewel of humanity.
- kmlx 4y ago
  > I think it'll eventually fragment into multiple segregated continental networks i think it already has. the Great Firewall of China is the classic example, but I think the trend started in the west with the Right to be forgotten/right to erasure in Europe, and subsequent HTTP Status 451 Unavailable For Legal Reasons. GDPR just further cemented the split between Europe and the rest, and the new DMA & DSA regulation in the European Union finally makes it clear. The writing is of course on the wall, so countries like India or Australia aren't too far behind. Places like California also have their own "right to be forgotten", and I'm sure the US will not be left behind for too long before we see regulation further splitting their internet from the RoW. And I don't think the RoW will hold off much longer till it also splits into multiple big blocks. It's the start of the new "nationalist" internet, and I'm sure we'll all be poorer because of it.
  - matheusmoreira 4y ago
    Exactly what I mean. There is no way to have an international network with national borders. Telecommunications providers have always been centralized and have always been in bed with the government. Only way we'll ever be free is if someone invents some kind of decentralized long range wireless mesh network.
    - politician 4y ago
      Like Starlink?
      - matheusmoreira 4y ago
Hnrobert42 4y ago
As you describe this, it makes me think about how populations tend to migrate to cities and away from rural areas. There’s even a parallel to white flight in the emerging popularity of the chan/gab fora.
black_puppydog 4y ago
These discussions always make me recal Jacob Applebaum. Think of him what you want, but this statement of his really stuck with me at the time. Paraphrasing: The real dark-net is facebook. Everything that goes in there never comes out again and is basically invisible to the world, except if you join facebook yourself. My own prime example of that used to be pinterest: it seems to be a 100% sink in the directed graph of internet links. But since Applebaum stated this, instagram (also facebook of course) is trying hard to push pinterest off that particular throne.
- LegitShady 4y ago
  to me this is also discord - which seems to have become the chose alternative tk online forums for many communities and basically hides what used to be the public face of those communities.
- samatman 4y ago
Cthulhu_ 4y ago
I don't believe the WWW internet is dead; there's still millions of webpages being made and published every day. However, the traffic numbers are skewed in favor of the big socials and aggregators; I wouldn't be surprised if the 80/20 rule applies there.
- pnutjam 4y ago
  There seems to be a tendancy towards video that undercuts the "old internet". I prefer instructions in a text or list format, but that's almost impossible to find for things like, changing the headlight bulb on my traverse. 1. turn the wheel so it is pointed hard in the direction of the bulb you are changing. 2. remove the hex screws from the shroud in the wheel well 3. pull the shroud down, it's pretty flexible plastic. 4. reach up and change the bulb. The wires are a bit short so you might need to get both hands in there. I have big hands and I'm able to do it. ---- There are innumerable videos explaining this process, but very few text directions.
  - Jiro 4y ago
    Even something like that flounders on the question "these instructions say to pull down the shroud, what is a shroud?" or "I can't find those hex screws, where are they located?" Repairs are inherently visual, although text with illustrations might work.
  - ElevenLathe 4y ago
    I think this is actually because real, fluent literacy is still rare even in highly developed places. It may be easier for a very literate someone to dash off those instructions but most people are 1000x more comfortable making a little video. Same goes for reading vs watching the video. This is my same theory about meetings being universally preferred to asynchronous email, even when literally all the questions someone asks at a meeting have already been answered in my long form email. Most people, even if they can read, are not really comfortable with it. Doubly so for writing. There used to be no choice to function in society, but increasingly we can use technology to substitute for reading and writing effectively, so people do.
DebtDeflation 4y ago
Unfortunately, correct. The average Internet user accesses it via a phone, not a desktop, laptop, or even tablet these days. Most of that access is through apps, not a browser. To the extent that a user is looking for a factoid answer and does a search, a Google Knowledge Graph result with a Wikipedia link is probably enough in most cases. If they want a technical question answered, Stack Exchange; a product review, Reddit; nearby restaurants with reviews, Google Maps; etc.
CTDOCodebases 4y ago
People are doing this already. You just have to include the site name in the search on google e.g reddit. Search on these platforms is often broken.
captainmuon 4y ago
But Twitter, Reddit, HN, and most other such places are just websites and can be indexed fine. Same with Wikipedia, which is very much a silo (they don't have regular links in text in the hypertext spirit, but only footnotes). Facebook and Instagram are more of a walled garden, like Quora, but there is a lot of junk there anyway. It's sad for the WWW, but I don't really think it is a fundamental problem for search engines. In fact Twitter for example gives a direct pipe to Google. If you tweet something, it is immediately findable. Similar for StackExchange, but there I think the site is so "small" that Google can afford to just continuously index it.
- TheRealDunkirk 4y ago
  And isn't great to get a link to Reddit or Twitter, and you click the link, and try to navigate to the comments for context or the answer, and you go to click the link to expand it, and then you get a demand to log in and install their app? Don't talk about walled gardens and not include Reddit or Twitter just because they let you look at one brick before demanding their tax.
  - deleted 4y ago
    [deleted]
- ratww 4y ago
  Twitter and Reddit still can be indexed, but they've also become increasingly hard to use without an account. Reddit doesn't let you fully expand threads when you're unlogged. Twitter limits the amount of things you can read and shows a modal. Both of them heavily limit usage on mobile devices without installing an app. Sure, an account is free but might require giving information you don't want to give. Twitter asks me for a phone number a few minutes after creating an account, even if I don't post anything). Reddit at least lets you skip giving an email. Sure, there are workarounds such as using lite versions (old Reddit, mobile Twitter), but that's not known to all people coming from a search engine. It feels as if HN are the only one that's not a partially walled garden yet (and Wikipedia of course).
  - 4y ago
shortformblog 4y ago
I think this is a tad reductive, but I will say that we sure let a lot of big companies convince a huge portion of the population to create all of their content on platforms that they have no real control over. The problem is, many of them didn’t realize this was a problem until recently. That said, plenty of exciting stuff is happening outside of the walled garden, as long as you know how to find it.
- Gravityloss 4y ago
  And not only did this happen already over a decade ago, a lot of the current internet users have never known anything else. We had a discussion with coworkers and somebody mentioned irc. Explaining to younger colleagues what it was and that it was not a product of a company, but operators had servers that formed a network, and it was more like infrastructure. Felt weird.
  - kasey_junk 4y ago
    Most of the kids in my 3rd graders peer group understand federated infrastructures quite well because of Minecraft. Perhaps it wasn’t the federated nature of irc that was surprising but the fact that it was irc?
    - mst 4y ago
      Isn't minecraft more decentralised than federated? IRC networks usually have multiple servers connected together (historically, often run by a bunch of different people) and I didn't think people self-hosting minecraft servers usually did that?
  - Elvie 4y ago
    isn't Discord a bit like IRC used to be?
    - ori_b 4y ago
      How do I connect to a self hosted discord, and then connect it to my friends self hosted one? And where do I get the RFC for the protocol so that I can write my own compatible implementation? IRC isn't a product. It's a standardized protocol sufficiently simple to implement in a day or two.
NicoJuicy 4y ago
The big siloed websites are just indexes of fresh content though. With a generic way to place comments on it.
PragmaticPulp 4y ago
> If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. Maybe we’re searching for different content, but I disagree. While Google results are not without noise, I think it’s a huge exaggeration to suggest it’s useless. I still regularly find quality results from a quick skim of the first or second page of Google results. Meanwhile places like Reddit, Twitter, and Hacker News are full of very strong opinions that feel truthy, but are mostly noise. Unless you go in with enough baseline knowledge to filter out 9/10 underinformed comments to dig out the 10% who actually have direct knowledge of the subject and aren’t just parroting some version of something they read from other comments, skipping straight to social sites becomes a source of misinformation.
lysecret 4y ago
I think this is a very "consumer focused" take. Yes. A lot of interesting people data is now "locked" behind these aggregators and platforms (and also hard to handle because of GDPR). But most interesting company data is still out there.
simion314 4y ago
This is not true, maybe for a subset of Internet users. For example you have Wikis and forums. Wikis are good for communities that are passionate about a topic and they collaborate on buidling content for their passion. Reddit is a valid alternative to forums but if the community s older and has members that are technical competent then they usually have the forum customized for their purpose and the forum will continue to exist , especially if you want to avoid some third party censorship. I never ever search for something and found answers on Facebook, sometimes very rare I find something that points to Instagram blogs/posts but never Facebook. Probably depends on your location and what you search for, so it might be possible that 99% of your Internet consumption is satisfied by 5-10 websites.
jrussbowman 4y ago
"Want to become rich? Make a search engine which indexes the fresh relevant data from the big siloed websites, and ignores the general dead Internet." Did that to some degree. Unscatter.com pulls from reddit and twitter to source links. I found reddit only created an echo chamber bubble of obvious bias and twitter only diluted it a little.
baxtr 4y ago
I am not so sure... I think what happened is this: the WWW was everything back in the days. But in the "old days," only 10% of all people were online, the web elite. Then, AOL came, and the rest came online slowly but surely. The so-called "mainstream" people were no geeks, and these people were "just" ordinary people. Almost all were captured by what you call "big websites". Now, we see the 100% being dominated by the 90%. That's why "Google results are bad". Bad for us! Not maybe (most probably) not for them.
- nl 4y ago
  Eternal September was Sep 1993. AOL hit the internet in March 1994. Netscape didn't launch until December 1994 (and the WWW was nothing before that. I subscribed to a mailing list with new sites that were released and I'd visit most new websites on the internet on most days with the Cello browser in my uni labs most days). AOL users have been there since the beginning of the WWW. https://en.m.wikipedia.org/wiki/Eternal_September https://en.m.wikipedia.org/wiki/Eternal_September
  - CWuestefeld 4y ago
    My recollection is that the AOL event you reference was only making usenet accessible - a point that makes good sense in the context of the eternal September. But when talking about the WWW, that's a very different story. I think that AOL didn't incorporate a web browser until quite some time after that.
    - nl 4y ago
      The WWW took off when Netscape shipped in late 1994. AOL users could use Netscape from the beginning.
psyc 4y ago
Based on my observations over the past year, I’m certain that Google and Bing choose not to show us most of the web anymore. I usually find what I’m looking for. It just takes literally three orders of magnitude longer than it used to for the same kind of stuff. I used to use Google a lot to jog my memory about various things I vaguely remembered. Type a few associative words and snippets, press Enter, done. Google’s useless for that now. If you’re looking for hot pop shit in trendy publications, things to buy, commercial services to subscribe to - G has you covered. That’s what they do now.
stackbutterflow 4y ago
I think you're generalizing your own behavior. I regularly use google to search for topics that cross my mind and I end up on many websites that are not one the giants in your list. It's a fun activity. If people stick to the same 10 websites that's on them. Nothing prevents you from exploring the web.
- MockObject 4y ago
  > Nothing prevents you from exploring the web. What prevents you from exploring the web is you can't find but the same 10 sites through search engines.
mrtksn 4y ago
It has been dead for a while now and the whole society feels it globally. Things were getting so good then things become horrible and whoever cracks the path to the goods stuff again will find great riches at the end of the path.
dotnet00 4y ago
I agree that this seems way too reductive. I was recently reflecting on this and noticed that I constantly run across new blogs and sites whenever trying to learn something. I just don't usually pay much attention to the site name in the way that I remember HN, Reddit, Twitter etc. So, while I would agree that some aspects of the old internet are dead (like 'small' ~1000 user forums focused on specific topics having largely been replaced by generally inferior subreddits and discord servers), I think it hasn't gotten as bad as you're making it out to be.
jerf 4y ago
"I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there." That is not the Dead Internet Theory. That's just something anyone can see by looking at the world. The Dead Internet Theory is that the Internet is already an echo chamber custom fed to you by a collection of bots and other such things, and that a lot of the "people" you think you're interacting with are already, today, faked. You're basically in a constructed echo chamber designed only with the interests of the creators of that chamber in mind, using the powerful social cues of homo sapiens effectively against you. In particular, those silos aren't where people are communicating. Those silos are where you think you're communicating. It is obviously not entirely true. When we physically meet friends, sometimes topics wander to "Did you see what I posted on Facebook?" So far, we've not caught Facebook actively forging posts from our real-life friends that we physically know. (Though we have caught them failing to disseminate posts in what seems to be a distinctly slanted manner.) I am also not terribly convinced that the bots have mastered long-form content like you see on HN. I think we've had some try, and while they can sort of pass, they seem to expend so much effort on merely "passing" that they don't have much left over to actually drive the conversation. HN probably still requires real humans to manipulate things. Where I do seriously wonder about this theory is Twitter. AI has progressed to the point that short-form content like that can be effectively generated and driven in a certain direction. There's been some chatter on the far-out rumor mills about just how bot-infested Twitter may be, how many people think they have thousands of followers, even having interacted with some of them as "people", and in fact may only have dozens of flesh-and-blood humans following them, if that. Stay tuned, this one is developing. (Note that while this could be "a big plan", it is also a possible outcome of many groups independently coming to the conclusion that a Twitter bot horde could be useful. A few hundred from X trying to nudge you one way, a few hundred from Y trying to nudge you another, another few thousand from Z trying to nudge you yet another, before you know it, the vast vast majority of everyone's "followers" is bots bots bots, and there was no grand plan to produce that result. It just so happens that Twitter's ancient decision to be dedicated to short-form content, with no particular real-world connection to the conversation participants, where everyone is isolated on their own feed (even if that is shared in some ways) made it the first place where this could happen. Things with real-world connections, things where everyone is in the same "area" like an HN conversation, and long-form content will all be three things that will be harder for AIs to manipulate. Twitter is like the agar dish for this sort of thing, by its structure.)
boplicity 4y ago
> No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. Really? We make our living running a small web based publication; around 40k readers a month. I know of many other sites like this. Google, and other search engines, depends on niche websites to provide quality search results. Without sites like ours, the internet would truly be dead, and search would be mostly useless. Our "traffic sources" come from a mix of Facebook, Search, Reddit, etc, in addition to our many loyal readers. Others in our niche are producing blog spam, which looks nearly identical to people who aren't experts in the field, but we have real experts, fact checkers, etc, as part of our production process. This is a big problem: These low quality websites get similar rankings to our own, which does make it much harder for people to get quality information via search. (Hence the general shift towards trusting social recommendations, such as from Reddit.) In short, the WWW is alive and well, it's just buried under a bunch of #$#$%.
- rchaud 4y ago
  > Our "traffic sources" come from a mix of Facebook, Search, Reddit, etc, in addition to our many loyal readers. 40k/mo is a pretty good number for an independent website. As a word of warning though, relying on social media reach is a dangerous game, as there is anecdotal evidence that tweets with outbound links don't get as many impressions as those that link to in-site content, like another Twitter post. As for Facebook, well, there's a good comic from The Oatmeal (enormously popular on FB back in 2010) that talks about what happened in the long run: https://twitter.com/Oatmeal/status/923250055540219904 https://twitter.com/Oatmeal/status/923250055540219904
rchaud 4y ago
> Want to become rich? Make a search engine which indexes the fresh relevant data from the big siloed websites, and ignores the general dead Internet. That would be a great service, but it certainly wouldn't make you rich. Where's the money going to come from? Google got rich because they acquired an ads platform (DoubleClick) and an analytics platform (Urchin) and started monetizing the vast amounts of data they had. That was years after Google had established goodwill as the best search engine.
- big_blind 4y ago
  I use beta search engines. On kagi.com and you.com you can preference and filter top sites. There's also no advertising on either. I've just stopped using Google altogether and its improved search so much.
jspaetzel 4y ago
This is so incredibly false, I've been working on a project for the last six months and MoM I've seen steady increase in usage. Tbh much much higher usage then I expected. Most users find my site via Google or Facebook however they are looking for content that's not in those silos and have no problems leaving them. If you have high quality content and you get it indexed properly by Google, users will come. There are reasons users are not using your website. 1. It's not solving a problem people have. 2. Users can't find it. Who, in their right mind searches for search engines? Nobody I know. If you want users you have to go out and get them (literally pound the pavement and talk to people) or create a LOT more content ironically, so they can find your site on the search engines they are using today.
throw10920 4y ago
> I agree: the WWW Internet is dead I've heard this claim a lot, with 0 supporting evidence. Do you have any? My own experience is that there are thousands of content-rich, high-quality blogs still being written by real humans, because I regularly find and bookmark new ones weekly, without even looking for them, so: please provide evidence for this claim that runs counter to my lived experience.
dageshi 4y ago
I agree with you to an extent. The web is less useful than it used to be. BUT I would say a lot of that usefulness has diverted into youtube. There are people who would previously have made sites who are making youtube videos instead which of course is owned by google.
altairprime 4y ago
If you want to be rich, solve search without full-text indexing of sites. Pagerank only ever worked because of human curation of webrings. Full-text search made is easier to find content, and opened the door for spammers. The only viable route forward for search will be to replace full-text indexing with human curation, somehow. Solve how to scale that up instead, so that when everyone else realizes we need it for the health of the Web, you’re ready.
soheil 4y ago
To a fish the world is made of water and there can't possibly be anything else worthwhile. This is more indicative of how you spend your time online vs reality.
ouid 4y ago
Google is still pretty good at searching reddit. Maybe reddit can acquire them.
- big_blind 4y ago
  site:reddit just is the best search engine at this point. I still don't like Google though.
derefr 4y ago
> Make a search engine which indexes the fresh relevant data from the big siloed websites, and ignores the general dead Internet I don’t understand why Google themselves don’t do this. LinkedIn v. hiQ demonstrated that they won’t get in trouble for scraping users’ subjective views of data within these silos and then stitching them together to form a cohesive whole. So where’s the effort to do so? It seems like the obvious step.
samstave 4y ago
This MUST be the reason that they threw their purchase of Postini in the garbage and my GMAIL INBOX is filled with spam, and my "social" and "promotions" tabs dont filter.... GMAIL is garbage now, I literally use it as my spam email any more. Which sucks because I have had it for a really long time. Annecdote on Yahoo! Mail ; years ago I wrote to yahoo support asking when I created my Yahoo Mail account (i'd had it from the 90s when it was very early available...) And support told me that they couldnt tell me when my account was created as that was *proprietary company information* So I deleted my Yahoo account. Im about to DL all my gmail and do the same.
omoikane 4y ago
I think the Dead Internet Theory bit is just a bait to get more comments. It's a bit of a stretch to conclude that the internet is mostly robots just because one website sees mostly robots. This extrapolation would be convincing if that one website is a high ranking website that sees a lot of traffic, but searchmysite.net does not appear to be one of the top websites.
deleted 4y ago
[deleted]
hombre_fatal 4y ago
I don't get how TFA shows evidence of the Dead Internet Theory just because their site manages to attract ~zero users. Just host a <form><textarea><button></form> at an IP address and notice it's just spambots submitting it with backlinks, not actual users. Doesn't mean the internet is dead nor that the indieweb is dead. It doesn't really show anything other than the only people able to extract value from your creation are the spammers.
Jenk 4y ago
> If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. High chances you will find a link to an external site over content actually on those big named sites though, right? That tells us the organic web isn't dead, it's just hard to discover/navigate - because of SEO wars, most probably... The problem isn't the lack of content, it's the number of shitty spammy sites standing in your way of the sites you actually want to see. Like a sleazy salesman trying to direct you to the crap laden three wheeled rust bucket when you were heading toward the family sedans.
heavyset_go 4y ago
I was once on this bandwagon, but I think it was just confirmation bias reflecting the way I used the internet at the time. The non-siloed internet is bigger than the pre-siloed internet ever was.
ClumsyPilot 4y ago
> If you think it's bad for you, imagine what it is like for Google Search! Their entire business is indexing a medium which no longer has any relevancy. Google was the one (among many) that killed it - so I am not gonna shed any tears.
Shinchy 4y ago
I think you're thinking too narrowly about general chit chat content. E-commerce for example is still very much in the function of using your own website. As I would say is documentation, e-learning, saas, company information, etc. It's a more purposeful web. What is dead though is the general blog like content and community platforms of old, the era of Wordpress blogs, forums and hobbyist websites is certainly gone.
Schroedingersat 4y ago
If they wanted good content, they shouldn't have coerced everyone with good content into turning it into illegible seo spam in order to appear in the top 10 pages. The writing was on the wall the first time a recipe site had to start writing stupid stories about their dog.

marginalia_nu 4y ago

I built my search engine in part to explore whether this was actually true, and I don't think it actually is. There's still a lot of organic human-made content still out there, possibly more than ever, it's just not able to compete with the SEO industry that completely displaces it from Google and social media.

fifticon 4y ago
I second that independent sites exist - I maintain my own website on a personally run server. There are dozens of us! to quote a quaint phrase.
api 4y ago
All open systems are destroyed by spam once they become popular enough to be profitable targets. This will eventually happen to the Fediverse too. If there is money to be made pissing all over the commons, the commons will be pissed all over. It even happens to proprietary silos if they are too open. Look at how many bots and spammers infest social media. Propaganda and disinformation can also be considered a form of spam. I realize this sounds cynical but don’t shoot the messenger. It’s just something I’ve learned watching the Internet evolve since the middle 1990s. Spam eats everything it can. IMHO the future is enclaves and invite only communities. The Internet is a dark forest.
- pwdisswordfish9 4y ago
  > This will eventually happen to the Fediverse too. Oh, don’t worry, the Fediverse will never catch on.
  - ffhhj 4y ago
    Why? Serious question.
- marginalia_nu 4y ago
  As old open systems are destroyed, new ones are created to replace them. The Internet exists in a constant state of rebirth and transformation. You really can't step into the same river twice.
  - nonrandomstring 4y ago
    > You really can't step into the same river twice. I love the maxim and philosophy of eternal refreshment. Seems like the problem is more akin to having nuclear waste dumped into our rivers though.
- pixl97 4y ago
  It's not cynical, is how every system in nature works. Everything alive must develop an immune system or it is attacked and eaten.
- NoGravitas 4y ago
  You are probably right about the future; not necessarily because of spam, though that's a part of it, but just because of the toxicity of global, open to the world, mostly public social media. The Fediverse has mostly coasted by so far on obscurity, but it's not great, and it's bound to get worse. All of my online socializing these days is either through short-lived pseuds on topic-oriented fora, or invite-only Matrix rooms.
indigochill 4y ago
How do you surface organic human content? I happen to linger around the fediverse/tildeverse sphere where I see organic content from people I personally have a direct (digital) connection to (and I started self-hosting my music after Epic bought Bandcamp), but I'm not clear on how I'd go about digging that kind of stuff up in the more general case.
- pixl97 4y ago
  Welcome to the billion dollar question. Any place that is authentic will face the zombie horde attempting to fake authenticity in order to capture attention.
  - tomxor 4y ago
    I think your almost right, but it's not necessarily authenticity... I think it's just money. Large "authentic" search engines can exist to serve the rest of the web, those personal blogs and other small communities. Those sites have a natural tendency to not be trying to turn everything into a revenue stream, so if that was the prerequisite for an engine, it would be a perfect match and naturally dissuade marketing types.
    - pixl97 4y ago
      Authenticity is worth money. When you have a 'real' community you're talking about real people with real salaries and desires, add in that you tend to develop a real trust between members. Think of this as fertilized soil. You can grow crops in it, but weed seeds will eventually land and try to take over it. HackerNews is a good example of this, it takes a healthy amount of moderation to keep things on topic where things like politics get peared pretty ruthlessly. If for a minute Dang gave in found ways to additionally monetize the forums, something that would be profitable for a while at least, things would start down a bad path.
- marginalia_nu 4y ago
  I do a traditional web crawl and exclude anything that looks too much like it wants a high google ranking. Nothing to it.
alxlaz 4y ago
This matches my findings 100%. The WWW is active and bubbling, but virtually all the cool websites I've found in the last 10 years or so came through friends, small IRC channels, or more recently through marginalia.nu :-). Google and friends are facilitators for the SEO and tracking industries, so of course they have zero interest to prioritize these things over content spam -- their whole business runs on content spam. But the WWW is as alive as it gets.
sdoering 4y ago
I can only agree with my sister comment. I find this industrialized web more and more shallow and taxing to use. While professionally I need to help (smaller, local) clients to reach their audiences I become more and more weary. It is like walking through a supermarket with industrialized fast convenience food shouting in bright colors and advertising while ultimately not nourishing me like slow, real food could. I am still looking for this digital slow food movement.
- nonrandomstring 4y ago
  > I am still looking for this digital slow food movement. https://digitalvegan.net https://digitalvegan.net Please read it, and if you enjoy it please suggest it to friends.
  - lovskogen 4y ago
    Read the intro. So you find vegans annoying (because they 'are the future'), and your not a vegan yourself – and you write that digital veganism is more important than actual veganism. Now that's a way to start off well!
ColinHayhurst 4y ago
Agreed. > If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. The general Internet is dead: it's just legacy content and spam. The "general" Internet is not dead. Though if you just want to participate in just Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord you might well think that. Users of marginalia (author above), Mojeek (disclosure: CEO) and others [0] are well aware that there are riches of organic human-made content; from years back and new. Yes, a lot of noise too, which Google has a bigger (SEO) struggle to compete against. But still there is good and different content available. To find good content, using search, you need to use "search" engines which enable discovery, as Google used to do so. I stress the "search" as the emphasis of Google, Bing and thus their syndicates is increasingly on being "answer" engines. [0] https://seirdy.one/2021/03/10/search-engines-with-own-indexes.html https://seirdy.one/2021/03/10/search-engines-with-own-indexe...
- mc32 4y ago
  Sounds like we’re back to AskJeeves and a number of failed answer engines from a couple of decades ago!
  - ColinHayhurst 4y ago
    AskBERT but now MUM knows best.
- tmaly 4y ago
  Everyone is trying to game the Google algorithm. The net result is all this long form content and cooking recipes that are 10 pages long. There seems to be a big disconnect with a typical users attention span and the length of a post.
  - ajmurmann 4y ago
    I thought the recipe thing was to be able to copyright them
pmontra 4y ago
I take myself as an example. People that know me and don't meet me regularly might know the URL of my web site and might care to look at it once per year and check if there is something new. Usually pictures and tales from holidays. Covid made those holidays less memorable so I didn't make any update since fall 2019. People that meet me regularly don't need that website, I'm telling them the tales first hand and showing them the pictures without being obnoxious. I guess that this website is a target for your search engine except it's not in English and your search engine seems to want English search phrases. I don't have anything of value to share on a public chat like Twitter and I don't have an ego to pretend I do. I also don't use Facebook anymore. I go there once per year to like the messages that wish me happy birthday. I think it's polite to do so. All my media production is on WhatsApp or Telegram in group chats with people I know in real life. If I really cared about producing content for the world I'd probably be using Twitter, Medium or the fad of the year and they'd take care of my SEO (do they?) or I'd be trying to score points on StackOverflow. To recap: I never intended to compete on SEO. I'm really OK that my website is only for friends and spreads by word of mouth. It probably never did, I bet it's been on a flatline since I created it 20+ years ago.
dylan604 4y ago
And who uses your search? I had never heard of "you" until just now. And there is the problem with "new" search engines. Unless you can come up with what would have to be one of the greatest ad campaigns the world has ever seen, no significant number of users will know you exist. Where does the money to pay for that ad campaign come from? How will a search engine generate money to stay relevant? Once people see you becoming relevant, they will figure out how to game your system. It's just the nature of the beast. I don't think I'm being overly cynical about this either.
- marginalia_nu 4y ago
  Why would I need to generate money to stay relevant?
  - dylan604 4y ago
    <edit>The first </edit>relevant was the wrong word. sustainable would be more appropriate. on the assumption that hosting the search engine isn't free, and unless it is supported by a generous benefactor it will need to have a way of generating money to keep the servers running.
    - marginalia_nu 4y ago
      I'm self hosting so my operational cost is like $50/mo.
- throwaway14356 4y ago
  then he must be relevant
kodah 4y ago
Agreed, the general internet is not dead, but the majority of internet users are on Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. From my perspective, we onboarded a lot (if not most) people to the internet after 2007 (the explosion of social media). People sticking to big sites really speaks to an inability to explore the larger internet and a lack of knowing why you would even want to.
- kthejoker2 4y ago
  I think the answer is in the name: "social" media. Most (99%) people use the Internet most (99%) of the time to see or hear what other people are up to. The big sites are where all the other people are. QED. (This comment falls into that space)
Vladimof 4y ago
I added it to my list of search engines on Firefox... your favicon is really small, that's on purpose?

lkxijlewlf 4y ago

> If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. Interesting. When I search for something topical I search those sites using Google because al(most) (I don't use some like FB and insta) all those sites have really shitty search.

hn_throwaway_99 4y ago

Doesn't this site, and all of the content it links to, pretty much disprove your theory? Yes, sure, I often do go to the "top sites" when searching for content, but I still usually start at Google. And, despite all the SEO spam, Google still does a fairly decent of landing me on, for example, the appropriate Wikipedia page, Stackoverflow post, travel site, etc.

mywaifuismeta 4y ago

I no longer see Google as a neutral "search engine" the way it used to be. Now it's just another company that owns and promotes certain types of content, no different from reddit. For some things Google has the best content, for some things Twitter or Reddit have the best content.

deleted 4y ago
[deleted]
maxwelldone 4y ago
Back in 2000s Google used to be the place for any type of search (IIRC). Now, I've been conditioned to use it only for specific use cases, mostly for convenience. Some examples include: 1. Anything programming related (searching for man pages, error codes etc) is straightforward. (I do have some UBO filters to exclude SO copycats) 2. Utility stuff like currency conversion, finding time in another city, weather etc. Where Google has really fallen behind is in multimedia search. Not sure if it's due to copyright issues or not but Bing and Yandex provide way better service in this regard. Not to mentions the "reddit" suffix I need to add to any search that even remotely calls for public opinion. In many cases, Google is just a shortcut to take me to the relevant subreddit.
- LegitShady 4y ago
  another thing seems to be prioritizing current news over past news which makes searching for old.articles youve read quite difficult.
- ufmace 4y ago
  Programming-related stuff seems to have gotten a lot worse in the last couple of years. Now most terms, at least for common things, return a ton of blogspam, when the official docs or SO are usually the best source.
photochemsyn 4y ago
I find one of the best ways to find interesting content on specific subjects using Google is now to start blocking all their top returns (a lot of SEO spam). This is somewhat tedious (lots of -site:seospam.com) and Google doesn't like automated queries. However, a few rounds of this often turns up interesting content down low in the search results. Just don't take what's on offer on page one of search results, basically. Where it's gotten really bad is on news searches as Google either now has some kind of shitlist of independent news sites that it won't allow to show op on, for example, site:youtube.com searches - or, it's filtered through a guest list. It's hard to tell which strategy they're using, but news is definitely being heavily filtered based on very dubious propaganda-smelling agendas.

Gigachad 4y ago

Interesting thought. I just went though my browser history and realised that almost every time I use google search, I already know what website I want, I just don’t know the exact link/page. I’ll use google because the search on stack overflow or reddit sucks but I know I’m looking for a page on one particular site.

Pelam 4y ago
I realized this too. I disabled search from address bar and started bookmarking everything even remotely sane I see. I often add a few personal keywords to the bookmark bar. It is starting to pay dividends. Instead of weird stuff thrown up by google when I type in something, I get the "oh yeah, that was the page" from a short list of bookmarks shown to match the words.
npilk 4y ago
I had the same realization and ended up setting up a simple Cloudflare script to automatically do an “I’m Feeling Lucky” style search to return the first result: https://notes.npilk.com/custom-search https://notes.npilk.com/custom-search

matheusmoreira 4y ago

The internet itself is probably gonna die soon anyway. Every country wants to impose its own laws on it. I think it'll eventually fragment into multiple segregated continental networks, if not national ones, all with heavy filtering at the borders. I'm happy to have experienced the free internet. Truly a jewel of humanity.

kmlx 4y ago
> I think it'll eventually fragment into multiple segregated continental networks i think it already has. the Great Firewall of China is the classic example, but I think the trend started in the west with the Right to be forgotten/right to erasure in Europe, and subsequent HTTP Status 451 Unavailable For Legal Reasons. GDPR just further cemented the split between Europe and the rest, and the new DMA & DSA regulation in the European Union finally makes it clear. The writing is of course on the wall, so countries like India or Australia aren't too far behind. Places like California also have their own "right to be forgotten", and I'm sure the US will not be left behind for too long before we see regulation further splitting their internet from the RoW. And I don't think the RoW will hold off much longer till it also splits into multiple big blocks. It's the start of the new "nationalist" internet, and I'm sure we'll all be poorer because of it.
- matheusmoreira 4y ago
  Exactly what I mean. There is no way to have an international network with national borders. Telecommunications providers have always been centralized and have always been in bed with the government. Only way we'll ever be free is if someone invents some kind of decentralized long range wireless mesh network.
  - politician 4y ago
    Like Starlink?
    - matheusmoreira 4y ago

Hnrobert42 4y ago

As you describe this, it makes me think about how populations tend to migrate to cities and away from rural areas. There’s even a parallel to white flight in the emerging popularity of the chan/gab fora.

black_puppydog 4y ago

These discussions always make me recal Jacob Applebaum. Think of him what you want, but this statement of his really stuck with me at the time. Paraphrasing: The real dark-net is facebook. Everything that goes in there never comes out again and is basically invisible to the world, except if you join facebook yourself. My own prime example of that used to be pinterest: it seems to be a 100% sink in the directed graph of internet links. But since Applebaum stated this, instagram (also facebook of course) is trying hard to push pinterest off that particular throne.

LegitShady 4y ago
to me this is also discord - which seems to have become the chose alternative tk online forums for many communities and basically hides what used to be the public face of those communities.
samatman 4y ago

Cthulhu_ 4y ago

I don't believe the WWW internet is dead; there's still millions of webpages being made and published every day. However, the traffic numbers are skewed in favor of the big socials and aggregators; I wouldn't be surprised if the 80/20 rule applies there.

pnutjam 4y ago
There seems to be a tendancy towards video that undercuts the "old internet". I prefer instructions in a text or list format, but that's almost impossible to find for things like, changing the headlight bulb on my traverse. 1. turn the wheel so it is pointed hard in the direction of the bulb you are changing. 2. remove the hex screws from the shroud in the wheel well 3. pull the shroud down, it's pretty flexible plastic. 4. reach up and change the bulb. The wires are a bit short so you might need to get both hands in there. I have big hands and I'm able to do it. ---- There are innumerable videos explaining this process, but very few text directions.
- Jiro 4y ago
  Even something like that flounders on the question "these instructions say to pull down the shroud, what is a shroud?" or "I can't find those hex screws, where are they located?" Repairs are inherently visual, although text with illustrations might work.
- ElevenLathe 4y ago
  I think this is actually because real, fluent literacy is still rare even in highly developed places. It may be easier for a very literate someone to dash off those instructions but most people are 1000x more comfortable making a little video. Same goes for reading vs watching the video. This is my same theory about meetings being universally preferred to asynchronous email, even when literally all the questions someone asks at a meeting have already been answered in my long form email. Most people, even if they can read, are not really comfortable with it. Doubly so for writing. There used to be no choice to function in society, but increasingly we can use technology to substitute for reading and writing effectively, so people do.

DebtDeflation 4y ago

Unfortunately, correct. The average Internet user accesses it via a phone, not a desktop, laptop, or even tablet these days. Most of that access is through apps, not a browser. To the extent that a user is looking for a factoid answer and does a search, a Google Knowledge Graph result with a Wikipedia link is probably enough in most cases. If they want a technical question answered, Stack Exchange; a product review, Reddit; nearby restaurants with reviews, Google Maps; etc.

CTDOCodebases 4y ago

People are doing this already. You just have to include the site name in the search on google e.g reddit. Search on these platforms is often broken.

captainmuon 4y ago

But Twitter, Reddit, HN, and most other such places are just websites and can be indexed fine. Same with Wikipedia, which is very much a silo (they don't have regular links in text in the hypertext spirit, but only footnotes). Facebook and Instagram are more of a walled garden, like Quora, but there is a lot of junk there anyway. It's sad for the WWW, but I don't really think it is a fundamental problem for search engines. In fact Twitter for example gives a direct pipe to Google. If you tweet something, it is immediately findable. Similar for StackExchange, but there I think the site is so "small" that Google can afford to just continuously index it.

TheRealDunkirk 4y ago
And isn't great to get a link to Reddit or Twitter, and you click the link, and try to navigate to the comments for context or the answer, and you go to click the link to expand it, and then you get a demand to log in and install their app? Don't talk about walled gardens and not include Reddit or Twitter just because they let you look at one brick before demanding their tax.
- deleted 4y ago
  [deleted]
ratww 4y ago
Twitter and Reddit still can be indexed, but they've also become increasingly hard to use without an account. Reddit doesn't let you fully expand threads when you're unlogged. Twitter limits the amount of things you can read and shows a modal. Both of them heavily limit usage on mobile devices without installing an app. Sure, an account is free but might require giving information you don't want to give. Twitter asks me for a phone number a few minutes after creating an account, even if I don't post anything). Reddit at least lets you skip giving an email. Sure, there are workarounds such as using lite versions (old Reddit, mobile Twitter), but that's not known to all people coming from a search engine. It feels as if HN are the only one that's not a partially walled garden yet (and Wikipedia of course).
- 4y ago

shortformblog 4y ago

I think this is a tad reductive, but I will say that we sure let a lot of big companies convince a huge portion of the population to create all of their content on platforms that they have no real control over. The problem is, many of them didn’t realize this was a problem until recently. That said, plenty of exciting stuff is happening outside of the walled garden, as long as you know how to find it.

Gravityloss 4y ago
And not only did this happen already over a decade ago, a lot of the current internet users have never known anything else. We had a discussion with coworkers and somebody mentioned irc. Explaining to younger colleagues what it was and that it was not a product of a company, but operators had servers that formed a network, and it was more like infrastructure. Felt weird.
- kasey_junk 4y ago
  Most of the kids in my 3rd graders peer group understand federated infrastructures quite well because of Minecraft. Perhaps it wasn’t the federated nature of irc that was surprising but the fact that it was irc?
  - mst 4y ago
    Isn't minecraft more decentralised than federated? IRC networks usually have multiple servers connected together (historically, often run by a bunch of different people) and I didn't think people self-hosting minecraft servers usually did that?
- Elvie 4y ago
  isn't Discord a bit like IRC used to be?
  - ori_b 4y ago
    How do I connect to a self hosted discord, and then connect it to my friends self hosted one? And where do I get the RFC for the protocol so that I can write my own compatible implementation? IRC isn't a product. It's a standardized protocol sufficiently simple to implement in a day or two.

NicoJuicy 4y ago

The big siloed websites are just indexes of fresh content though. With a generic way to place comments on it.

PragmaticPulp 4y ago

> If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. Maybe we’re searching for different content, but I disagree. While Google results are not without noise, I think it’s a huge exaggeration to suggest it’s useless. I still regularly find quality results from a quick skim of the first or second page of Google results. Meanwhile places like Reddit, Twitter, and Hacker News are full of very strong opinions that feel truthy, but are mostly noise. Unless you go in with enough baseline knowledge to filter out 9/10 underinformed comments to dig out the 10% who actually have direct knowledge of the subject and aren’t just parroting some version of something they read from other comments, skipping straight to social sites becomes a source of misinformation.

lysecret 4y ago

I think this is a very "consumer focused" take. Yes. A lot of interesting people data is now "locked" behind these aggregators and platforms (and also hard to handle because of GDPR). But most interesting company data is still out there.

simion314 4y ago

This is not true, maybe for a subset of Internet users. For example you have Wikis and forums. Wikis are good for communities that are passionate about a topic and they collaborate on buidling content for their passion. Reddit is a valid alternative to forums but if the community s older and has members that are technical competent then they usually have the forum customized for their purpose and the forum will continue to exist , especially if you want to avoid some third party censorship. I never ever search for something and found answers on Facebook, sometimes very rare I find something that points to Instagram blogs/posts but never Facebook. Probably depends on your location and what you search for, so it might be possible that 99% of your Internet consumption is satisfied by 5-10 websites.

jrussbowman 4y ago

"Want to become rich? Make a search engine which indexes the fresh relevant data from the big siloed websites, and ignores the general dead Internet." Did that to some degree. Unscatter.com pulls from reddit and twitter to source links. I found reddit only created an echo chamber bubble of obvious bias and twitter only diluted it a little.

baxtr 4y ago

I am not so sure... I think what happened is this: the WWW was everything back in the days. But in the "old days," only 10% of all people were online, the web elite. Then, AOL came, and the rest came online slowly but surely. The so-called "mainstream" people were no geeks, and these people were "just" ordinary people. Almost all were captured by what you call "big websites". Now, we see the 100% being dominated by the 90%. That's why "Google results are bad". Bad for us! Not maybe (most probably) not for them.

nl 4y ago
Eternal September was Sep 1993. AOL hit the internet in March 1994. Netscape didn't launch until December 1994 (and the WWW was nothing before that. I subscribed to a mailing list with new sites that were released and I'd visit most new websites on the internet on most days with the Cello browser in my uni labs most days). AOL users have been there since the beginning of the WWW. https://en.m.wikipedia.org/wiki/Eternal_September https://en.m.wikipedia.org/wiki/Eternal_September
- CWuestefeld 4y ago
  My recollection is that the AOL event you reference was only making usenet accessible - a point that makes good sense in the context of the eternal September. But when talking about the WWW, that's a very different story. I think that AOL didn't incorporate a web browser until quite some time after that.
  - nl 4y ago
    The WWW took off when Netscape shipped in late 1994. AOL users could use Netscape from the beginning.

psyc 4y ago

Based on my observations over the past year, I’m certain that Google and Bing choose not to show us most of the web anymore. I usually find what I’m looking for. It just takes literally three orders of magnitude longer than it used to for the same kind of stuff. I used to use Google a lot to jog my memory about various things I vaguely remembered. Type a few associative words and snippets, press Enter, done. Google’s useless for that now. If you’re looking for hot pop shit in trendy publications, things to buy, commercial services to subscribe to - G has you covered. That’s what they do now.

stackbutterflow 4y ago

I think you're generalizing your own behavior. I regularly use google to search for topics that cross my mind and I end up on many websites that are not one the giants in your list. It's a fun activity. If people stick to the same 10 websites that's on them. Nothing prevents you from exploring the web.

MockObject 4y ago
> Nothing prevents you from exploring the web. What prevents you from exploring the web is you can't find but the same 10 sites through search engines.

mrtksn 4y ago

It has been dead for a while now and the whole society feels it globally. Things were getting so good then things become horrible and whoever cracks the path to the goods stuff again will find great riches at the end of the path.

dotnet00 4y ago

I agree that this seems way too reductive. I was recently reflecting on this and noticed that I constantly run across new blogs and sites whenever trying to learn something. I just don't usually pay much attention to the site name in the way that I remember HN, Reddit, Twitter etc. So, while I would agree that some aspects of the old internet are dead (like 'small' ~1000 user forums focused on specific topics having largely been replaced by generally inferior subreddits and discord servers), I think it hasn't gotten as bad as you're making it out to be.

jerf 4y ago

"I agree: the WWW Internet is dead, that is your problem. No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there." That is not the Dead Internet Theory. That's just something anyone can see by looking at the world. The Dead Internet Theory is that the Internet is already an echo chamber custom fed to you by a collection of bots and other such things, and that a lot of the "people" you think you're interacting with are already, today, faked. You're basically in a constructed echo chamber designed only with the interests of the creators of that chamber in mind, using the powerful social cues of homo sapiens effectively against you. In particular, those silos aren't where people are communicating. Those silos are where you think you're communicating. It is obviously not entirely true. When we physically meet friends, sometimes topics wander to "Did you see what I posted on Facebook?" So far, we've not caught Facebook actively forging posts from our real-life friends that we physically know. (Though we have caught them failing to disseminate posts in what seems to be a distinctly slanted manner.) I am also not terribly convinced that the bots have mastered long-form content like you see on HN. I think we've had some try, and while they can sort of pass, they seem to expend so much effort on merely "passing" that they don't have much left over to actually drive the conversation. HN probably still requires real humans to manipulate things. Where I do seriously wonder about this theory is Twitter. AI has progressed to the point that short-form content like that can be effectively generated and driven in a certain direction. There's been some chatter on the far-out rumor mills about just how bot-infested Twitter may be, how many people think they have thousands of followers, even having interacted with some of them as "people", and in fact may only have dozens of flesh-and-blood humans following them, if that. Stay tuned, this one is developing. (Note that while this could be "a big plan", it is also a possible outcome of many groups independently coming to the conclusion that a Twitter bot horde could be useful. A few hundred from X trying to nudge you one way, a few hundred from Y trying to nudge you another, another few thousand from Z trying to nudge you yet another, before you know it, the vast vast majority of everyone's "followers" is bots bots bots, and there was no grand plan to produce that result. It just so happens that Twitter's ancient decision to be dedicated to short-form content, with no particular real-world connection to the conversation participants, where everyone is isolated on their own feed (even if that is shared in some ways) made it the first place where this could happen. Things with real-world connections, things where everyone is in the same "area" like an HN conversation, and long-form content will all be three things that will be harder for AIs to manipulate. Twitter is like the agar dish for this sort of thing, by its structure.)

boplicity 4y ago

> No-one visits websites anymore, everyone has moved to the 10 biggest websites and all data is now siloed there. Really? We make our living running a small web based publication; around 40k readers a month. I know of many other sites like this. Google, and other search engines, depends on niche websites to provide quality search results. Without sites like ours, the internet would truly be dead, and search would be mostly useless. Our "traffic sources" come from a mix of Facebook, Search, Reddit, etc, in addition to our many loyal readers. Others in our niche are producing blog spam, which looks nearly identical to people who aren't experts in the field, but we have real experts, fact checkers, etc, as part of our production process. This is a big problem: These low quality websites get similar rankings to our own, which does make it much harder for people to get quality information via search. (Hence the general shift towards trusting social recommendations, such as from Reddit.) In short, the WWW is alive and well, it's just buried under a bunch of #$#$%.

rchaud 4y ago
> Our "traffic sources" come from a mix of Facebook, Search, Reddit, etc, in addition to our many loyal readers. 40k/mo is a pretty good number for an independent website. As a word of warning though, relying on social media reach is a dangerous game, as there is anecdotal evidence that tweets with outbound links don't get as many impressions as those that link to in-site content, like another Twitter post. As for Facebook, well, there's a good comic from The Oatmeal (enormously popular on FB back in 2010) that talks about what happened in the long run: https://twitter.com/Oatmeal/status/923250055540219904 https://twitter.com/Oatmeal/status/923250055540219904

rchaud 4y ago

> Want to become rich? Make a search engine which indexes the fresh relevant data from the big siloed websites, and ignores the general dead Internet. That would be a great service, but it certainly wouldn't make you rich. Where's the money going to come from? Google got rich because they acquired an ads platform (DoubleClick) and an analytics platform (Urchin) and started monetizing the vast amounts of data they had. That was years after Google had established goodwill as the best search engine.

big_blind 4y ago
I use beta search engines. On kagi.com and you.com you can preference and filter top sites. There's also no advertising on either. I've just stopped using Google altogether and its improved search so much.

jspaetzel 4y ago

This is so incredibly false, I've been working on a project for the last six months and MoM I've seen steady increase in usage. Tbh much much higher usage then I expected. Most users find my site via Google or Facebook however they are looking for content that's not in those silos and have no problems leaving them. If you have high quality content and you get it indexed properly by Google, users will come. There are reasons users are not using your website. 1. It's not solving a problem people have. 2. Users can't find it. Who, in their right mind searches for search engines? Nobody I know. If you want users you have to go out and get them (literally pound the pavement and talk to people) or create a LOT more content ironically, so they can find your site on the search engines they are using today.

throw10920 4y ago

> I agree: the WWW Internet is dead I've heard this claim a lot, with 0 supporting evidence. Do you have any? My own experience is that there are thousands of content-rich, high-quality blogs still being written by real humans, because I regularly find and bookmark new ones weekly, without even looking for them, so: please provide evidence for this claim that runs counter to my lived experience.

dageshi 4y ago

I agree with you to an extent. The web is less useful than it used to be. BUT I would say a lot of that usefulness has diverted into youtube. There are people who would previously have made sites who are making youtube videos instead which of course is owned by google.

altairprime 4y ago

If you want to be rich, solve search without full-text indexing of sites. Pagerank only ever worked because of human curation of webrings. Full-text search made is easier to find content, and opened the door for spammers. The only viable route forward for search will be to replace full-text indexing with human curation, somehow. Solve how to scale that up instead, so that when everyone else realizes we need it for the health of the Web, you’re ready.

soheil 4y ago

To a fish the world is made of water and there can't possibly be anything else worthwhile. This is more indicative of how you spend your time online vs reality.

ouid 4y ago

Google is still pretty good at searching reddit. Maybe reddit can acquire them.

big_blind 4y ago
site:reddit just is the best search engine at this point. I still don't like Google though.

derefr 4y ago

> Make a search engine which indexes the fresh relevant data from the big siloed websites, and ignores the general dead Internet I don’t understand why Google themselves don’t do this. LinkedIn v. hiQ demonstrated that they won’t get in trouble for scraping users’ subjective views of data within these silos and then stitching them together to form a cohesive whole. So where’s the effort to do so? It seems like the obvious step.

samstave 4y ago

This MUST be the reason that they threw their purchase of Postini in the garbage and my GMAIL INBOX is filled with spam, and my "social" and "promotions" tabs dont filter.... GMAIL is garbage now, I literally use it as my spam email any more. Which sucks because I have had it for a really long time. Annecdote on Yahoo! Mail ; years ago I wrote to yahoo support asking when I created my Yahoo Mail account (i'd had it from the 90s when it was very early available...) And support told me that they couldnt tell me when my account was created as that was *proprietary company information* So I deleted my Yahoo account. Im about to DL all my gmail and do the same.

omoikane 4y ago

I think the Dead Internet Theory bit is just a bait to get more comments. It's a bit of a stretch to conclude that the internet is mostly robots just because one website sees mostly robots. This extrapolation would be convincing if that one website is a high ranking website that sees a lot of traffic, but searchmysite.net does not appear to be one of the top websites.

deleted 4y ago

[deleted]

hombre_fatal 4y ago

I don't get how TFA shows evidence of the Dead Internet Theory just because their site manages to attract ~zero users. Just host a <form><textarea><button></form> at an IP address and notice it's just spambots submitting it with backlinks, not actual users. Doesn't mean the internet is dead nor that the indieweb is dead. It doesn't really show anything other than the only people able to extract value from your creation are the spammers.

Jenk 4y ago

> If I want to search for something topical and relevant, I go to Facebook, Twitter, Reddit, HackerNews, Instagram, Google Maps, Discord etc. High chances you will find a link to an external site over content actually on those big named sites though, right? That tells us the organic web isn't dead, it's just hard to discover/navigate - because of SEO wars, most probably... The problem isn't the lack of content, it's the number of shitty spammy sites standing in your way of the sites you actually want to see. Like a sleazy salesman trying to direct you to the crap laden three wheeled rust bucket when you were heading toward the family sedans.

heavyset_go 4y ago

I was once on this bandwagon, but I think it was just confirmation bias reflecting the way I used the internet at the time. The non-siloed internet is bigger than the pre-siloed internet ever was.

ClumsyPilot 4y ago

> If you think it's bad for you, imagine what it is like for Google Search! Their entire business is indexing a medium which no longer has any relevancy. Google was the one (among many) that killed it - so I am not gonna shed any tears.

Shinchy 4y ago

I think you're thinking too narrowly about general chit chat content. E-commerce for example is still very much in the function of using your own website. As I would say is documentation, e-learning, saas, company information, etc. It's a more purposeful web. What is dead though is the general blog like content and community platforms of old, the era of Wordpress blogs, forums and hobbyist websites is certainly gone.

Schroedingersat 4y ago

If they wanted good content, they shouldn't have coerced everyone with good content into turning it into illegible seo spam in order to appear in the top 10 pages. The writing was on the wall the first time a recipe site had to start writing stupid stories about their dog.