6 ms·

> rather than moving to a challenge system that actually impacts scrapers At this point isn't it basically auth-only? Rant: (genuinely wondering too, and RFC,

> rather than moving to a challenge system that actually impacts scrapers At this point isn't it basically auth-only? Rant: (genuinely wondering too, and RFC, request for conversation) at this point don't we have Google, etc. basically doing Real World ID Verification, but without an open protocol backing, using it to corral users into their ecosystem and gather data, and leaving us without some open and distributed alternative like the (fairly well working it seems to me) Public Key infrastructure? (PKI) I know Sam Altman (with Proof of Human or w/e) and others have suggested and also have their own alternative (which I think is basically what Google already does), but I think this should simply be built based on an open protocol and then hopefully not-too-large (and with a healthy competition) local companies could do it for you.

GoblinSlayer 17d ago
Botnets will just borrow your id.
- darkmighty 17d ago
  Maybe, but at least you can more easily assign responsibility in this case. If you soft-block or hard-block someone because they're in a botnet, you can get them to change (expire the old ones) their credentials and start with clean systems. Maybe they'll realize their TV is part of a botnet if they have to keep doing it, or their PC has malware.
  - GoblinSlayer 17d ago
    The pattern described in the blog is that one ip makes 4-5 requests and never comes back.
    - darkmighty 17d ago
      Indeed, but then you can (for example) just basically blocklist this ID and share this blocklist with other servers (and likewise import blocklists) -- a blocklist curated by one or more NGOs or the sort would be ideal. Fixing a malware infection should be like fixing up anything else broken around your house (clean things up, re-auth yourself with the ID verifier, done). Also, bots cannot simply automate renewing the credentials of course, after expiry you should be required to physically (or in a trustworthy digital medium) re-authenticate yourself preventing any kind of mass attack, you'd need millions of people to perform authentication multiple times a day (by that point it seems safe to assume they'd rather fix the malware source in their personal devices).
      - xena 17d ago
        I am actually working on this kind of shared reputation database: https://github.com/TecharoHQ/reputationdb https://github.com/TecharoHQ/reputationdb (warning: very prototype, not documented, such bugs, wow). I just have more to do than I have free time at this point.

GoblinSlayer 17d ago

Botnets will just borrow your id.

darkmighty 17d ago
Maybe, but at least you can more easily assign responsibility in this case. If you soft-block or hard-block someone because they're in a botnet, you can get them to change (expire the old ones) their credentials and start with clean systems. Maybe they'll realize their TV is part of a botnet if they have to keep doing it, or their PC has malware.
- GoblinSlayer 17d ago
  The pattern described in the blog is that one ip makes 4-5 requests and never comes back.
  - darkmighty 17d ago
    Indeed, but then you can (for example) just basically blocklist this ID and share this blocklist with other servers (and likewise import blocklists) -- a blocklist curated by one or more NGOs or the sort would be ideal. Fixing a malware infection should be like fixing up anything else broken around your house (clean things up, re-auth yourself with the ID verifier, done). Also, bots cannot simply automate renewing the credentials of course, after expiry you should be required to physically (or in a trustworthy digital medium) re-authenticate yourself preventing any kind of mass attack, you'd need millions of people to perform authentication multiple times a day (by that point it seems safe to assume they'd rather fix the malware source in their personal devices).
    - xena 17d ago
      I am actually working on this kind of shared reputation database: https://github.com/TecharoHQ/reputationdb https://github.com/TecharoHQ/reputationdb (warning: very prototype, not documented, such bugs, wow). I just have more to do than I have free time at this point.