8 ms·

Bot or human? Creating an invisible Turing test for the internet

141 points by timshell 1y ago

adityaagr 1y ago
This is a super clean research post! Absolutely loved the demos too
BobbyTables2 1y ago
Ironic that we are so intent on creating bots that ask and check questions unsolvable by other bots.
JimDabell 1y ago
This is interesting stuff, but I’d be seriously concerned about this accidentally catching people who have accessibility needs. How is it going to handle somebody using the keyboard to tab through controls instead of the mouse? Is a typing cadence detector going to flag people who use voice interfaces?
- lofaszvanitt 1y ago
  Well, they need a doctor signed key pair that shows that they are legit cripples. Feed it to the browser, win.
  - areyourllySorry 1y ago
    you joke but this is pretty much how hcaptcha handles this, no doctor needed though https://www.hcaptcha.com/accessibility https://www.hcaptcha.com/accessibility
qoez 1y ago
I totally assumed typing cadence and mouse behaviour was incorperated into bot detection for years before this already, interesting.
- NoMoreNicksLeft 1y ago
  You can never go wrong betting on laziness and aversion to ambition for excellence.
- bgwalter 1y ago
  chess.com had this a long time ago.
- lq9AJ8yrfs 1y ago
  You are not wrong. The article is more of an intro piece for newcomers and doesn't discuss at all the state of the art or where the competition is--the high end of the market is pretty saturated already but the low end is wide open. There is a bit of a spread in the market, and the specific detection techniques are ofc proprietary and dynamic. Until you have stewed on it quite a bit, it is reasonable to assume everything you can think of has a- been tried b- is either mainstream or doesn't work well c- what working well means is subtle. Bots are adversarial and nasty ones play the field. Sources of truth are scarce and expensive to consult, and the costs of false positives are felt acutely by the users and the buyers, vs false negatives are more of a slow burn and a nagging suspicion.
  - hinkley 1y ago
    As I understand it detection software is also at great pains to make it difficult for bots to analyze the patterns of rejections to figure out what rule is catching them. If they can narrow down the possibilities to quadratic space then you lose.
- timshell 1y ago
  That's definitely been the marketing. The point of Section 1 is to refute that point
  - lucb1e 1y ago
    I had a security manager at a big bank (one of my first clients) tell straight to my face that the website decides whether to let me in before I even start typing the password(-equivalent) and that the password is just a formality not to scare people. Near as I could tell, he believed it himself Marketing indeed. He had me doubting for a while what magic they weren't sharing with the rest of us to avoid countermeasures being developed, but I know better now (working in infosec, seeing what these systems catch, don't catch, and bycatch)
chromatin 1y ago
"When a measure becomes a target, it ceases to be a good measure". https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law BRB, changing the simulated latency in my bot.
- timshell 1y ago
  Agreed. Section 3 takes the idea to the extreme -- can a bot replicate human cognition? Traditional OCR CAPTCHAs were a good 'measure' that couldn't be fully gamed. That is, while the rise of computer vision made them eventually ineffective, the gains in computer vision did not come from bot farms
- deleted 1y ago
  [deleted]
imiric 1y ago
I applaud the effort. We need human-friendly CAPTCHAs, as much as they're generally disliked. They're the only solution to the growing spam and abuse problem on the web. Proof-of-work CAPTCHAs work well for making bots expensive to run at scale, but they still rely on accurate bot detection. Avoiding both false positives and negatives is crucial, yet all existing approaches are not reliable enough. One comment re: > While AI agents can theoretically simulate these patterns, the effort likely outweighs other alternatives. For now. Behavioral and cognitive signals seem to work against the current generation of bots, but will likely also be defeated as AI tools become cheaper and more accessible. It's only a matter of time until attackers can train a model on real human input, and inference to be cheap enough. Or just for the benefit of using a bot on a specific target to outweigh the costs. So I think we will need a different detection mechanism. Maybe something from the real world, some type of ID, or even micropayments. I'm not sure, but it's clear that bot detection is at the opposite, and currently losing, side of the AI race.
- JimDabell 1y ago
  > So I think we will need a different detection mechanism. Maybe something from the real world, some type of ID, or even micropayments. I'm not sure, but it's clear that bot detection is at the opposite, and currently losing, side of the AI race. I think the most likely long-term solution is something like DIDs. https://en.wikipedia.org/wiki/Decentralized_identifier https://en.wikipedia.org/wiki/Decentralized_identifier A small number of trusted authorities (e.g. governments) issue IDs. Users can identify themselves to third-parties without disclosing their real-world identity to the third-party and without disclosing their interaction with the third-party to the issuing body. The key part of this is that the identity is persistent. A website might not know who you are, but they know when it’s you returning. So if you get banned, you can’t just register a new account to evade the ban. You’d need to do the equivalent of getting a new passport from your government.
  - freeone3000 1y ago
    It also allows automated software to act on behalf of a person, which is excellent for assistive technologies and something most current bot detection leaves behind.
NoMoreNicksLeft 1y ago
The problem has never been that some bots could eventually seem like they were human. The problem is and will continue to be that many humans (millions upon millions) look like bots. Have you never once looked at the captcha and couldn't decide whether the 3 pixels of the motorcycle sticking out into the grid square meant that you should select that grid square too? Not once? As the tests become ever more sophisticated, more and more of you all will be locked out.
- baby_souffle 1y ago
  Or you'll get the "click all squares with a stop light" prompt and it's a closeup of a signal light so you just click everything... But if you get it correct _and_ too quick l, you're a bot!
- gus_massa 1y ago
  Is the guy on the motorcycle part of the motorcycle? I guess no. Is the big box on the back seat part of the motorcycle? I guess yes. Who can be sure???
TechDebtDevin 1y ago
I personally work on this all day everyday, you're never going to find my crawlers, stop trying lmfao.
- erekp 1y ago
  same. good luck finding us out there - we can replicate all the patterns you point out there. been in this industry for 10 years now :)
  - Dylan16807 1y ago
    Just don't cause problems on purpose and almost nobody will care about blocking you. Don't be an asshole.
    - metladsl 1y ago
      [flagged]
- deleted 1y ago
  [deleted]
- yvely 1y ago
  I like that you chime in on this and think better cooperation makes more sense here. I assume you also have no interest in enshittified, slower, more expensive, environmentally worse, lower quality Internet. From your point of view, what are potential options/directions to approach to solve some of the issues arising from f.x. crawlers, bota, and other detection-avoiders? I am genuinely curious here
kjok 1y ago
Solutions relying on JavaScript that runs in user-controlled browsers are vulnerable to attacks and manipulation.
- _df 1y ago
  >Solutions relying on JavaScript ... ... break the Web. ftfy
charcircuit 1y ago
>How much can these behavioral patterns be spoofed? This remains an ongoing question, but the evidence to date is optimistic. Academic studies have found behavioral biometrics to be robust against attacks under adversarial conditions, and industry validation from top financial institutions demonstrates real-world resilience I have the opposite view. This already played out in the Minecraft community and it turns out ghost clients are effective in spoofing such behavioral signals and avoiding anticheat. Also I doubt you can get any meaningful signal from the couple of a seconds a user's ai agent is scrolling through a site.
joshmarinacci 1y ago
I feel like we are fighting the wrong battle here. Eventually AI bot behavior online will be indistinguishable from human, but so what?! We've had teams of underpaid humans being paid to be organic bots for years now. Whether the person interacting with your website is human or not isn't relevant anymore. What matters is what they are doing; be they human, bot, or AI agent.
- butundstand 1y ago
  You have to understand the motive to understand why this is a problem; their startups haven’t unicorned yet. They never had a fallback plan so humanity must cling to web app driven economics until they unicorn. See also Elon demanding ad spend on his platform or like it’s literally just like when the Nazis invaded Poland. Anyone got some E? PLUR, bro but also fewer vacay days for you. Empty economic activity driven by fiat decree of wealth hoarders suffering from post war and Cold War and leaded gas fume, lead water fueled paranoias and psychosis. People made insane by memorization of illusory social obligations to history always run the world.
- Terr_ 1y ago
  IMO in most cases, the real need is to ensure the new account has "skin in the game", so that their requests are not frivolous and they will "care" about the good standing of their account.
bwfan123 1y ago
We also need an inverse turing test. ie, detect humans pretending to be AI. Like the case recently of builder.ai which had humans pretending to be ai. Turing was a visionary - but even he could not imagine a time when humans pretend to be bots.
- hobs 1y ago
  Not so far fetched, The Mechanical Turk was created in the 1700s, so that already happened a long time before Turing was born.
- jenadine 1y ago
  Yet, human pretending to be machine have existed for centuries https://en.m.wikipedia.org/wiki/Mechanical_Turk https://en.m.wikipedia.org/wiki/Mechanical_Turk
hinkley 1y ago
I’ve wanted to create a wiki for a hobby for a long time, but I don’t want to get stuck in spam and abuse reports, which just becomes more of a given with each passing year. With a hobby wiki, eventual consistency is fine. I believe ghost bans and quarantine and some sort of invisible captcha would go a long way toward my goal, but it’s hard to find invisible captcha. There was a research project long ago that used high resolution data from keyboards to determine who was typing. The idea was not to use the typing pattern as a password, but to flag suspicious activity. To have someone walk past that desk to see if Sally hurt her arm playing tennis this weekend of if Dave is fucking around on her computer while she’s in a meeting That’s about the level I’m looking for. Assume everyone is a bot during a probationary period and put accounts into buckets of likely human, likely bot, and unknown. What I’d have to work out though is temporary storage for candidate edits in a way they cannot fill up my database. A way to throttle them and throw some away if they hit a limit. Otherwise it’s still a DOS attack.
- lucb1e 1y ago
  How does one graduate from probation, while being hellbanned / having your contribution quarantined? Since I'm certainly not wasting my time doing a second contribution so long as the first one isn't getting approved, it sounds like this would have to be a manual process or you'd lose out on new contributors that are seeing their work go to /dev/null and never returning
  - hinkley 1y ago
    Do you believe what we are doing now is working? Because with the exception of places like this the internet sure looks pretty Dead to me. You always have to show people their own edits. It's a common form of proofreading. But what's added and how often does matter. Misinformation is one thing. External links are potentially something much worse. I used to think SO had it figured out as far as mutual policing, but that's not working so well now either.
    - lucb1e 1y ago
      I'm not sure what e.g. showing one one's own change answers. Do you manually review submissions or how does get one out of this initial "put everyone in quarantine" state? I'm also not sure what "we" are doing now that makes the web look dead to you. I receive no more email spam than ten years ago, less if anything, and I haven't seen any spam on the places that I frequent like HN, stackexchange, wikipedia, mastodon, signal, github, etc.
hinkley 1y ago
I think the real purpose of Google’s recaptcha is to punish people who have privacy settings turned on, and gather training data for AI research.
logsr 1y ago
In a few more years there will probably be virtually no human users of web sites and apps. Everything will be through an AI agent mediation layer. Building better CAPTCHAs is interesting technically, but it is doubling down on a failed solution that nobody actually wants. What is needed is an authentication layer that allows agents to act on behalf of registered users with economic incentives to control usage. CAPTCHA has always been an economic bar only, since they are easy to farm out to human solvers, and it is a very low bar. Having an agent API with usage charges is a much better solution because it compensates operators instead of wasting the cost of solving CAPTCHAs. Maybe this will finally be the era of micro payments?
- mdahardy 1y ago
  Co-founder of Roundtable here. I agree that better authentication methods for AI agents are needed. But right now bots and malicious agents are a real problem for anyone running sites with significant traffic. In the long run I don’t think human traffic will go to zero even if its relative proportion is reduced.
- contagiousflow 1y ago
  > Building better CAPTCHAs is interesting technically, but it is doubling down on a failed solution that nobody actually wants I want it. I don't want my message boards to be people's AI agents...
- emporas 1y ago
  Certainly. An authentication layer, and everything else customizable by the user. The web, HTML that is, is a grammar, an app is a grammar, the buttons of my car are a grammar, I want each grammar served, transformed to my grammar however I like it, probably org-mode file grammar. I don't want each website's colors, or clickable elements to be determined by any other person than the user. There are themes, I want to select exactly what theme I am browsing the internet today. I also want my fridge to be connected to the internet, accessed using an authentication layer on top of IPv6, and using it's functionality with a grammar. In other words, the web, browsers, apps and physical buttons will go down the drain soon and they will be replaced by something which can open and manipulate org filetypes. The web was/is a huge financial bubble anyway, and it will burst quickly when that happens.
mzmzmzm 1y ago
All of the behavioral analysis stuff going on in the background makes me wonder if big accessibility problems are brewing. If we're looking at how naturally keystrokes are input, what does that mean for someone who uses dictation tools that generate text in chunks? Will this strategy make accessibility worse in unforeseen ways?
_el1s7 1y ago
It's pointless, it's just a matter of time when AI agents will be able to mimic human behavior exactly (they probably already do, it's just not public). These tests here are easily bypassable, just adding a random delay somewhere during the action phases to mimic humans, and there's already tools for mimicking human mouse movements.
koalaman 1y ago
I'm not sure reCAPTCHA is really trying to detect automated vs human interaction with a browser. The primary use-case is to detect abusive use. The distinction here is if I automate my own browser to do things for me on sites using my personal account may not be a problem for site owners, while a spam operation or reselling operation which generates thousands of false accounts using automation is a big problem that they'd want to be able to block. I think reCAPTCHA is tailored towards the latter, and for it not to block the former might be more of a feature than a bug.
- roguecoder 1y ago
  LinkedIn, for example, doesn't care if you as a human are manually looking at all your connections one-by-one or if you have automated a bot to do it: it will lock you out the same either way.
b0a04gl 1y ago
[dead]
loandbehold 1y ago
Aren't those distinctions only work because bots aren't specifically designed to circumvent them? If you have an arms race between bots and bot detectors, eventually bots will learn to overcome them to the point that you can't distinguish human and bot.
lucb1e 1y ago
And so what am I supposed to do if a false positive happens? I use keyboard navigation on many pages. Using the firefox setting "search when you start typing", I don't have to hit ctrl+f to search on the page, I just type what I want to click on and press enter or ctrl+enter for a new browser tab, or press (shift+)tab to go to the nearest (previous/next) input field. When I open HN, it's muscle memory: ctrl+t (new tab) new enter (autocompletes to the domain) thr enter (go to threads page) anything new? type first few chars of username, shift+tab+tab enter to upvote. Done? Backspace to go back. View comments of a link? Type last char of a word in the link, space, and first char of next word, that's almost always unique on the page, then escape, type men, enter, to almost always activate the comment link. Or shift+tab enter instead to upvote. On the comments page, reading top-level comments is either searching for [ and then enter+f3 when I want to collapse the next one, space for page down... Don't have to take my hands off the home row etc. on lots of website, also ones I've never visited before (it'll be slower and less habitual of course, but still: if there is text near to where I want to go, I'm typing it). I use the mouse as well, but I find it harder to use than the keys that are always in the same place, much easier to press So will it tell me that my mouse movements don't look human enough or will I see a "Sorry, something went wrong" http 403 error and have no clue if it's tracking cookies, my IP address, that I don't use Google Chrome®, that I went through pages too fast, that I didn't come past the expected page (where a cookie gets set) but clicked on a search result directly, that I have a bank in country A but residence in country B, that I now did too many tries in figuring out which of these factors is blocking me.... I can give examples of websites where I got blocked in the last ~2 months for each of these. It's such a minefield. The only thing that always passes is proof-of-work CPU challenges, but I dread to think what poor/eco people with slow/old computers are facing. Will this "invisible" captcha (yeah, invisible until you get banned) at least tell me how I'm supposed to give my money to whatever service or webshop will use this?
- ivanjermakov 1y ago
  I also use keyboard navigation and Vimium's quick link actions, and I often catch CAPCHAs and rate limiting on some websites because I'm too fast. Fun times!
Animats 1y ago
Previous CAPTCHAs were based on tasks humans could do but machines could not. The machines caught up and passed humans on those tasks. These new tasks are based on the concept that humans are dumber than AI agents, making more mistakes and showing more randomness. It might work for a while, but that's a losing battle.
- timshell 1y ago
  > These new tasks are based on the concept that humans are dumber than AI agents, making more mistakes and showing more randomness. Hi this is incorrect. Different =/= dumber. The insight is that humans and computers have different constraints / algorithmic capabilities / objective functions / etc.
  - Animats 1y ago
    For a few more years, humans who haven't been laid off yet can believe that.
guddlickspls 1y ago
[dead]
avoutos 1y ago
Anyone know how this compares to Cloudflare Turnstile?
renegat0x0 1y ago
So recently two things have happened. I have been banned on reddit technology, and warned on other subreddit that I behave like a bot. Maybe it was my fault to advertise my own solution in comments. Such behavior however triggered bot detection. I might have behaved like a NPC. So currently a human can be identified as a bot, and banned on that premise. Crazy times. Currently I feel I must act like a human.
- ivanjermakov 1y ago
  I read your comment with a robot voice, kidding. False positives are always gonna be the problem.
ATechGuy 1y ago
Please don't deploy this on the internet, it may block real users and lock them out.
throwaway48476 1y ago
Not all automation is malicious. AI promised us agents that will browse the web for us. PoW is useful in that the difficulty can be scaled to prevent egregious abuse but still lower the cost enough to allow non malicious use.
- Nextgrid 1y ago
  It's malicious if your compensation depends on wasting human time. Sadly, a lot of people's careers and compensation does depend on that.
lugu 1y ago
It is late and I am thinking out load. How about a reputation system where users bring proof that other websites haven't found them abusive. Visit a website that require identification. Generate a random unique identifier in your user agent. Live your life on that site. Download from that site a certificate that prove that your didn't abuse their site. Repeat that a few times. Visit the site that wants to know if you are an abusive user. Share your certificates. They get to choose if they accept you. If you abuse that site, it reports the abuse to the other sites that delivered you a certificate. Those sites gets to decide if they revoke their certificate or not. It is a self policying system that require some level of cooperation. Users make themselves vulnerable to the risk of having sites they like loose trust in them.
- rcstank 1y ago
  Sounds like a privacy nightmare. Also, what one site calls abuse, another wouldn't.
- spondylosaurus 1y ago
  Some stuff would definitely either slip through the cracks OR tarnish the reputation of legitimate users. What happens when someone's device gets compromised by a botnet that silently clicks ads in the background or turns that device into part of a DDoS army?
  - MichaelZuo 1y ago
    Why would anyone even expect a perfectly zero false-positive and false-negative rate in the first place?
- lq9AJ8yrfs 1y ago
  > It is a self policying system that require some level of cooperation. How hard is it to obtain one of these certificates as a bot? What you are describing though is possibly comparable to Privacypass. Apple seems to be on board with Privacypass, perhaps they'll include a digital voucher of some kind with their devices and that presumably contributes to old devices getting worse as the voucher is spent down. Just imagine if the whole web can contribute to planned obsolescence and you can pay for a fast, hassle free internet experience again just by buying a new phone. And then you can dump the old ones on eBay for cheap as long as you don't plan on using them to access online services. Unless you are willing to settle for basic economy web experience.
thatcat 1y ago
If the general Internet was based on torrents, then the required upload ratio enforcement would have ensured bots contribute to the reliability rather than destabilize the infrastructure.
- ivanjermakov 1y ago
  The problem would move from verifying legitimacy of agent actions to legitimacy of agent reputation, which might even be easier to spoof.
- areyourllySorry 1y ago
  torrents don't have a required upload ratio though.
  - thatcat 1y ago
    There are tracker ecosystems that enforce ratio, what.cd for example had developed a pretty advanced torrent economic rule set.
    - areyourllySorry 1y ago
      I know what you mean but it's not part of the protocol and I don't think you could enforce it on an internet scale
curtisblaine 1y ago
AFAIK, reCaptcha is not based on user behaviour anymore since v3, but uses proprietary network host information from Google. See this issue opened in 2018: https://github.com/google/recaptcha/issues/235 https://github.com/google/recaptcha/issues/235
Nevermark 1y ago
The problem becomes simpler if you turn it around. It is getting easier and easier to create questions/problems that humans can't answer at LLM speed. Of course, that solves a complementary problem, not the original. But in terms of instances, by any definition, the demographics are quickly moving in one direction.
protocolture 1y ago
I have noticed that a particular website will tell me I fail captcha half the time, until I resize my browser from a square to a rectangle. Took me ages to figure out what its issue was.
userbinator 1y ago
Or you could just ask it to count how many letters are in certain words.
xarope 1y ago
> Take for example the Stroop task . It's a classic psychology experiment where humans select the color a word is written it and not what the word says. Humans typically show slower responses when the meaning of a word conflicts with its color (e.g., the word "BLUE" written in green), reflecting an overriding of automatic behavior . Bots and AI agents, by contrast, are not subject to such interference and can respond with consistent speed regardless of stimuli. So I completely disagree with this; you can train youself to completely ignore the color and just read/act on the word very fast. In fact, this is a game that people play.
sly010 1y ago
First off, I always thought the type of things described (tracking mouse movements, keypress jitter, etc) are already done by ReCacpha to decide when to present the user with a captcha. I am surprised they are not already doing this. Second, I am surprised AI agents are this naive. I thought they would emulate human behavior better. In fact, just based on this article, very little effort has been put into this race on either side. So I wonder if is has to do with the fact that if companies like google reliably filtered out bot traffic, they would loose 90% of their AD revenue. This way they have plausible deniability.
- fredfish 1y ago
  They were very proud of this mouse movement stuff when the desktop was 70% of traffic.. It's not worth as much investment as its been given since there's no group limiting people to one HID method and removing accessibility from world.
dsekz 1y ago
Plenty of improvements to mouse movement algorithms have already been made and they’re still evolving. While the blog post and the product it introduces offer some interesting ideas, they don’t yet reach the robustness of modern anti-bot solutions and still trail current industry standards. I doubt it would take me - or any average reverse engineer - more than five seconds to bypass something like this. There are already numerous open source mouse movement libraries available; and even if they didn’t exist, writing one wouldn’t be difficult. Yes, mouse movement or keyboard data can be quite powerful in a modern anti-bot stack and an in depth analysis of it is genuinely valuable, but on its own it’s still insufficient. Relying on this data alone isn’t costly for the attacker and offers little real protection.
- klaussilveira 1y ago
  > they don’t yet reach the robustness of modern anti-bot solutions Like what?
randomtoast 1y ago
Isn’t it possible to emulate mouse movements and keypress jitter using a neural network trained on human data in order to simulate human behavior?
Saris 1y ago
I feel like analyzing keystrokes or mouse movements is just going to punish people who use password managers that autofill for them. It does seem like I get more captchas when on sites because of that.
PaulHoule 1y ago
Personally I think CAPTCHAs are harmful. They defend the enshittification economy, preventing the development of tools that protect human users of the web right in a time when it is more practical than ever to develop those tools. Even in 2009 I knew people who were using neural networks (in PHP no less!) to decode CAPTCHAs with superhuman peformance. I see the whole thing as performative as those things get in my way tens or hundreds of times a day when I browse the web as a human but in years of webcrawling they didn't give me any trouble until the last two weeks.
codedokode 1y ago
Recently I started getting a captcha when trying to use Google, probably because of VPN or Linux. I decided to switch to bing and duckduckgo. Dear Google, go solve your captchas yourself.
- instagib 1y ago
  Turning on Firefox anti fingerprinting triggered it for me.
instagib 1y ago
Anyone tried creating a new GitHub account lately? I considered using AI to help get past the proof of work requirement. The imagery test had me click dragging the finger pointing to see if I could line up with the animal. Probably 80% success. Audio detect two different people with heavy accents speaking vs singular people speaking was my lowest score probably 60-70%. Music switching instruments was around 70% also. Ultimately I passed after many attempts on the imagery detection.
Kate5477 1y ago
[dead]

141 points by timshell 1y ago

adityaagr 1y ago

This is a super clean research post! Absolutely loved the demos too

BobbyTables2 1y ago

Ironic that we are so intent on creating bots that ask and check questions unsolvable by other bots.

JimDabell 1y ago

This is interesting stuff, but I’d be seriously concerned about this accidentally catching people who have accessibility needs. How is it going to handle somebody using the keyboard to tab through controls instead of the mouse? Is a typing cadence detector going to flag people who use voice interfaces?

lofaszvanitt 1y ago
Well, they need a doctor signed key pair that shows that they are legit cripples. Feed it to the browser, win.
- areyourllySorry 1y ago
  you joke but this is pretty much how hcaptcha handles this, no doctor needed though https://www.hcaptcha.com/accessibility https://www.hcaptcha.com/accessibility

qoez 1y ago

I totally assumed typing cadence and mouse behaviour was incorperated into bot detection for years before this already, interesting.

NoMoreNicksLeft 1y ago
You can never go wrong betting on laziness and aversion to ambition for excellence.
bgwalter 1y ago
chess.com had this a long time ago.
lq9AJ8yrfs 1y ago
You are not wrong. The article is more of an intro piece for newcomers and doesn't discuss at all the state of the art or where the competition is--the high end of the market is pretty saturated already but the low end is wide open. There is a bit of a spread in the market, and the specific detection techniques are ofc proprietary and dynamic. Until you have stewed on it quite a bit, it is reasonable to assume everything you can think of has a- been tried b- is either mainstream or doesn't work well c- what working well means is subtle. Bots are adversarial and nasty ones play the field. Sources of truth are scarce and expensive to consult, and the costs of false positives are felt acutely by the users and the buyers, vs false negatives are more of a slow burn and a nagging suspicion.
- hinkley 1y ago
  As I understand it detection software is also at great pains to make it difficult for bots to analyze the patterns of rejections to figure out what rule is catching them. If they can narrow down the possibilities to quadratic space then you lose.
timshell 1y ago
That's definitely been the marketing. The point of Section 1 is to refute that point
- lucb1e 1y ago
  I had a security manager at a big bank (one of my first clients) tell straight to my face that the website decides whether to let me in before I even start typing the password(-equivalent) and that the password is just a formality not to scare people. Near as I could tell, he believed it himself Marketing indeed. He had me doubting for a while what magic they weren't sharing with the rest of us to avoid countermeasures being developed, but I know better now (working in infosec, seeing what these systems catch, don't catch, and bycatch)

chromatin 1y ago

"When a measure becomes a target, it ceases to be a good measure". https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law BRB, changing the simulated latency in my bot.

timshell 1y ago
Agreed. Section 3 takes the idea to the extreme -- can a bot replicate human cognition? Traditional OCR CAPTCHAs were a good 'measure' that couldn't be fully gamed. That is, while the rise of computer vision made them eventually ineffective, the gains in computer vision did not come from bot farms
deleted 1y ago
[deleted]

imiric 1y ago

I applaud the effort. We need human-friendly CAPTCHAs, as much as they're generally disliked. They're the only solution to the growing spam and abuse problem on the web. Proof-of-work CAPTCHAs work well for making bots expensive to run at scale, but they still rely on accurate bot detection. Avoiding both false positives and negatives is crucial, yet all existing approaches are not reliable enough. One comment re: > While AI agents can theoretically simulate these patterns, the effort likely outweighs other alternatives. For now. Behavioral and cognitive signals seem to work against the current generation of bots, but will likely also be defeated as AI tools become cheaper and more accessible. It's only a matter of time until attackers can train a model on real human input, and inference to be cheap enough. Or just for the benefit of using a bot on a specific target to outweigh the costs. So I think we will need a different detection mechanism. Maybe something from the real world, some type of ID, or even micropayments. I'm not sure, but it's clear that bot detection is at the opposite, and currently losing, side of the AI race.

JimDabell 1y ago
> So I think we will need a different detection mechanism. Maybe something from the real world, some type of ID, or even micropayments. I'm not sure, but it's clear that bot detection is at the opposite, and currently losing, side of the AI race. I think the most likely long-term solution is something like DIDs. https://en.wikipedia.org/wiki/Decentralized_identifier https://en.wikipedia.org/wiki/Decentralized_identifier A small number of trusted authorities (e.g. governments) issue IDs. Users can identify themselves to third-parties without disclosing their real-world identity to the third-party and without disclosing their interaction with the third-party to the issuing body. The key part of this is that the identity is persistent. A website might not know who you are, but they know when it’s you returning. So if you get banned, you can’t just register a new account to evade the ban. You’d need to do the equivalent of getting a new passport from your government.
- freeone3000 1y ago
  It also allows automated software to act on behalf of a person, which is excellent for assistive technologies and something most current bot detection leaves behind.

NoMoreNicksLeft 1y ago

The problem has never been that some bots could eventually seem like they were human. The problem is and will continue to be that many humans (millions upon millions) look like bots. Have you never once looked at the captcha and couldn't decide whether the 3 pixels of the motorcycle sticking out into the grid square meant that you should select that grid square too? Not once? As the tests become ever more sophisticated, more and more of you all will be locked out.

baby_souffle 1y ago
Or you'll get the "click all squares with a stop light" prompt and it's a closeup of a signal light so you just click everything... But if you get it correct _and_ too quick l, you're a bot!
gus_massa 1y ago
Is the guy on the motorcycle part of the motorcycle? I guess no. Is the big box on the back seat part of the motorcycle? I guess yes. Who can be sure???

TechDebtDevin 1y ago

I personally work on this all day everyday, you're never going to find my crawlers, stop trying lmfao.

erekp 1y ago
same. good luck finding us out there - we can replicate all the patterns you point out there. been in this industry for 10 years now :)
- Dylan16807 1y ago
  Just don't cause problems on purpose and almost nobody will care about blocking you. Don't be an asshole.
  - metladsl 1y ago
    [flagged]
deleted 1y ago
[deleted]
yvely 1y ago
I like that you chime in on this and think better cooperation makes more sense here. I assume you also have no interest in enshittified, slower, more expensive, environmentally worse, lower quality Internet. From your point of view, what are potential options/directions to approach to solve some of the issues arising from f.x. crawlers, bota, and other detection-avoiders? I am genuinely curious here

kjok 1y ago

Solutions relying on JavaScript that runs in user-controlled browsers are vulnerable to attacks and manipulation.

_df 1y ago
>Solutions relying on JavaScript ... ... break the Web. ftfy

charcircuit 1y ago

>How much can these behavioral patterns be spoofed? This remains an ongoing question, but the evidence to date is optimistic. Academic studies have found behavioral biometrics to be robust against attacks under adversarial conditions, and industry validation from top financial institutions demonstrates real-world resilience I have the opposite view. This already played out in the Minecraft community and it turns out ghost clients are effective in spoofing such behavioral signals and avoiding anticheat. Also I doubt you can get any meaningful signal from the couple of a seconds a user's ai agent is scrolling through a site.

joshmarinacci 1y ago

I feel like we are fighting the wrong battle here. Eventually AI bot behavior online will be indistinguishable from human, but so what?! We've had teams of underpaid humans being paid to be organic bots for years now. Whether the person interacting with your website is human or not isn't relevant anymore. What matters is what they are doing; be they human, bot, or AI agent.

butundstand 1y ago
You have to understand the motive to understand why this is a problem; their startups haven’t unicorned yet. They never had a fallback plan so humanity must cling to web app driven economics until they unicorn. See also Elon demanding ad spend on his platform or like it’s literally just like when the Nazis invaded Poland. Anyone got some E? PLUR, bro but also fewer vacay days for you. Empty economic activity driven by fiat decree of wealth hoarders suffering from post war and Cold War and leaded gas fume, lead water fueled paranoias and psychosis. People made insane by memorization of illusory social obligations to history always run the world.
Terr_ 1y ago
IMO in most cases, the real need is to ensure the new account has "skin in the game", so that their requests are not frivolous and they will "care" about the good standing of their account.

bwfan123 1y ago

We also need an inverse turing test. ie, detect humans pretending to be AI. Like the case recently of builder.ai which had humans pretending to be ai. Turing was a visionary - but even he could not imagine a time when humans pretend to be bots.

hobs 1y ago
Not so far fetched, The Mechanical Turk was created in the 1700s, so that already happened a long time before Turing was born.
jenadine 1y ago
Yet, human pretending to be machine have existed for centuries https://en.m.wikipedia.org/wiki/Mechanical_Turk https://en.m.wikipedia.org/wiki/Mechanical_Turk

hinkley 1y ago

I’ve wanted to create a wiki for a hobby for a long time, but I don’t want to get stuck in spam and abuse reports, which just becomes more of a given with each passing year. With a hobby wiki, eventual consistency is fine. I believe ghost bans and quarantine and some sort of invisible captcha would go a long way toward my goal, but it’s hard to find invisible captcha. There was a research project long ago that used high resolution data from keyboards to determine who was typing. The idea was not to use the typing pattern as a password, but to flag suspicious activity. To have someone walk past that desk to see if Sally hurt her arm playing tennis this weekend of if Dave is fucking around on her computer while she’s in a meeting That’s about the level I’m looking for. Assume everyone is a bot during a probationary period and put accounts into buckets of likely human, likely bot, and unknown. What I’d have to work out though is temporary storage for candidate edits in a way they cannot fill up my database. A way to throttle them and throw some away if they hit a limit. Otherwise it’s still a DOS attack.

lucb1e 1y ago
How does one graduate from probation, while being hellbanned / having your contribution quarantined? Since I'm certainly not wasting my time doing a second contribution so long as the first one isn't getting approved, it sounds like this would have to be a manual process or you'd lose out on new contributors that are seeing their work go to /dev/null and never returning
- hinkley 1y ago
  Do you believe what we are doing now is working? Because with the exception of places like this the internet sure looks pretty Dead to me. You always have to show people their own edits. It's a common form of proofreading. But what's added and how often does matter. Misinformation is one thing. External links are potentially something much worse. I used to think SO had it figured out as far as mutual policing, but that's not working so well now either.
  - lucb1e 1y ago
    I'm not sure what e.g. showing one one's own change answers. Do you manually review submissions or how does get one out of this initial "put everyone in quarantine" state? I'm also not sure what "we" are doing now that makes the web look dead to you. I receive no more email spam than ten years ago, less if anything, and I haven't seen any spam on the places that I frequent like HN, stackexchange, wikipedia, mastodon, signal, github, etc.

hinkley 1y ago

I think the real purpose of Google’s recaptcha is to punish people who have privacy settings turned on, and gather training data for AI research.

logsr 1y ago

In a few more years there will probably be virtually no human users of web sites and apps. Everything will be through an AI agent mediation layer. Building better CAPTCHAs is interesting technically, but it is doubling down on a failed solution that nobody actually wants. What is needed is an authentication layer that allows agents to act on behalf of registered users with economic incentives to control usage. CAPTCHA has always been an economic bar only, since they are easy to farm out to human solvers, and it is a very low bar. Having an agent API with usage charges is a much better solution because it compensates operators instead of wasting the cost of solving CAPTCHAs. Maybe this will finally be the era of micro payments?

mdahardy 1y ago
Co-founder of Roundtable here. I agree that better authentication methods for AI agents are needed. But right now bots and malicious agents are a real problem for anyone running sites with significant traffic. In the long run I don’t think human traffic will go to zero even if its relative proportion is reduced.
contagiousflow 1y ago
> Building better CAPTCHAs is interesting technically, but it is doubling down on a failed solution that nobody actually wants I want it. I don't want my message boards to be people's AI agents...
emporas 1y ago
Certainly. An authentication layer, and everything else customizable by the user. The web, HTML that is, is a grammar, an app is a grammar, the buttons of my car are a grammar, I want each grammar served, transformed to my grammar however I like it, probably org-mode file grammar. I don't want each website's colors, or clickable elements to be determined by any other person than the user. There are themes, I want to select exactly what theme I am browsing the internet today. I also want my fridge to be connected to the internet, accessed using an authentication layer on top of IPv6, and using it's functionality with a grammar. In other words, the web, browsers, apps and physical buttons will go down the drain soon and they will be replaced by something which can open and manipulate org filetypes. The web was/is a huge financial bubble anyway, and it will burst quickly when that happens.

mzmzmzm 1y ago

All of the behavioral analysis stuff going on in the background makes me wonder if big accessibility problems are brewing. If we're looking at how naturally keystrokes are input, what does that mean for someone who uses dictation tools that generate text in chunks? Will this strategy make accessibility worse in unforeseen ways?

_el1s7 1y ago

It's pointless, it's just a matter of time when AI agents will be able to mimic human behavior exactly (they probably already do, it's just not public). These tests here are easily bypassable, just adding a random delay somewhere during the action phases to mimic humans, and there's already tools for mimicking human mouse movements.

koalaman 1y ago

I'm not sure reCAPTCHA is really trying to detect automated vs human interaction with a browser. The primary use-case is to detect abusive use. The distinction here is if I automate my own browser to do things for me on sites using my personal account may not be a problem for site owners, while a spam operation or reselling operation which generates thousands of false accounts using automation is a big problem that they'd want to be able to block. I think reCAPTCHA is tailored towards the latter, and for it not to block the former might be more of a feature than a bug.

roguecoder 1y ago
LinkedIn, for example, doesn't care if you as a human are manually looking at all your connections one-by-one or if you have automated a bot to do it: it will lock you out the same either way.

b0a04gl 1y ago

[dead]

loandbehold 1y ago

Aren't those distinctions only work because bots aren't specifically designed to circumvent them? If you have an arms race between bots and bot detectors, eventually bots will learn to overcome them to the point that you can't distinguish human and bot.

lucb1e 1y ago

And so what am I supposed to do if a false positive happens? I use keyboard navigation on many pages. Using the firefox setting "search when you start typing", I don't have to hit ctrl+f to search on the page, I just type what I want to click on and press enter or ctrl+enter for a new browser tab, or press (shift+)tab to go to the nearest (previous/next) input field. When I open HN, it's muscle memory: ctrl+t (new tab) new enter (autocompletes to the domain) thr enter (go to threads page) anything new? type first few chars of username, shift+tab+tab enter to upvote. Done? Backspace to go back. View comments of a link? Type last char of a word in the link, space, and first char of next word, that's almost always unique on the page, then escape, type men, enter, to almost always activate the comment link. Or shift+tab enter instead to upvote. On the comments page, reading top-level comments is either searching for [ and then enter+f3 when I want to collapse the next one, space for page down... Don't have to take my hands off the home row etc. on lots of website, also ones I've never visited before (it'll be slower and less habitual of course, but still: if there is text near to where I want to go, I'm typing it). I use the mouse as well, but I find it harder to use than the keys that are always in the same place, much easier to press So will it tell me that my mouse movements don't look human enough or will I see a "Sorry, something went wrong" http 403 error and have no clue if it's tracking cookies, my IP address, that I don't use Google Chrome®, that I went through pages too fast, that I didn't come past the expected page (where a cookie gets set) but clicked on a search result directly, that I have a bank in country A but residence in country B, that I now did too many tries in figuring out which of these factors is blocking me.... I can give examples of websites where I got blocked in the last ~2 months for each of these. It's such a minefield. The only thing that always passes is proof-of-work CPU challenges, but I dread to think what poor/eco people with slow/old computers are facing. Will this "invisible" captcha (yeah, invisible until you get banned) at least tell me how I'm supposed to give my money to whatever service or webshop will use this?

ivanjermakov 1y ago
I also use keyboard navigation and Vimium's quick link actions, and I often catch CAPCHAs and rate limiting on some websites because I'm too fast. Fun times!

Animats 1y ago

Previous CAPTCHAs were based on tasks humans could do but machines could not. The machines caught up and passed humans on those tasks. These new tasks are based on the concept that humans are dumber than AI agents, making more mistakes and showing more randomness. It might work for a while, but that's a losing battle.

timshell 1y ago
> These new tasks are based on the concept that humans are dumber than AI agents, making more mistakes and showing more randomness. Hi this is incorrect. Different =/= dumber. The insight is that humans and computers have different constraints / algorithmic capabilities / objective functions / etc.
- Animats 1y ago
  For a few more years, humans who haven't been laid off yet can believe that.

guddlickspls 1y ago

[dead]

avoutos 1y ago

Anyone know how this compares to Cloudflare Turnstile?

renegat0x0 1y ago

So recently two things have happened. I have been banned on reddit technology, and warned on other subreddit that I behave like a bot. Maybe it was my fault to advertise my own solution in comments. Such behavior however triggered bot detection. I might have behaved like a NPC. So currently a human can be identified as a bot, and banned on that premise. Crazy times. Currently I feel I must act like a human.

ivanjermakov 1y ago
I read your comment with a robot voice, kidding. False positives are always gonna be the problem.

ATechGuy 1y ago

Please don't deploy this on the internet, it may block real users and lock them out.

throwaway48476 1y ago

Not all automation is malicious. AI promised us agents that will browse the web for us. PoW is useful in that the difficulty can be scaled to prevent egregious abuse but still lower the cost enough to allow non malicious use.

Nextgrid 1y ago
It's malicious if your compensation depends on wasting human time. Sadly, a lot of people's careers and compensation does depend on that.

lugu 1y ago

It is late and I am thinking out load. How about a reputation system where users bring proof that other websites haven't found them abusive. Visit a website that require identification. Generate a random unique identifier in your user agent. Live your life on that site. Download from that site a certificate that prove that your didn't abuse their site. Repeat that a few times. Visit the site that wants to know if you are an abusive user. Share your certificates. They get to choose if they accept you. If you abuse that site, it reports the abuse to the other sites that delivered you a certificate. Those sites gets to decide if they revoke their certificate or not. It is a self policying system that require some level of cooperation. Users make themselves vulnerable to the risk of having sites they like loose trust in them.

rcstank 1y ago
Sounds like a privacy nightmare. Also, what one site calls abuse, another wouldn't.
spondylosaurus 1y ago
Some stuff would definitely either slip through the cracks OR tarnish the reputation of legitimate users. What happens when someone's device gets compromised by a botnet that silently clicks ads in the background or turns that device into part of a DDoS army?
- MichaelZuo 1y ago
  Why would anyone even expect a perfectly zero false-positive and false-negative rate in the first place?
lq9AJ8yrfs 1y ago
> It is a self policying system that require some level of cooperation. How hard is it to obtain one of these certificates as a bot? What you are describing though is possibly comparable to Privacypass. Apple seems to be on board with Privacypass, perhaps they'll include a digital voucher of some kind with their devices and that presumably contributes to old devices getting worse as the voucher is spent down. Just imagine if the whole web can contribute to planned obsolescence and you can pay for a fast, hassle free internet experience again just by buying a new phone. And then you can dump the old ones on eBay for cheap as long as you don't plan on using them to access online services. Unless you are willing to settle for basic economy web experience.

thatcat 1y ago

If the general Internet was based on torrents, then the required upload ratio enforcement would have ensured bots contribute to the reliability rather than destabilize the infrastructure.

ivanjermakov 1y ago
The problem would move from verifying legitimacy of agent actions to legitimacy of agent reputation, which might even be easier to spoof.
areyourllySorry 1y ago
torrents don't have a required upload ratio though.
- thatcat 1y ago
  There are tracker ecosystems that enforce ratio, what.cd for example had developed a pretty advanced torrent economic rule set.
  - areyourllySorry 1y ago
    I know what you mean but it's not part of the protocol and I don't think you could enforce it on an internet scale

curtisblaine 1y ago

AFAIK, reCaptcha is not based on user behaviour anymore since v3, but uses proprietary network host information from Google. See this issue opened in 2018: https://github.com/google/recaptcha/issues/235 https://github.com/google/recaptcha/issues/235

Nevermark 1y ago

The problem becomes simpler if you turn it around. It is getting easier and easier to create questions/problems that humans can't answer at LLM speed. Of course, that solves a complementary problem, not the original. But in terms of instances, by any definition, the demographics are quickly moving in one direction.

protocolture 1y ago

I have noticed that a particular website will tell me I fail captcha half the time, until I resize my browser from a square to a rectangle. Took me ages to figure out what its issue was.

userbinator 1y ago

Or you could just ask it to count how many letters are in certain words.

xarope 1y ago

> Take for example the Stroop task . It's a classic psychology experiment where humans select the color a word is written it and not what the word says. Humans typically show slower responses when the meaning of a word conflicts with its color (e.g., the word "BLUE" written in green), reflecting an overriding of automatic behavior . Bots and AI agents, by contrast, are not subject to such interference and can respond with consistent speed regardless of stimuli. So I completely disagree with this; you can train youself to completely ignore the color and just read/act on the word very fast. In fact, this is a game that people play.

sly010 1y ago

First off, I always thought the type of things described (tracking mouse movements, keypress jitter, etc) are already done by ReCacpha to decide when to present the user with a captcha. I am surprised they are not already doing this. Second, I am surprised AI agents are this naive. I thought they would emulate human behavior better. In fact, just based on this article, very little effort has been put into this race on either side. So I wonder if is has to do with the fact that if companies like google reliably filtered out bot traffic, they would loose 90% of their AD revenue. This way they have plausible deniability.

fredfish 1y ago
They were very proud of this mouse movement stuff when the desktop was 70% of traffic.. It's not worth as much investment as its been given since there's no group limiting people to one HID method and removing accessibility from world.

dsekz 1y ago

Plenty of improvements to mouse movement algorithms have already been made and they’re still evolving. While the blog post and the product it introduces offer some interesting ideas, they don’t yet reach the robustness of modern anti-bot solutions and still trail current industry standards. I doubt it would take me - or any average reverse engineer - more than five seconds to bypass something like this. There are already numerous open source mouse movement libraries available; and even if they didn’t exist, writing one wouldn’t be difficult. Yes, mouse movement or keyboard data can be quite powerful in a modern anti-bot stack and an in depth analysis of it is genuinely valuable, but on its own it’s still insufficient. Relying on this data alone isn’t costly for the attacker and offers little real protection.

klaussilveira 1y ago
> they don’t yet reach the robustness of modern anti-bot solutions Like what?

randomtoast 1y ago

Isn’t it possible to emulate mouse movements and keypress jitter using a neural network trained on human data in order to simulate human behavior?

Saris 1y ago

I feel like analyzing keystrokes or mouse movements is just going to punish people who use password managers that autofill for them. It does seem like I get more captchas when on sites because of that.

PaulHoule 1y ago

Personally I think CAPTCHAs are harmful. They defend the enshittification economy, preventing the development of tools that protect human users of the web right in a time when it is more practical than ever to develop those tools. Even in 2009 I knew people who were using neural networks (in PHP no less!) to decode CAPTCHAs with superhuman peformance. I see the whole thing as performative as those things get in my way tens or hundreds of times a day when I browse the web as a human but in years of webcrawling they didn't give me any trouble until the last two weeks.

codedokode 1y ago

Recently I started getting a captcha when trying to use Google, probably because of VPN or Linux. I decided to switch to bing and duckduckgo. Dear Google, go solve your captchas yourself.

instagib 1y ago
Turning on Firefox anti fingerprinting triggered it for me.

instagib 1y ago

Anyone tried creating a new GitHub account lately? I considered using AI to help get past the proof of work requirement. The imagery test had me click dragging the finger pointing to see if I could line up with the animal. Probably 80% success. Audio detect two different people with heavy accents speaking vs singular people speaking was my lowest score probably 60-70%. Music switching instruments was around 70% also. Ultimately I passed after many attempts on the imagery detection.

Kate5477 1y ago

[dead]