18 ms·
Chrome's hidden X-Browser-Validation header reverse engineered
- dsekz 1y agoDug into chrome.dll and figured out how the x-browser-validation header is generated. Full write up and PoC code here: https://github.com/dsekz/chrome-x-browser-validation-header https://github.com/dsekz/chrome-x-browser-validation-header Why do you think Chrome bothers with this extra headers. Anti-spoofing, bot detection, integrity or something else?
- twapi 1y agoSeems like they are using these headers only for google.com requests.
- xnx 1y agoYes I think it is part of their multi level testing of for new version rollouts. In addition to all the internal unit and performance tests, they want an extra level of verification that weird things aren't happening in the wild
- AznHisoka 1y agoThey probably are using it to block bots scraping Google results is my theory
- userbinator 1y agoMaking it easier to reject "unapproved" or "unsupported" browsers and take away user freedom. Trying to make it harder for other browsers to compete.
- ajross 1y agoThat can be done already based on User-Agent, though. Other browsers don't spoof their agent strings to look like Chrome, and never have (or, they do, but only in the sense that everyone still claims to be Mozilla). And browsers have always (for obvious reasons) been very happy to identify themselves correctly to backend sites. The purpose here is surely to detect sophisticated spoofing by non-user-browser software, like crawlers and robots. Robots are in fact required by the net's Geneva Convention equivalent to identify themselves and respect limitations, but obviously many don't. I have a hard time understanding robot detection as an issue of "user freedom" or "browser competition".
- soulofmischief 1y agoIt's easy to change the User Agent and we cannot handwave this fact away for the sake of argument.
- Sayrus 1y ago> I have a hard time understanding robot detection as an issue of "user freedom" or "browser competition". In the name of robot detection, you can lock down device, require device attestation, prevent users from running non-standard devices/OS/software, prevent them from accessing websites (CloudFlare dislikes non-chrome browser and hates non-standard browsers, ReCaptcha blocks you out if you're not on Chrome-like/Safari/Firefox). Web Environment Integrity[1] is also a good example of where robot detection ends up affecting the end user. [1] https://en.wikipedia.org/wiki/Web_Environment_Integrity https://en.wikipedia.org/wiki/Web_Environment_Integrity
- jml7c5 1y ago>I have a hard time understanding robot detection as an issue of "user freedom" or "browser competition". The big one is that running a browser other than Chrome (or Safari) could come to mean endless captchas, degrading the experience. "Chrome doesn't have as many captchas" is a pretty good hook.
- Forgeties79 1y agoNot to mention how often you can get stuck in an infinite loop where it just will not accept your captcha results and keeps making you do it over and over. Especially if you’re using a VPN. It’s maddening sometimes. Can’t even do a basic search
- exiguus 1y agoI have two questions: 1. Do I understand it correctly and the validation header is individual for each installation? 2. Is this header only in Google Chrome or also in Chromium?
- gruez 1y ago>1. Do I understand it correctly and the validation header is individual for each installation? I'm not sure how you got that impression. It's generated from fixed constants. https://github.com/dsekz/chrome-x-browser-validation-header?tab=readme-ov-file#how-the-header-is-made https://github.com/dsekz/chrome-x-browser-validation-header?...
- exiguus 1y agoIt's still not clear to me because it's called the default API key. And for me, default means that this is normally overwritten. And if overwritten, during build or during install? That's what I'm asking myself.
- dlenski 1y agoI had the same question (2). https://news.ycombinator.com/item?id=44560664 https://news.ycombinator.com/item?id=44560664 If it's only in the closed-source Chrome, then it seems it's intended to help Google's servers distinguish between Google's own products and others. But I've never seen a Google site which worked less-well in Chromium than in Chrome, so I'm somewhat skeptical of this. Perhaps there are exceptions
- deleted 1y ago[deleted]
- wernerb 1y agoIs it not likely that it protects against AI bot Llama?
- wut42 1y agoI don't see how you can "protect" against a large language model that cannot do browsing.
- Avamander 1y ago> Why do you think Chrome bothers with this extra headers. Anti-spoofing, bot detection, integrity or something else? Bot detection. It's a menace to literally everyone. Not to piss anyone off, but if you haven't dealt with it, you don't have anything of value to scrape or get access to.
- lxgr 1y agoDo you mean bot and non-Chrome-using human detection?
- ohdeargodno 1y agoBullshit. You don't have anything of value either. Scrapers will ram through _anything_, and figure out if it's useful later.
- motorest 1y ago> Bot detection. It's a menace to literally everyone. Not to piss anyone off, but if you haven't dealt with it, you don't have anything of value to scrape or get access to. What leads you to believe that bit developers are unable to set a request header? They managed fine to set Chrome's user agent. Why do you think something like X-Browser-Validation is off limits?
- Sophira 1y agoBecause you would need to reproduce an explicit Google copyright statement which states that you don't have the right to copy it ("All rights reserved.") in order to do it fully. That presumably gives Google the legal ammunition it needs to sue you if you do it.
- tomsonj 1y agoIt seems like the requirement to reproduce this copyright header alone, nevermind the validation hash, would be enough to scare off scrapers?
- 1y ago
- cebert 1y agoI have to imagine Google added these headers to make it easier for them to identify agentic requests vs human requests. What angers me is that this is yet another signal that can be used to uniquely fingerprint users.
- gruez 1y agoIt doesn't really meaningfully increase the fingerprinting surface. As the OP mentioned the hash is generated from constants that are the same for all chrome builds. The only thing it really does is help distinguish chrome from other chromium forks (eg. edge or brave), but there's already enough proprietary bits inside chrome that you can easily tell it apart.
- thayne 1y ago> The only thing it really does is help distinguish chrome from other chromium forks (eg. edge or brave) You could already do that with the user agent string. What this does is distinguishes between chrome and something else pretending to be chrome. Like say a firefox user who is spoofing a chrome user agent on a site that blocks, or reduces functionality for the firefox user agent.
- bobbiechen 1y agoPlenty of bots pretend to be Chrome via user agent, but if you look closely are actually running Headless Chromium. This is a very useful signal for fraud and abuse prevention.
- thayne 1y agoLet's ignore for the moment that this has been reverse engineered. If they only look at this header, then legitimate users using non-chrome browsers will get treated as bots. If the these headers are only used for chrome user agents, then it would be easy to bypass by using headless chromium with a user agent that spoofs firefox or safari.
- userbinator 1y agoThis should be somewhat alarming to anyone who already knows about WEI. I wonder if "x-browser-copyright" is an attempt at trying to use the legal system to stifle competition and further their monopoly. If so, have they not heard of Sega v. Accolade ? I'm a bit amused that they're using SHA-1. Why not MD5, CRC32, or (as the dumb security scanners would recommend) even SHA256?
- Retr0id 1y agoSHA-1 is a head-scratcher for sure. I can only assume it's the flawed logic that it's "reasonably secure, but shorter than sha256". Flawed because SHA1 is broken, and SHA256 is faster on most hardware, and you can just truncate your SHA256 output if you really want it to be shorter.
- pinoy420 1y ago[dead]
- adrian_b 1y agoSHA-1 is broken for being used in digital signature algorithms or for any other application that requires collision resistance. There are a lot of applications for which collision resistance is irrelevant and for which the use of SHA-1 is fine, for instance in some random number generators. On the CPUs where I have tested this (with hardware instructions for both hashes, e.g. some Ryzen and some Aarch64), SHA-1 is faster than SHA-256, though the difference is not great. In this case, collision resistance appears irrelevant. There is no point in finding other strings that will produce the same validation hash. The correct input strings can be obtained by reverse engineering anyway, which has been done by the author. Here the hash was used just for slight obfuscation.
- Retr0id 1y agoThe perf difference between SHA1 and SHA256 was marginal on the systems I tested (3950x, M1 Pro), which makes SHA256 a no-brainer to me if you're just picking between those two (collision resistance is nice to have even if you "don't need it"). You're right that collision resistance doesn't really matter here, but there's a fair chance SHA1 will end up deprecated or removed from whatever cryptography library you're using for it, at some point in the future.
- _imnothere 1y agoAnd why should anyone with a sane mind (except for Googlers) allow this kind of validation bs to exist?
- rs186 1y agoAt this point I am fully convinced that Google is abusing Chrome's dominant position to push their own agenda and shape the whole Internet the way they want. Privacy sandbox, manifest v3, you name it. Sadly nobody can do anything about it, so far. We'll yet need to see the outcome of the antitrust trial.
- orphea 1y ago> to push their own agenda and shape the whole Internet the way they want It is Chrome's raison d'être from the very beginning. You don't think Google made its own browser because they felt generous, right?
- thayne 1y agoWhy would they think this was a good idea after losing the chrome anti-trust trial? I don't know the intended purpose is for this, but I can see several ways this could be used anti-competitive way, although now it has been reverse engineered, an extension could spoof it. On the other hand, I wonder if they intend to claim the header is a form of DRM and such spoofing is a DMCA violation...
- krackers 1y ago>an extension could spoof it not if they make it dynamic somehow (e.g. include current day in hash). Then with MV3 changes that prevent dynamic header manipulation there is no way for an extension to spoof it.
- Retr0id 1y agox-browser-copyright seems like an attempt at something similar to the Gameboy's nintendo-logo DRM (wherein cartridges are required to have the nintendo logo bitmap before they can boot, so any unlicensed carts would be trademark infringement)
- userbinator 1y agohttp://en.wikipedia.org/wiki/Sega_Enterprises_Ltd._v._Accolade%2C_Inc%2E http://en.wikipedia.org/wiki/Sega_Enterprises_Ltd._v._Accola... is the legal precedent that says trying to do that won't work, but then again maybe Google thinks it's invincible and can do whatever it wants after it ironically defeated Oracle in a case about interoperability and copyright.
- Everdred2dx 1y agoIf you were using a user agent spoofing extension couldn't this be used to guess your "real" UA?
- jedimastert 1y agoIt looks like it's an SHA hash, so working backwards would probably be prohibitively irritating.
- dataflow 1y agoThat's not how it works. The combination of valid inputs is a small set. You just try each one until you get the hash.
- jedimastert 1y agoIt's not all that small, although probably small enough to make a rainbow table or something. You would have to maintain the code to generate character-perfect strings (or maybe just keep a very large library of the current most popular ones) and also make sure you have the up to date API key salt values (which they probably going to start rotating regularly), which–as I said before–wouldn't be impossible, just prohibitively irritating to maintain for comparatively little benefit. And besides, it won't be too long before people just start spoofing the hash too, probably shorter than getting the generator up and running
- giingyui 1y ago[dead]
- aussieguy1234 1y agoSo this is basically hidden client attestation?
- Aaargh20318 1y agoNot really. It's just an API key + the user agent. There is no mechanism to detect the browser hasn't been tampered with. If you wanted to do that you'd at least include a hash over the browser binary, or better yet the in-memory application binary.
- delusional 1y agoThat would provide not extra capability. Anybody smart enough to modify the chrome executable could just patch the hash generation to also return a static (but correct) hash.
- jakub_g 1y agoFYI: Google enterprise workspace admins can enable policies which e.g. prevent login ability to google.com properties to only Chrome browsers. I wonder if this is header is not connected in some way to that feature.
- cj 1y agoSeems unnecessary. The same policies also offer the ability to force-install an official Google "Endpoint Verification" chrome extension which validates browser/OS integrity using Enterprise Chrome Extension APIs ("chrome.enterprise") [0] only available in force-installed enterprise extensions. FWIW, in my years of managing enterprise chrome deployments, I haven't come across the feature to force people to use Chrome (there are a lot of settings, maybe I've missed this one). But, there definitely is the ability to prevent users from mixing their work and non-work gmail accounts in the same chrome profile. [0] https://developer.chrome.com/docs/extensions/reference/api/enterprise/hardwarePlatform https://developer.chrome.com/docs/extensions/reference/api/e... Edit: Okay, maybe one hole in my logic is the first-sign in experience. When signing into google for the first time in a new chrome browser, the force-installed extension wouldn't be there yet. Although Google could hypothetically still allow the login initially, but then abort/cancel the sign in process as part of the login flow if the extension doesn't sync and install (indicating non-chrome use).
- jakub_g 1y agoIn my current job we do have force-Chrome setting enabled. I can't log in to Gmail through any other browser. Neither SSO login to GitHub via Google.
- cj 1y agoThis might be their “context aware” security feature. Which can prevent access to certain things based on device, browser, etc. I don’t see why any of that can’t rely on a chrome extension implementation using the privileged APIs to verify OS, Browser, etc. Struggling to understand why they need special headers for any of this functionality.
- binary132 1y agoI think it’s difficult to argue that Google doesn’t have the right and capability to build their own private internet, I just also think they’d like to make the entire internet their own private internet, and do away with the public internet, and I’d really prefer they not do that.
- TacticalCoder 1y ago[dead]
- delusional 1y agoIs an "api key" like this covered by copyright? Would that technically mean that spoofing this random sequence of numbers would require me to agree to whatever source license they offer it under, since I wouldn't know the random sequence unless I read it in their source? That's an odd possibility.
- userbinator 1y agoAnti-reverse-engineering clauses in EULAs are limited and exceptions are always present for interoperability. The same goes for copyright. It's hard to argue that this key is secret if it's widely and publicly distributed. Ironically, Google just fought with Oracle a case around similar concepts.
- RandyOrion 1y agoTwo questions: Which version of chrome is the first to implement these headers? What are the potential effects of these headers on chromium forks, e.g. ungoogled chromium?
- kuschkufan 1y agoWell, what did you find out?
- Larrikin 1y agoHow do I set this in Firefox?
- egorfine 1y agoIs there a way to prevent Chrome from sending those headers?
- dlenski 1y agoDoes the open-source Chromium generate this header as well? (Perhaps with a slightly different UA as input.) Or is it exclusive to the closed-source Chrome codebase?