6 ms·
Identifying Airtel middleboxes that censor HTTPS traffic
- ThreeFx 6y agoThe TTL field isn't cryptographically protected though, so it should be very possible that there's an actor after Airtel which does the censoring, no?
- justDankin 6y agoUsing the TTL, we figured out that the censor kicks in at the kth hop. This kth box belonged to Airtel. If a box was censoring after Airtel, we would have received a clean response (ICMP timeout) at hop k as well. Of course, the TTL itself can change during the run, but that wouldn't happen for so many cases :)
- iso1631 6y agoIf you used an allowed domain name, did you get TTL expired messages back when using TTL=5/6/7/8?
- justDankin 6y agoYes, that was the case
- davidhyde 6y agoDoes Airtel block TLS 1.3 with the encrypted SNI extension enabled? https://blog.cloudflare.com/encrypted-sni/ https://blog.cloudflare.com/encrypted-sni/
- justDankin 6y agoI haven't been able to test that yet, neither am aware of any research which answers that question. IMO the only way to do that would be to either (i) block the IP (high collateral blocking) or (ii) block TLS 1.3 itself (GFC does this). A major blocker in answering this is finding a potentially blocked website that also supports TLS 1.3
- davidhyde 6y agoAgreed and since TLS 1.3 is still work in progress the chances are slim that you will find a blocked website that meets your criteria. Great article, very accessible. Thanks!
- tialaramex 6y agoTLS 1.3 was published as RFC 8446 back in 2018. I know it sometimes seems like this is still 2016 somehow, but it is not.
- davidhyde 6y agoThat RFC is marked as "PROPOSED STANDARD" which is why I saw it as work in progress but you're right, that seems to be the end of the road for RFC's (for example RFC 6455 December 2011 (websockets) is also marked as proposed standard but this is what everyone has implemented)
- tialaramex 6y agoThe IETF deliberately has no power whatsoever. Whether an IETF standards track proposal in fact becomes a standard everybody implements is entirely up to the implementers. This is in contrast to many standards development organisations (and indeed whether the IETF is even an organisation is open to doubt) which are government functions and can produce de jure standards you're obliged to implement or in the worst case force may be exercised against you by those with a monopoly on its use. As a result IETF standards are only proposed and that is as you say "the end of the road".
- cesarb 6y ago> As a result IETF standards are only proposed and that is as you say "the end of the road". Not really, after PROPOSED STANDARD there's INTERNET STANDARD, the STD series. For instance, IPv4/ICMPv4 (RFC 791/792) is STD 5, UDP (RFC 768) is STD 6, TCP (RFC 793) is STD 7, DNS is STD 13, and so on (STD 1 has the full list). However, an IETF standard only reaches that level after it's been in use for a while; according to RFC 2026 (BCP 9), "A specification for which significant implementation and successful operational experience has been obtained may be elevated to the Internet Standard level. An Internet Standard (which may simply be referred to as a Standard) is characterized by a high degree of technical maturity and by a generally held belief that the specified protocol or service provides significant benefit to the Internet community."
- airtelblock 6y agoSetting esni to enabled in Firefox partially works. But Airtel really, really wants to run scripts and show ads on blocked pages. Duckduckgo - https://i.postimg.cc/SqkRhpRC/Mozilla-Firefox-29-09-20-w-PA.png https://i.postimg.cc/SqkRhpRC/Mozilla-Firefox-29-09-20-w-PA.... Pirate Bay - https://i.postimg.cc/qMmwMXVY/t-29-09-20-9-Dv.png https://i.postimg.cc/qMmwMXVY/t-29-09-20-9-Dv.png
- dannyw 6y agoDuckDuckGo is blocked in India??
- vinay_ys 6y agoNope
- uname_amiy 6y agoYes, ISPs are blocking DDG. Airtel(ISP) is blocking DDG but HTTPS version is accessible. https://imgur.com/a/y7wnOjD https://imgur.com/a/y7wnOjD
- smolder 6y agoWhat reason could they have for blocking DDG? Is it easier to find pirated content there than on Google or something? That's my best guess. I can't imagine they'd block on behalf of a competitor or something.
- justDankin 6y agoYou never know. Could be a mistake where they were trying to block a certain path due to some search result of copyright infringement, but ended up banning the domain itself. One can only guess. Reddit and Github have previously been temporarily banned in India due to similar "mistakes"
- pjc50 6y agoI suspect it's political censorship/surveillance of some sort. Some suggest it's collateral damage from the ban on Chinese apps. https://www.themobileindian.com/news/duckduckgo-blocked-by-multiple-service-providers-in-india-31958 https://www.themobileindian.com/news/duckduckgo-blocked-by-m... Whether that's a mistake or they've made an enemy of some sort is not clear. India is a democratic country but not an especially free one.
- airtelblock 6y agoDuckduckgo is blocked in India with Airtel. HTTP url refuses to connect. I have to manually type https://duckduckgo.com https://duckduckgo.com to connect. https://i.postimg.cc/SqkRhpRC/Mozilla-Firefox-29-09-20-w-PA.png https://i.postimg.cc/SqkRhpRC/Mozilla-Firefox-29-09-20-w-PA....
- justDankin 6y agoYeah there was a huge outcry about it on Twitter some time back. P.S. From the screenshot, it looks like you're trying to connect to [http]://duck.., hence shifting to https works. This also hints towards a mixture of plain old http censorship and https censorship, which Airtel (in fact all ISPs) do randomly
- airtelblock 6y agoThis is not about the block a month ago. This is a new block. When it was blocked a month ago, there was no notice. Now, there is a notice that it is a TRAI order. usually seen on sites that the govt themselves ask to block (piratebay, torrentz.eu, etc)
- deleted 6y ago[deleted]
- IceWreck 6y agoI guess its only limited to some regions. I'm using Airtel for my home connection as well mobile and get redirected to https everytime I visit the http version of ddg on both connections.
- crazy_hombre 6y agoThat's weird. I just get a 301 redirect to the HTTPS version when I visit http://duckduckgo.com http://duckduckgo.com.
- Santosh83 6y ago
- xrisk 6y agoYou should consider using HTTPS :)
- gammarays_ 6y agoPlease read the article :)
- xrisk 6y agoI did. It was just a suggestion for OP to make his blog accessible via HTTPS.
- kube-system 6y agoI think they're talking about the blog itself not being configured to use HTTPS. I had to add an exception in my HTTPS everywhere extension to read the article.
- justDankin 6y agoHaha yeah, I've been procrastinating for the last 5 years now
- galliher 6y agoThis is pretty clever! The reset within airtel_103.224.212.222_fullhd720.com.pcap arrives with IP time-to-live of fifty-seven while the segment carrying synchronize | acknowledge flags arrived with a time-to-live of forty-four. So without any active probing, and some educated guesses around default IP time-to-live values @ 1<<[6..8] you could could conclude that the reset originated fourteen hops closer to the capture than other packets in same five-tuple defined "flow".
- justDankin 6y agoYes! That's a great observation However, the reason I went for probing the entire path is because the TTL itself can be spoofed
- galliher 6y agoAgreed, it's flimsy. Certainly a bit more effort for them spoof it correctly though. Would need to watch traffic on the path back per flow to isolate the number of prior decrements to the TTL leading up to MitM, and then store that value until such time that it sees an SNI it cares about / it's time to generate a reset.
- segfaultbuserr 6y agoTor Project closely cooperates with The Open Observatory of Network Interference. OONI provides a standardized testsuit that automatically probes a wide range of websites for evaluating the censorship status in different parts of the world, and volunteers can contribute data by running an instance. But the objective of the project is only to identify governmental and ISP-wide censorships. And its capabilities for checking protocol-level censorship techniques are rather limited [0]. Perhaps we need a similar tool or fork for probing middleboxes and censorship in private networks. [0] https://ooni.org/nettest/ https://ooni.org/nettest/
- myself248 6y agoOooo. How does that compare with something like RIPE ATLAS? Is there a list of such monitoring projects somewhere?
- deleted 6y ago[deleted]
- tyingq 6y agoSome older research suggests Airtel uses Netsweeper: https://citizenlab.ca/2018/04/planet-netsweeper-section-1-methodology-technical-findings/ https://citizenlab.ca/2018/04/planet-netsweeper-section-1-me... Netsweeper is a Canadian company in the business of content filtering: https://www.netsweeper.com/ https://www.netsweeper.com/
- aeyes 6y agoAnother possible supplier is this company: https://www.ipoque.com/ https://www.ipoque.com/ They openly state that they supply ISPs with DPI hardware, I talked to them in person 15 years ago and they had no problem to admit that they also supply government institutions.
- walrus01 6y agoMy job overlaps with microwave and millimeter wave RF engineering somewhat. R&S also has no qualms about selling high-end spectrum analysis equipment to authoritarian regimes. Probably done through middlemen. For instance, you can find the Iranian government using their equipment in Tehran to hunt down things they don't like. To be fair, there's probably less than ten manufacturers of their category of spectrum analysis gear (for commercial/non-military use) in the world, and their stuff is top quality.
- tyingq 6y agoInteresting, so triangulating people forwarding "open" internet over consumer-grade microwave (like Ubiquiti or similar)? I assume they can't do much about Toosheh since it's "read only", multiplexed with legitimate TV channels on the same transponder, and uplinked from the UAE.
- walrus01 6y agoThings that transmit generally, in all sorts of bands, lots of countries where the government holds an armed monopoly on connections to the outside world. Try setting up an independent two-way satellite based C or Ku band earth station in Ethiopia, offer service to your neighbours and armed men will come to dismantle it. Commercial spectrum analysis tools are an essential and important things in the hands of network engineers, but also a tool to crack down on anything that transmits that an authoritarian regime doesn't like.
- amitheonlyone 6y agoThis is completely off-topic but the strike-through on links had me confused for a good few minutes. I was not clicking on those because I thought the link were not valid today and hence it has strike-through.
- Semaphor 6y agoWhat strike-through?
- folmar 6y agoAll links have a thin red line through them: https://pasteimg.com/images/2020/09/29/strika.png https://pasteimg.com/images/2020/09/29/strika.png
- Semaphor 6y agoWeird, I have that neither in Chrome nor FF: https://pasteimg.com/images/2020/09/30/image.png https://pasteimg.com/images/2020/09/30/image.png
- walrus01 6y agoI wonder what they intend to do (other than just blocking entire IP ranges of non-Indian hosting providers, which I would not be surprised by) when things increasingly move to TLS1.3 with ESNI.
- justDankin 6y agoWell, they could block TLS1.3 entirely (which would force hosts to drop down to 1.2 for connections) GFC does this, I really hope it doesn't happen here
- tialaramex 6y agoThe Great Firewall does not block TLS 1.3. You may have seen headlines which claim it does, but they're based on a report that actually says it doesn't. Remember journalists probably know even less than you do about most things they write about! In this case the report says the Great Firewall was determined to block the following specific combination: * A ClientHello for TLS 1.3 that * Includes the 0xffce extension value (used for experimenting with an earlier SNI draft) If you add a 0xffce extension full of random noise, the Great Firewall blocks it. If you use the same random noise but pick a different extension value (do not do this in production code - those aren't for your meddling!) the Great Firewall doesn't interfere at all. We have yet to discover what happens if a big bang release of Encrypted Client Hello (the current iteration of the encrypted SNI work) just deluges the Great Firewall with ECH connections. But we do know TLS 1.3 has been used successfully for years from China. You also mention this idea that it would "force hosts to drop down to 1.2 for connections". It is hard to tell what you intended here, it would of course be possible to force the humans using a computer to downgrade, or to disable encryption, or to cease using a computer altogether, perhaps you could put a gun to their heads for example. But TLS 1.3 has an anti-downgrade design. A [edited to add] modern TLS 1.3 capable web browser which connects to a TLS 1.3 capable web site but finds that the connection has been negotiated as TLS 1.2 instead will reject the connection as clearly under attack, you cannot reach that site until the problem is remedied. I think you would notice if all TLS 1.3 capable sites (about a third of popular sites) suddenly did not work from China, even the Chinese government might struggle to silence such confusion and dismay from their people.
- walrus01 6y agoOn a meta level, this is one of the reasons why I tell every junior/entry level person I encounter in the ISP business the following: Ethics is important in network engineering. You can and should refuse to do things that cause measurable harm to the Internet. You should understand why certain things are bad, and should make a conscious choice not to aid and abet them. It is regretful that organizations like NANOG, RIPE, ARIN, APNIC and others do not take a stronger stance (on a global geopolitical level) against censorship of the Internet, and attempts to create walled gardens and national firewalls. The people who possess the equivalent of 'enable'/'configure' on the core routers of gigantic ASNs have real power to refuse to further harm the Internet. We have seen a number of discussions on HN about ethics in software engineering. A quick search for posts with 'ethics' in the subject line turns up a number of things. In my opinion, ethics in the ISP, telecom and network engineering industry is equally important.
- jacquesm 6y agoThis goes for any position with responsibility for and access to end user data. For instance: administering a mail server or hosted mail account for the company, dealing with the fall-out of security incidents and so on.
- joosters 6y agoAll well and good, but are these kind of 'middleboxes' unequivocally unethical? For example, some ISPs might want to block highly illegal content - let's use the typical examples, e.g. child porn sites, malware domains, and so on. It's not inherently unethical (or, at least, there are plenty of reasonable people who would say it is ethical) to install a middlebox that will make it more difficult for users to access these sites. So now your company has got a content blocker installed. What exactly are your network engineers meant to do now? Demand personal refusal over any additions to the sites that these boxes will block? That seems highly unlikely to happen, and how could that even work in practice? Are all the engineers meant to vote on blocks, and only those unopposed sites get added to the list? Can ethical network engineers usefully oppose content blocking?
- walrus01 6y ago
- Kaknut 6y agoReally loved your article buy why no SSL/HTTPS? It's free afterall.
- cnst 6y agoHTTPS is not free. It has a very significant management/maintenance and compatibility overhead, which is unavoidable by the very nature of HTTPS.
- sroussey 6y agoWhich can be automated.
- cnst 6y agoYes, you gotta automate a whole bunch of things if you need HTTPS, you have to update the protocols every few years, certificates as often as every few months, OpenSSL versions on a moment's notice. Or, you could decide to just go HTTP-only for your blog, and never bother doing any of the above, never worry about any automation failing for any reason, never worry about any expired or revoked certificates, never worry about the extra compatibility issues that TLS brings. There's no benefit for HTTPS for a personal blog. It's only there to restrict the access, increase attack surface, and cause compatibility issues.
- ac29 6y agoOn the scale of a personal blog, its approximately 0 minutes per year to maintain HTTPS certificates using Lets Encrypt with something like certbot (or use Caddy, which handles it on it own).
- cnst 6y agoAbsolutely false. On the scale of a personal blog, the cost of HTTPS is enormous, but the benefit is approximately 0.
- commandlinefan 6y agoWe have SNI because it's needed to support virtual hosting, which we really only need because IPv4 addresses are scarce. If we could ever get to IPv6, SNI could be retired completely.
- mindslight 6y agoSure, but the security vulnerability of unencrypted SNI is still present with IP based vhosts - the destination IP precisely identifies the site.
- occamrazor 6y agoTrue, but then IP based blocking would be highly effective.
- hedora 6y agoI wish the world would adopt tcpcrypt. SSL provides two services, encryption, and authentication. tcpcrypt just does the encryption part. Once the connection is established, userspace on both sides can invoke an ioctl that provides a session nonce. If the nonce matches on both sides, the connection is not man-in-the-middled. It’s easy to confirm the nonce matches: Both sides sign it and send it to the other party. This has two big advantages: (1) operating systems can opportunistically encrypt traffic for unmodified legacy applications and network protocols, allowing the endpoints to detect mass surveillance without requiring any certificates. (2) Like newer versions of SSL, it encrypts the information currently sent in cleartext in the SNI, preventing the type of censorship in the article.
- PowerBar 6y agoThat would still rely on A) some kind of secret that only the server knows that the client can verify in order to ensure it's not trading nonces with the MITM. B) A way for the client to ensure that the nonce isn't being passed through a second tcpcrypt session between the MITM and the server with the connection being in cleartext between the 2 tcpcrypt streams. Currently the best supported method of implementing both A and B is certificates, which means you may as well use TLS.
- Dylan16807 6y ago> which means you may as well use TLS Even if you don't authenticate at all, it makes it much more expensive to intercept all these connections. And TLS lacks a way to automatically apply it to all connections. Also I don't understand what scenario you're outlining with B.
- Spivak 6y agoClient <-> Evil Middlebox <-> Real Web Server Client establishes a tcpcrypt session with what it thinks is Real Web Server but is actually Evil Middlebox replaying the request to the server and the response back to the client.
- totaldude87 6y agoTheoretically, lets say am in India, using a tor browser, or an opera browser wit inbuilt VPN, would I see different results? my point is to see whether a VPN of sort can circumvent, in that case, what if all those browsers decides to create inbuilt vpn (for connection origins).. does that makes sense?
- dj_mc_merlin 6y agoA VPN would prevent SNI snooping since traffic over it is encrypted. Of course, the ISP could block the TLS handshake between you and the VPN server if that's how your VPN functions. Tor would similarly work.