8 ms·
Possible Vendetta Behind the East Coast Web Slowdown
- thesteverichey 10y agoAny evidence this is using the IoT botnet that was reported on earlier this year?
- micaksica 10y agoMirai? With the source of that being public, there are probably quite a few Mirai botnets now.
- meira 10y agoNot working, is bloomberg down too?
- jerf 10y agoFor a long time, I've wondered what would finally be the Securitypocalypse, the thing that finally caused our industry as a whole to take security seriously. These IoT DDoS attacks are as good a candidate as any I've seen in a long time. They are fundamentally very difficult to fix in light of the non-updateability of many of these devices, and this is only the beginning, because the IoT has hardly begun to develop. And in the short-term, I'm not sure I see any hope, because the forces that make people throw out cheap devices with broken firmwares with no update capability aren't going away. If we could somehow mandate that these devices were supported with firmware updates for the indefinite future, that would simply destroy the entire market. And you can't do that, because even the devices created by an entity that no longer exists and didn't sell its IP to anybody else will eventually be enough to do these DDoSes, if they aren't already.
- seanp2k2 10y agoYep, and manufacturers have not much incentive to update firmware for a device which is not their latest greatest or update firmware while not adding more features to help them sell more. Security isn't a feature that the vast majority of consumers would pay extra for or know how to verify anyway. There was plenty of demand for that one "unhackable" android phone, but I'd be blown away if it wasn't 100% snake oil. My prediction is that it'll get worse before it gets better and that these type of botnets will be around for at least 5 years. Look at what happened to unsecured-by-default routers, android phones, Windows PCs, cars...the way consumers will get more secure stuff is by manufacturers being publicly embarrassed / sued over problems until caring about security makes business sense, then they'll have it in their hands when their old insecure gadgets die. My cynical side side thinks this will be a problem until all the old endpoints supporting these insecure things are shut down eventually in 5-10 years.
- Angostura 10y agoThis isn't just small manufacturers either. I bought a new Samsung tablet for my kid two weeks ago. It is running a three year old version of Android with no updates available. Pretty shocking.
- JoelBennett 10y agoThere'd be something ironic about a manufacturer's website being made unavailable because of a DDoS caused by their own poorly secured devices.
- NKCSS 10y agoIt's easy to fix; back in the day when a machine was infected; an ISP would just block outgoing traffic, contact line owner and re-enable when the issue is resolved.
- gaius 10y agoIf the "machine" in question is my ADSL router as supplied by my ISP, I will be deeply unimpressed if they block me due to their own negligence in updating it!
- Jtsummers 10y agoSimilarly, a single bad device on my network would block the whole of my network from the internet. It's another sort of denial of service attack. We need IPv6 and have devices either access the internet with their own IP address or not access it at all. This solution, then, would only impact bad actor devices, not your other (non-compromised) devices. Still, not easy.
- cpncrunch 10y agoI think it's fair to block the entire network. It is then up to the network administrator to fix the problematic device.
- Jtsummers 10y agoWhile technically accurate to describe them as such, the vast majority of consumers (and internet service subscribers) lack the actual technical expertise to be network administrators. Where these devices are being attacked inside, ostensibly, professional organizations (companies, schools, government buildings), I agree. But there you have, again ostensibly, an actual network administrator capable of dealing with the issue (and paid to do so).
- matthewmacleod 10y agoI think that's okay. We don't expect all homeowners to be, say, experts in electrical wiring, or gas supply, plumbing, drainage, or waste management. But all of these things—if they are poorly modified, managed, or maintained—can cause impacts on third parties. In the case of networked devices, the possible impact on third parties is even greater. We also enforce strong regulation on these systems – defining what may and may not be legally connected to public utility networks, for example. We would probably expect a homeowner to hire a tradesperson to maintain these services, and in some cases it's legally mandated that only a qualified person may install or modify these systems. Is it then unreasonable to kick consumers off of the Internet when they install poorly-maintained devices, and require them to resolve the problem – perhaps by hiring the networking equivalent of a qualified plumber?
- gtrubetskoy 10y agoThese attacks are mostly possible because of the complacency of operators at many sites and companies. This is not a new problem and many of RFC's talk about methods for preventing and mitigating them, but most people don't care and prefer to just outsource everything to a single provider, which becomes the weakest link. The Internet wasn't envisioned with a single email provider, single DNS provider, single app container provider. (Ok, for most of these you have two, sometimes three choices, but still, that is too few). The centralization makes everything very vulnerable - imagine what would happen when Gmail is knocked out for a day.
- supergeek133 10y agoThe problem with these devices in particular is the weak point is the user. As is the case in most attacks. Your average user says "Sure I can setup cameras" then sees "remote access" in the menu, sets it up, maybe it has some UPNP to the router and BOOM. Magic remote login without any type of mitigation.
- cmdrfred 10y agoExactly, I have tons of IOT devices. I put them on a separate subnet that does not have a gateway to the internet then I VPN into that network to access them. Perhaps a product that makes that a simple process will solve the problem?
- RickS 10y agoFWIW, I would definitely be interested in paying for a service like this. I'm technical enough to care about this, but not technical enough to solve it myself. Similar to where I was before dropbox.
- NetStrikeForce 10y agoMy comment here might be relevant to your interests: https://news.ycombinator.com/item?id=12765051 https://news.ycombinator.com/item?id=12765051 It could suit your needs or we can help with custom deployments. In any case I'd like to learn more about your needs and your expectations. Can I drop you an email?
- HipHopHacker 10y ago> For a long time, I've wondered what would finally be the Securitypocalypse, the thing that finally caused our industry as a whole to take security seriously. Nothing. If the economic system revolves around capital's valorization of itself, security is a distraction from that. I have to spend five seconds typing my password in every time I sit at my desk? I can't just easily e-mail this executable file to my co-worker and have them run it? My desktop is locked down by the desktop admins to prevent me being able to do this, and many other things? Every implementation of security costs money for the personnel to do it and possibly the product cost. Plus any lost productivity it might cause (15 seconds to type in a password each time one sits at their desk, compounded). Donn Parker wrote one of the first books on computer security in 1976, Crime by Computer. The opening words are as apt for corporate security now as it was then. The #1 fear for the corporate manager are the employees of that company. They are the ones with the greatest control over the means of production, so to speak, even more than the managers themselves who are de jure in charge, but are de facto one step away from actual control. Look at how much access someone like Snowden had at Booz Allen. Obviously, if all products have wide open holes, script kiddies will be able to get control. Some minimal security will always be done to stop this sort of thing. On the other hand, one (or better yet, several) dedicated people who want to get past some security arrangement can almost always get in. Even if the firewall is supposedly impenetrable, the wifi or the building security or the social engineering credulity of employees or something will be there. There will be some weak link in the chain. Especially for a company that needs to make a profit. The real security is that semi-intelligent, persistent agents that seek to access and control systems without authorization are lacking. Things depend on the conditions that cause this to rise or diminish. Because once it rises, there is little that can be done. I forget who said that the czar's Russian Okhrana was one of the largest, most extensive security forces that existed. That meant little when Russia began collapsing in 1916 though - all it meant was that they were even more aware that virtually everyone in the country was becoming the czar's enemy. Securitypocalypse events due result in business and government putting more focus on security for a while, but time moves on, and attention drifts back to the main focus. These things go in waves, and total security is never something of the highest priority.
- trhway 10y ago>They are fundamentally very difficult to fix in light of the non-updateability of many of these devices as you proved, fixing the situation by fixing the devices wouldn't be a feasible approach. The traffic from those devices is carried by ISPs and this is there this traffic should be stopped. To me the situation reminds about email spam. We didn't get rid of spammers, instead the email traffic is analyzed and dealt with accordingly. I'm sure that ISPs easily see the patterns of such massive DDoS attacks and could just drop (or throttle down into oblivion, like 100s times down) the participating traffic.
- kakarot 10y agoUnfortunately, forced firmware updating is an area our governments should not be mandating. That puts unnecessary strain on small companies and creates a larger gap that companies must cross to become commercially viable
- snovv_crash 10y agoJust like with bridges. Getting certification by a professional engineer is just too much barrier to entry for small construction companies. Edit: forgot the /s
- Ar-Curunir 10y agoI'm not sure if you're being sarcastic, but isn't that a good thing?
- deleted 10y ago[deleted]
- the_watcher 10y agoYea, it's a hard problem. While there's clearly a lot of vulnerabilities out there that emerge because it's cheaper to ignore security until you're large enough for a breach to be a big issue, forcing mandatory updates is a great way to discourage anyone from attempting to try something new. There might be a tipping point at which the costs of a breach outweighs the benefit, and maybe we've hit it already, but government mandates should be something we discuss cautiously and should prefer to avoid.
- DashRattlesnake 10y agoI don't think that's necessarily a bad thing. If a company doesn't have the resources to create secure products, then maybe it shouldn't be in that business in the first place.
- deleted 10y ago[deleted]
- trendia 10y agoIf you are unable to connect because of DNS problems, switch your DNS server to 8.8.8.8 (Google). Edit: sorry there, this worked for me but apparently it's not guaranteed.
- lell 10y agoDidn't help me, heroku is still unreachable with DNS servers changed to 8.8.8.8 and 8.8.4.4.
- cafard 10y agoThat would resolve neither python.org nor cpan.org this morning.
- rashkov 10y agoHaving good luck with opendns 208.67.222.222 They have other IPs as well, but that's what I'm using
- deleted 10y ago[deleted]
- esharte 10y agoI had to move my DNS servers off google today to get twitter to load.
- LeoPanthera 10y agoThat didn't help me this morning. Switching to OpenDNS worked for Github but not for Twitter.
- losvedir 10y agoI switched temporarily from those to Open DNS's 208.67.222.222 and things are working for now. But, just to be clear, it's not Google's fault: 8.8.8.8 are not the authoritative name servers for the sites that are down. Rather, Dyn, the provider of the NS is down, and I presume Google (8.8.8.8) is correctly not returning any IP address because the underlying authoritative name server is not. Presumably Open DNS is working because it's not abiding by the TTL it's supposed to? It's caching the underlying authoritative name server longer than it was told?
- zzleeper 10y agoTo be honest, I wouldn't be surprised at all if the BackConnect kid decided to launch the DDoS: https://www.crunchbase.com/person/marshal-webb https://www.crunchbase.com/person/marshal-webb Edit: Maybe this helps with the downvotes: http://www.cbsnews.com/news/lulzsec-takes-revenge-on-alleged-snitches/ http://www.cbsnews.com/news/lulzsec-takes-revenge-on-alleged... https://www.reddit.com/r/cincinnati/comments/ibwbz/fbi_hacking_probe_leads_to_hamilton_teen/ https://www.reddit.com/r/cincinnati/comments/ibwbz/fbi_hacki...
- csdrane 10y agoThis is totally inappropriate.
- thechowchowman 10y agoWhy inappropriate?
- zzleeper 10y agoVery young person so possibly impulsive; started college at age 12 so might not have developed enough emotional intelligence to avoid doing these things. I mean, Bloomberg is pointing fingers, I'm just trying to understand why an anti-DDoS firm would be DDoSing other firms.. EDIT: Also, "Marshal Webb, 18, whose Hamilton, Ohio home was raided this week by FBI agents as part of the LulzSec investigation". Maybe he did it or maybe not, but if he did, it wouldn't be the first time http://www.thesmokinggun.com/documents/internet/hackers-who-tried-sink-lulz-boat-071289 http://www.thesmokinggun.com/documents/internet/hackers-who-...
- elmigranto 10y agoNo luck with Google DNS for me, but Yandex seems to work: 77.88.8.8 77.88.8.1 https://dns.yandex.ru https://dns.yandex.ru
- drinchev 10y agoProbably, but I would definitely avoid giving all my DNS resolutions to a *.ru domain. The reputation of the government - shutting down access to websites that hurt them is kind-a no-go for me.
- Jerry2 10y agoUS government is not much better... or have you forgotten all those hundreds of FBI/ICE domain seizures. How many have Russians taken down? If your'e gonna use DNS servers and you don't want someone to track you, use the DNS server based somewhere where your government cannot access them. If you're in the US, it's easy to assume that US DoJ/FBI will not be able to subpoena Yandex or some Chinese internet provider.
- CalChris 10y agoYeah, that worked for me in SV. But I'd rather not rely on anything Putin related.
- nik736 10y agoThanks!
- gaur 10y agoClickbait headline, followed by a two-bullet "summary" that does not actually deliver what the headline promises. Fuck this.
- eridius 10y agoWhat are you taking about? There's a whole article there, not just 2 bullet points.
- gaur 10y agoNeither the headline nor the bullet point "summary" actually delivers the promised information about a possible vendetta. The goal is obviously to bury the information as deep as possible in the article to increase the likelihood that readers will click on ads.
- eridius 10y agoYou're upset that the headline doesn't deliver info on what the headline tells you? That doesn't make any sense. And the bullets are providing context for the article, not trying to answer the headline.
- twelve40 10y agoUpon reading TFA three times, it is still unclear to me who is waging a vendetta against whom. I vote clickbait as well.
- projektfu 10y agoIn a news article, the lede (first 1-3 paragraphs) typically contains the who, what, where, why. The rest of the article is usually interesting background and speculation. This article doesn't even "bury the lede". It contains no who. If you read the whole article and you think about it deeply, you might come to the conclusion that someone is DDOS'ing the people who publicly connect the dots between security researchers and bad actors (on a theoretical level). This is a vendetta apparently, but the word "vendetta" doesn't appear in the story. In other words, the story doesn't deliver the headline. We could write the lede differently to support the headline. "A major attack is underway targeting another company who publicly drew an association between hackers and security researchers. Dyn, a provider of DNS services, is experiencing a DDOS attack similar to the one experienced by Bruce Schneier. "The attack occurred coincidentally with a suggestion by Dyn's security director that some hackers and security researchers may be the one and the same. Mr. Schneier also made similar connections before the attack on his servers." Now, I know what might be going on.
- jpeg_hero 10y agobloomberg was down for me. I had disabled adblock at their insistence... i re-enabled adblock and I could get the article. hmmmm. maybe something about the 50 unrelated js calls?? perhaps?
- gorbachev 10y agoHere's a better article from Mr. Krebs: https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twitter-spotify-reddit/ https://krebsonsecurity.com/2016/10/ddos-on-dyn-impacts-twit... Personally I think his case is pretty convincing.
- dates 10y agokrebs is loading reaaaaal slow for me...i wonder if its related? or just a lot of people linking to it today.
- ComodoHacker 10y ago502 already. Google cache: http://webcache.googleusercontent.com/search?q=cache%3Ahttps%3A%2F%2Fkrebsonsecurity.com%2F2016%2F10%2Fddos-on-dyn-impacts-twitter-spotify-reddit%2F http://webcache.googleusercontent.com/search?q=cache%3Ahttps... And the article about BackConnect mentioned by Bloomberg: http://webcache.googleusercontent.com/search?q=cache%3Ahttps%3A%2F%2Fkrebsonsecurity.com%2F2016%2F09%2Fddos-mitigation-firm-has-history-of-hijacks%2F&ie=utf-8&oe=utf-8 http://webcache.googleusercontent.com/search?q=cache%3Ahttps...
- dom0 10y agoGuess what, Krebs' site also receives a spanking at the moment. (Given that it's hosted by Google I find it highly unlikely to go down under normal traffic)
- brightball 10y agoFrom the article: "Last month, a hacker by the name of Anna_Senpai released the source code for Mirai, a crime machine that enslaves IoT devices for use in large DDoS attacks. The 620 Gbps attack that hit my site last month was launched by a botnet built on Mirai, for example." I repeatedly hear people refer to IoT devices that are notoriously difficult to update...yet this Mirai code is technically able to access millions of devices and bend them to its will. So what I'm wondering is just, what prevents the good guys from using Mirai to slurp down every available device to patch the vulnerability that allowed Mirai to work in the first place? It seems like if vulnerabilities in these devices can destabilize the entire internet that it should be perfectly viable as a response to actively look for those vulnerabilities, patch/minimize them and notify their creators of the issue.
- inostia 10y agoMore specifics about Mirai bots and their numbers: https://threatpost.com/mirai-bots-more-than-double-since-source-code-release/121368/ https://threatpost.com/mirai-bots-more-than-double-since-sou...
- raverbashing 10y agoNot only East Coast, Twitter can't be resolved in Ireland/UK right now (I assume the mobile app uses some kind of 'dns pinning' as that is working)
- profmonocle 10y ago> (I assume the mobile app uses some kind of 'dns pinning' as that is working) The app was down for me until I switched my WiFi network to use OpenDNS. It's possible your phone has the DNS record cached, or it's using a different DNS server. (Is it on cellular?) Hardcoding IPs into a mobile app typically isn't done because it makes changing your infrastructure extremely painful.
- ilaksh 10y agoDNS is actually fairly centralized the way it is actually used. We need protocols and systems that are designed to be distributed from the outset.
- bcheung 10y agoI know the TTL is set really low for a lot of DNS entries but this recent outage got me wondering if it makes sense for servers further down the chain to hold onto it for longer than the TTL, honor it when they are able to get a new DNS entry within a reasonable amount of time, but fall back to the "expired" version if the authoritative server is not reachable. I'm wondering what would be the negative consequences of this and if they outweigh the benefit of being more resilient to these types of attacks.
- idlewords 10y agoThere was a good discussion on this in a sibling thread earlier today: https://news.ycombinator.com/item?id=12762110 https://news.ycombinator.com/item?id=12762110
- nastyasiwannabe 10y agoI'm suggesting this just so someone more knowledgeable can debunk it. Suppose FBI or someone up there had a meeting and said "in three weeks, there could be millions of armed Americans who believe that democracy was just stolen from them by some evil dictator in a massive globalist conspiracy. These people love twitter. Is there a way to make twitter go down without making it look like we're suddenly pulling the plug?" The answer was yes, we'll do a test run Friday.
- galdosdi 10y agoI'll bite. It would take a lot longer than a couple of hours of twitter being down for that to have a useful effect. For something as major as the presidential election result, it would probably take minimum a week before people got bored and moved on to a different topic. So this kind of attack that only takes something out for a few hours would have no useful effect for an actor that wants to prevent people from discussing a recent event. IMHO, it would be hard in general to take out a service run by a serious IT organization (of which there are admittedly few, by my definition of serious) for more than a few days unless the attacker carried out non-trivial physical damage (eg, bombing multiple datacenters, murdering multiple system administrators, etc) or managed to somehow destroy enough backups (which in a serious IT shop, should be hard, as there should be some offline cold backups that require physical human activity to destroy)
- nastyasiwannabe 10y agoSure it wouldnt work for an extended time. I'm just thinking that in an unpredictable situation, a few hours might be all you need to diffuse it. For example suppose someone claims they have evidence of some crazy shit happening at the polling places, and the only thing that can be done is to for Patriots to seize the equipment at the polling places before the globalists can cover their tracks.
- deleted 10y ago[deleted]
- oldmanjay 10y ago
- pc2g4d 10y agoI always thought DNS had enough redundancy built-in that this sort of thing wouldn't really have much effect. But here I am unable to access websites, simply because name resolution isn't working. If my local DNS server were caching things longer there would largely be no issue.
- Falkon1313 10y agoYeah, DNS entries are usually (or at least used to be) cached for what would seem like long enough, but I guess it doesn't really work the way it sounds. "a hierarchical decentralized naming system [that] provides distributed and fault tolerant service and was designed to avoid a single large central database" doesn't sound like it should be so fragile. Having single 'authoritative' servers for the sort of thing that should be inherently distributed sounds more like an Achilles heel.
- rrggrr 10y agoThese attacks are possible because the US Congress hasn't extended tort liability to manufacturers of software and network hardware. The full weight of the US products liability bar will quickly and rapidly motivate manufacturers to ship secure devices. The lack of accountability is enabling vulnerability.
- egypturnash 10y agoI am a non-programmer who reads HN and keeps up with tech news in general. And every time I read about the IoT botnet, my immediate response is to look around my apartment at my Internet-connected lights, and wonder if they're part of it. How can I find this out? Is anyone making a tool that a non-technical user can run to squint at their network and look for evidence of Mirai, or anything else trying to take advantage of this niche? There are plenty of tools with a reasonably simple interface that will tell me if my laptop/desktop computer is infected with something. But what can I use to diagnose the health of all of the other computers proliferating around my house? How can a non-technical user easily monitor the overall health of their connected household? Is this a project anyone is building? Because I think it's definitely something that needs to exist now.
- dmourati 10y agoThe best place to do this is at your border. You probably have a cable modem or router or some such that connects your home to the internet. You would typically install software known as IDS (Intrusion Detection System) such as Snort there and look for anomalous traffic. As for a non-technical solution, it will be difficult to implement. It requires some computer know how and time. Such a secure device could be created or better yet offered by the manufacturers of the modems/routers frequently deployed in homes.
- pmlnr 10y agoThe last time I played around with Snort[1] I realized I'm lightyears away from being paranoid compared to the default settings :) It would be good to have an IDS with bare minimum settings, easy to turn on layer after layer, though I understand it's tricky. [1]: https://www.snort.org/ https://www.snort.org/
- Retr0spectrum 10y agoI might be wrong, but in the case of Mirai I'm fairly sure you're safe if all your devices are behind NAT.
- 10y ago
- reacharavindh 10y agoPerhaps a naive question, but Why can't a DNS provider identify such participants in a DDOS and ban their IPs forever?
- JayNeely 10y agoBecause IP addresses are often shared resources. Your ISP gives each customer an IP address (often a temporary one), and then that customer's router system handles assigning private, local-network-only IP addresses to any devices connecting through the network. So if a DNS provider starts banning public IPs (which are the only IPs it sees), you could end up with an entire college getting banned because of one hacked webcam in one student's dorm room. Or someone in an apartment somewhere with (unknowingly) a hacked thermostat finds their internet no longer works (DNS provider has banned them), so they reboot their modem, which causes their ISP to provide them with a new IP address. Guess what happens to their old IP address? It goes back into the pool of available IPs that that ISP can assign to other customers, and more and more banned-from-DNS addresses keep getting passed along to innocent, un-hacked customers.
- reacharavindh 10y agoAh, (inter)networking 101. Thanks! Then, is there a way for the DNS providers to know the ultimate recepient at all? MAC address? (or does it get truncated at the lower levels and not passed over IP protocol?)
- jlgaddis 10y agoNope. Assuming you have a router connecting your home network to your ISP, for example, the MAC addresses of your "internal" devices are not visible to the ISP. The only MAC address they see is the MAC address of your router's "WAN" interface. The source/destination MAC addresses in an Ethernet frame (layer 2) are rewritten at every router (layer 3) hop. The original IP source/destination addresses in the IP packet, however, do not change (exception: NAT, which does exactly that). Another problem -- in many (most?) DDoS attacks where UDP traffic is involved -- is that the source IP addresses are "spoofed". That is, IP packet that the victim receives says that it's coming from Alice but it really came from Bob. There are also "amplification" attacks, where an "innocent third-party" is used, unknowingly, to "help" perform the attack.
- anotherevan 10y agoDid any one else find the style of writing in this article really annoying? Things like using prefacing statements with "so-called" or putting terms in quotes to make them seem suspect. e.g.s: a so-called distributed denial-of-service (DDoS) attack York said Dyn was “actively” dealing with a “third wave” of the attack.
- GrinningFool 10y agoI tend to assume that the larger publications use it in the underlying sense of "..as it is so called".
- davidf18 10y agoThe failing here as in many cases such as a number of security breaches was a lack of investment. As someone with an engineering degree that worked as a VLSI design engineer, good engineering requires * backup systems *. This costs money that people don't want to spend. In some cases such as a startup they might be cash short, but many firms have the money but don't want to spend it ensuring that they have well engineered software that includes backups, up-to-date software and security upgrades, hiring (expensive) highly competent software engineers and consulting firms. The mistake in this case was relying on one vendor for DNS. Amazon Route 53 would be a good alternate vendor for DNS, for example.
- patrickg_zill 10y agoI think even basic home routers these days, have enough cpu power to handle egress filtering. If you have an iot device, by its nature it only needs to connect to a few services and hosts. The manufacturer can provide this in their docs, and give an automatic config url that the router uses to load its egress rules. The rules to load are displayed and the user checks they are legit by comparing to the printed version in the manual, then clicks ok. Or something like that. Rate limits in terms of packets per second, total bandwidth both instantaneous and over time, are set also.
- woliveirajr 10y agoI love those comments about IoT and who should be responsible for error-proof products, or ISP monitoring traffic, or ... Internet, in the beginning, was even more insecure. Including the computers and OSes. There were less abuse because few had resources and knowledge. Read some old software and you'll find all bad designs in it. Software didn't become worst, it's just targeted with more knowledge and intensity.