17 ms·
All United Flights Grounded Due to Mysterious Problem
- krschultz 11y agoI was flying from Dublin to Newark on Saturday on a United flight. At some point during our flight the entertainment system needed to be rebooted. When it came back up, the splash screen hit me with a huge amount of nostalgia. It was RedBoot with a kernel build date from 2004. Obviously this is the entertainment system and not something more critical, but it's telling. There is a huge cadence mismatch between software cycles and capital good replacement cycles. Airplanes, factories, HVAC systems, even home appliances last for decades. Software on these systems needs to get upgraded, I can't even imagine the number of security patches that have gone into the Linux kernel in the last 11 years.
- theandrewbailey 11y agoThis is the Achilles heel of the entire Internet of Things and smart appliance trend, and I think this will bite everyone bad. After 50+% of these vendors go out of business in the next decade, their products won't get updated, and people will wonder why their "smart" TV can't watch movies from whatever replaced Netflix/new whizbang video service. They won't be as likely to buy any "smart" thing again.
- stcredzero 11y agoProbably the only reason why your ca. 2005 linux media pc won't work in your current living room is that its hardware can't support your new 4K flatscreen. Open software could go some way towards addressing the update problem, but it would need to have a much more robust code signing infrastructure behind it, to avoid becoming an even worse morass of security problems than the morass we have today. (Generally, not in open software.)
- jkestner 11y agoThat will only matter if companies don't train customers to expect frequent upgrade cycles. It's worked for consumer electronics, but there will be strain in previously "came with the house" objects. You'll need to sidestep people's habits, in the way that the microwave did, perhaps. And we're only worried about vendors going out of business because it's the early days and it's largely startups pushing the trend. With a Samsung or Apple, it's more that they'll quickly (by home equipment standards) stop supporting whatever doesn't stick to the wall. There is a case to be made for self-contained objects that don't derive most of their value from an ecosystem, but work normally with no network. Work up from a toaster, not down from a computer.
- rcthompson 11y agoAre you arguing that frequent upgrade cycles are a good thing?
- jkestner 11y agoDefinitely not, for the consumer. For the manufacturers, sure. Supporting a cheap piece of hardware for several years when consumers are going to demand it works with HomeKit, Google @Home, or whatever it's called now, and their descendants, and Microsoft's play, and the many competitors yet to come — that'll suck. Traditional hardware makers are going to have to factor the support of this software component into their prices now.
- a3n 11y agoRoughly OT: I have yet to have seen anything that makes me look forward to the IoT, and when I talk to the voices of vendors in my head I'm basically telling them "Stay out of my refrigerator, my furnace, my toilet and my smart doorlock that I'll never have." I just don't feel that anything relevant to the IoT is missing from my life. At all.
- wlesieutre 11y agoI can see "Check if I left the stove on and turn it off remotely" being a neat thing to have. But not neat enough to let a 10 year old appliance with no software updates and the capability to burn my house down be accessible to the internet.
- exelius 11y agoYeah, but chances are you won't use the functionality often enough to configure it in the first place. That's the real problem with the Internet of Things: most of the things we own are not all that useful when we're not in close proximity to them. Thus, not only are users and manufacturers unlikely to update them in the future; users are just as unlikely to connect the thing in the first place. Home automation through things like light switches, etc. has a use case, but those products have been available and Internet-connected for over a decade and we still haven't seen wide adoption. I recently priced it out -- it would cost me over $5000 to swap out the outlets and switches in my house for Insteon devices. And that's just the hardware; not the electrician required to connect it all or the time I would spend configuring everything. Home builders aren't going to spend that kind of money building this into anything but the most high-end homes -- the IoT hardware alone blows through the fixtures and appliances budget that most home builders allocate. People want systems that "just work". IoT does not "just work", and none of the current or announced implementations address the big problems around configuration (namely, every house is different so every implementation is custom). And in some cases like a stove or a refrigerator, any amount of configuration is going to be too much.
- wlesieutre 11y ago
- snowwrestler 11y agoI bought an LG smart TV a while ago. It stopped receiving firmware updates after a couple years and is now way behind current models in terms of features--and, I can only imagine, security patches. I will go out of my way to buy a dumb TV next time.
- cma 11y agoMine updated itself to add a big banner advertisement to the main screen. It originally had no ads.
- nodata 11y ago> I will go out of my way to buy a dumb TV next time. How? I don't see any dumb TVs for sale.
- morsch 11y agoIsn't dumb TV just another word for monitor these days?
- nodata 11y agoBecause you no longer need a tuner? Could be. But I doubt you could buy a 60" glass front monitor for the price they sell TVs :)
- sliverstorm 11y agoThey certainly seem to be similar! But, a number of differences do arise from intended use. - TVs don't need anywhere near the same level of display quality. They are viewed from ten feet away and do not render small text, so they don't need as clear a picture. They also don't really have to go over 30fps, and latency is less of a concern. Basically, they have looser constraints in many ways, making them cheaper - TVs have a plethora of inputs of many formats - TVs have remote controls
- dragonwriter 11y ago> Isn't dumb TV just another word for monitor these days? A dumb TV still has a TV tuner, so they aren't equivalent. But unless you plan on plugging it directly into an antenna for OTA broadcasts, they are basically equivalent.
- rcraft 11y agoThis is exactly why I prefer buying "dumb" tvs and simply adding chromecast/appletv/firetv, etc. Much better experience.
- sliverstorm 11y agoIt doesn't really matter that much if it's dumb or smart. You can use a Chromecast with either, and refusing to buy one kind limits your choices.
- antsar 11y agoYou can use a Chromecast with either, but one of them can also spy on you without your consent.
- kstenerud 11y agoNot if it isn't connected to your network.
- antsar 11y agoValid point. But then you're left with a bunch of crap in the UI that is unnecessary and annoying at best.
- kstenerud 11y agoReally? I'm not. My smart TV defaults to just displaying whatever the incoming signal tells it to. And if some manufacturer decides to clutter things, I just won't buy it. Problem solved.
- rcthompson 11y agoIt's not too much of a stretch for smart TVs to start including cellular connectivity (and advertising it as "zero-setup").
- 11y ago
- saosebastiao 11y agoI know it is just an OS kernel and security is far more comprehensive than having a secure kernel, but this is an amazing start: https://sel4.systems/ https://sel4.systems/ I'm surprised IoT conversations are still happening with Linux as a contender for the OS, let alone Windows.
- wazoox 11y agoActually one month ago in Amsterdam I've used a Philips smart TV that complained about "youtube is not supported anymore yadda yadda". Programmed obsolescence at its best (this TV probably wasn't more than 3 or 4 years old).
- danudey 11y agoHeck, just look at the huge number of Android phones that you can buy at retail with an out-of-date version of Android with known security holes which will never be patched or updated. And those are relatively complex devices which are trivial for users to update if given the option, based on the adoption rates for e.g. iOS updates. A company doesn't have to be out of business to not do security updates; they can not do security updates starting day one. There was an article a while ago about tons of home router vendors with insecure software from a third party, where the third party had resolved security issues years ago but the vendors had never bothered to update, leaving hundreds of thousands of devices vulnerable over the last few years.
- logfromblammo 11y agoThey don't even need to go bankrupt. I have an LG television that has been promising new widgets ever since I first got it, and no new widget has ever appeared. The impetus there was that LG changed its net-connected TV platform in 2011, and instantly dropped all support for older devices. One would think that a final update could remove that "coming soon" box from their proprietary added-feature screen, but they haven't even bothered to do that. So I can watch NetFlix and YouTube on that device, but not Amazon instant video, or Crackle, or Crunchyroll, or Vimeo, or any of the dozens of selections available to better supported platforms. Having learned my lesson, and aware of the increasingly stalkerish behavior of "smart" televisions, my next TV purchase was very specifically a dumb screen. If I want an internet-connected service now, I use the Wii, or XBox, or the extended desktop from the nearest computer. I will likely refuse to buy any network-enhanced appliance in the future, unless I am able to root/jailbreak it and install software without the manufacturer's stamp of approval. I probably wouldn't do much beyond installing ChillBox, or FridgeBSD, or CryogenMod, or whatever, but it feels like the possibility might keep them a little more honest. Because you know that refrigerator hackers would be capturing and picking apart every packet that thing sends out, quickly discovering that every time someone closes the door, it sends a tattle out to fridge-use.org about how long you stood there with the fridge door open, along with before-and-after photos of your food. Though it would also be embarrassing if they marketed value models of a product line by disabling features in software/firmware, and some NetBSD-loving punks could come along and write a simple script that turns the doohickey that retails at $200 into the one that sells for $800. So it's already too late for me. "Smart" appliances are just another low-capability computer that I will have to support as the in-home IT guy. And I will have to presume that they come pre-loaded with all manner of crapware and spyware. I would forever need to be checking on chipsets and revision numbers and compatibility lists. No thanks. It's hard enough managing the congestion on the home WiFi already.
- pubnub 11y agoUse https://resin.io/ https://resin.io/ yo
- stcredzero 11y agoThere is a huge cadence mismatch between software cycles and capital good replacement cycles. How well would you say that Tesla is coping with this as a company? For that matter, what about Apple? I can't even imagine the number of security patches that have gone into the Linux kernel in the last 11 years. Let's take a step back and think about this statement. Isn't this insane? We know enough to be able to build something much better than this. The reason that we don't, is that we've just kept on pragmatically building on what we had before. We're like a corporation that keeps pouring money into its "stovepipe" system because we keep on making short-term decisions. (Somehow "stovepipe" has come to mean "vertically isolated," but I seem to remember that it also used to refer to the tendency of iron stovepipes to corrode and need constant patching.)
- bitwize 11y agoHow well would you say that Tesla is coping with this as a company? For that matter, what about Apple? Apple just kind of assumes that you have the latest shiny, because why wouldn't you? This induces a phenomenon I call the Apple Turnover: when a software update aimed at new Apple things comes out and makes your old Apple thing not run so good anymore. Sluggish iPhones are the hallmark example today, but I was bitten badly by this in the mid-2000s when Panther would no longer compile C++ files. You see, one of Apple's OS updates for Panther came with Tiger's libstdc++, which used the new Itanium ABI. This was so Xcode for Tiger could compile programs to run on Panther, but without heroic efforts to set up compiler flags in every package you built to link against the old static libstdc++, compiling on Panther would link against the new libstdc++ by default and fail horribly, rendering C++ code uncompilable. (Deleting or renaming the new libstdc++ was not an option; it was a heavily depended on system component and I think even the header files were changed for the new library.) And a lot of stuff depended on C++, including C-API stuff like SDL. And Apple did fuck all to fix it. So if you buy a shiny Apple toy, your choices are to commit to upgrading early in the new product cycle or risk an Apple Turnover rendering your purchase, if not useless, then with degraded functionality even relative to the same device when you bought it. And the pisser is during the 80s and 90s, Apple gear was legendary for running well, and being supported, many years if not more than a decade after its purchase date.
- dba7dba 11y agoConsidering the ridiculous amount of testing required (rightly so) for any change on the jetliners, they probably cannot keep up with the patches. One Windows running company I worked for long time ago simply didn't apply the patches. They said it broke things...
- stcredzero 11y agoI've had demos ruined by iOS 8.x patches! (By changes to pretty well established code, like NSTimer and UIImageView updating.) I could easily believe that Windows patches would break things.
- TheCapn 11y agoDepends on the risk it poses and the attack vectors available. If you can assume jetliners use a custom hardware with no web tie ins and no physical access (USB/otherise) then security patches covering attacks that require those vectors are kind of moot aren't they? A whole lot of testing and verification would go into applying a patch that is rendered useless by other security precautions.
- nostromo 11y ago> Obviously this is the entertainment system and not something more critical According to recent reports, the entertainment system is not fully isolated from the plane's navigation systems. However, Boeing has denied this. http://www.cnn.com/2015/05/17/us/fbi-hacker-flight-computer-systems/ http://www.cnn.com/2015/05/17/us/fbi-hacker-flight-computer-...
- Cacti 11y agoThat's because the researcher didn't do that on an actual plane. In fact it wasn't even a simulator, it was a virtualized simulator where he put the systems all on the same bus. On top of that the guy was only able to listen in on traffic. The entire story is bogus. The systems are isolated just fine.
- jfaat 11y agoCan you provide a source? [edit] I've seen the claim made more than once in this thread without a source. I understand that the article was probably very wrong. I'm just interested to know if a reputable publication has confirmed this or if it's just conjecture.
- burger_moon 11y agoI know this doesn't mean diddly squat, but my sister worked for a manufacturer of airline wiring harnesses up until very recently, and when I asked about it, she mentioned that they were not interconnected or exposed to one another. I don't believe they did the really huge jets, but regional and private sized jet harnesses instead, so it could be different in that aspect also.
- nradov 11y agoCan you provide a reputable source to prove that President Obama is not a disguised lizard alien? ☺ I mean the original article was so obviously nonsense on the face of it that no refutation should be necessary.
- EvanAnderson 11y ago
- pedrocr 11y agoExactly because of this I think Fred Wilson is spot on in saying expensive things will stay dumb and the smarts will be in the cheaper replaceable gadgets[1]. Cars have already done that with audio (bluetooth works reasonably well). Hopefully one of the video casting solutions will solve video as well so we can stop mounting smartphones on the dash and just cast the images into the car's screen. [1] http://avc.com/2011/12/cheap-willl-be-smart-expensive-will-be-dumb/ http://avc.com/2011/12/cheap-willl-be-smart-expensive-will-b...
- petercooper 11y agoThe IFE system on a 747 I was on a couple of years ago was running Windows 98, complete with graphical boot screen :-) (Edit: Removed airline, I like them too much.)
- cletus 11y agoThis reminds me of the rumours that Apple would release a TV. Frankly I never believed it and (IMHO) you have to be pretty ignorant of Apple to think the rumours had any credence whatsoever. The upgrade cycles are simply too long for TVs. But more importantly, Apple's whole model is to treat things like this as just "dumb complements". Your mobile device is, from the carrier's perspective, is increasingly becoming an a dumb Internet pipe (first with the App Store, later with the likes of iMessages and how LTE works, etc). The TV for Apple is simply a dumb display to stick an Apple TV into. A sub-$100 device you can replace every other year if need be. A $3000 TV is replaced a whole lot less often. Why would Apple want to be in the business of (eventually) supporting 5+ year old TVs for such a low-margin business? Or what makes you think users would pay for the Apple brand and/or upgrade more often to make it worthwhile? So as far as IoT goes, I have trouble seeing a future where someone says "I need to buy new lightbulbs because mine don't get firmware updates anymore" or "I need to buy a new fridge because it can't talk to my new phone".
- ghaff 11y agoSmart TVs doubtless sound like a great idea if you're a TV manufacturer who desperately wants to shorten upgrade cycles and sell higher-margin "value add" in a world where so many people already have HD TVs that are pretty much as big as they have room for. For everyone else? Not so much. Not only does it make sense to put the smarts and connectivity in devices that are either cheap, that people already have anyway, and can be easily upgraded but the user interface in a phone/tablet/etc. tends to be far better than a typical remote. In general, I tend to prefer the Chromecast model of just casting video from a general purpose device, but the Kindle stick and Apple TV are OK as well. By contrast, I rarely used the Smart TV features on my Panasonics because they were just so painful to use.
- keeperofdakeys 11y ago19 January 2038 will be a very interesting day, since that's when signed 32 bit unix time will overflow. I don't want to think about how much software would rely on that, especially critical embedded hardware.
- srtjstjsj 11y agoAn embedded system with a 30-year lifespan and no maintenance plan?
- arca_vorago 11y agoFrom what I understand of the Chris Roberts fiasco, their avionics systems weren't airgapped from the other systems. If that is the case and not just hype, then no fucking wonder shit like this can happen.
- rubicon33 11y agoWhat does "air gapped" mean?
- cheald 11y agohttp://en.wikipedia.org/wiki/Air_gap_(networking) http://en.wikipedia.org/wiki/Air_gap_(networking)
- deleted 11y ago[deleted]
- AngrySkillzz 11y agoMeans that they are not physically connected to the other systems, that there should be no possible way to reach the avionics from e.g. the in-flight entertainment system.
- Zikes 11y agoRoughly, not connected in any way.
- minot 11y ago> An air gap or air wall is a network security measure that consists of ensuring that a secure computer network is physically isolated from unsecured networks, such as the public Internet or an unsecured local area network. It is often taken for computers and networks that must be extraordinarily secure. Frequently the air gap is not completely literal, such as via the use of dedicated cryptographic devices that can tunnel packets over untrusted networks while avoiding packet rate or size variation; even in this case, there is no ability for computers on opposite sides of the air gap to communicate. The case above is an example cited as a life-critical system: > Computers used in aviation, such as FADECs and avionics https://en.wikipedia.org/wiki/Air_gap_(networking) https://en.wikipedia.org/wiki/Air_gap_(networking)
- jsingleton 11y agoCould be a similar issue to the recent electronic flight bag issue. Certainly reads that way from the article. They seem very quick to blame a hack when it could very easily be a bug in the system or an administrator error. http://www.theguardian.com/technology/2015/apr/29/apple-ipad-fail-grounds-few-dozen-american-airline-flights http://www.theguardian.com/technology/2015/apr/29/apple-ipad...
- a3n 11y agoHave they actually officially blamed anything?
- cgy1 11y agoHope it's not due to people using unopened cans of Diet Cokes as weapons.
- georgeglue1 11y agoInteresting, United recently added a pretty lucrative bug bounty program (a rarity among airlines) a couple of weeks ago. http://www.united.com/web/en-US/content/Contact/bugbounty.aspx http://www.united.com/web/en-US/content/Contact/bugbounty.as... It would be ironic if the bug bounty program directly/indirectly lead to this.
- theseatoms 11y agoIronic? Maybe "fitting" instead?
- toxicFork 11y agoI think the use of the word "ironic" was quite fitting. Ironic: A state of affairs or an event that seems deliberately contrary to what one expects and is often wryly amusing as a result.
- lucaspiller 11y agoExcept: Bugs that are not eligible for submission: * Bugs on internal sites for United employees or agents (not customer-facing) * Bugs on onboard Wi-Fi, entertainment systems or avionics
- saryant 11y agoThe onboard wi-fi and IFE are provided by external vendors (Panasonic, LiveTV, etc) and United probably doesn't want to pay for their bugs.
- jfaat 11y agoMore importantly, I think they are trying to avoid giving any incentive to hack a plane mid-flight. The policy also stated that criminal actions may be persued in cases of attempting to access these systems.
- NoMoreNicksLeft 11y agoWhether or not they pay bounties for them, they're definitely paying for them.
- jobu 11y agoNot sure if it's related, but the United website was down this morning for a while as well.
- dredmorbius 11y agoSpoke to folks at another airline a couple of years back. Flight registration, SABRE, and aircraft maintenance were all managed through the same systems. I found that both surprising and unacceptably risky.
- djcapelis 11y agoA lot of people seem to be jumping to the conclusion that their systems are malfunctioning because of being hacked rather than their systems malfunctioning on their own. Hard to know what is happening from the outside, but their systems may just merely be bad. That said, the plane communication protocols aren't terribly secure, so it's certainly feasible someone is playing around with them. Maybe they'll decide it's in our interest for us to know at some point.
- cryoshon 11y agoIs this what an actual cyberattack / cyberwar looks like? Imagine how much money is being lost right now as a result of this disruption. Somewhere hackers are popping champagne.
- chatmasta 11y agoUAL stock took a $1 nosedive (hah) at 10am.
- glesica 11y agoFind out who made money by shorting it, there's your list of suspects, or at least co-conspirators (if this turns out to be a hack).
- caskance 11y agoOr people who use twitter and like to gamble.
- getsat 11y agoOr bots who subscribe to Twitter feeds and open trades based on sentiment
- chatmasta 11y agoIn that case the suspects could be the ones buying it after it dropped $1, since it went up afterward. As long as securities react to hacks, there will be a massive incentive to 1) hack, and 2) overstate the hack's significance. Furthermore, as bots become more sophisticated, confusing them becomes easier. If you know bots will short CompanyX when "CompanyX hacked" hits the headlines, then you have an unfair advantage just by being the first to know of the hack.
- glesica 11y agoThat's why I said "list of suspects" instead of "culprit".
- onyxraven 11y agoThe initial descriptions sound more like someone pointed a testing tool at the wrong environment, rather than a hack.
- philip1209 11y agoI'm honestly surprised that critical software at the core of more operations-heavy companies does not go down more often. Possible causes range from a software bug to database master failover to a data center outage, but realistically there are single points of failure at delivery companies, airline companies, and more that could stall everything. I'm surprised this software doesn't break more often. When was the last time that UPS had delivery delays due to a software outage?
- vonklaus 11y agoWell, if 9/11 is any indication, commercial jets can deliver pretty destructive payloads. Avi Rubin summarized some hacks in his TED talk[0], where hackers gain complete control of vehicles. It is unwise to just spread FUD this early, however, if these systems bare any resemblance to cars (and it is likely that they have many of the same characteristics i.e digital control of key steering/speed/avionics) then it is possible someone has the information to control a fleet of missiles on American soil. Unlike 9/11, these people will not be the US government, and could be actual terrorists. Who knows, maybe this is just a 16 year old who got accosted going through security and wanted to burn off some steam. [0] https://www.youtube.com/watch?v=BHHCvcCUOWU https://www.youtube.com/watch?v=BHHCvcCUOWU
- netizzio 11y agoIt's interesting that the Wired article mentions the recent controversy about claims that aircraft systems can be hacked, but explicitly ignores the incident last week as having any possible relation to these events, where a bigoted employee denied a Muslim passenger an open can of Diet Coke because it "might be used as a weapon", while giving the passenger in the adjacent seat an open can of beer. The response from United was unapologetic and absolutely disgraceful: https://hub.united.com/en-us/News/Company-Operations/Pages/shuttle-america-flight-3504.aspx https://hub.united.com/en-us/News/Company-Operations/Pages/s....
- saryant 11y agoA few things about that: 1) The beer isn't free, the passenger paid for the entire can or used a 1K drink chit. 2) UA flight attendants are famous for making up rules and many try to avoid handing out entire cans of soda, and this one wasn't even a United flight attendant. 3) What on earth would that have to do with today's event?
- netizzio 11y agoWhether or not the beer was free has absolutely nothing to do with this. It's not about the beer or diet coke, but about the blatant bigotry exhibited by an employee against a passenger on a United flight, as well as inexcusable behavior by another passenger. Similarly, whether or not this was a United flight attendant is also of absolutely zero relevance. They may have technically been an employee of Shuttle America, but were part of the cabin crew and a representative of United on that flight, working under the United brand and wearing United uniforms. Therefore, when United releases a statement making no apology for abhorrent behavior exhibited by their representative, it reflects directly on them. It may have nothing to do with this event, just as Chris Roberts tweeting that he hacked into the in-flight entertainment system may have nothing to do with this event. It's merely interesting that Wired explicitly ignored the actions of United as having any possible relationship to this event.
- deleted 11y ago[deleted]
- deleted 11y ago
- dmazin 11y agoI was one of the people grounded this morning. They said they were having mechanical problems. They took the plane out to the runway and taxied it around "to try to figure out what's wrong," then let us on.
- gkanapathy 11y agoPretty sure that if it was a hack or credible bomb threat, that they would not have been flying again after only an hour. A scenario like that would suggest just a normal IT glitch and a reboot/restart to fix and validate that it's back to normal.
- clmns 11y agoInteresting. I worked on the support team for the software that creates and files the flightplans for UAL. It was a horrible piece of SW/architecture with many outages. We tested in production daily and had direct access to the databases. I'm pretty sure they never changed their policies.. So yeah, this sounds very much like it! Edit: Just got confirmation, this software was the root cause. No hacks/whatsoever!
- evo_9 11y agoCould this be related to the resent articles about a researcher taking control of a play via the entertainment system? http://www.wired.com/2015/05/feds-say-banned-researcher-commandeered-plane/ http://www.wired.com/2015/05/feds-say-banned-researcher-comm...
- simonebrunozzi 11y agoWhat I really HATE about things like this one is that United will not refund us (I was heavily affected yesterday), or will offer offensive amounts of dollars/miles as refund.