7 ms·

Really interesting to me that none of the commentators I've seen in the press have even hinted that maybe an OS that requires frequent security patches shouldn'

Really interesting to me that none of the commentators I've seen in the press have even hinted that maybe an OS that requires frequent security patches shouldn't be used for infrastructure in the first place. For just one example, I've seen photos of BSODs on airport monitors that show flight lists -- why aren't those built on Linux or even OpenBSD? Security is not a feature that can be layered on. It has to be built in. We now have an entire industry dedicated to trying to layer security onto Windows -- but it still doesn't work.

dopylitty 2y ago
Or don't use an OS at all. We need to think about minimizing the use of software in critical infrastructure. If that means less efficiency because you have to be near something to maintain it then so be it. That would be good for jobs anyway.
- antihero 2y ago
  How exactly would a lot of end user systems function without one?
  - advael 2y ago
    You can run single-purpose software on bare metal, and many OS-agnostic toolkits for things like user interfaces exist
    - sabas123 2y ago
      I'd like to pose two questions: 1. How does the software obtain new data at run time? 2. How do you make sure that thing doesn't pose a security hole when a vulnerability gets discovered? (assuming this never happens is unrealistic)
      - advael 2y ago
        The answer to both questions is robust organizational infrastructure. To be frank, I think a minimal linux system as a baseline OS serves most use cases better than a bare metal application, but many applications have self-contained update systems and can connect to networks. Self-repairable infrastructure is a necessity, both in terms of tooling and staffing, for any organization for which an outage or a breach could be catastrophic, and the rise of centralized, cloud-reliant infrastructure in these contexts should be seen as a massive and unacceptable risk for those organizations to take on. Organizations being subject to unpatched vulnerabilities and inability to manage their systems competently are direct results of replacing internal competency and purpose-built systems with general-purpose systems maintained and controlled by unaccountable distant tech monopolies
        MetaWhirledPeas 2y ago
        > the rise of centralized, cloud-reliant infrastructure in these contexts should be seen as a massive and unacceptable risk for those organizations to take on I agree with you but I also want to play the devil's advocate: using software like CrowdStrike is not what I would call being "cloud-reliant". It's simply using highly-privileged software that appears to have the ability to update itself. And that is likely far more common than cloud-reliant setups.
        advael 2y ago
        Yea, and use of highly privileged software with the ability to update itself that the organization has no oversight of should be the most suspect. Software is used by nearly every organization for drastically different needs, and I think there will never be adequate security or reliability for any of them if software providers continue to consolidate, generalize, and retain ever more control of their offerings. Personally, I think the solution is local-first software, either open-source or grown within the organizations using them, which necessitates having that capability within orgs. The whole "buy all our infrastructure from some shady vendor" model is a recipe for disaster
      - ssivark 2y ago
        Vulnerabilities in what though? If you make an application so simple that it can only fetch data through an API and display, there's simply not much more that it can do. And a simple application is easy to audit. So it would be ideal if we could bundle this (akin to compiling) and deploy on bare metal.
    - antihero 2y ago
      Most of these OS need to run a variety of applications from different vendors though.
- Osiris 2y ago
  Even unikernel applications have an OS compiled into the application. It's necessary to initialize the hardware it's running on, including the CPU and GPU and storage. I suppose you could build it as a UEFI module that relies on the UEFI firmware to initialize the hardware but then you get a text only interface. But then the UEFI is the OS. But this outage was not an OS problem. It was an application bug that used invalid pointers. If it was a unikernel it still would have crashed.
  - ScottBurson 2y ago
    I've read that it was a driver of some kind, not an application. Applications can't cause BSODs (I hope).
stefan_ 2y ago
I read the T&C of this CrowdStroke garbage and they have the usual blurb about not using it in critical industry. Maybe we just charge & arrest the people that put it there and this checkbox-software mess stops real quick.
- AceyMan 2y ago
  /set Devil's Advocate mode: from the reporting so far, no one has died as a result of the Crowdstrike botch. For my money, that sounds like it's not being used in 'critical industry'. /unset There were several 911 service outages included in the news yesterday, so I would definitely say agree those fall into the category. I haven't seen how many hospitals were deeply affected; I know there were several reports of facilities that were deferring any elective procedures.
  - logbiscuitswave 2y ago
    I almost had to defer a procedure for one of my cats because my vet’s systems were all down. This meant they couldn’t process payments, schedule appointments, use their X-ray machine, or dispense prescriptions. (Thankfully, they had the ingenuity to get their diagnostic equipment online through other means, and our prescriptions had already been dispensed so we didn’t have to reschedule.) I would imagine it’s the same story at human hospitals too that ran afoul of this. I wouldn’t expect life-critical systems to go offline, but there’s many other more mundane systems that also need to function.
- delfinom 2y ago
  The public T&C is for small businesses. Any large business is going to be negotiating very different terms which are not public.
nullindividual 2y ago
> why aren't those built on Linux or even OpenBSD The vendor who makes the software has always written for Windows (or in reality, wrote for either DOS or OS/2 then transitioned to NT4). History, momentum, familiarity, cost, and ease of support all are factors (among others, I'm sure). Security is a process, not a product. And yes, distros require frequent updates, though more to your point, you can limit the scope of installed software. I'm sure airport displays don't need MPEG2, VP1 and so on codecs, for instance. It's also important to remember that there is a lot of 'garageware' out there with these specialized systems. Want SAML/OIDC support? We only support LDAP over cleartext, or Active Directory at best. Want the latest and greatest version of Apache Tomcat? Sorry, the vendor doesn't know how to troubleshoot either, so they only "support" a three year old vulnerable version. Ran into that more than a few times. Given the hypothesis of what caused the BSOD with Crowdstrike (NUL pointer), using a safe language would have been appropriate -- it's fairly easy in this case to lay the blame with CS. Microsoft supplies the shotgun. It's the vendors responsibility to point it away from themselves.
- pwg 2y ago
  > I'm sure airport displays don't need MPEG2, VP1 and so on codecs, for instance. They don't, until the day the airport managers are approached by an advertising company waving the wads of cash the airport could be 'earning' if only they let "AdCo" display, in the top 1/4 of each screen, a video advertising loop. At which point, those displays need the codecs for "AdCo's" video ads.
  - nullindividual 2y ago
    Absolutely (sigh)! But with a deployment of devices like that, the operator has a solid central management system from which they could push software as-needed.
  - westpfelia 2y ago
    Boy do I sure hate you for saying that. I mean at some point you are right. That is the future. But god am I mad at you for reminding me this is the world we live in.
hedora 2y ago
This sort of emergent behavior is a feature, not a bug. Operating systems that don't require frequent security patches aren't profitable. Anyway, this is the step of late-phase capitalism that comes after enshittification. Ghost in the Shell 2045 calls it "sustainable war". I'd link to an article, but they're all full of spoilers in the first paragraph. It probably suffices to say that the series refers to it as capitalism in its most elegant form: It is an economic device that can continue to function without any external inputs, and it has some sort of self-regulatory property that means the collateral damage it causes is just below the threshold where society collapses. In the case of Cloud Strike, the body count is low enough, and plausible deniability is low enough that the government can get away with not jailing anyone. Instead, the event will increase the money spent on security theater, and probably lead to a new regulatory framework that leads to yet-another layer of mandatory buggy security crapware (which Cloud Strike apparently is). In turn, that'll lower the margins of anyone that uses computers in the US by something like 0.1%, and that wealth will be transferred into the industry segment responsible for the debacle in the first place. Ideally, the next layer of garbage will have a bigger blast radius, allowing the computer security complex to siphon additional margins.
- Ylpertnodi 2y ago
  [flagged]
  - hedora 2y ago
    I would like to link to one of the many articles about it (singular), but all the articles (plural) about it ... Two example pages (both have lots of spoilers; the second is worse about that): https://www.reddit.com/r/Ghost_in_the_Shell/comments/og7ags/how_does_sustainable_war_work/ https://www.reddit.com/r/Ghost_in_the_Shell/comments/og7ags/... https://ghostintheshell.fandom.com/wiki/Sustainable_War https://ghostintheshell.fandom.com/wiki/Sustainable_War
  - Drygord 2y ago
    [flagged]
- 2y ago
wil421 2y ago
Airport staff need to be able to support them. Not HN types. Most people know how to use a windows computer. Most IT desktop support knows how to use and manage windows. Even building facilities folks can help support them. Microsoft makes it easy to manage a fleet of computers. They also provide first party (along with thousands of 3rd parties) training and certifications for it. Windows are the de facto Business Machines. Most signage companies use windows. Finding someone who knows a BSD is not easy.
- mkoubaa 2y ago
  Airport staff don't maintain infrastructure, at best they maintain front ends to it
  - wil421 2y ago
    You consider signage infra? Same with conference rooms. Most of the places I have worked have facilities type people working on it. Tier 3 is usually a direct phone call away for them You would send an engineer into an airport to reboot a sign?
    - mkoubaa 2y ago
      Of course not. Thats the point I was making
  - NegativeK 2y ago
    At some airports, staff does maintain infrastructure. At others, airline staff is responsible for it. And just like airport staff, a tech who can deal with Firefox on Windows is cheaper than someone who can troubleshoot the same in Linux or a more custom system.
- fifteen1506 2y ago
  Yup. Another take to be done here is: computers shouldn't have unfiltered internet access all the time. Whitelist it and once every 3 days open the internet gates. (Easier said than done)
- 2y ago
giancarlostoro 2y ago
> why aren't those built on Linux or even OpenBSD? Or even ChromeOS which has insane security. > but it still doesn't work. It works momentarily but there will always be 0-days the people who make the exploits intimately know the windows API internals.
- echoangle 2y ago
  > Or even ChromeOS ChromeOS is a Linux distro BTW
delfinom 2y ago
I'm sorry but even Linux requires frequent security updates due it's large ecosystem of dependencies. It's more or less required by every cyber security standard to update them just like windows.
- blablabla123 2y ago
  On the other hand OpenBSD doesn't require very frequent patching assuming a default install which comes with batteries included. For a web server there's just one relevant patch since April for 7.5: https://www.openbsd.org/errata75.html https://www.openbsd.org/errata75.html
  - citrin_ru 2y ago
    OpenBSD is a non-starter for many companies because they don't have LTS and releases are relative frequent.
- advael 2y ago
  I agree that all dependencies should be treated as attack surface. For that reason, systems for which dependencies can be more tightly controlled are inherently more secure than ones for which they can't. The monolithic and opaque nature of windows and other proprietary software makes them harder to minimize risk about in this way
- deleted 2y ago
  [deleted]
V__ 2y ago
Right now on the frontpage: 'CrowdStrike broke Debian and Rocky Linux months ago, but no one noticed' [1] https://news.ycombinator.com/item?id=41018029 https://news.ycombinator.com/item?id=41018029
marban 2y ago
Security is not a feature that can be layered on. There's an entire industry for guard-railing LLMs now. Go figure.
- advael 2y ago
  In the current economic environment, something doesn't have to be wise or even feasible to have an "industry"
politelemon 2y ago
> an OS that requires frequent security patches > Security is not a feature that can be layered on. It has to be built in This is a common misunderstanding, an OS that receives frequent security updates is a very good thing. That means attention is being paid to issues being raised, and risks are being mitigated. Security is not a 'checkbox' it's more of a neverending process because the environment is always in a state of flux. So to flip it, if an OS is not receiving updates, or not being updated frequently, that's not great. What you want is updates that don't destabilize an OS, and behind that is a huge history and layers of decisions at each 'shop' that runs these machines. Security is meant to be in layers and needs to be built in. > but it still doesn't work. It does work because the 'scene' has been silent for so long, but what we as humans notice is the incident where it didn't.
- dotancohen 2y ago
  Remote update is a nice way of saying remote code execution. It is really really hard to ensure that only the entity that you want to update your system, can update your system, when facing a state-funded adversary. Sometimes that state adversary might even work in concert with your OS vendor. That's before even addressing mistakes.
  - echoangle 2y ago
    Is that really so hard? Isn’t the problem mostly solved by signing your update and verifying the update at the client? As long as you can keep the private key secret, that should be enough, right? Or are we assuming you can’t keep a single key private from your adversary?
    - g_p 2y ago
      Yes, this is really hard. You could get a Solarwinds type situation where the adversary has the signing keys and ability to publish to the website. You might also find that the vendor ships a library (like libxz) as a part of their invisible or hidden supply chain, that is able to be compromised. You might find that one of the people working at the company makes a change to the code to enable remote access by the adversary in a targeted collaboration/attack. The problem isn't that signing key (although I could delve into the lengths you'd need to go to to keep that secret under these threat models) - the problem is what they sign. A signed end release binary or series of packages isn't going to address the software source code itself having something added, or the dependencies of it being compromised.
beefnugs 2y ago
Layering is absolutely possible, but more at the network layer than the individual computer layer. Minimal software and OS running on linux as a layer between any windows/whatever and internet connectivity. Minimize and control the exact information that gets to the less hardened and trustworthy/complicated computers
lr4444lr 2y ago
That's beyond their level of comprehension.
akira2501 2y ago
> I've seen photos of BSODs on airport monitors that show flight lists The kiosk display terminal is not something I care about that much. > We now have an entire industry dedicated to trying to layer security onto Windows Too bad we have no such layering in our networks, our internet connections, or in our authentication systems. Thinking about it another way there's actually no specific system in place to ensure your pilot does not show up drunk. We don't give them breathalyzers before the flight. We absolutely could do this even without significant disruption to current operations. We have no need to actually do this because we've layered so many other systems on top of your pilot that they all serve as redundant checks on their state of mind and current capabilities to safely conduct the flight. These checks are broader and tend to identify a wider range of issues anyways. This type of thinking is entirely missing at the computer network and human usability layer.
ta1243 2y ago
Those sorts of things just need to boot to a web browser in full screen with some watchdog software in the background, launching from a read only disk (or network image). Get a problem, just unplug it and plug it back in. Make it POE based so you can easily do it automatically, stick them on a couple of distros (maybe even half on bsd, half on linux, half using chrome, half on firefox)
- polski-g 2y ago
  A web browser is an unbelievably complex piece of software. So complex that there are now only two. And also so complex that there are weekly updates because there's so many security holes.
  - autoexec 2y ago
    > So complex that there are now only two There are more than two, and the vast majority of the time people don't need anywhere near the complexity that modern browsers have shoved into them. A lean browser that supported only a bare minimum of features would go a long way to reducing attack surface. As it is now, I already find myself disabling more and more functionality from my browsers (service workers, WebRTC, JS, SVG, webgl, PDF readers, prefetch, mathml, etc)
    - echoangle 2y ago
      There are more than 2 browsers, but only 2 rendering engines, which are the complicated part of the browser.
      - autoexec 2y ago
        More than two there too. For example: WebKit, Blink, Gecko, LibWeb, Servo, Goanna, Presto, and Libwww or whatever Lynx is using these days.
        Osiris 2y ago
        The first three cover basically 99% of browsers.
jijji 2y ago
every year multiple times per year there's reports of Microsoft Windows systems having either mass downtime or exploitation.... it's kind of amazing that critical systems would rely on something that causes so much frustration on a regular basis.... I've been running systems under Linux and Unix for decades and never had any down time... so I don't know I mean it's nice to know that Linux is pretty solid and always has been the worst that's ever happened has been like a process that might go down during an upgrade, but never the whole system.
balls187 2y ago
I've never managed linux IT departments--how well are the management tools compared to what Microsoft offers such as tooling for managing thousands of computers across hundreds of offices.
Drygord 2y ago
Linux is vulnerable too (but not as vulnerable as windows of course) it’s just not targeted by hackers because it’s market share is so small. That wouldn’t be the case if, say, half of all users ran Linux.
- makapuf 2y ago
  It's market share on servers (a juicy target) is not small at all.
  - Rinzler89 2y ago
    And that sees plenty of attacks too. But here Windows wasn't under attack or a Windows vulnerability exploited, CS just fucked up and companies were stupid enough to put all their trust in CS.
- smcleod 2y ago
  There are far more servers running linux/bsd than there are Windows.
pjmlp 2y ago
Sure, https://x.com/shantanugoel/status/1814567750289006686 https://x.com/shantanugoel/status/1814567750289006686
Rinzler89 2y ago
>Really interesting to me that none of the commentators I've seen in the press have even hinted that maybe an OS that requires frequent security patches shouldn't be used for infrastructure in the first place. Nobody's commenting on that because it's the wrong thing to focus on. 1) This fuckup was on CrowdStrike's Falcon tool (basically a rootkit) bricking Windows due to a bad kernel driver they pushed out without proper hygiene, not on Windows's security patches being bad. 2) Linux also needs to get patches all the time to be secure (remember XZ?) It's not just magically secure by default because of the chubby penguin but is only as secure as it's most vulnerable component, and XZ proved it has a lot of components. I'd be scared if a long period goes by and I see no security patches being pushed to my OS. Modern software is complex and vulnerabilities are everywhere. No OS is ever bug-free and fully bullet proof in order to believe it can be secure without regular patches. Other than TempleOS of course. The lesson is whichever OS you use, don't surrender your security to a single third party vendor who you now have to trust with the keys of your kingdom as that now becomes your single point of failure. Or if you do be sure you can sue them for the damages.
- citrin_ru 2y ago
  > Linux gets security patches all the time 1) While CrowdStrike can be run on Linux it is less of a risk to use Linux without it than Windows. I don't think most Linux/BSD boxes would benefit from it. It could be useful for a Linux with remotely accessible software of questionable quality (or a desktop working with untrusted files) but this should not be the case for any critical system. 2) There is a difference between auto-updates (common in Windows world) and updates triggered manually only when it is necessary (and after testing in non-prod environment). Also while Linux is far from being bug-free, remotely exploitable vulnerabilities are rare.
  - Rinzler89 2y ago
    >2) There is a difference between auto-updates (common in Windows world) and updates triggered manually only when it is necessary (and after testing in non-prod environment). Again, those auto updates that caused this issue were developed and pushed from Crowdstrike not from Windows. That tool does the same auto updates on Linux too. On Windows side you can have sys-admins delay Windows updates until they get tested in non-production instances, but again, this update was not pushed by Windows for sysadmins to be able to do anything about it.
citrin_ru 2y ago
For many CTO/CISO it is more important to have a good target to shift responsibility when things go awry than to have a reliable/secure system. A Big Brand is a good target, an open-source project like OpenBSD is not. I doubt any CTO will be fired for choosing Widnows+CrowdStrike (instead of Linux/BSD) despite many million losses. "Nobody ever gets fired for buying IBM" is as true as ever at least in the corporate world.
- deleted 2y ago
  [deleted]
- commercialnix 2y ago
  > I doubt any CTO will be fired for choosing Widnows+CrowdStrike (instead of Linux/BSD) I was personally involved in a meeting where my firm's leadership advised a client who did fire their CTO and a bunch of other people for what was ultimately putting what they thought were smart career moves over their actual responsibilities. Unfortunately, as you did just point out, the CEO, other execs, and board are often just as incompetent as the CTO/CISO who have such shit-brained mindset.
LVB 2y ago
To pick on your airport example a bit… all of the times I’ve gotten to enjoy a busted in-seat entertainment system, I’ve found myself staring at a stuck Linux boot process. This goes well beyond the OS.
- fxtentacle 2y ago
  It's typically Android.
  - echoangle 2y ago
    To clarify: Android is based on Linux.
    - pjmlp 2y ago
      To clarify, the Linux kernel, nothing else from GNU/Linux is exposed as official userspace API.
  - Osiris 2y ago
    The ones I've seen are definitely not Android. But I don't have any data to argue which OS is most common among in flight entertainment systems.
hi_hi 2y ago
I'm sure we've all heard the phrase "We're a Windows shop" in some variation. I understand the reasons for it, and why large, billion dollar companies try to create some sort of efficiency by centralising on one "vendor", but, then this happens. I don't know how to fix the problem of following "Industry Trends" when every layer above me in the organisation is telling me not to spend the time (money) to investigate alternative software choices which don't fit into their nice box.
- Osiris 2y ago
  The outage was not because of the OS. It was a kernel driver that attempted to use invalid memory. The same thing crash could happen with any kernel driver in any operating system. You've never seen Linux crash because of a driver bug?
  - hi_hi 2y ago
    Yes, I'm well aware. I wasn't trying to conflate a CrowdStrike problem with a Microsoft problem. Having said that, in this particular incident, the problems were specifically limited to Windows OS.
tester756 2y ago
>We now have an entire industry dedicated to trying to layer security onto Windows -- but it still doesn't work. What makes you think so? How is Linux better in that area?
dheera 2y ago
> why aren't those built on Linux or even OpenBSD? Because in the non-Silicon-Valley world of software, if you pick Linux and it has issues, fingers will get pointed at you. If you pick Windows and it has issues, fingers will get pointed at Microsoft.
Osiris 2y ago
Remember when operating systems only got updates through service packs? We moved to a more frequent update cycle because when a critical vulnerability was found, no one wanted to wait 6-12 months for the service pack.
tinytime 2y ago
There's no excuse in today's world to not write fantastic unit tests especially with LLMs. Plug for how we enable that here https://github.com/codeintegrity-ai/mutahunter https://github.com/codeintegrity-ai/mutahunter

dopylitty 2y ago

Or don't use an OS at all. We need to think about minimizing the use of software in critical infrastructure. If that means less efficiency because you have to be near something to maintain it then so be it. That would be good for jobs anyway.

antihero 2y ago
How exactly would a lot of end user systems function without one?
- advael 2y ago
  You can run single-purpose software on bare metal, and many OS-agnostic toolkits for things like user interfaces exist
  - sabas123 2y ago
    I'd like to pose two questions: 1. How does the software obtain new data at run time? 2. How do you make sure that thing doesn't pose a security hole when a vulnerability gets discovered? (assuming this never happens is unrealistic)
    - advael 2y ago
      The answer to both questions is robust organizational infrastructure. To be frank, I think a minimal linux system as a baseline OS serves most use cases better than a bare metal application, but many applications have self-contained update systems and can connect to networks. Self-repairable infrastructure is a necessity, both in terms of tooling and staffing, for any organization for which an outage or a breach could be catastrophic, and the rise of centralized, cloud-reliant infrastructure in these contexts should be seen as a massive and unacceptable risk for those organizations to take on. Organizations being subject to unpatched vulnerabilities and inability to manage their systems competently are direct results of replacing internal competency and purpose-built systems with general-purpose systems maintained and controlled by unaccountable distant tech monopolies
      - MetaWhirledPeas 2y ago
        > the rise of centralized, cloud-reliant infrastructure in these contexts should be seen as a massive and unacceptable risk for those organizations to take on I agree with you but I also want to play the devil's advocate: using software like CrowdStrike is not what I would call being "cloud-reliant". It's simply using highly-privileged software that appears to have the ability to update itself. And that is likely far more common than cloud-reliant setups.
        advael 2y ago
        Yea, and use of highly privileged software with the ability to update itself that the organization has no oversight of should be the most suspect. Software is used by nearly every organization for drastically different needs, and I think there will never be adequate security or reliability for any of them if software providers continue to consolidate, generalize, and retain ever more control of their offerings. Personally, I think the solution is local-first software, either open-source or grown within the organizations using them, which necessitates having that capability within orgs. The whole "buy all our infrastructure from some shady vendor" model is a recipe for disaster
    - ssivark 2y ago
      Vulnerabilities in what though? If you make an application so simple that it can only fetch data through an API and display, there's simply not much more that it can do. And a simple application is easy to audit. So it would be ideal if we could bundle this (akin to compiling) and deploy on bare metal.
  - antihero 2y ago
    Most of these OS need to run a variety of applications from different vendors though.
Osiris 2y ago
Even unikernel applications have an OS compiled into the application. It's necessary to initialize the hardware it's running on, including the CPU and GPU and storage. I suppose you could build it as a UEFI module that relies on the UEFI firmware to initialize the hardware but then you get a text only interface. But then the UEFI is the OS. But this outage was not an OS problem. It was an application bug that used invalid pointers. If it was a unikernel it still would have crashed.
- ScottBurson 2y ago
  I've read that it was a driver of some kind, not an application. Applications can't cause BSODs (I hope).

stefan_ 2y ago

I read the T&C of this CrowdStroke garbage and they have the usual blurb about not using it in critical industry. Maybe we just charge & arrest the people that put it there and this checkbox-software mess stops real quick.

AceyMan 2y ago
/set Devil's Advocate mode: from the reporting so far, no one has died as a result of the Crowdstrike botch. For my money, that sounds like it's not being used in 'critical industry'. /unset There were several 911 service outages included in the news yesterday, so I would definitely say agree those fall into the category. I haven't seen how many hospitals were deeply affected; I know there were several reports of facilities that were deferring any elective procedures.
- logbiscuitswave 2y ago
  I almost had to defer a procedure for one of my cats because my vet’s systems were all down. This meant they couldn’t process payments, schedule appointments, use their X-ray machine, or dispense prescriptions. (Thankfully, they had the ingenuity to get their diagnostic equipment online through other means, and our prescriptions had already been dispensed so we didn’t have to reschedule.) I would imagine it’s the same story at human hospitals too that ran afoul of this. I wouldn’t expect life-critical systems to go offline, but there’s many other more mundane systems that also need to function.
delfinom 2y ago
The public T&C is for small businesses. Any large business is going to be negotiating very different terms which are not public.

nullindividual 2y ago

> why aren't those built on Linux or even OpenBSD The vendor who makes the software has always written for Windows (or in reality, wrote for either DOS or OS/2 then transitioned to NT4). History, momentum, familiarity, cost, and ease of support all are factors (among others, I'm sure). Security is a process, not a product. And yes, distros require frequent updates, though more to your point, you can limit the scope of installed software. I'm sure airport displays don't need MPEG2, VP1 and so on codecs, for instance. It's also important to remember that there is a lot of 'garageware' out there with these specialized systems. Want SAML/OIDC support? We only support LDAP over cleartext, or Active Directory at best. Want the latest and greatest version of Apache Tomcat? Sorry, the vendor doesn't know how to troubleshoot either, so they only "support" a three year old vulnerable version. Ran into that more than a few times. Given the hypothesis of what caused the BSOD with Crowdstrike (NUL pointer), using a safe language would have been appropriate -- it's fairly easy in this case to lay the blame with CS. Microsoft supplies the shotgun. It's the vendors responsibility to point it away from themselves.

pwg 2y ago
> I'm sure airport displays don't need MPEG2, VP1 and so on codecs, for instance. They don't, until the day the airport managers are approached by an advertising company waving the wads of cash the airport could be 'earning' if only they let "AdCo" display, in the top 1/4 of each screen, a video advertising loop. At which point, those displays need the codecs for "AdCo's" video ads.
- nullindividual 2y ago
  Absolutely (sigh)! But with a deployment of devices like that, the operator has a solid central management system from which they could push software as-needed.
- westpfelia 2y ago
  Boy do I sure hate you for saying that. I mean at some point you are right. That is the future. But god am I mad at you for reminding me this is the world we live in.

hedora 2y ago

This sort of emergent behavior is a feature, not a bug. Operating systems that don't require frequent security patches aren't profitable. Anyway, this is the step of late-phase capitalism that comes after enshittification. Ghost in the Shell 2045 calls it "sustainable war". I'd link to an article, but they're all full of spoilers in the first paragraph. It probably suffices to say that the series refers to it as capitalism in its most elegant form: It is an economic device that can continue to function without any external inputs, and it has some sort of self-regulatory property that means the collateral damage it causes is just below the threshold where society collapses. In the case of Cloud Strike, the body count is low enough, and plausible deniability is low enough that the government can get away with not jailing anyone. Instead, the event will increase the money spent on security theater, and probably lead to a new regulatory framework that leads to yet-another layer of mandatory buggy security crapware (which Cloud Strike apparently is). In turn, that'll lower the margins of anyone that uses computers in the US by something like 0.1%, and that wealth will be transferred into the industry segment responsible for the debacle in the first place. Ideally, the next layer of garbage will have a bigger blast radius, allowing the computer security complex to siphon additional margins.

Ylpertnodi 2y ago
[flagged]
- hedora 2y ago
  I would like to link to one of the many articles about it (singular), but all the articles (plural) about it ... Two example pages (both have lots of spoilers; the second is worse about that): https://www.reddit.com/r/Ghost_in_the_Shell/comments/og7ags/how_does_sustainable_war_work/ https://www.reddit.com/r/Ghost_in_the_Shell/comments/og7ags/... https://ghostintheshell.fandom.com/wiki/Sustainable_War https://ghostintheshell.fandom.com/wiki/Sustainable_War
- Drygord 2y ago
  [flagged]
2y ago

wil421 2y ago

Airport staff need to be able to support them. Not HN types. Most people know how to use a windows computer. Most IT desktop support knows how to use and manage windows. Even building facilities folks can help support them. Microsoft makes it easy to manage a fleet of computers. They also provide first party (along with thousands of 3rd parties) training and certifications for it. Windows are the de facto Business Machines. Most signage companies use windows. Finding someone who knows a BSD is not easy.

mkoubaa 2y ago
Airport staff don't maintain infrastructure, at best they maintain front ends to it
- wil421 2y ago
  You consider signage infra? Same with conference rooms. Most of the places I have worked have facilities type people working on it. Tier 3 is usually a direct phone call away for them You would send an engineer into an airport to reboot a sign?
  - mkoubaa 2y ago
    Of course not. Thats the point I was making
- NegativeK 2y ago
  At some airports, staff does maintain infrastructure. At others, airline staff is responsible for it. And just like airport staff, a tech who can deal with Firefox on Windows is cheaper than someone who can troubleshoot the same in Linux or a more custom system.
fifteen1506 2y ago
Yup. Another take to be done here is: computers shouldn't have unfiltered internet access all the time. Whitelist it and once every 3 days open the internet gates. (Easier said than done)
2y ago

giancarlostoro 2y ago

> why aren't those built on Linux or even OpenBSD? Or even ChromeOS which has insane security. > but it still doesn't work. It works momentarily but there will always be 0-days the people who make the exploits intimately know the windows API internals.

echoangle 2y ago
> Or even ChromeOS ChromeOS is a Linux distro BTW

delfinom 2y ago

I'm sorry but even Linux requires frequent security updates due it's large ecosystem of dependencies. It's more or less required by every cyber security standard to update them just like windows.

blablabla123 2y ago
On the other hand OpenBSD doesn't require very frequent patching assuming a default install which comes with batteries included. For a web server there's just one relevant patch since April for 7.5: https://www.openbsd.org/errata75.html https://www.openbsd.org/errata75.html
- citrin_ru 2y ago
  OpenBSD is a non-starter for many companies because they don't have LTS and releases are relative frequent.
advael 2y ago
I agree that all dependencies should be treated as attack surface. For that reason, systems for which dependencies can be more tightly controlled are inherently more secure than ones for which they can't. The monolithic and opaque nature of windows and other proprietary software makes them harder to minimize risk about in this way
deleted 2y ago
[deleted]

V__ 2y ago

Right now on the frontpage: 'CrowdStrike broke Debian and Rocky Linux months ago, but no one noticed' [1] https://news.ycombinator.com/item?id=41018029 https://news.ycombinator.com/item?id=41018029

marban 2y ago

Security is not a feature that can be layered on. There's an entire industry for guard-railing LLMs now. Go figure.

advael 2y ago
In the current economic environment, something doesn't have to be wise or even feasible to have an "industry"

politelemon 2y ago

> an OS that requires frequent security patches > Security is not a feature that can be layered on. It has to be built in This is a common misunderstanding, an OS that receives frequent security updates is a very good thing. That means attention is being paid to issues being raised, and risks are being mitigated. Security is not a 'checkbox' it's more of a neverending process because the environment is always in a state of flux. So to flip it, if an OS is not receiving updates, or not being updated frequently, that's not great. What you want is updates that don't destabilize an OS, and behind that is a huge history and layers of decisions at each 'shop' that runs these machines. Security is meant to be in layers and needs to be built in. > but it still doesn't work. It does work because the 'scene' has been silent for so long, but what we as humans notice is the incident where it didn't.

dotancohen 2y ago
Remote update is a nice way of saying remote code execution. It is really really hard to ensure that only the entity that you want to update your system, can update your system, when facing a state-funded adversary. Sometimes that state adversary might even work in concert with your OS vendor. That's before even addressing mistakes.
- echoangle 2y ago
  Is that really so hard? Isn’t the problem mostly solved by signing your update and verifying the update at the client? As long as you can keep the private key secret, that should be enough, right? Or are we assuming you can’t keep a single key private from your adversary?
  - g_p 2y ago
    Yes, this is really hard. You could get a Solarwinds type situation where the adversary has the signing keys and ability to publish to the website. You might also find that the vendor ships a library (like libxz) as a part of their invisible or hidden supply chain, that is able to be compromised. You might find that one of the people working at the company makes a change to the code to enable remote access by the adversary in a targeted collaboration/attack. The problem isn't that signing key (although I could delve into the lengths you'd need to go to to keep that secret under these threat models) - the problem is what they sign. A signed end release binary or series of packages isn't going to address the software source code itself having something added, or the dependencies of it being compromised.

beefnugs 2y ago

Layering is absolutely possible, but more at the network layer than the individual computer layer. Minimal software and OS running on linux as a layer between any windows/whatever and internet connectivity. Minimize and control the exact information that gets to the less hardened and trustworthy/complicated computers

lr4444lr 2y ago

That's beyond their level of comprehension.

akira2501 2y ago

> I've seen photos of BSODs on airport monitors that show flight lists The kiosk display terminal is not something I care about that much. > We now have an entire industry dedicated to trying to layer security onto Windows Too bad we have no such layering in our networks, our internet connections, or in our authentication systems. Thinking about it another way there's actually no specific system in place to ensure your pilot does not show up drunk. We don't give them breathalyzers before the flight. We absolutely could do this even without significant disruption to current operations. We have no need to actually do this because we've layered so many other systems on top of your pilot that they all serve as redundant checks on their state of mind and current capabilities to safely conduct the flight. These checks are broader and tend to identify a wider range of issues anyways. This type of thinking is entirely missing at the computer network and human usability layer.

ta1243 2y ago

Those sorts of things just need to boot to a web browser in full screen with some watchdog software in the background, launching from a read only disk (or network image). Get a problem, just unplug it and plug it back in. Make it POE based so you can easily do it automatically, stick them on a couple of distros (maybe even half on bsd, half on linux, half using chrome, half on firefox)

polski-g 2y ago
A web browser is an unbelievably complex piece of software. So complex that there are now only two. And also so complex that there are weekly updates because there's so many security holes.
- autoexec 2y ago
  > So complex that there are now only two There are more than two, and the vast majority of the time people don't need anywhere near the complexity that modern browsers have shoved into them. A lean browser that supported only a bare minimum of features would go a long way to reducing attack surface. As it is now, I already find myself disabling more and more functionality from my browsers (service workers, WebRTC, JS, SVG, webgl, PDF readers, prefetch, mathml, etc)
  - echoangle 2y ago
    There are more than 2 browsers, but only 2 rendering engines, which are the complicated part of the browser.
    - autoexec 2y ago
      More than two there too. For example: WebKit, Blink, Gecko, LibWeb, Servo, Goanna, Presto, and Libwww or whatever Lynx is using these days.
      - Osiris 2y ago
        The first three cover basically 99% of browsers.

jijji 2y ago

every year multiple times per year there's reports of Microsoft Windows systems having either mass downtime or exploitation.... it's kind of amazing that critical systems would rely on something that causes so much frustration on a regular basis.... I've been running systems under Linux and Unix for decades and never had any down time... so I don't know I mean it's nice to know that Linux is pretty solid and always has been the worst that's ever happened has been like a process that might go down during an upgrade, but never the whole system.

balls187 2y ago

I've never managed linux IT departments--how well are the management tools compared to what Microsoft offers such as tooling for managing thousands of computers across hundreds of offices.

Drygord 2y ago

Linux is vulnerable too (but not as vulnerable as windows of course) it’s just not targeted by hackers because it’s market share is so small. That wouldn’t be the case if, say, half of all users ran Linux.

makapuf 2y ago
It's market share on servers (a juicy target) is not small at all.
- Rinzler89 2y ago
  And that sees plenty of attacks too. But here Windows wasn't under attack or a Windows vulnerability exploited, CS just fucked up and companies were stupid enough to put all their trust in CS.
smcleod 2y ago
There are far more servers running linux/bsd than there are Windows.

pjmlp 2y ago

Sure, https://x.com/shantanugoel/status/1814567750289006686 https://x.com/shantanugoel/status/1814567750289006686

Rinzler89 2y ago

>Really interesting to me that none of the commentators I've seen in the press have even hinted that maybe an OS that requires frequent security patches shouldn't be used for infrastructure in the first place. Nobody's commenting on that because it's the wrong thing to focus on. 1) This fuckup was on CrowdStrike's Falcon tool (basically a rootkit) bricking Windows due to a bad kernel driver they pushed out without proper hygiene, not on Windows's security patches being bad. 2) Linux also needs to get patches all the time to be secure (remember XZ?) It's not just magically secure by default because of the chubby penguin but is only as secure as it's most vulnerable component, and XZ proved it has a lot of components. I'd be scared if a long period goes by and I see no security patches being pushed to my OS. Modern software is complex and vulnerabilities are everywhere. No OS is ever bug-free and fully bullet proof in order to believe it can be secure without regular patches. Other than TempleOS of course. The lesson is whichever OS you use, don't surrender your security to a single third party vendor who you now have to trust with the keys of your kingdom as that now becomes your single point of failure. Or if you do be sure you can sue them for the damages.

citrin_ru 2y ago
> Linux gets security patches all the time 1) While CrowdStrike can be run on Linux it is less of a risk to use Linux without it than Windows. I don't think most Linux/BSD boxes would benefit from it. It could be useful for a Linux with remotely accessible software of questionable quality (or a desktop working with untrusted files) but this should not be the case for any critical system. 2) There is a difference between auto-updates (common in Windows world) and updates triggered manually only when it is necessary (and after testing in non-prod environment). Also while Linux is far from being bug-free, remotely exploitable vulnerabilities are rare.
- Rinzler89 2y ago
  >2) There is a difference between auto-updates (common in Windows world) and updates triggered manually only when it is necessary (and after testing in non-prod environment). Again, those auto updates that caused this issue were developed and pushed from Crowdstrike not from Windows. That tool does the same auto updates on Linux too. On Windows side you can have sys-admins delay Windows updates until they get tested in non-production instances, but again, this update was not pushed by Windows for sysadmins to be able to do anything about it.

citrin_ru 2y ago

For many CTO/CISO it is more important to have a good target to shift responsibility when things go awry than to have a reliable/secure system. A Big Brand is a good target, an open-source project like OpenBSD is not. I doubt any CTO will be fired for choosing Widnows+CrowdStrike (instead of Linux/BSD) despite many million losses. "Nobody ever gets fired for buying IBM" is as true as ever at least in the corporate world.

deleted 2y ago
[deleted]
commercialnix 2y ago
> I doubt any CTO will be fired for choosing Widnows+CrowdStrike (instead of Linux/BSD) I was personally involved in a meeting where my firm's leadership advised a client who did fire their CTO and a bunch of other people for what was ultimately putting what they thought were smart career moves over their actual responsibilities. Unfortunately, as you did just point out, the CEO, other execs, and board are often just as incompetent as the CTO/CISO who have such shit-brained mindset.

LVB 2y ago

To pick on your airport example a bit… all of the times I’ve gotten to enjoy a busted in-seat entertainment system, I’ve found myself staring at a stuck Linux boot process. This goes well beyond the OS.

fxtentacle 2y ago
It's typically Android.
- echoangle 2y ago
  To clarify: Android is based on Linux.
  - pjmlp 2y ago
    To clarify, the Linux kernel, nothing else from GNU/Linux is exposed as official userspace API.
- Osiris 2y ago
  The ones I've seen are definitely not Android. But I don't have any data to argue which OS is most common among in flight entertainment systems.

hi_hi 2y ago

I'm sure we've all heard the phrase "We're a Windows shop" in some variation. I understand the reasons for it, and why large, billion dollar companies try to create some sort of efficiency by centralising on one "vendor", but, then this happens. I don't know how to fix the problem of following "Industry Trends" when every layer above me in the organisation is telling me not to spend the time (money) to investigate alternative software choices which don't fit into their nice box.

Osiris 2y ago
The outage was not because of the OS. It was a kernel driver that attempted to use invalid memory. The same thing crash could happen with any kernel driver in any operating system. You've never seen Linux crash because of a driver bug?
- hi_hi 2y ago
  Yes, I'm well aware. I wasn't trying to conflate a CrowdStrike problem with a Microsoft problem. Having said that, in this particular incident, the problems were specifically limited to Windows OS.

tester756 2y ago

>We now have an entire industry dedicated to trying to layer security onto Windows -- but it still doesn't work. What makes you think so? How is Linux better in that area?

dheera 2y ago

> why aren't those built on Linux or even OpenBSD? Because in the non-Silicon-Valley world of software, if you pick Linux and it has issues, fingers will get pointed at you. If you pick Windows and it has issues, fingers will get pointed at Microsoft.

Osiris 2y ago

Remember when operating systems only got updates through service packs? We moved to a more frequent update cycle because when a critical vulnerability was found, no one wanted to wait 6-12 months for the service pack.

tinytime 2y ago

There's no excuse in today's world to not write fantastic unit tests especially with LLMs. Plug for how we enable that here https://github.com/codeintegrity-ai/mutahunter https://github.com/codeintegrity-ai/mutahunter