6 ms·

I am running a box built with Fedora Core 4 (2007 vintage). Never patch any systems. Why would I? If I am running a service facing the internet, it's custom bu

I am running a box built with Fedora Core 4 (2007 vintage). Never patch any systems. Why would I? If I am running a service facing the internet, it's custom built and patches would do it no good. Why would I wait for vendor to release a patch? If a service is external, I will watch out for vulnerabilities and rebuild ASAP before any patches are out. Besides, 90% of the time my custom build is not even vulnerable to a particular problem. If I am NOT running a service, why would I care about patches for it? Why would I wholesale patch a server anyway? If somebody breaks in and gets a local shell, all is lost anyway. If they are not in, they are dealing with externally facing services only, see above. There are specific and counted number of daemons on every machine. This whole patch-update thing is misguided and for people that want assurances and no responsibility.

ars 13y ago
Please tell me you are joking. The is no way you could possibly keep up with every vulnerability of everything installed on your server. > If somebody breaks in and gets a local shell, all is lost anyway. That is not true at all. You should run your server such that someone could get a shell running as the apache user - and still be able to do very little. They could read files and the database (which is bad), but not modify any files (which would be worse).
- andrewflnr 13y ago
  WRT local shells, it might well be a good idea to assume someone who got a shell as apache could use a privilege escalation 0-day and do some more damage. Hopefully your deployment process is such that starting from scratch isn't a huge hardship. I'd appreciate of someone with actual security experience (not me) weighed in...
  - foohbarbaz 13y ago
    It's very naive to think that you can protect a box from somebody with a local shell. Never worked.
- foohbarbaz 13y ago
  Of course I can and will keep up with every vulnerability for every service that is running and facing the Internet. I do not accept the risk of waiting for some vendor to release a patch. If there's a hole, read the report, determine whether your config/build is vulnerable, rebuild. Why would want to patch something you are not running or use?
andrewflnr 13y ago
I can't make head or tail of this comment. Are you saying you think it's a bad idea to keep my Linux kernel and Nginx up to date? What good does it do to "rebuild ASAP" unless you've at least downloaded source updates from the developers? Or are you telling me you write your own security fixes for all the software you use in public-facing services?
- foohbarbaz 13y ago
  Of course it's a bad idea to make unnecessary system changes (install patches) that bring system to essentially unknown state that nobody ever tested (the order and set of patches installed over your specific OS configuration). You only patch what you need to patch. Most of the time for every production service you end up building a custom version anyway. Patching does no good to those. So, by patching you only bring potential harm and overhead of going through change control processes.
brownbat 13y ago
As a security researcher, this approach just confounds me. I've never had an update break my system, and if someone pushed updates that were broken, I wouldn't trust any old versions of their software any more than the current one. And we keep finding that people don't update and miss critical vulnerabilities. There may be some admins out there that can independently track and patch every known vulnerability... but that seems like an impossible task for a box with any nontrivial amount of software on it. And a lot of vulnerabilities aren't widely released. Updates sometimes coincidentally break zero days that were never publicly revealed. I remember the world where everyone stubbornly refused to leave early versions of IE. Massive problem for security. The Chrome team looked at that and made the call to move to automatic updates. I'm still pretty convinced that's a better world. You want to run a small box that barely faces the internet where you constantly write your own patches in parallel with the primary software developers, while also researching and patching new vulnerabilities before they are deployed, go for it... but when that becomes the industry norm, I consider it extremely harmful. Maybe you can pull that off, but most people are not nearly that cool.
- cpncrunch 13y ago
  I think you misunderstand. It's not that people are pushing out crap updates. Rather, the problem is that when you update one thing on linux you usually end up having to update 100 other things. I'm in a similar position to the OP, in that I don't generally update linux systems. The problem is that there is no way to simply 'update everything' in linux (at least, not in Centos). yum update certainly doesn't do it - in Centos 5.5 it only gets you php 5.1.x. To get a newer version you have to update it manually or bodge yum. Then the problem is that many newer packages require a newer glibc or whatever, and that is something that can break your entire system very easily. I think the root of the problem is that linux isn't very easy to update, unlike Windows. As long as your linux system is well locked down and you regularly keep an eye on it, I don't see a problem with not updating regularly.
  - brownbat 13y ago
    That makes a lot of sense, thanks. I had always sort of seen Linux as easier to update, since it's a single command, but you're right... that command doesn't necessarily get you all the way. Things are going to vary from distro to distro, and none of them will necessarily roll in the bleeding edge version of whatever thing you want the day it launches. And then, custom code is vital on a lot of machines for a lot of applications, and it will introduce its own dependencies. That said, these factors really complicate security advice on patch management. If customers could be trusted to lock things down and keep an eye on them, that would be a much better world. And I'm sure a lot of admins out there are more than capable, but I worry about the Dunning Krueger effect catching some admins off guard. But ultimately, this is just a battle of emphasis more than disagreement. The answer isn't "everyone should always patch everything," it just depends on a lot of factors.

ars 13y ago

Please tell me you are joking. The is no way you could possibly keep up with every vulnerability of everything installed on your server. > If somebody breaks in and gets a local shell, all is lost anyway. That is not true at all. You should run your server such that someone could get a shell running as the apache user - and still be able to do very little. They could read files and the database (which is bad), but not modify any files (which would be worse).

andrewflnr 13y ago
WRT local shells, it might well be a good idea to assume someone who got a shell as apache could use a privilege escalation 0-day and do some more damage. Hopefully your deployment process is such that starting from scratch isn't a huge hardship. I'd appreciate of someone with actual security experience (not me) weighed in...
- foohbarbaz 13y ago
  It's very naive to think that you can protect a box from somebody with a local shell. Never worked.
foohbarbaz 13y ago
Of course I can and will keep up with every vulnerability for every service that is running and facing the Internet. I do not accept the risk of waiting for some vendor to release a patch. If there's a hole, read the report, determine whether your config/build is vulnerable, rebuild. Why would want to patch something you are not running or use?

andrewflnr 13y ago

I can't make head or tail of this comment. Are you saying you think it's a bad idea to keep my Linux kernel and Nginx up to date? What good does it do to "rebuild ASAP" unless you've at least downloaded source updates from the developers? Or are you telling me you write your own security fixes for all the software you use in public-facing services?

foohbarbaz 13y ago
Of course it's a bad idea to make unnecessary system changes (install patches) that bring system to essentially unknown state that nobody ever tested (the order and set of patches installed over your specific OS configuration). You only patch what you need to patch. Most of the time for every production service you end up building a custom version anyway. Patching does no good to those. So, by patching you only bring potential harm and overhead of going through change control processes.

brownbat 13y ago

As a security researcher, this approach just confounds me. I've never had an update break my system, and if someone pushed updates that were broken, I wouldn't trust any old versions of their software any more than the current one. And we keep finding that people don't update and miss critical vulnerabilities. There may be some admins out there that can independently track and patch every known vulnerability... but that seems like an impossible task for a box with any nontrivial amount of software on it. And a lot of vulnerabilities aren't widely released. Updates sometimes coincidentally break zero days that were never publicly revealed. I remember the world where everyone stubbornly refused to leave early versions of IE. Massive problem for security. The Chrome team looked at that and made the call to move to automatic updates. I'm still pretty convinced that's a better world. You want to run a small box that barely faces the internet where you constantly write your own patches in parallel with the primary software developers, while also researching and patching new vulnerabilities before they are deployed, go for it... but when that becomes the industry norm, I consider it extremely harmful. Maybe you can pull that off, but most people are not nearly that cool.

cpncrunch 13y ago
I think you misunderstand. It's not that people are pushing out crap updates. Rather, the problem is that when you update one thing on linux you usually end up having to update 100 other things. I'm in a similar position to the OP, in that I don't generally update linux systems. The problem is that there is no way to simply 'update everything' in linux (at least, not in Centos). yum update certainly doesn't do it - in Centos 5.5 it only gets you php 5.1.x. To get a newer version you have to update it manually or bodge yum. Then the problem is that many newer packages require a newer glibc or whatever, and that is something that can break your entire system very easily. I think the root of the problem is that linux isn't very easy to update, unlike Windows. As long as your linux system is well locked down and you regularly keep an eye on it, I don't see a problem with not updating regularly.
- brownbat 13y ago
  That makes a lot of sense, thanks. I had always sort of seen Linux as easier to update, since it's a single command, but you're right... that command doesn't necessarily get you all the way. Things are going to vary from distro to distro, and none of them will necessarily roll in the bleeding edge version of whatever thing you want the day it launches. And then, custom code is vital on a lot of machines for a lot of applications, and it will introduce its own dependencies. That said, these factors really complicate security advice on patch management. If customers could be trusted to lock things down and keep an eye on them, that would be a much better world. And I'm sure a lot of admins out there are more than capable, but I worry about the Dunning Krueger effect catching some admins off guard. But ultimately, this is just a battle of emphasis more than disagreement. The answer isn't "everyone should always patch everything," it just depends on a lot of factors.