5 ms·

>"Use a build tool to do dependency checks!" > >And then what? > >"Only update libraries if they have vulnerabilities." > >So, should this be a manual process w

>"Use a build tool to do dependency checks!" > >And then what? > >"Only update libraries if they have vulnerabilities." > >So, should this be a manual process where I have to dig through obscure warnings every time I build something? > >"No, you can automate it!" > > Congrats, you've just outsourced the decision making process about which versions of libraries to use to some 3d party. You've also made your build process reliant on yet another online service. Sorry, this is a total strawman. Automation doesn't need to be integrated with the build in such a way that it creates noise, and it doesn't mean you need to let changes be applied without human intervention. I'm hooked up with Snyk which scans all of my dependencies and sends me a summary email if it discovers any vulnerabilities. It creates PRs that can patch dependency vulnerabilities, and I can apply them if I approve of the change. If I don't think the change is worth it or the vulnerability is relevant, I can go to the Snyk site and mark a vulnerability to be ignored, or even tell it to ignore it for a month or whatever. I'm sure Snyk isn't unique, either. Yes, a human should be involved in decisions. No, it doesn't need to be noisy. I'm on Node.js and working on multiple projects with crazy numbers of dependencies, and the number of "new vulnerabilities" I see is typically a few per month at most. Some months there are no new notifications at all. It simply doesn't change that quickly, and it's not constant noise unless you ignore the notifications, and that's just bad software engineering.

bastardoperator 4y ago
This is basically what Dependabot on GitHub does too. If it finds a vulnerability it creates a PR that you can decide to apply. It's automated to the extent that it's easy but it's not making or forcing changes on you. A human is still in control.
- SomeCallMeTim 4y ago
  True enough, though Snyk will sometimes offer a patch rather than a PR to a new version of the dependency. The Snyk tools apply to the patch directly to (e.g.) `node_modules` after the latest code has been downloaded.
kodah 4y ago
I think people's experiences will vary with tools like this due to company policy. For instance, your company allows you to make decisions about whether a vulnerability pertains to you, but many don't. I think that's how these tools become "noise", because the only option an engineer is given by leadership is to comply to the tool. At this point, I think a lot of problems in software are actually misaligned or misincentivized policy gone awry. When you read about them on forums they look technical, but they're not.
- SomeCallMeTim 4y ago
  You can't use a technical solution to fix a management problem, no. :)
taeric 4y ago
I don't see how you have really fixed the break much. Especially if you can "push back" and say no to an update, you are just setting yourself up for when the update is even harder to accept. (That is, the further behind on an update you are, the harder to take it. Typically.) And then there is the "I want the fix, but I don't want the added attack vector of features that comes with it."
- SomeCallMeTim 4y ago
  99% of the time it's a minor update patch that you really should just apply, or Snyk will add a patch to the build to fix the security hole directly. Honestly it's just not worth worrying about whether a security hole is exploitable when the decision is whether to accept a minor update patch. Just accept the patch PR and move on. The times you need to do a major update to get a patch fix it's equally important to just make that upgrade, since that means the ancient version you're using is likely out of maintenance and likely is throwing warnings that you're using a deprecated version. Yes, it's more work to perform a major upgrade. But that doesn't mean it isn't good engineering to keep your tools up to date.
- kelnos 4y ago
  Sure, but you're in control, and can decide to do what's best for you and your organization. Taking on the technical debt of allowing your dependencies to get further and further behind isn't a decision I usually make, but it can be a valid one depending on the individual circumstances.
  - taeric 4y ago
    I mean, you aren't wrong. But this is akin to saying you don't need a memory safe language, because you can decide what is best for your applications memory needs. That is, I think the ask is more of "why haven't we come up with something that is a bit better at mitigating risks, here?"
    - SomeCallMeTim 4y ago
      Software engineering is complex. Some of the challenges are simply irreducible. I believe this is one of them.
aahortwwy 4y ago
> If I don't think the change is worth it or the vulnerability is relevant, I can go to the Snyk site and mark a vulnerability to be ignored, or even tell it to ignore it for a month or whatever. I am currently fighting with our security team because they refuse to allow developers to do this, and instead require my team to file tickets with their team, wait hours for that ticket to be picked up, explain why a thing is a false positive to a person who doesn't understand the codebase they're working on, and hope that person feels that the finding should be ignored. I do not work at a large company. This is greatly exacerbated by the fact that Snyk - like all package scanners - primarily flags false positives. When you start digging in to what it flags you'll find that a lot of the vulnerabilities aren't reasonably exploitable, or the impact is questionable, or you can't repro the issue. Those that do legitimately affect the underlying library often don't affect your usage of that library. I'd estimate our false positive rate is >90%. So when we get a finding our options are to go through that costly and disruptive process I mentioned above, or just upgrade the dependency (which carries associated risk of breaking shit in production). These tools emphasize the quantity of findings they produce, and quietly ignore the quality. As soon as you have any divergence in motivations or incentives between the people running the scanner and the people being blocked by the scanner those low quality findings become incredibly costly.
- freeqaz 4y ago
  This is compliance vs security. Finding vulns checks a box for SOC2, but in reality detection is the easy part. Figuring out what to fix, based on real-world usage and risk, requires much more work and is often ignored. I'm sorry you're on the receiving end of this problem! Shill notice: I'm working on an Open Source tool[0] that makes this problem less horrible. My colleague wrote a post about our hypothesis[1] about how we can avoid this false positive trap. I'd love to chat with anybody feeling this pain (even just as therapy lol). 0: https://github.com/lunasec-io/lunasec https://github.com/lunasec-io/lunasec 1: https://www.lunasec.io/docs/blog/the-issue-with-vuln-scanners/ https://www.lunasec.io/docs/blog/the-issue-with-vuln-scanner...

bastardoperator 4y ago

This is basically what Dependabot on GitHub does too. If it finds a vulnerability it creates a PR that you can decide to apply. It's automated to the extent that it's easy but it's not making or forcing changes on you. A human is still in control.

SomeCallMeTim 4y ago
True enough, though Snyk will sometimes offer a patch rather than a PR to a new version of the dependency. The Snyk tools apply to the patch directly to (e.g.) `node_modules` after the latest code has been downloaded.

kodah 4y ago

I think people's experiences will vary with tools like this due to company policy. For instance, your company allows you to make decisions about whether a vulnerability pertains to you, but many don't. I think that's how these tools become "noise", because the only option an engineer is given by leadership is to comply to the tool. At this point, I think a lot of problems in software are actually misaligned or misincentivized policy gone awry. When you read about them on forums they look technical, but they're not.

SomeCallMeTim 4y ago
You can't use a technical solution to fix a management problem, no. :)

taeric 4y ago

I don't see how you have really fixed the break much. Especially if you can "push back" and say no to an update, you are just setting yourself up for when the update is even harder to accept. (That is, the further behind on an update you are, the harder to take it. Typically.) And then there is the "I want the fix, but I don't want the added attack vector of features that comes with it."

SomeCallMeTim 4y ago
99% of the time it's a minor update patch that you really should just apply, or Snyk will add a patch to the build to fix the security hole directly. Honestly it's just not worth worrying about whether a security hole is exploitable when the decision is whether to accept a minor update patch. Just accept the patch PR and move on. The times you need to do a major update to get a patch fix it's equally important to just make that upgrade, since that means the ancient version you're using is likely out of maintenance and likely is throwing warnings that you're using a deprecated version. Yes, it's more work to perform a major upgrade. But that doesn't mean it isn't good engineering to keep your tools up to date.
kelnos 4y ago
Sure, but you're in control, and can decide to do what's best for you and your organization. Taking on the technical debt of allowing your dependencies to get further and further behind isn't a decision I usually make, but it can be a valid one depending on the individual circumstances.
- taeric 4y ago
  I mean, you aren't wrong. But this is akin to saying you don't need a memory safe language, because you can decide what is best for your applications memory needs. That is, I think the ask is more of "why haven't we come up with something that is a bit better at mitigating risks, here?"
  - SomeCallMeTim 4y ago
    Software engineering is complex. Some of the challenges are simply irreducible. I believe this is one of them.

aahortwwy 4y ago

> If I don't think the change is worth it or the vulnerability is relevant, I can go to the Snyk site and mark a vulnerability to be ignored, or even tell it to ignore it for a month or whatever. I am currently fighting with our security team because they refuse to allow developers to do this, and instead require my team to file tickets with their team, wait hours for that ticket to be picked up, explain why a thing is a false positive to a person who doesn't understand the codebase they're working on, and hope that person feels that the finding should be ignored. I do not work at a large company. This is greatly exacerbated by the fact that Snyk - like all package scanners - primarily flags false positives. When you start digging in to what it flags you'll find that a lot of the vulnerabilities aren't reasonably exploitable, or the impact is questionable, or you can't repro the issue. Those that do legitimately affect the underlying library often don't affect your usage of that library. I'd estimate our false positive rate is >90%. So when we get a finding our options are to go through that costly and disruptive process I mentioned above, or just upgrade the dependency (which carries associated risk of breaking shit in production). These tools emphasize the quantity of findings they produce, and quietly ignore the quality. As soon as you have any divergence in motivations or incentives between the people running the scanner and the people being blocked by the scanner those low quality findings become incredibly costly.

freeqaz 4y ago
This is compliance vs security. Finding vulns checks a box for SOC2, but in reality detection is the easy part. Figuring out what to fix, based on real-world usage and risk, requires much more work and is often ignored. I'm sorry you're on the receiving end of this problem! Shill notice: I'm working on an Open Source tool[0] that makes this problem less horrible. My colleague wrote a post about our hypothesis[1] about how we can avoid this false positive trap. I'd love to chat with anybody feeling this pain (even just as therapy lol). 0: https://github.com/lunasec-io/lunasec https://github.com/lunasec-io/lunasec 1: https://www.lunasec.io/docs/blog/the-issue-with-vuln-scanners/ https://www.lunasec.io/docs/blog/the-issue-with-vuln-scanner...