7 ms·

Pentests where people actually get out of bed to do stuff (read code, read API docs etc) and then try to really hack your system are rare. Pentests where people

by mymac 3y ago

Pentests where people actually get out of bed to do stuff (read code, read API docs etc) and then try to really hack your system are rare. Pentests where people go through the motions, send you report with a few unimportant bits highlit while patting you on the back for your exemplary security so you can check the box on whatever audit you're going through are common.

iamflimflam1 3y ago
Yep, most pentests go through the OWASP list and call it done.
- ganoushoreilly 3y ago
  The problem is that is what most companies want. They don't want to spend the money nor get the feedback beyond "Best case standards". It's a calculated risk.
- fomine3 3y ago
  It's great if it's done exhaustively
- Faelian2 3y ago
  Honestly, the OWASP top ten is generic enough that most vulnerability fit in it : "injection", "security misconfiguration", "insecure design". The problem is 1. knowing the gazillion of web vulnerabilities, and technologies 2. being good enough to tests them 3. kick yourself and go through the laborious process of understand and test every key feature of the target.
j245 3y ago
From my understanding as a non security expert: Pentest comes across more as checking all the common attack vectors don’t exist. Getting out of bed to do the so-called “real stuff” is typically called a bug bounty program or security researching. Both exist and I don’t see why most companies couldn’t start a bug bounty program if they really cared a lot about the “real stuff”
- mymac 3y ago
  > From my understanding as a non security expert: That certainly helps.
  - j245 3y ago
    What a shame, HackerNews typically has more insightful comments than garbage like this. Edit: thanks to everyone who wrote some insightful responses, and there are indeed many. Faith in HackerNews restored !
  - bee_rider 3y ago
    People are going to chit-chat about things only tangentially related to their areas of expertise; it is good when we’re honest about our limitations. If nothing else, an obviously wrong take is a nice setup for a correction.
- NegativeK 3y ago
  The checkbox form exists because crooked vendors are catering to organizations who are intentionally lazy about their cybersecurity. Real penetration tests provide valuable insight that a bug bounty program won't.
- prmoustache 3y ago
  pentest means penetration testing which mean one need to take the attacker hat and try to enter your network or the app infrastructure and get as much data as he can, be it institutionnal or customer data. It can be through technical means as well as social engineering practices. And then report back. This is in no way related to a bug bounty program.
  - nbk_2000 3y ago
    Counter point: Most of the top rated Bug Bounty hunters have a background in penetration testing. I think it's more accurate to say Bug Bounty only covers a small subset of penetration testing (mainly in that escalation and internal pivoting are against the BB policy of most companies).
_jal 3y ago
Let me tell you about the laptop connected to our network with a cellular antenna we found in a locked filing cabinet after getting a much-delayed forced-door alert. This, after some social engineering attempts that displayed unnerving familiarity with employees and a lot of virtual doorknob-rattling. They may be rare, but "real" pentests are still a thing.
- mymac 3y ago
  Ouch. How did that ended up?
  - deleted 3y ago
    [deleted]
nbk_2000 3y ago
If you're a large company that's actually serious about security, you'll have a Red Team that is intimately familiar with your tech stacks, procedures, business model, etc. This team will be far better at emulating motivated attackers (as well as providing bespoke mitigation advice, vetting and testing solutions, etc.). Unfortunately, compliance/customer requirements often stipulate having penetration tests performed by third parties. So for business reasons, these same companies, will also hire low-quality pen-tests from "check-box pen-test" firms. So when you see that $10K "complete pen-test" being advertised as being used by [INSERT BIG SERIOUS NAME HERE], good chance this is why.
- pixl97 3y ago
  Ugh, in the work I do I run into so much of this kind of stuff. Customer: "We had a pentest/security scan/whatever find this issue in your software" Me: "And they realized that mitigations are in place as per the CVE that keep that issue from being an exploitable issue, right" Customer: "Uhhhh" Testing group: "Use smaller words please, we only click some buttons and this is the report that gets generated"
evntdrvn 3y ago
what I always want to know when people talk about this is "what reputable companies can I actually pay to do a real pentest (without costing hundreds of thousands of dollars)."
- pnt12 3y ago
  I think hiring a security specialist is the way to go.
- amlozano 3y ago
  The problem is security is a "Market for lemons" https://en.wikipedia.org/wiki/The_Market_for_Lemons https://en.wikipedia.org/wiki/The_Market_for_Lemons. Just like when trying to buy a used car, you need someone who is basically an expert in selling used cars. In order to purchase a reputable pentest, you basically have to have a security team that is mature enough to have just done it themselves. I can throw out some names for some reputable firms, but you are still going to need to do some leg work vetting the people they will staff your project with, and who knows if those firms will be any good next year or the year after. Here's a couple generic tips from an old pentester: * Do not try and schedule your pentest in Q4, everyone is too busy. Go for late Q1 or Q2. Also say you are willing to wait for the best fit testers to be available. * Ask to review resumes of the testing team. They should have some experience with your tech and at least one of them needs to have at least 2 years experience pen-testing. * Make sure your testing environment is set up, as production like as possible, and has data in it already. Test the external access. Test all the credentials, once after you generated them, again the night before the test starts. The most common reason to lose your good pentest team and get some juniors swapped in that have no idea what they are doing is you delayed the project by not being ready day 1.
  - evntdrvn 3y ago
    thank you!

by mymac 3y ago

iamflimflam1 3y ago

Yep, most pentests go through the OWASP list and call it done.

ganoushoreilly 3y ago
The problem is that is what most companies want. They don't want to spend the money nor get the feedback beyond "Best case standards". It's a calculated risk.
fomine3 3y ago
It's great if it's done exhaustively
Faelian2 3y ago
Honestly, the OWASP top ten is generic enough that most vulnerability fit in it : "injection", "security misconfiguration", "insecure design". The problem is 1. knowing the gazillion of web vulnerabilities, and technologies 2. being good enough to tests them 3. kick yourself and go through the laborious process of understand and test every key feature of the target.

j245 3y ago

From my understanding as a non security expert: Pentest comes across more as checking all the common attack vectors don’t exist. Getting out of bed to do the so-called “real stuff” is typically called a bug bounty program or security researching. Both exist and I don’t see why most companies couldn’t start a bug bounty program if they really cared a lot about the “real stuff”

mymac 3y ago
> From my understanding as a non security expert: That certainly helps.
- j245 3y ago
  What a shame, HackerNews typically has more insightful comments than garbage like this. Edit: thanks to everyone who wrote some insightful responses, and there are indeed many. Faith in HackerNews restored !
- bee_rider 3y ago
  People are going to chit-chat about things only tangentially related to their areas of expertise; it is good when we’re honest about our limitations. If nothing else, an obviously wrong take is a nice setup for a correction.
NegativeK 3y ago
The checkbox form exists because crooked vendors are catering to organizations who are intentionally lazy about their cybersecurity. Real penetration tests provide valuable insight that a bug bounty program won't.
prmoustache 3y ago
pentest means penetration testing which mean one need to take the attacker hat and try to enter your network or the app infrastructure and get as much data as he can, be it institutionnal or customer data. It can be through technical means as well as social engineering practices. And then report back. This is in no way related to a bug bounty program.
- nbk_2000 3y ago
  Counter point: Most of the top rated Bug Bounty hunters have a background in penetration testing. I think it's more accurate to say Bug Bounty only covers a small subset of penetration testing (mainly in that escalation and internal pivoting are against the BB policy of most companies).

_jal 3y ago

Let me tell you about the laptop connected to our network with a cellular antenna we found in a locked filing cabinet after getting a much-delayed forced-door alert. This, after some social engineering attempts that displayed unnerving familiarity with employees and a lot of virtual doorknob-rattling. They may be rare, but "real" pentests are still a thing.

mymac 3y ago
Ouch. How did that ended up?
- deleted 3y ago
  [deleted]

nbk_2000 3y ago

If you're a large company that's actually serious about security, you'll have a Red Team that is intimately familiar with your tech stacks, procedures, business model, etc. This team will be far better at emulating motivated attackers (as well as providing bespoke mitigation advice, vetting and testing solutions, etc.). Unfortunately, compliance/customer requirements often stipulate having penetration tests performed by third parties. So for business reasons, these same companies, will also hire low-quality pen-tests from "check-box pen-test" firms. So when you see that $10K "complete pen-test" being advertised as being used by [INSERT BIG SERIOUS NAME HERE], good chance this is why.

pixl97 3y ago
Ugh, in the work I do I run into so much of this kind of stuff. Customer: "We had a pentest/security scan/whatever find this issue in your software" Me: "And they realized that mitigations are in place as per the CVE that keep that issue from being an exploitable issue, right" Customer: "Uhhhh" Testing group: "Use smaller words please, we only click some buttons and this is the report that gets generated"

evntdrvn 3y ago

what I always want to know when people talk about this is "what reputable companies can I actually pay to do a real pentest (without costing hundreds of thousands of dollars)."

pnt12 3y ago
I think hiring a security specialist is the way to go.
amlozano 3y ago
The problem is security is a "Market for lemons" https://en.wikipedia.org/wiki/The_Market_for_Lemons https://en.wikipedia.org/wiki/The_Market_for_Lemons. Just like when trying to buy a used car, you need someone who is basically an expert in selling used cars. In order to purchase a reputable pentest, you basically have to have a security team that is mature enough to have just done it themselves. I can throw out some names for some reputable firms, but you are still going to need to do some leg work vetting the people they will staff your project with, and who knows if those firms will be any good next year or the year after. Here's a couple generic tips from an old pentester: * Do not try and schedule your pentest in Q4, everyone is too busy. Go for late Q1 or Q2. Also say you are willing to wait for the best fit testers to be available. * Ask to review resumes of the testing team. They should have some experience with your tech and at least one of them needs to have at least 2 years experience pen-testing. * Make sure your testing environment is set up, as production like as possible, and has data in it already. Test the external access. Test all the credentials, once after you generated them, again the night before the test starts. The most common reason to lose your good pentest team and get some juniors swapped in that have no idea what they are doing is you delayed the project by not being ready day 1.
- evntdrvn 3y ago
  thank you!