6 ms·

I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia t

I get what you are saying but that’s exactly how security and compliance work. And I don’t think amount of people who want to be ethical in an organization ia that different than people who want to build secure software (but possibly no one gives a shit compliance other than it’s something that needs to be done)

tgv 1mo ago
There are at least some penalties for failed security and compliance. Ethics is the first victim of greed.
dannyw 1mo ago
Security and compliance tends to be a lot less subjective than ethics. Not saying it’s objective, but there’s a huge difference. The downside is it can often feel like a box-checking exercise than actual security or compliance, but “you need 2FA” is less debatable than, say, AI and copyright.
- antupis 1mo ago
  Big corporations there is politics on play and too often you see CABs and other bureaucratic stuff instead of checkboxes for gated releases, 2FA etc.
lnsru 1mo ago
During my years at big corp we did tons of work just to go around the law department. While for core business it made absolute sense to have lawyers involved in daily business we were satellite office decoupled from core business. Local management failed to communicate that and we were stalked by lawyers from main office. Pretty sure others do the same with compliance, law, ethics departments. Just work around to get things done instead of stuck for weeks in stupid meetings.
hnlmorg 1mo ago
And organisations fail at security when security and compliance is only considered important by that one teamnn Security requires the whole business to buy in. And it requires processes that allow people to get shit done without people resorting to shadow IT; thus working around that one team. So the GPs point still works.
- ivan_gammel 1mo ago
  Rule number one of CIO: become friends with CFO and chief lawyer. And nothing else matters.
- close04 1mo ago
  The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. Poor security practices harm your teams, your data, and usually you make moderate savings at best. Poor ethics "only" harm your customers while making bank for the company. This is the real problem with ethics in a large corporation. You're not saying "no" to another team, you're saying no to large profits, you're saying no to the company's leadership. That is what never works.
  - hnlmorg 1mo ago
    > The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. I don’t agree with this. Data breaches affect customers more than businesses. If your point were true, we’d see fewer breaches. Plus not all breaches are a result of software engineering teams. For example product managers sharing customer details. I’ve managed plenty of teams where I’ve had to instil the importance of secure best practices at all stages of development. So it’s definitely not something inherently important to all people who work in organisations. Just like with ethics. It’s very easy to dismiss either as an inconvenience if you don’t instil the right company culture at all levels of the organisation. This is why European financial organisations have such strict onboarding procedures to teach new hires about fraud, bribery and other financial misconduct even for issues that are ethical grey rather than outright illegal. Similarly many organisations will have onboarding procedures to teach new hires their security best practices too
    - close04 1mo ago
      I think we're both thinking of different examples and drawing wider conclusions. I'm thinking of a company losing the data that makes it money, you're thinking of losing the customer data and thus their trust. Probably equally valid points of view. The lack of ethics hugely contributes to companies collecting more and more user data that they normally shouldn't have. This makes the data a more attractive target. > If your point were true, we’d see fewer breaches. But this doesn't follow. There are way more factors at play that influence the number of attacks and the number of successes. Companies hold more and more data with ever higher value (so more liability), and hacking tools and hacker determination advanced faster than defensive measures. The result is expected and the solution isn't only "more security", but also "hold less data". While security is a double edged sword, ethics had been proven to be very single edged. History shows that for startups and large companies alike, the lack of ethics is actually a competitive advantage for the company.
nine_k 1mo ago
The structure is untenable when the work of one team is to say "no" to other teams, when these teams are not asking. A good infrastructure team would seek a competent security review that would say "no" to problematic things before an intruder says "aha" to them. If feedback from the ethics team is not sought, nobody is going to heed its opinion anyway.
- pyuser583 1mo ago
  I’ve never known a security team to be in the position to say “no.” The role of security teams vary a lot from one organization to another. But generally they can make you aware of the tradeoffs you are making, or serve as a kind of quality control that generally does not want to be asked questions.
a_bonobo 1mo ago
Security/compliance have the external hammer: if they fail, your org will have to pay fines and someone may end up being criminally liable (depends on country). The ethics department; if they fail, there may be some negative journalism, but who which AI company has positive journalism these days? There's no external hammer for ethics.
jiggawatts 1mo ago
> exactly how security and compliance Not everywhere. I go out of my way to assist teams to achieve a secure outcome with less effort. Things like: “instead of admin access to the production servers the devs can have fully automated deployment pipelines combined with OpenTelemetry for observability so they don’t have to spend half the day scrolling through gigabytes of logs.” That’s more secure and and more better. Nobody had to be told “no”. Similarly, I replace key store access with secret-less managed identity, etc.
seanhunter 1mo ago
In a high-functioning security and compliance team they tend to say “no” only in really dire circumstances. Good security and compliance teams spend a lot of time asking exactly what it is the people are really trying to do and then find a way to say “yes, and…” as in “yes you can and here’s how you do it without compromising security/breaking the law etc”. And as a sibling said, orgs fail at security when they make security only the infosec teams’ job. This is also why I said “whose only job”. In a good org, the security team doesn’t only say no to devsecops requests, they also do trainings to skill up other teams, keep the network secure, proactively seek out and understand external threats, work with external vendors etc etc …
mnahkies 1mo ago
I think the key part is about aligning incentives and outcomes - if your security and compliance departments are only judged on "were we breached, did we pass our audits" then there's a risk they won't weigh the tradeoffs associated with the decisions around controls implemented to achieve those outcomes. To use a ridiculous extreme you can't breach a web app that isn't exposed to the internet, but the users can't access it either. If you can connect/balance those goals to other metrics around cost and productivity, usability, and a realistic threat model, as guardrails then you incentatize cross-team collaboration to achieve the shared org outcomes.
mchinen 1mo ago
SRE is a 'say no to powerful people' job as well. I think for this to work the leadership needs to show support for it. The friction is still there, but in more tolerable areas. I bet for ethics this isn't the case at OAI, but everyone values security and stability. For an SRE there can be more directed hate received from the junior employees, that want to release new features they developed. Especially because there is less accountability across orgs. Security is an interesting one because it seems to have less of this friction, maybe because it's more clear cut what is an issue.
nobodywillobsrv 1mo ago
Markets solve this by pricing risk. Ideally you would have some kind of notion of selling insurance internally and track things. But ultimately existential risk is hard to negotiate from the inside. And companies are supposed to go bust or succeed. They are not really the same as a population trying to survive forever. At least that is one take.
scelerat 1mo ago
The compliance team seeks to maximize company profit/success within the confines of hard legal boundaries. There are no "hard" boundaries for ethics or "doing the right thing," and so those boundaries will always be pushed until they are useless.

tgv 1mo ago

There are at least some penalties for failed security and compliance. Ethics is the first victim of greed.

dannyw 1mo ago

Security and compliance tends to be a lot less subjective than ethics. Not saying it’s objective, but there’s a huge difference. The downside is it can often feel like a box-checking exercise than actual security or compliance, but “you need 2FA” is less debatable than, say, AI and copyright.

antupis 1mo ago
Big corporations there is politics on play and too often you see CABs and other bureaucratic stuff instead of checkboxes for gated releases, 2FA etc.

lnsru 1mo ago

During my years at big corp we did tons of work just to go around the law department. While for core business it made absolute sense to have lawyers involved in daily business we were satellite office decoupled from core business. Local management failed to communicate that and we were stalked by lawyers from main office. Pretty sure others do the same with compliance, law, ethics departments. Just work around to get things done instead of stuck for weeks in stupid meetings.

hnlmorg 1mo ago

And organisations fail at security when security and compliance is only considered important by that one teamnn Security requires the whole business to buy in. And it requires processes that allow people to get shit done without people resorting to shadow IT; thus working around that one team. So the GPs point still works.

ivan_gammel 1mo ago
Rule number one of CIO: become friends with CFO and chief lawyer. And nothing else matters.
close04 1mo ago
The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. Poor security practices harm your teams, your data, and usually you make moderate savings at best. Poor ethics "only" harm your customers while making bank for the company. This is the real problem with ethics in a large corporation. You're not saying "no" to another team, you're saying no to large profits, you're saying no to the company's leadership. That is what never works.
- hnlmorg 1mo ago
  > The difference between "ethics" and "security and compliance" is that the latter is something that hits inside the company, while the former usually hits outside of the company. I don’t agree with this. Data breaches affect customers more than businesses. If your point were true, we’d see fewer breaches. Plus not all breaches are a result of software engineering teams. For example product managers sharing customer details. I’ve managed plenty of teams where I’ve had to instil the importance of secure best practices at all stages of development. So it’s definitely not something inherently important to all people who work in organisations. Just like with ethics. It’s very easy to dismiss either as an inconvenience if you don’t instil the right company culture at all levels of the organisation. This is why European financial organisations have such strict onboarding procedures to teach new hires about fraud, bribery and other financial misconduct even for issues that are ethical grey rather than outright illegal. Similarly many organisations will have onboarding procedures to teach new hires their security best practices too
  - close04 1mo ago
    I think we're both thinking of different examples and drawing wider conclusions. I'm thinking of a company losing the data that makes it money, you're thinking of losing the customer data and thus their trust. Probably equally valid points of view. The lack of ethics hugely contributes to companies collecting more and more user data that they normally shouldn't have. This makes the data a more attractive target. > If your point were true, we’d see fewer breaches. But this doesn't follow. There are way more factors at play that influence the number of attacks and the number of successes. Companies hold more and more data with ever higher value (so more liability), and hacking tools and hacker determination advanced faster than defensive measures. The result is expected and the solution isn't only "more security", but also "hold less data". While security is a double edged sword, ethics had been proven to be very single edged. History shows that for startups and large companies alike, the lack of ethics is actually a competitive advantage for the company.

nine_k 1mo ago

The structure is untenable when the work of one team is to say "no" to other teams, when these teams are not asking. A good infrastructure team would seek a competent security review that would say "no" to problematic things before an intruder says "aha" to them. If feedback from the ethics team is not sought, nobody is going to heed its opinion anyway.

pyuser583 1mo ago
I’ve never known a security team to be in the position to say “no.” The role of security teams vary a lot from one organization to another. But generally they can make you aware of the tradeoffs you are making, or serve as a kind of quality control that generally does not want to be asked questions.

a_bonobo 1mo ago

Security/compliance have the external hammer: if they fail, your org will have to pay fines and someone may end up being criminally liable (depends on country). The ethics department; if they fail, there may be some negative journalism, but who which AI company has positive journalism these days? There's no external hammer for ethics.

jiggawatts 1mo ago

> exactly how security and compliance Not everywhere. I go out of my way to assist teams to achieve a secure outcome with less effort. Things like: “instead of admin access to the production servers the devs can have fully automated deployment pipelines combined with OpenTelemetry for observability so they don’t have to spend half the day scrolling through gigabytes of logs.” That’s more secure and and more better. Nobody had to be told “no”. Similarly, I replace key store access with secret-less managed identity, etc.

seanhunter 1mo ago

In a high-functioning security and compliance team they tend to say “no” only in really dire circumstances. Good security and compliance teams spend a lot of time asking exactly what it is the people are really trying to do and then find a way to say “yes, and…” as in “yes you can and here’s how you do it without compromising security/breaking the law etc”. And as a sibling said, orgs fail at security when they make security only the infosec teams’ job. This is also why I said “whose only job”. In a good org, the security team doesn’t only say no to devsecops requests, they also do trainings to skill up other teams, keep the network secure, proactively seek out and understand external threats, work with external vendors etc etc …

mnahkies 1mo ago

I think the key part is about aligning incentives and outcomes - if your security and compliance departments are only judged on "were we breached, did we pass our audits" then there's a risk they won't weigh the tradeoffs associated with the decisions around controls implemented to achieve those outcomes. To use a ridiculous extreme you can't breach a web app that isn't exposed to the internet, but the users can't access it either. If you can connect/balance those goals to other metrics around cost and productivity, usability, and a realistic threat model, as guardrails then you incentatize cross-team collaboration to achieve the shared org outcomes.

mchinen 1mo ago

SRE is a 'say no to powerful people' job as well. I think for this to work the leadership needs to show support for it. The friction is still there, but in more tolerable areas. I bet for ethics this isn't the case at OAI, but everyone values security and stability. For an SRE there can be more directed hate received from the junior employees, that want to release new features they developed. Especially because there is less accountability across orgs. Security is an interesting one because it seems to have less of this friction, maybe because it's more clear cut what is an issue.

nobodywillobsrv 1mo ago

Markets solve this by pricing risk. Ideally you would have some kind of notion of selling insurance internally and track things. But ultimately existential risk is hard to negotiate from the inside. And companies are supposed to go bust or succeed. They are not really the same as a population trying to survive forever. At least that is one take.

scelerat 1mo ago

The compliance team seeks to maximize company profit/success within the confines of hard legal boundaries. There are no "hard" boundaries for ethics or "doing the right thing," and so those boundaries will always be pushed until they are useless.