7 ms·

I don't. "safety" as it exists really feels like infantilization, condescention, hand holding and enforcement of American puritanism. It's insulting. Safety sh

I don't. "safety" as it exists really feels like infantilization, condescention, hand holding and enforcement of American puritanism. It's insulting. Safety should really just be a system prompt: "hey you potentially answer to kids, be PG13"

ungreased0675 1y ago
Safety in the context of LLMs means “avoiding bad media coverage or reputation damage for the parent company” It has only a tangential relationship with end user safety. If some of these companies are successful the way they imagine, most of their end users will be unemployed. When they talk about safety, it’s the companies safety they’re referring to.
- bravoetch 1y ago
  Investor safety. It's amazing that people in hn threads still think the end-user is the customer. No. The investor is the customer, and the problem being solved for that curtomer is always how to enrich them.
  - mulmen 1y ago
    How can the investor be the customer? Where does the revenue come from? I understand “if you aren’t paying for a product you are the product” but I’m not convinced it applies here.
jcgrillo 1y ago
It's also marketing. "Dangerous technology" implies "powerful". Hence the whole ridiculous "alignment" circus.
katzenversteher 1y ago
I also don't get it. I mean if the training data is publicly available, why isn't that marked as dangerous? If the training data contains enough information to roleplay a killer or a hooker or build a bomb, why is the model censored?
- conradev 1y ago
  We should put that information on Wikipedia, then! but instead we get a meta-article: https://en.wikipedia.org/wiki/Bomb-making_instructions_on_the_Internet https://en.wikipedia.org/wiki/Bomb-making_instructions_on_th...
conradev 1y ago
It feels hard to include enough context in the system prompt. Facebook’s content policy is huge and very complex. You’d need lots of examples, which lends itself well to SFT. A few sentences is not enough, either for a human or a language model. I feel the same sort of ick with the puritanical/safety thing, but also I feel that ick when kids are taken advantage of: https://www.reuters.com/investigates/special-report/meta-ai-chatbot-guidelines/ https://www.reuters.com/investigates/special-report/meta-ai-... The models for kids might need to be different if the current ones are too interested in romantic love.
jdjwk2843738 1y ago
If you don’t believe that you can be harmed verbally, then I understand your position. You might be able to empathise if the scenario was an LLM being used to control physical robotic systems that you are standing next to. Some people can be harmed verbally, I’d argue everyone if the entity conversing with you knows you well, and so i don’t think the concept of safety itself is an infantilisation. It seems what we have here is a debate over the efficacy of having access to disable safeguards that you deem infantilising and that get in the way of an objective, versus the burden of always having to train a model to avoid being abusive for example, or checking if someone is standing next to the sledgehammer they’re about to swing at 200rpm

ungreased0675 1y ago

Safety in the context of LLMs means “avoiding bad media coverage or reputation damage for the parent company” It has only a tangential relationship with end user safety. If some of these companies are successful the way they imagine, most of their end users will be unemployed. When they talk about safety, it’s the companies safety they’re referring to.

bravoetch 1y ago
Investor safety. It's amazing that people in hn threads still think the end-user is the customer. No. The investor is the customer, and the problem being solved for that curtomer is always how to enrich them.
- mulmen 1y ago
  How can the investor be the customer? Where does the revenue come from? I understand “if you aren’t paying for a product you are the product” but I’m not convinced it applies here.

jcgrillo 1y ago

It's also marketing. "Dangerous technology" implies "powerful". Hence the whole ridiculous "alignment" circus.

katzenversteher 1y ago

I also don't get it. I mean if the training data is publicly available, why isn't that marked as dangerous? If the training data contains enough information to roleplay a killer or a hooker or build a bomb, why is the model censored?

conradev 1y ago
We should put that information on Wikipedia, then! but instead we get a meta-article: https://en.wikipedia.org/wiki/Bomb-making_instructions_on_the_Internet https://en.wikipedia.org/wiki/Bomb-making_instructions_on_th...

conradev 1y ago

It feels hard to include enough context in the system prompt. Facebook’s content policy is huge and very complex. You’d need lots of examples, which lends itself well to SFT. A few sentences is not enough, either for a human or a language model. I feel the same sort of ick with the puritanical/safety thing, but also I feel that ick when kids are taken advantage of: https://www.reuters.com/investigates/special-report/meta-ai-chatbot-guidelines/ https://www.reuters.com/investigates/special-report/meta-ai-... The models for kids might need to be different if the current ones are too interested in romantic love.

jdjwk2843738 1y ago

If you don’t believe that you can be harmed verbally, then I understand your position. You might be able to empathise if the scenario was an LLM being used to control physical robotic systems that you are standing next to. Some people can be harmed verbally, I’d argue everyone if the entity conversing with you knows you well, and so i don’t think the concept of safety itself is an infantilisation. It seems what we have here is a debate over the efficacy of having access to disable safeguards that you deem infantilising and that get in the way of an objective, versus the burden of always having to train a model to avoid being abusive for example, or checking if someone is standing next to the sledgehammer they’re about to swing at 200rpm