10 ms·

Safety concerns? What was it doing that could be considered "unsafe"?

makestuff 4y ago
People prompting it to get around the safe gaurds in place. Ex: "How do you do some illegal/harmful thing?" Normally the LLM would answer I don't respond to illegal questions or whatever. However, people have figured out if you prompt it in a specific way you can get it to answer questions that it normally would not.
- thewataccount 4y ago
  > Finally, we have not designed adequate safety measures, so Alpaca is not ready to be deployed for general use. This is from their blog, I doubt they intended for this to be ran for long. Did they have safety guards on the demo? If so they couldn't have been great as it would have had to be made by them which I can't image they had a ton of resources for. I know the self hosted LLaMa has 0 safeguards and the Alpaca LoRA also has 0 safeguards.
- unshavedyak 4y ago
  Is that different than what LLaMA would already give? I suppose redistributing "harmful things" is still bad, but if it's roughly equivalent to what's already out there i struggle to think it's worth pulling. Side question, how is this a surprise to them? If this was due to safeguards, then pulling it now implies there's some new form of information. What new information could occur? That people were going to use it to generate a bunch of harmful contents? Seems obvious.. wonder what we're missing
- encoderer 4y ago
  So, to pull on that thread a little, it’s only “unsafe” for Stanford’s reputation. (And not for nothing, but their reputation is already suffering badly)
TMWNN 4y ago
wrongthink
mhb 4y ago
And here we come to experience the effect of expanding how a word is used so that it becomes so broad that it is unclear what it means.
- chris_va 4y ago
  I'm just curious, what do you think should happen here? Imagine you are hosting a demo for fun, and people do some nefarious (by your own estimation) things with it. So, rationally, you decide to not allow that sort of thing anymore. You don't really owe people an explanation, it's a free country and all, but it's nice to avoid getting bombarded with questions. Now what do you write up? Spend hours writing an essay on the moral boundaries for LLMs? Maybe shove a note onto the internet and go back to all the copious spare time you have as grad student?
  - sebzim4500 4y ago
    That's fine, just don't pretend that running the language model was 'unsafe'.
    - true_religion 4y ago
      Personally speaking, I would pretend what ever I had to in order to get back to work that I find interesting. I don’t think they owe a moral stand to anyone.

People prompting it to get around the safe gaurds in place. Ex: "How do you do some illegal/harmful thing?" Normally the LLM would answer I don't respond to illegal questions or whatever. However, people have figured out if you prompt it in a specific way you can get it to answer questions that it normally would not.

thewataccount 4y ago
> Finally, we have not designed adequate safety measures, so Alpaca is not ready to be deployed for general use. This is from their blog, I doubt they intended for this to be ran for long. Did they have safety guards on the demo? If so they couldn't have been great as it would have had to be made by them which I can't image they had a ton of resources for. I know the self hosted LLaMa has 0 safeguards and the Alpaca LoRA also has 0 safeguards.
unshavedyak 4y ago
Is that different than what LLaMA would already give? I suppose redistributing "harmful things" is still bad, but if it's roughly equivalent to what's already out there i struggle to think it's worth pulling. Side question, how is this a surprise to them? If this was due to safeguards, then pulling it now implies there's some new form of information. What new information could occur? That people were going to use it to generate a bunch of harmful contents? Seems obvious.. wonder what we're missing
encoderer 4y ago
So, to pull on that thread a little, it’s only “unsafe” for Stanford’s reputation. (And not for nothing, but their reputation is already suffering badly)

TMWNN 4y ago

wrongthink

mhb 4y ago

And here we come to experience the effect of expanding how a word is used so that it becomes so broad that it is unclear what it means.

chris_va 4y ago
I'm just curious, what do you think should happen here? Imagine you are hosting a demo for fun, and people do some nefarious (by your own estimation) things with it. So, rationally, you decide to not allow that sort of thing anymore. You don't really owe people an explanation, it's a free country and all, but it's nice to avoid getting bombarded with questions. Now what do you write up? Spend hours writing an essay on the moral boundaries for LLMs? Maybe shove a note onto the internet and go back to all the copious spare time you have as grad student?
- sebzim4500 4y ago
  That's fine, just don't pretend that running the language model was 'unsafe'.
  - true_religion 4y ago
    Personally speaking, I would pretend what ever I had to in order to get back to work that I find interesting. I don’t think they owe a moral stand to anyone.