6 ms·

> We're developing a responsible access framework that makes models available to researchers for scholarly purposes while preventing misuse. The idea of traini

> We're developing a responsible access framework that makes models available to researchers for scholarly purposes while preventing misuse. The idea of training such a model is really a great one, but not releasing it because someone might be offended by the output is just stupid beyond believe.

fkdk 9mo ago
Maybe the authors are overly careful. Maybe avoiding to publish aspects of their work gives an edge over academic competitors. Maybe both. In my experience "data available upon request" doesn't always mean what you'd think it does.
nine_k 9mo ago
Public access, triggering a few racist responses from the model, a viral post on Xitter, the usual outrage, a scandal, the project gets publicly vilified, financing ceases. The researchers carry the tail of negative publicity throughout their remaining careers. Why risk all this?
- Forgeties79 9mo ago
  > triggering a few racist responses from the mode I feel like, ironically, it would be folks less concerned with political correctness/not being offensive that would abuse this opportunity to slander the project. But that’s just my gut.
  - dingnuts 9mo ago
    [dead]
- NuclearPM 9mo ago
  That’s ridiculous. There is no risk.
- teaearlgraycold 9mo ago
  Sure but Grok already exists.
- cj 9mo ago
  Because there are easy workarounds. If it becomes an issue, you can quickly add large disclaimers informing people that there might be offensive output because, well, it's trained on texts written during the age of racism. People typically get outraged when they see something they weren't expecting. If you tell them ahead of time, the user typically won't blame you (they'll blame themselves for choosing to ignore the disclaimer). And if disclaimers don't work, rebrand and relaunch it under a different name.
  - nine_k 9mo ago
    I wonder is you're being ironic here. You speak as if the people who play to an outrage wave are interested in achieving truth, peace, and understanding. Instead the rage-mongers are there to increase their (perceived) importance, and for lulz. The latter factor should not be underappreciated; remember "meme stocks". The risk is not large, but very real: the attack is very easy, and the potential downside, quite large. So not giving away access, but having the interested parties ask for it is prudent.
    - cj 9mo ago
      While I agree we live in a time of outrage, that also works in your favor. When there’s so much “outrage” every day, it’s very easy to blend in to the background. You might have a 5 minute moment of outrage fame, but it fades away quick. If you truly have good intentions with your project, you’re not going to get “canceled”, your career won’t be ruined Not being ironic. Not working on a LLM project because you’re worried about getting canceled by the outrage machine is an overreaction IMO. Are you able to name any developer or researcher who has been canceled because of their technical project or had their careers ruined? The only ones I can think of are clearly criminal and not just controversial (SBF, Snowden, etc)
- gnarbarian 9mo ago
  this is FUD.
- why-o-why 9mo ago
  I think you are confusing research with commodification. This is a research project, and it is clear how it was trained, and targeted at experts, enthusiasts, historians. Like if I was studying racism, the reference books explicitly written to dissect racism wouldn't be racist agents with a racist agenda. And as a result, no one is banning these books (except conservatives that want to retcon american history). Foundational models spewing racist white supremecist content when the trillion-dollar company forces it in your face is a vastly different scenario. There's a clear difference.
  - aidenn0 9mo ago
    > And as a result, no one is banning these books (except conservatives that want to retcon american history). My (very liberal) local school district banned English teachers from teaching any book that contained the n-word, even at a high-school level, and even when the author was a black person talking about real events that happened to them. FWIW, this was after complaints involving Of Mice and Men being on the curriculum.
    - zoky 9mo ago
      Banning Huckleberry Finn from a school district should be grounds for immediate dismissal.
      - somenameforme 9mo ago
        Even more so as the lesson of that story is perhaps the single most important one for people to learn in modern times. Almost everybody in that book is an awful person, especially the most 'upstanding' of types. Even the protagonist is an awful person. The one and only exception is 'N* Jim' who is the only kind-hearted and genuinely decent person in the book. It's an entire story about how the appearances of people, and the reality of those people, are two very different things. It being banned for using foul language, as educational outcomes continue to deteriorate, is just so perfectly ironic.
      - why-o-why 9mo ago
- kurtis_reed 9mo ago
  If people start standing up to the outrage it will lose its power
  - deleted 9mo ago
    [deleted]
- nofriend 9mo ago
  People know that models can be racist now. It's old hat. "LLM gets prompted into saying vile shit" hasn't been notable for years.
- vintermann 9mo ago
  Because the problem of bad faith attacks can only get worse if you fold every time. Sooner or later society has to come emotionally to terms with the fact that other times and places value things completely different from us, hold as important things we don't care about and are indifferent to things we do care about. Intellectually I'm sure we already know, but e.g. banning old books because they have reprehensible values (or even just use nasty words) - or indeed, refusing to release a model trained on historic texts "because it could be abused" is a sign that emotionally we haven't. It's not that it's a small deal, or should be expected to be easy. It's basically what Popper called "the strain of civilization" and posited as explanation for the totalitarianism which was rising in his time. But our values can't be so brittle that we can't even talk or think about other value systems.
- Alex2037 9mo ago
  nobody gives a shit about the journos and the terminally online. the smear campaign against AI is a cacophony, background noise that most people have learned to ignore, even here. consider this: https://news.ycombinator.com/from?site=nytimes.com https://news.ycombinator.com/from?site=nytimes.com HN's most beloved shitrag. day after day, they attack AI from every angle. how many of those submissions get traction at this point?
dash2 9mo ago
You have to understand that while the rest of the world has moved on from 2020, academics are still living there. There are many strong leftists, many of whom are deeply censorious; there are many more timeservers and cowards, who are terrified of falling foul of the first group. And there are force multipliers for all of this. Even if you yourself are a sensible and courageous person, you want to protect your project. What if your manager, ethics committee or funder comes under pressure?

fkdk 9mo ago

Maybe the authors are overly careful. Maybe avoiding to publish aspects of their work gives an edge over academic competitors. Maybe both. In my experience "data available upon request" doesn't always mean what you'd think it does.

nine_k 9mo ago

Public access, triggering a few racist responses from the model, a viral post on Xitter, the usual outrage, a scandal, the project gets publicly vilified, financing ceases. The researchers carry the tail of negative publicity throughout their remaining careers. Why risk all this?

Forgeties79 9mo ago
> triggering a few racist responses from the mode I feel like, ironically, it would be folks less concerned with political correctness/not being offensive that would abuse this opportunity to slander the project. But that’s just my gut.
- dingnuts 9mo ago
  [dead]
NuclearPM 9mo ago
That’s ridiculous. There is no risk.
teaearlgraycold 9mo ago
Sure but Grok already exists.
cj 9mo ago
Because there are easy workarounds. If it becomes an issue, you can quickly add large disclaimers informing people that there might be offensive output because, well, it's trained on texts written during the age of racism. People typically get outraged when they see something they weren't expecting. If you tell them ahead of time, the user typically won't blame you (they'll blame themselves for choosing to ignore the disclaimer). And if disclaimers don't work, rebrand and relaunch it under a different name.
- nine_k 9mo ago
  I wonder is you're being ironic here. You speak as if the people who play to an outrage wave are interested in achieving truth, peace, and understanding. Instead the rage-mongers are there to increase their (perceived) importance, and for lulz. The latter factor should not be underappreciated; remember "meme stocks". The risk is not large, but very real: the attack is very easy, and the potential downside, quite large. So not giving away access, but having the interested parties ask for it is prudent.
  - cj 9mo ago
    While I agree we live in a time of outrage, that also works in your favor. When there’s so much “outrage” every day, it’s very easy to blend in to the background. You might have a 5 minute moment of outrage fame, but it fades away quick. If you truly have good intentions with your project, you’re not going to get “canceled”, your career won’t be ruined Not being ironic. Not working on a LLM project because you’re worried about getting canceled by the outrage machine is an overreaction IMO. Are you able to name any developer or researcher who has been canceled because of their technical project or had their careers ruined? The only ones I can think of are clearly criminal and not just controversial (SBF, Snowden, etc)
gnarbarian 9mo ago
this is FUD.
why-o-why 9mo ago
I think you are confusing research with commodification. This is a research project, and it is clear how it was trained, and targeted at experts, enthusiasts, historians. Like if I was studying racism, the reference books explicitly written to dissect racism wouldn't be racist agents with a racist agenda. And as a result, no one is banning these books (except conservatives that want to retcon american history). Foundational models spewing racist white supremecist content when the trillion-dollar company forces it in your face is a vastly different scenario. There's a clear difference.
- aidenn0 9mo ago
  > And as a result, no one is banning these books (except conservatives that want to retcon american history). My (very liberal) local school district banned English teachers from teaching any book that contained the n-word, even at a high-school level, and even when the author was a black person talking about real events that happened to them. FWIW, this was after complaints involving Of Mice and Men being on the curriculum.
  - zoky 9mo ago
    Banning Huckleberry Finn from a school district should be grounds for immediate dismissal.
    - somenameforme 9mo ago
      Even more so as the lesson of that story is perhaps the single most important one for people to learn in modern times. Almost everybody in that book is an awful person, especially the most 'upstanding' of types. Even the protagonist is an awful person. The one and only exception is 'N* Jim' who is the only kind-hearted and genuinely decent person in the book. It's an entire story about how the appearances of people, and the reality of those people, are two very different things. It being banned for using foul language, as educational outcomes continue to deteriorate, is just so perfectly ironic.
    - why-o-why 9mo ago
kurtis_reed 9mo ago
If people start standing up to the outrage it will lose its power
- deleted 9mo ago
  [deleted]
nofriend 9mo ago
People know that models can be racist now. It's old hat. "LLM gets prompted into saying vile shit" hasn't been notable for years.
vintermann 9mo ago
Because the problem of bad faith attacks can only get worse if you fold every time. Sooner or later society has to come emotionally to terms with the fact that other times and places value things completely different from us, hold as important things we don't care about and are indifferent to things we do care about. Intellectually I'm sure we already know, but e.g. banning old books because they have reprehensible values (or even just use nasty words) - or indeed, refusing to release a model trained on historic texts "because it could be abused" is a sign that emotionally we haven't. It's not that it's a small deal, or should be expected to be easy. It's basically what Popper called "the strain of civilization" and posited as explanation for the totalitarianism which was rising in his time. But our values can't be so brittle that we can't even talk or think about other value systems.
Alex2037 9mo ago
nobody gives a shit about the journos and the terminally online. the smear campaign against AI is a cacophony, background noise that most people have learned to ignore, even here. consider this: https://news.ycombinator.com/from?site=nytimes.com https://news.ycombinator.com/from?site=nytimes.com HN's most beloved shitrag. day after day, they attack AI from every angle. how many of those submissions get traction at this point?

dash2 9mo ago

You have to understand that while the rest of the world has moved on from 2020, academics are still living there. There are many strong leftists, many of whom are deeply censorious; there are many more timeservers and cowards, who are terrified of falling foul of the first group. And there are force multipliers for all of this. Even if you yourself are a sensible and courageous person, you want to protect your project. What if your manager, ethics committee or funder comes under pressure?