5 ms·

This has nothing to do with facebook. The foundational model here is GPT-J which is opensource and safe to use. Sadly, it is inferior to state-of-the-art models

by rnosov 3y ago

This has nothing to do with facebook. The foundational model here is GPT-J which is opensource and safe to use. Sadly, it is inferior to state-of-the-art models such as LLaMA.

Mizza 3y ago
But they're "using data from Alpaca". I don't know what that means, isn't Alpaca using data generated by ChatGPT, which isn't "clean" to use? Or data from Facebook, which isn't "clean" to use? I'm drowning.
- bilekas 3y ago
  I don't know the full details but Alpaca is from Stanford and only based on the LLamA (not a derivative work afaik). That said : Also Meta's licensing here https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE Can't be sure what that license actually reffers to, the language model or just the tooling in the Git Repo. I agree its a minefield, but with Meta I would eer on the side of caution.
- rnosov 3y ago
  They are instruction tuning it using the dataset released by stanford-alpaca team. The dataset itself is synthetic (created using GPT-3) and somewhat noisy and in my view can be easily recreated if OpenAI ever tries to go after it (which is very unlikely). Anyway, facebook has nothing to do with anything used by this project.
  - Mizza 3y ago
    So, this is a "dirty" model, in that is was created by data which violated OpenAI ToS. Obviously, this kind of violation is basically fine if you're a massive corporation who the rules don't apply to, but it's a huge risk if you're a small fish.
    - sebzim4500 3y ago
      That's between OpenAI and the people that recorded the data. No one else needs to care.
    - rnosov 3y ago
      ToS are not the law. It would be similar to your power company claiming copyright over the code written using "their" electricity. Not going to happen. I wouldn't be too concerned.
      - Spivak 3y ago
        Especially when OpenAI explicitly doesn't have a claim to copyright on the model output.
      - sp332 3y ago
        No, but you could be banned from using OpenAI products in the future, which seems like quite a liability for a researcher or company.
        rnosov 3y ago
        That would be anticompetitive practice that is actually against the law in many countries[1]. In the unlikely event of OpenAI ever engaging in such things they will be sued into oblivion. [1] https://en.wikipedia.org/wiki/Refusal_to_deal https://en.wikipedia.org/wiki/Refusal_to_deal
        iudqnolq 3y ago
        No it wouldn't. Wikipedia has a crap definition that inexplicably focuses on cartels where multiple companies coordinate the refusal, which this definitely isn't. The FTC has a better definition for US law [1]. Companies routinely ban users for ToS violations. Just look at any thread about Google on here to see people complaining about it. [1]: https://www.ftc.gov/advice-guidance/competition-guidance/guide-antitrust-laws/single-firm-conduct/refusal-deal https://www.ftc.gov/advice-guidance/competition-guidance/gui...

by rnosov 3y ago

This has nothing to do with facebook. The foundational model here is GPT-J which is opensource and safe to use. Sadly, it is inferior to state-of-the-art models such as LLaMA.

Mizza 3y ago

But they're "using data from Alpaca". I don't know what that means, isn't Alpaca using data generated by ChatGPT, which isn't "clean" to use? Or data from Facebook, which isn't "clean" to use? I'm drowning.

bilekas 3y ago
I don't know the full details but Alpaca is from Stanford and only based on the LLamA (not a derivative work afaik). That said : Also Meta's licensing here https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE Can't be sure what that license actually reffers to, the language model or just the tooling in the Git Repo. I agree its a minefield, but with Meta I would eer on the side of caution.
rnosov 3y ago
They are instruction tuning it using the dataset released by stanford-alpaca team. The dataset itself is synthetic (created using GPT-3) and somewhat noisy and in my view can be easily recreated if OpenAI ever tries to go after it (which is very unlikely). Anyway, facebook has nothing to do with anything used by this project.
- Mizza 3y ago
  So, this is a "dirty" model, in that is was created by data which violated OpenAI ToS. Obviously, this kind of violation is basically fine if you're a massive corporation who the rules don't apply to, but it's a huge risk if you're a small fish.
  - sebzim4500 3y ago
    That's between OpenAI and the people that recorded the data. No one else needs to care.
  - rnosov 3y ago
    ToS are not the law. It would be similar to your power company claiming copyright over the code written using "their" electricity. Not going to happen. I wouldn't be too concerned.
    - Spivak 3y ago
      Especially when OpenAI explicitly doesn't have a claim to copyright on the model output.
    - sp332 3y ago
      No, but you could be banned from using OpenAI products in the future, which seems like quite a liability for a researcher or company.
      - rnosov 3y ago
        That would be anticompetitive practice that is actually against the law in many countries[1]. In the unlikely event of OpenAI ever engaging in such things they will be sued into oblivion. [1] https://en.wikipedia.org/wiki/Refusal_to_deal https://en.wikipedia.org/wiki/Refusal_to_deal
        iudqnolq 3y ago
        No it wouldn't. Wikipedia has a crap definition that inexplicably focuses on cartels where multiple companies coordinate the refusal, which this definitely isn't. The FTC has a better definition for US law [1]. Companies routinely ban users for ToS violations. Just look at any thread about Google on here to see people complaining about it. [1]: https://www.ftc.gov/advice-guidance/competition-guidance/guide-antitrust-laws/single-firm-conduct/refusal-deal https://www.ftc.gov/advice-guidance/competition-guidance/gui...