7 ms·

The next major leap in LLMs (in the next year) is probably going to be the prompt context size. Right now we have 2k, 4k, 8k ... but OpenAI also has a 32k mode

The next major leap in LLMs (in the next year) is probably going to be the prompt context size. Right now we have 2k, 4k, 8k ... but OpenAI also has a 32k model that they're not really giving access to unfortunately. The 8k model is nice but it's GPT4 so it's slow. I think the thing that you're missing is that zero shot learning is VERY hard but anything > GPT3 is actually pretty good once you give it some real world examples. I think prompt engineering is going to be here for a while just because, on a lot of task, examples are needed. Doesn't mean it needs to be a herculean effort of course. Just that you need to come up with some concrete examples. This is going to be ESPECIALLY true with Open Source LLMs that aren't anywhere near as sophisticated as GPT4. In fact, I think there's a huge opportunity to use GPT4 to train the prompts of smaller models, come up with more examples, and help improve their precision/recall without massive prompt engineering efforts.

kiratp 3y ago
You can’t commercially use anything you train off OpenAI outputs.
- rufius 3y ago
  Can you elaborate?
  - kiratp 3y ago
    Their API TOS basically forbid it. Simple as that.
    - MacsHeadroom 3y ago
      Someone who acquires these outputs who has never consented to their ToS is not bound by their ToS.
      - reissbaker 3y ago
        Sure, but the ways of acquiring those outputs legally have vampiric licensing that bind you to those ToS, since the re-licenser is bound by the original ToS. It's like distributing GPL code in a nonfree application. Even if you didn't "consent to [the original author's] ToS," you are still going to be bound to it via the redistributors license.
        MacsHeadroom 3y ago
        No you are not, the outputs of an LLM are afforded no intellectual property rights protections. The internet is increasingly covered with OpenAI's outputs and they do not have grounds for a claim to prevent people from using them.
        Moissanite 3y ago
        > Even if you didn't "consent to [the original author's] ToS," you are still going to be bound to it via the redistributors license. In the context of the GPL, are there real examples of judgements which bind defendants to a license they never saw or knew anything about, because of the errant actions of an intermediary?
        wokwokwok 3y ago
        It doesn’t matter. It gives OpenAI a legal basis to launch a law suit if they want to. Would it succeed? Is it right? Do they care? Eh. …but, if I as some random reddit user say I might sue you for making a LLM you for training on data that may or may not have my posts in it, you can probably safely ignore me. If you go and build a massive LLM using high quality data that couldn’t possibly come from anywhere other than openai, and they have a log of all the content that api key XXX generated; they both know and have a legal basis for litigation. There’s a difference, even if you’re a third party (not the owner of the api key) or don’t care. (And I’m not saying they would, or even they would win; but it’s sufficient cause for them to be able to make a case if they want to)
        uoaei 3y ago
    - andygeorge 3y ago
      [flagged]
  - 411111111111111 3y ago
    They're probably talking about the TOS a user would've had to agree to when using their services. It's actually a lot more permissive then I expected > Restrictions. You may not (i) use the Services in a way that infringes, misappropriates or violates any person’s rights; (ii) reverse assemble, reverse compile, decompile, translate or otherwise attempt to discover the source code or underlying components of models, algorithms, and systems of the Services (except to the extent such restrictions are contrary to applicable law); (iii) use output from the Services to develop models that compete with OpenAI;
    - kiratp 3y ago
      > use output from the Services to develop models that compete with OpenAI It can be argued that if you build a model using their outputs such that you can then stop using their API, your model is effectively competing with their’s. Let’s just say that if you’re a startup or SMB, you do not want to be the one dragged to court to iron out whether this holds or not.
- sebzim4500 3y ago
  You can as long as the resulting model does not compete with OpenAI.
  - kiratp 3y ago
    It can be argued that if you build a model using their outputs such that you can then stop using their API, your model is effectively competing with their’s. Let’s just say that if you’re a startup or SMB, you do not want to be the one dragged to court to iron out whether this holds or not.
- wanderingmind 3y ago
  This is just ToS violation which will just result in loss of access to OpenAI. There is nothing they can do to stop you from commercially competing, given there is no copyright law precedence
throwawayForMe2 3y ago
>> The next major leap in LLMs (in the next year) is probably going to be the prompt context size. Right now we have 2k, 4k, 8k ... but OpenAI also has a 32k model that they're not really giving access to unfortunately. Saw this article today about a different approach that opens up orders of magnitude larger contexts https://hazyresearch.stanford.edu/blog/2023-03-07-hyena https://hazyresearch.stanford.edu/blog/2023-03-07-hyena

kiratp 3y ago

You can’t commercially use anything you train off OpenAI outputs.

rufius 3y ago
Can you elaborate?
- kiratp 3y ago
  Their API TOS basically forbid it. Simple as that.
  - MacsHeadroom 3y ago
    Someone who acquires these outputs who has never consented to their ToS is not bound by their ToS.
    - reissbaker 3y ago
      Sure, but the ways of acquiring those outputs legally have vampiric licensing that bind you to those ToS, since the re-licenser is bound by the original ToS. It's like distributing GPL code in a nonfree application. Even if you didn't "consent to [the original author's] ToS," you are still going to be bound to it via the redistributors license.
      - MacsHeadroom 3y ago
        No you are not, the outputs of an LLM are afforded no intellectual property rights protections. The internet is increasingly covered with OpenAI's outputs and they do not have grounds for a claim to prevent people from using them.
      - Moissanite 3y ago
        > Even if you didn't "consent to [the original author's] ToS," you are still going to be bound to it via the redistributors license. In the context of the GPL, are there real examples of judgements which bind defendants to a license they never saw or knew anything about, because of the errant actions of an intermediary?
        wokwokwok 3y ago
        It doesn’t matter. It gives OpenAI a legal basis to launch a law suit if they want to. Would it succeed? Is it right? Do they care? Eh. …but, if I as some random reddit user say I might sue you for making a LLM you for training on data that may or may not have my posts in it, you can probably safely ignore me. If you go and build a massive LLM using high quality data that couldn’t possibly come from anywhere other than openai, and they have a log of all the content that api key XXX generated; they both know and have a legal basis for litigation. There’s a difference, even if you’re a third party (not the owner of the api key) or don’t care. (And I’m not saying they would, or even they would win; but it’s sufficient cause for them to be able to make a case if they want to)
        uoaei 3y ago
  - andygeorge 3y ago
    [flagged]
- 411111111111111 3y ago
  They're probably talking about the TOS a user would've had to agree to when using their services. It's actually a lot more permissive then I expected > Restrictions. You may not (i) use the Services in a way that infringes, misappropriates or violates any person’s rights; (ii) reverse assemble, reverse compile, decompile, translate or otherwise attempt to discover the source code or underlying components of models, algorithms, and systems of the Services (except to the extent such restrictions are contrary to applicable law); (iii) use output from the Services to develop models that compete with OpenAI;
  - kiratp 3y ago
    > use output from the Services to develop models that compete with OpenAI It can be argued that if you build a model using their outputs such that you can then stop using their API, your model is effectively competing with their’s. Let’s just say that if you’re a startup or SMB, you do not want to be the one dragged to court to iron out whether this holds or not.
sebzim4500 3y ago
You can as long as the resulting model does not compete with OpenAI.
- kiratp 3y ago
  It can be argued that if you build a model using their outputs such that you can then stop using their API, your model is effectively competing with their’s. Let’s just say that if you’re a startup or SMB, you do not want to be the one dragged to court to iron out whether this holds or not.
wanderingmind 3y ago
This is just ToS violation which will just result in loss of access to OpenAI. There is nothing they can do to stop you from commercially competing, given there is no copyright law precedence

throwawayForMe2 3y ago

>> The next major leap in LLMs (in the next year) is probably going to be the prompt context size. Right now we have 2k, 4k, 8k ... but OpenAI also has a 32k model that they're not really giving access to unfortunately. Saw this article today about a different approach that opens up orders of magnitude larger contexts https://hazyresearch.stanford.edu/blog/2023-03-07-hyena https://hazyresearch.stanford.edu/blog/2023-03-07-hyena