9 ms·

Every tech company minus the few doing core research have been doing this for at least half a year. Generate training data with GPT4 or sometimes even 3.5 -> us

Every tech company minus the few doing core research have been doing this for at least half a year. Generate training data with GPT4 or sometimes even 3.5 -> use it to do a QLoRA finetune on a llama or mistral base -> roll it out as a "proprietary" AI model -> management claims a big win and talks about how they're leaders in "[industry name] AI". It is remarkably easy - it takes practically zero knowledge of ML and can usually be done with less than <$1k of cloud compute costs. The issue is that for most realistic tasks you can expect to end up with something roughly on the level of GPT-3.5, and its actually really hard to compute with GPT-3.5 on a cost level, at least if you use cloud GPUs.

padolsey 3y ago
> Every tech company minus the few doing core research have been doing this for at least half a year I'm assuming you mean all those new 'AI wrapper' startups popping up.? I wouldn't say "every tech company". But yeh it seems incredibly easy, definitely an easy win and leaders get to feel ahead of the curve on AI.
- deleted 3y ago
  [deleted]
- phillipcarter 3y ago
  I agree, in fact I would wager the opposite, that most who claim to be a tech company are simply using OpenAI or another vendor with a turnkey API for things launched recently. Over time I expect more use of fine-tuned models, but fine-tuning is not easy, especially if your goal is GPT parity (or better).
  - atleta 3y ago
    The thing I don't understand about this strategy is that it itself shows that there really is no money to be made here. I mean it's a pretty obvious giveaway that: 1. they don't have the resources to build their own technology and probably never will 2. even if they did have, the best they could do is come up with something very similar to OpenAI's GPT, i.e. a (somewhat) generic AI model. This means that OpenAI can also easily compete with them. All these companies are doing (if anything) is that they test the market for OpenAI (or Google, MS) for free.
    - ignoramous 3y ago
      The flaw in your assumption is that perfect tech or tech powerhouses win. I mean, sure when they do, they win big; but the endgame for b2b SaaS is mostly M&A, powered by sales, which is mostly down to c-suite relationships and perception of being one among the market leaders ("nobody ever got fired for buying IBM"). If you can move fast, deliver, expand, and raise money, there's a good chance the AI wrapper lands a nice exit and/or morphs into a tech behemoth. Those outcomes (among others), even if mutually exclusive, are equally possible.
      - atleta 3y ago
        So, if I understand you correctly, the business strategy for an AI wrapper company would be that they acquire customers quickly from a specific niche, build a name, while having very little custom technology and then get acquired by some of the larger players who do have the actual AI tech in-house. And, for them, it would be worth it for the brand/market/existing client base. Assuming that the advance made in the meanwhile in AI doesn't eradicate the whole thing. I mean say some company builds a personal assistant for managers to supplant secretaries, they become the go-to name and then Google buys them in 2-3-5 years. Unless Google's AI becomes so good in the meantime that you can just instruct it in 1-2 sentences to do this for you.
        ignoramous 3y ago
        > get acquired by some of the larger players who do have the actual AI tech in-house. And, for them, it would be worth it for the brand/market/existing client base. The key is, if the incumbents truly feel they can't breach whatever moat, M&A is the safer bet over agonizing what if (I am thinking "git wrapper" startups that saw plenty competition from BigTech; remember Microsoft CodePlex, Google Code, AWS CodeCommit?). Given Meta's push and other prolific upstarts (OpenAI, Mistral), I don't believe access to SoTA AI itself (in the short term) will be an hindrance for product-based utility AI businesses (aka wrappers).
- pclmulqdq 3y ago
  No, as far as I have seen, the "AI wrapper" companies have been clinging to GPT-4 a lot faster than other tech companies. Many bigger companies deploy GPT-4 very sparingly if at all.
Dr_Birdbrain 3y ago
I have a question about this—-isn’t it against the OpenAI Terms of Service to do this?
- ssalka 3y ago
  Have the same question - I mean, for training an open source model with no monetization attached, not much Open AI can do besides ban the user, but they can make another account. For a company doing this with the intent to sell it as a capability... seems risky.
- phillipcarter 3y ago
  Yes, it's explicitly against their TOS. > What You Cannot Do. [...] > Use Output to develop models that compete with OpenAI.
  - jacquesm 3y ago
    What's good for the goose...
  - raverbashing 3y ago
    If you're not selling/ putting your model out there as a generic competitor to OpenAi then you're not competing with them
    - phillipcarter 3y ago
      That's a moving target :)
  - throwaway5959 3y ago
    But didn’t X do that with their ML model Grok?
    - phillipcarter 3y ago
      Burning bridges and getting sued isn't uncharted territory for Elon Musk.
nextworddev 3y ago
And even this analysis is optimistic as it doesn’t factor in the $$$$ it costs to hire a data scientist to fine tune the models. Just use off the shelf models with RAG until you really need custom models
- ignoramous 3y ago
  You've a point. Doesn't look like RAG (w/ in-context learning) or fine-tuning add domain knowledge to the LLM, so some claim they're equivalent: https://twitter.com/Shahules786/status/1748059074556760421 https://twitter.com/Shahules786/status/1748059074556760421 / https://archive.is/iRN5j https://archive.is/iRN5j Proxy-tuning (https://twitter.com/rasbt/status/1748021765790376385 https://twitter.com/rasbt/status/1748021765790376385 / https://archive.is/oQs0m https://archive.is/oQs0m) and other such merged models(https://twitter.com/osanseviero/status/1745121420353454219 https://twitter.com/osanseviero/status/1745121420353454219 / https://archive.is/hFYbh https://archive.is/hFYbh) are an interesting area of study, too
siliconc0w 3y ago
I'm curious if you can actually get better than 3.5 though considering how meh it is at most applications. I'd be nice to know whether I could actually get a better model without the effort of trying this.

padolsey 3y ago

> Every tech company minus the few doing core research have been doing this for at least half a year I'm assuming you mean all those new 'AI wrapper' startups popping up.? I wouldn't say "every tech company". But yeh it seems incredibly easy, definitely an easy win and leaders get to feel ahead of the curve on AI.

deleted 3y ago
[deleted]
phillipcarter 3y ago
I agree, in fact I would wager the opposite, that most who claim to be a tech company are simply using OpenAI or another vendor with a turnkey API for things launched recently. Over time I expect more use of fine-tuned models, but fine-tuning is not easy, especially if your goal is GPT parity (or better).
- atleta 3y ago
  The thing I don't understand about this strategy is that it itself shows that there really is no money to be made here. I mean it's a pretty obvious giveaway that: 1. they don't have the resources to build their own technology and probably never will 2. even if they did have, the best they could do is come up with something very similar to OpenAI's GPT, i.e. a (somewhat) generic AI model. This means that OpenAI can also easily compete with them. All these companies are doing (if anything) is that they test the market for OpenAI (or Google, MS) for free.
  - ignoramous 3y ago
    The flaw in your assumption is that perfect tech or tech powerhouses win. I mean, sure when they do, they win big; but the endgame for b2b SaaS is mostly M&A, powered by sales, which is mostly down to c-suite relationships and perception of being one among the market leaders ("nobody ever got fired for buying IBM"). If you can move fast, deliver, expand, and raise money, there's a good chance the AI wrapper lands a nice exit and/or morphs into a tech behemoth. Those outcomes (among others), even if mutually exclusive, are equally possible.
    - atleta 3y ago
      So, if I understand you correctly, the business strategy for an AI wrapper company would be that they acquire customers quickly from a specific niche, build a name, while having very little custom technology and then get acquired by some of the larger players who do have the actual AI tech in-house. And, for them, it would be worth it for the brand/market/existing client base. Assuming that the advance made in the meanwhile in AI doesn't eradicate the whole thing. I mean say some company builds a personal assistant for managers to supplant secretaries, they become the go-to name and then Google buys them in 2-3-5 years. Unless Google's AI becomes so good in the meantime that you can just instruct it in 1-2 sentences to do this for you.
      - ignoramous 3y ago
        > get acquired by some of the larger players who do have the actual AI tech in-house. And, for them, it would be worth it for the brand/market/existing client base. The key is, if the incumbents truly feel they can't breach whatever moat, M&A is the safer bet over agonizing what if (I am thinking "git wrapper" startups that saw plenty competition from BigTech; remember Microsoft CodePlex, Google Code, AWS CodeCommit?). Given Meta's push and other prolific upstarts (OpenAI, Mistral), I don't believe access to SoTA AI itself (in the short term) will be an hindrance for product-based utility AI businesses (aka wrappers).
pclmulqdq 3y ago
No, as far as I have seen, the "AI wrapper" companies have been clinging to GPT-4 a lot faster than other tech companies. Many bigger companies deploy GPT-4 very sparingly if at all.

Dr_Birdbrain 3y ago

I have a question about this—-isn’t it against the OpenAI Terms of Service to do this?

ssalka 3y ago
Have the same question - I mean, for training an open source model with no monetization attached, not much Open AI can do besides ban the user, but they can make another account. For a company doing this with the intent to sell it as a capability... seems risky.
phillipcarter 3y ago
Yes, it's explicitly against their TOS. > What You Cannot Do. [...] > Use Output to develop models that compete with OpenAI.
- jacquesm 3y ago
  What's good for the goose...
- raverbashing 3y ago
  If you're not selling/ putting your model out there as a generic competitor to OpenAi then you're not competing with them
  - phillipcarter 3y ago
    That's a moving target :)
- throwaway5959 3y ago
  But didn’t X do that with their ML model Grok?
  - phillipcarter 3y ago
    Burning bridges and getting sued isn't uncharted territory for Elon Musk.

nextworddev 3y ago

And even this analysis is optimistic as it doesn’t factor in the $$$$ it costs to hire a data scientist to fine tune the models. Just use off the shelf models with RAG until you really need custom models

ignoramous 3y ago
You've a point. Doesn't look like RAG (w/ in-context learning) or fine-tuning add domain knowledge to the LLM, so some claim they're equivalent: https://twitter.com/Shahules786/status/1748059074556760421 https://twitter.com/Shahules786/status/1748059074556760421 / https://archive.is/iRN5j https://archive.is/iRN5j Proxy-tuning (https://twitter.com/rasbt/status/1748021765790376385 https://twitter.com/rasbt/status/1748021765790376385 / https://archive.is/oQs0m https://archive.is/oQs0m) and other such merged models(https://twitter.com/osanseviero/status/1745121420353454219 https://twitter.com/osanseviero/status/1745121420353454219 / https://archive.is/hFYbh https://archive.is/hFYbh) are an interesting area of study, too

siliconc0w 3y ago

I'm curious if you can actually get better than 3.5 though considering how meh it is at most applications. I'd be nice to know whether I could actually get a better model without the effort of trying this.