7 ms·

The landgrab going on in this space is ferocious. If you weren't convinced this is mania, and a 10M "seed" round for Langchain doesn't do it for you, nothing wi

The landgrab going on in this space is ferocious. If you weren't convinced this is mania, and a 10M "seed" round for Langchain doesn't do it for you, nothing will. Well done Harrison on the cash grab (take as much money off the table every round as you can), it's a smart move. But I can't shake the feeling that this ever increasing mania will sweep up anything OSS with vague traction and this whole AI space that used to be religiously defaulted to open and sharing will fairly quickly end up in VC-funded fiefdoms with pay-to-play being all that's left modulo the "forever free" community hobbled versions. Hope I'm wrong.

chpatrick 3y ago
I'm also worried that courts will decide that model weights are copyrightable and the open source free-for-all will be over.
- nwienert 3y ago
  If models can’t prove they are fully free of copyrighted data I don’t think they’ll have a leg to stand on there.
  - yuuuuyu 3y ago
    If you are holding copyright to something, it will be on you to prove it's in there.
    - manojlds 3y ago
      I am obviously clueless, but if it's a case, can't one demand what the training data is?
      - arthurcolle 3y ago
        Based on what case law? In what jurisdiction?
        nwienert 3y ago
        It's very easy to show the models generating near-identical data to copyrighted data, which is enough to get courts to force them to allow discovery.
        yuuuuyu 3y ago
        This not have happened all over the place yet is evidence against this being as easy as you make it sound.
  - nl 3y ago
    This is clearly not a given. Search engines are good decided case law in the opposite direction.
    - Nullabillity 3y ago
      Search engines aren't a replacement for the original data, they're a way to direct traffic towards it.
      - nl 3y ago
        The business model doesn't really change the IP considerations though. (Additionally, newer LLMs like Perplexity.AI's correctly cite content sources, so that is even more similar to search engines)
        nwienert 3y ago
        These models will readily generate near identical outputs to copyrighted data, at length. This is not comparable to search.
        nl 3y ago
        I'd invite you to read "Foundation Models and Fair Use"[1] which is a paper written as a collaboration between Standford's law school and computer science department. It talks at length about this specific problem and migration techniques for it: Existing foundation models are trained on copyrighted material. Deploying these models can pose both legal and ethical risks when data creators fail to receive appropriate attribution or compensation. In the United States and several other countries, copyrighted content may be used to build foundation models without incurring liability due to the fair use doctrine. However, there is a caveat: If the model produces output that is similar to copyrighted data, particularly in scenarios that affect the market of that data, fair use may no longer apply to the output of the model. In this work, we emphasize that fair use is not guaranteed, and additional work may be necessary to keep model development and deployment squarely in the realm of fair use. First, we survey the potential risks of developing and deploying foundation models based on copyrighted content. We review relevant U.S. case law, drawing parallels to existing and potential applications for generating text, source code, and visual art. Experiments confirm that popular foundation models can generate content considerably similar to copyrighted material. Second, we discuss technical mitigations that can help foundation models stay in line with fair use. We argue that more research is needed to align mitigation strategies with the current state of the law. [1] https://arxiv.org/abs/2303.15715 https://arxiv.org/abs/2303.15715
TylerE 3y ago
The question is if the 2nd AI Winter kills python the way the first killed lisp.
- FractalHQ 3y ago
  God I hope so.
  - captaincaveman 3y ago
    Could it swap, so we can have lisp back?
    - capableweb 3y ago
      It would be absolute godsend if we could, so one measly error wouldn't require restarting a entire process just to rewrite some small part.
      - TylerE 3y ago
        That's really more erlang's niche.
        freilanzer 3y ago
        No, that's really Lisps thing.
        capableweb 3y ago
        I meant more like dynamically re-evaluating specific functions without having to restart the process. Haven't done much Erlang, but my experience is around doing so in lisp rather, which definitely can do that.
- cossatot 3y ago
  Python has huge uptake outside of the ML domain. I can't see an AI winter affecting the many people using Numpy for non-ML purposes (i.e., scientists in academia, most of whom still deal with normal numerical and data analytical modeling, not ML) much less Django.
dbish 3y ago
Interestingly this is shortly after another huge “seed” round in this space from fixie.ai (17mm). A lot of money being thrown at making it easier to chain/give LLMs access to other applications and tools.
- itsaquicknote 3y ago
  Yeah, wow, a 17M seed round and not a hint of irony. Ferocious. Capital is DESPERATE to throw money into anything that appears to be related to LLMs. "Value creation event of our lifetimes" etc. There's grift money to made here and I'll be damned if some decent proportion of the "API wrapper + template UI" startups masquerading as AI companies aren't cashing in. Not sure I blame them.
  - james-revisoai 3y ago
    I agree with you, and people miss that December 2022 is not when GPT became commercially possible. Look at the startup OpenAI generations 1-2 years back - they have largely sank at comparable or worse rates than any other startup from 2020/2021. The GPT-3 first wave companies, around translations, basic quizzes, summarization tools, language learning apps, and ofcourse the notorious paraphrase tools (almost entirely obsolete since ChatGPT) can't be found in that form anymore, they've all been forced to shut down or move functionality significantly. Early on, OpenAI limited output to 300 tokens max - and less for most usecases, often 50-150. Chatbots were not allowed IIRC for over a year. If ChatGPT hadn't came along, much of what langchain enables wouldn't have either, nor so many big companies willing to now risk. I can count none over 2 years old which have not since been made obsolete by raw ChatGPT access or are now default dead due to competition from existing unicorn (e.g. Duolingo, Quizlet Q chat) who are now crushing them. It must be painful to have spent $xx,xxx on GPT-3 at $0.06 a token to obtain users, and now have your market ripped from you by a $B+ company paying $0.006... So I doubt the template UI startups will sustain retention or stay about long term, unless they really find traditional startup ways to nuggle into niches and use cases/vendor lock in. This isn't an innovators dilemma for most companies, it's just an obvious sensible thing to try at this point, so startups don't have much to balance with on risk. That being the case, the market surely should seem less appealing than the open ended "blue ocean new value" of 2021 GPT tech, but - I guess not.
legitimayzer 3y ago
I think you're absolutely correct. But this is by 'design', I have very unreasonable suspicions that indeed, the whole VC 'world of entrepreneurs' is just the way the USA government does R & D on an industrial-corporate scale. The 'brilliance' behind this way of doing R&D is that they only pick up the winners after they won, so they don't "waste" money on R&D death ends nor moonshots. on the other hand, this is a good way to 'explode' for cheap the technological applications of already developed scientific innovations. meaning none of those VC-backed startups are doing innovative research, but in fact are devleoping commercial applications for corporate overlords who having seen who won, step in to buy them out. even music industry is shifting to that model, they are now only signing bands/artists/influencers who already build their audience.
- xiphias2 3y ago
  It's far superior to the EU model where politicians are tasked with creating requirements for ,,the next innovations'' and creating tenders, businesses popping up / recycled to win those tenders, do the minimum to satisfy the criterion that nobody really cares about, and try to siphon out as much money as possible using trusted subcontractors, meanwhile paying a huge part of the money for the privilage of winning the tender. At least that's what's going on in Hungary, the most corrupt government in EU, I hope other parts are a bit better.

chpatrick 3y ago

I'm also worried that courts will decide that model weights are copyrightable and the open source free-for-all will be over.

nwienert 3y ago
If models can’t prove they are fully free of copyrighted data I don’t think they’ll have a leg to stand on there.
- yuuuuyu 3y ago
  If you are holding copyright to something, it will be on you to prove it's in there.
  - manojlds 3y ago
    I am obviously clueless, but if it's a case, can't one demand what the training data is?
    - arthurcolle 3y ago
      Based on what case law? In what jurisdiction?
      - nwienert 3y ago
        It's very easy to show the models generating near-identical data to copyrighted data, which is enough to get courts to force them to allow discovery.
        yuuuuyu 3y ago
        This not have happened all over the place yet is evidence against this being as easy as you make it sound.
- nl 3y ago
  This is clearly not a given. Search engines are good decided case law in the opposite direction.
  - Nullabillity 3y ago
    Search engines aren't a replacement for the original data, they're a way to direct traffic towards it.
    - nl 3y ago
      The business model doesn't really change the IP considerations though. (Additionally, newer LLMs like Perplexity.AI's correctly cite content sources, so that is even more similar to search engines)
      - nwienert 3y ago
        These models will readily generate near identical outputs to copyrighted data, at length. This is not comparable to search.
        nl 3y ago
        I'd invite you to read "Foundation Models and Fair Use"[1] which is a paper written as a collaboration between Standford's law school and computer science department. It talks at length about this specific problem and migration techniques for it: Existing foundation models are trained on copyrighted material. Deploying these models can pose both legal and ethical risks when data creators fail to receive appropriate attribution or compensation. In the United States and several other countries, copyrighted content may be used to build foundation models without incurring liability due to the fair use doctrine. However, there is a caveat: If the model produces output that is similar to copyrighted data, particularly in scenarios that affect the market of that data, fair use may no longer apply to the output of the model. In this work, we emphasize that fair use is not guaranteed, and additional work may be necessary to keep model development and deployment squarely in the realm of fair use. First, we survey the potential risks of developing and deploying foundation models based on copyrighted content. We review relevant U.S. case law, drawing parallels to existing and potential applications for generating text, source code, and visual art. Experiments confirm that popular foundation models can generate content considerably similar to copyrighted material. Second, we discuss technical mitigations that can help foundation models stay in line with fair use. We argue that more research is needed to align mitigation strategies with the current state of the law. [1] https://arxiv.org/abs/2303.15715 https://arxiv.org/abs/2303.15715

TylerE 3y ago

The question is if the 2nd AI Winter kills python the way the first killed lisp.

FractalHQ 3y ago
God I hope so.
- captaincaveman 3y ago
  Could it swap, so we can have lisp back?
  - capableweb 3y ago
    It would be absolute godsend if we could, so one measly error wouldn't require restarting a entire process just to rewrite some small part.
    - TylerE 3y ago
      That's really more erlang's niche.
      - freilanzer 3y ago
        No, that's really Lisps thing.
      - capableweb 3y ago
        I meant more like dynamically re-evaluating specific functions without having to restart the process. Haven't done much Erlang, but my experience is around doing so in lisp rather, which definitely can do that.
cossatot 3y ago
Python has huge uptake outside of the ML domain. I can't see an AI winter affecting the many people using Numpy for non-ML purposes (i.e., scientists in academia, most of whom still deal with normal numerical and data analytical modeling, not ML) much less Django.

dbish 3y ago

Interestingly this is shortly after another huge “seed” round in this space from fixie.ai (17mm). A lot of money being thrown at making it easier to chain/give LLMs access to other applications and tools.

itsaquicknote 3y ago
Yeah, wow, a 17M seed round and not a hint of irony. Ferocious. Capital is DESPERATE to throw money into anything that appears to be related to LLMs. "Value creation event of our lifetimes" etc. There's grift money to made here and I'll be damned if some decent proportion of the "API wrapper + template UI" startups masquerading as AI companies aren't cashing in. Not sure I blame them.
- james-revisoai 3y ago
  I agree with you, and people miss that December 2022 is not when GPT became commercially possible. Look at the startup OpenAI generations 1-2 years back - they have largely sank at comparable or worse rates than any other startup from 2020/2021. The GPT-3 first wave companies, around translations, basic quizzes, summarization tools, language learning apps, and ofcourse the notorious paraphrase tools (almost entirely obsolete since ChatGPT) can't be found in that form anymore, they've all been forced to shut down or move functionality significantly. Early on, OpenAI limited output to 300 tokens max - and less for most usecases, often 50-150. Chatbots were not allowed IIRC for over a year. If ChatGPT hadn't came along, much of what langchain enables wouldn't have either, nor so many big companies willing to now risk. I can count none over 2 years old which have not since been made obsolete by raw ChatGPT access or are now default dead due to competition from existing unicorn (e.g. Duolingo, Quizlet Q chat) who are now crushing them. It must be painful to have spent $xx,xxx on GPT-3 at $0.06 a token to obtain users, and now have your market ripped from you by a $B+ company paying $0.006... So I doubt the template UI startups will sustain retention or stay about long term, unless they really find traditional startup ways to nuggle into niches and use cases/vendor lock in. This isn't an innovators dilemma for most companies, it's just an obvious sensible thing to try at this point, so startups don't have much to balance with on risk. That being the case, the market surely should seem less appealing than the open ended "blue ocean new value" of 2021 GPT tech, but - I guess not.

legitimayzer 3y ago

I think you're absolutely correct. But this is by 'design', I have very unreasonable suspicions that indeed, the whole VC 'world of entrepreneurs' is just the way the USA government does R & D on an industrial-corporate scale. The 'brilliance' behind this way of doing R&D is that they only pick up the winners after they won, so they don't "waste" money on R&D death ends nor moonshots. on the other hand, this is a good way to 'explode' for cheap the technological applications of already developed scientific innovations. meaning none of those VC-backed startups are doing innovative research, but in fact are devleoping commercial applications for corporate overlords who having seen who won, step in to buy them out. even music industry is shifting to that model, they are now only signing bands/artists/influencers who already build their audience.

xiphias2 3y ago
It's far superior to the EU model where politicians are tasked with creating requirements for ,,the next innovations'' and creating tenders, businesses popping up / recycled to win those tenders, do the minimum to satisfy the criterion that nobody really cares about, and try to siphon out as much money as possible using trusted subcontractors, meanwhile paying a huge part of the money for the privilage of winning the tender. At least that's what's going on in Hungary, the most corrupt government in EU, I hope other parts are a bit better.