8 ms·
Corporate America is getting hooked on open-source AI
https://archive.is/kmOqm https://archive.is/kmOqm
- deleted 13d ago[deleted]
- slowin 13d agoI'd love to! For real coding though, SOTA models barely get the job done. It wasn't until Opus 4.5 that you could really get decent results. I'm sure this will change (and I can't wait for it!) but as of today, open models might be fine for summarizing and writing docs, but you need SOTA to work on code if you want to be competitive.
- kbwal7 13d agoI do think there are now open weight models that are on par with (or beating) Opus 4.5 by now (e.g. Kimi K3, GLM5.3). But yeah obviously the frontier closed source models seem to have pulled away once again, so open weight seems to be a few months behind right now (which might be too long to wait for a lot of people!).
- redox99 13d agoThose two you mentioned completely demolish opus 4.5. It's not even close. I'd say they are between opus 4.8 and opus 5. And better in some tasks.
- water-drummer 13d agoOpus 5 feels like a regression compared to 4.8
- happycube 12d agoAnd if you run it yourself, you know it will be generally consistent with no random quality dips.
- nemomarx 13d agoEvery time I check in on this I hear a more recent model is the one where they started doing good work. I'm excited to here that Astra is where it got capable enough to work on code next year
- tomashubelbauer 13d agoI think it truly just was Opus 4.5 where LLMs became usable for coding.
- horsawlarway 13d agoYou're not corporate America (and trust me, I mostly mean that as a plus). I also work in software, and while I vaguely disagree that open models can't be used (they absolutely fit into productive niches here, and holy hell are the last generation [ex laguna s1, kimi k3, glm 5.3, etc] actually decent) - I will agree that SOTA are a better fit for software development, especially when used in conjunction with an already very expensive employee who's driving them. But for "Corporate America"... no. You absolutely don't need SOTA. They're doing things like transcription, summarization, customer interaction, minor technical tasks like form creation in existing tools, report generation (ex - powerpoint, pdf, docs, etc) and other general "white collar tasks". Think about roles in business that are in the 60-85k compensation range. It's mostly busy work that keeps existing processes flowing and the business on the rails. Important, but not research/novel. And cheap ai... is a wonderful fit for a lot of this. No one wants to replace an employee making 80k with a less reliable AI that costs 45k a year in tokens (SOTA). But they're absolutely willing to drop 2-3k/year on AI (~100/month - right in the open model cost range) for that employee if they can get a 10% bump in productivity or happiness.
- slowin 13d agoYeah, I totally agree. I'm sure these open models are more than good enough for non-dev work. I'm also sure they'll be good enough for dev work soon enough (and some people are saying the latest already are). My point was that given the difficulty of writing great code and dealing with large systems, SOTA just recently emerged as a viable option. I expect open models to catch up soon.
- tinyplanets 13d agoGood luck to the poor shleps trying to make a living performing "white collar tasks" I guess, right? They can all go be poets or painters...
- patja 12d ago"real coding" carries a lot of the weight in that comment. Seems like if you ask 5 different people what "real coding" means you might get 5 different answers. Not everyone is building the next framework or compiler. Self-hosted Qwen 3.8 @Q4 on my RTX 3090 can produce beautiful functional CRUD pages and apps all day long. And that is 90% of the "real coding" being done in corporate settings. The quote in the article about Mazda vs. Maserati captures this. Many might want the Maserati and drool over its specs and capabilities, but balk at the cost and how often are they really going to run it up to full performance limits on their daily commute to their cubicle?
- transdev12 13d agoI generally use sonnet 5 for most coding tasks, a lot of coding is really pretty straightforward.
- spopejoy 11d agoI'm having no difficulty getting deepseek v4 to blast out good code. I guess devs want to outsource all of their thinking now? Yes open models maybe can't design the whole thing soup to nuts but why is that necessary?
- gflh73 13d agoHow is the NYT's copyright lawsuit against OpenAI going and why have you abandoned your start witness Suchir Balaji? Have you been brought into line? Open source AI also violates copyrights.
- Der_Einzige 13d agoMost open source fans are also hostile to copyrights existence and are openly IP abolitionists. As such, they collectively respond with "good." This is actual communism, and the fact that Bernie Sanders and every other member of the DSA isn't actively fighting for open source and is often fighting against all AI shows how fake their purported movements are and have always been.
- fcarraldo 13d agoWhile I don’t disagree that they should be fighting for open source AI, Sanders is hardly “fighting against all AI”. https://jacobin.com/2026/07/ai-nationalization-sanders-libertarians-property https://jacobin.com/2026/07/ai-nationalization-sanders-liber...
- shimman 13d agoI've been disappointed with Sanders lately, his age is showing and how he got convinced to give these LLM companies a massive federal bailout is bonkers.
- 2948154 13d agoWho cares about power user fans? They don't own the copyright. Open source authors have always been protective of their copyright. There are numerous examples when drivers have been copied between BSD/Linux (I forget which direction) which led to huge flame wars. The whole point of the GPL is that it uses copyright and copyright assignment to the FSF to protect what it calls software freedom. BSD authors are very upset if the attribution clause isn't observed. And so on. It is communism to exploit poor open source authors? I have to read Marx again.
- 13d ago
- petcat 13d ago> Some U.S. firms remain reluctant to use Chinese A.I. models because of concerns over regulation and data privacy. AT&T researches Chinese models but is not using them, Mr. Markus said. Instead, it is working with popular alternatives made by American companies such as the Gemma A.I. model from Google and the Llama A.I. models from Meta. This makes sense since corporations require legal certainty, and using an open model from an American company (probably) provides them some level of indemnity, and also someone to sue.
- Waterluvian 13d agoThe current U.S. regime is also replacing some amount of that legal certainty with regime fealty. Picking Chinese options over American ones probably runs a risk of upsetting their leader. I've got to imagine American companies are weighing this factor in their decisions.
- petcat 13d agoThat's certainly one factor, yes. But even before getting to that part I think the bigger issues for Big Corp legal teams is mostly around the legal ambiguity of the models themselves. What representations are made about the training data? What jurisdiction governs the license? If somebody alleges that the model infringes their IP, what rights does AT&T have? Counterparty risk is a lot more straight-forward to evaluate when dealing entirely within the US, with US companies.
- honr 13d agoTo me, the legal concerns seem blown out of proportion. If you use open weight models (of dubious origin) to generate code, you can still verify them with code review and tests and other methods, used to verify human output, right? That is still a great win, maybe not as much as having AI write all the code, but that would be a reasonable point in control vs quantity spectrum for most solid and well made products. But if you let LLMs talk to people (customers, for example) directly, then yes, you need an LLM provider that you can hold responsible.
- 13d ago
- optimalsolver 13d agoGoogle can make themselves the heroes of the AI story by releasing a 120B dense Gemma model.
- amelius 13d agoThey already released the transformers paper, and I'm sure they are now scratching their head about it.
- _doctor_love 13d agoEasy prediction: LLMs will get shrunk down further and further until GenAI is just something that ships on a chip as part of your hardware. In the future it will seem quaint that we needed a network connection to talk to our LLM. Adoption of open-source models to my mind is a similar step in that direction. In all cases, the goal is to become untethered from a mercurial vendor.
- honr 13d agoLike taalas.com (very recently acquired by AMD), or cerebras.ai (whole wafer is a chip)? As you said, I also think that is one of the main direction many companies (and academia) is moving to.
- wnmurphy 13d agoYeah, I'm looking forward to this actually. https://chatjimmy.ai/ https://chatjimmy.ai/ blew my mind at how fast etched model weights can be. For on-device LLMs, there's a point of diminishing returns, meaning you don't need to have the latest frontier model for most operations.
- hparadiz 12d agoI currently have a small TTS model running in the background on my machine through which my agent(s) speak to me as they work. If that can be baked into an ASIC along with a few thousand voices in every major language then it should just be a utility chip on your mobo for anything that needs it. And yes, I too, am looking forward to it.
- andriy_koval 13d agofor every small GenAI model there will be larger model or cluster of models which are smarter than small model
- hparadiz 12d agoWe're gonna start baking in models like TTS with thousands of voices available in any language as a chip on device. They just need to hit 99% accuracy and then it's a done deal.
- syntaxing 13d agoI swear, Qwen 3.8 27B @ Q8 is smarter than Sonnet 5 most of the time. Why wouldn’t corporate America self host at this point, especially with better options like Deepseek Flash and GLM 5.3 flash that’s a middle ground between Sonnet and Opus
- SamInTheShell 13d ago@q4 is definitely smarter than sonnet from what I’ve seen so far. It’s even caught problems in code made by fable, when using it as a code reviewer.
- r_lee 13d agois this actually the case? I haven't kept up with the small models but if there's roughly Sonnet 4.6 level capable open small models, then I'd be impressed
- Zambyte 13d agoQwen 3.8 27b has the juice. Try it.
- jhonof 13d agoQwen 3.8 27B is better than Sonnet 4.6
- syntaxing 12d agoQwen 3.8 27B is the real deal BUT remember to use froggeric template and/or medium reasoning. https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
- majorchord 13d agoAgreed. And conversely, American models can also just as easily be secretly influenced for bad things, or be more tightly controlled by the government, to corporate America's own detriment.
- 13d ago
- cmiles8 13d agoEvery larger company I talk to these days has an active project on moving away from OpenAI and Anthropic to open models. And they’re actively shifting, as the article says, so the threat is far from theoretical. Unless they both dramatically slash prices then they’re in big trouble. Neither of them can afford to do that and both desperately need to convince the street that the opposite will happen if they want any hope at a successful IPO. However the cold reality for both is that there is zero moat to a model anymore. It’s a pure commodity. Those selling compute and access to open models are gearing up to wipe the floor with Open AI and Anthropic.
- r_lee 13d agoI knew that there was no real moat from the very start, I mean, these things were close enough from the very start, how could it not result in a race to the bottom, especially as you can't really prevent distillation reliably?
- 0xbadcafebee 13d agoA race to the bottom is where you lower standards, wages, or regulations to cut costs and attract business. What's actually happening is the opposite: a race to the top. Every model is trying to get better. Simultaneously they also happen to be getting more cost effective, but it's sort of a coincidence. Companies still require very good models, but they are not picking the "absolute best at any cost" anymore, because it turns out "any cost" isn't worth it.
- calebkaiser 13d agoI support and use open models as much as possible, but I'm not totally convinced that OAI or Anthropic have no moat, even as open models catch up to the frontier. Serving and inference are still hard problems when you're talking about a 2 trillion parameter model. Fine-tuning, if that remains a realistic need for businesses, is also a difficult infra problem at that scale. In the most bearish case, where there is no competitive advantage to using their models, big labs still have an advantage in this area. Maybe there is some threshold where the price/quality math for your standard business tips in favor of smaller models and self-hosting the entire stack. I'd certainly love that.
- natebc 13d ago[dead]
- chasd00 13d agoI'm not seeing it. Corporate America needs someone they can sue if anything goes sideways with AI given the rate of change and legal ambiguities. It took years/decades for actual, real, open source to be widely adopted in corporations for the same reasons.
- dboreham 12d agoNot seeing it either. I do see a torrent of astroturf saying everyone is using self hosted LLMs, but in reality nobody I know is.
- jjfoooo4 11d agoWhat monied interest is trying to gin up enthusiasm for open source models?
- cmiles8 13d agoCorporate America has been using open source for decades, and it wasn’t anywhere as slow as you portray. This argument simply doesn’t hold water. Besides, in their present rather dire financial state there isn’t much to sue these companies for anyway cash wise. NYTimes is suing on IP grounds.
- chasd00 12d agoit holds all the water, to this very day using open source in my client's projects requires approval from legal.
- therealdrag0 12d agoAnd does legal ever give approval?
- nozzlegear 11d agoI fear we may never know the answer.
- MrResearcher 13d agoWhat IDE/extensions do you use for open-source LLMs? I tried VSCode with ollama and lm studio, and the experience is very subpar to the built-in copilot. It's not very usable.
- hparadiz 13d agohttps://huggingface.co/Qwen/Qwen3.8-27B https://huggingface.co/Qwen/Qwen3.8-27B No one I know uses the built in VSCode extensions anymore. It's all TUIs now. You can use Opencode as a TUI now for local.
- Zambyte 13d agoSeconding the TUI usage. Opencode, codex, and pi are ones that I have / my friends have had success with using locally. I mostly use pi. It feels like the most "boring tool that just does its job" out of the big options.
- Quibblingeek 12d agoI’ve been using Opencode, but recently migrated to oh-my-pi and have really enjoyed it. I’ve also had luck with copilot-cli, but find that to be more limiting and it’s only really worth using if you are paying for GitHub Copilot (or your company pays for it, as is my case).
- cisrockandroll 13d agoJust use OpenRouter
- arbuge 13d agoI think what they're really getting hooked on is the lowest cost provider. Which makes the Muse 1.3 launch this week particularly interesting, although to get the low cost version you do need to agree to share data with Meta.
- AnotherGoodName 13d agoI actually think they’re hooked on models they can fine tune. You can’t further train the closed models. The open models can be fine tuned for your company. Big companies fine tune models on all the internal systems and documentation, not just through .md files (you’d blow up the context trying it that way) but actual fine tuning of open weights models. A low tier but open weights model actually beats frontier models when you do this for a specific task. I think the frontier providers need to have a way to isolate instances (bedrock style?) and allow fine tuning to compete. Big companies are absolutely fine tuning models right now and getting better results than even the best frontier models for their use cases.
- motbus3 13d agoI had a conversation with a large group of friends and we independently came to the conclusion that Openai/Claude does not deliver more than a open source model. It takes about the same and the quality is about the same, and this does not mean it is good
- overfeed 12d agoI'm just glad the dweebs who were parroting "OMG, Distillation attacks!!1!" have been empirically proven wrong, and discussion around open-weight models are more rational now.
- happycube 12d agoDistilled data or no, the Chinese labs are bringing a lot of solid original work in training and inference efficiency. Deepseek's work has probably made everyone's AI cheaper to run by this point.
- schopra909 13d agoIMO this will be a blip. There’s a lot of talk in the wake of all the Uber handwringing about token spend. Legacy enterprises want to look innovative to Wall Street without spooking them, so it’s easy to hop on the narrative and “show” that they’re innovating in a cost responsible manner. This feels reminiscent of the big push to RAG a few years ago. And, more broadly the skunkworks projects that big companies tout in the press before they end up killing, when the operational overhead becomes too much for their liking. Ultimately, the narrative is good for the consumer and the enterprise. It’ll mean OpenAI and anthropic will have to keep prices low. But ultimately, in the course of the next 10 years, I don’t see enterprises wanting to do this themselves. It’ll just be simpler (and eventually safer in their eyes) to send traffic to the big labs.
- CodingJeebus 12d agoI don't see this being a blip, for multiple reasons. A) the uptime matrix between Github, OpenAI, and Anthropic means that we've faced multiple entire days of not being able to ship org-wide due to our reliance on automated code review and other tooling. Every cloud service baked into our CI pipeline becomes a point of failure. We can't live without AI anymore, but it too often either directly or indirectly gets interferes with our ability to ship, and I don't see this improving any time soon. B) Locally hosted AI has serious advantages with regard to PII/sensitive data management, and there's not much the frontier models can do to overcome this. There are so many things I want to build and let loose in a sensitive data environment but can't due to data governance around frontier models. C) Anthropic and OpenAI cannot keep prices low forever. They're still burning insane amounts of cash and at some point, they're going to have to transition from growth mode to profit mode. They're already juicing their sales pipelines to the max with introductory pricing and other things to get people in the door. But those are all short-term online marketing plays.
- schopra909 12d agoTotally hear you on point a/b! I think ultimately the folks with the purses won’t care enough about a for it to be taken seriously, even if it’s an engineering bottleneck. B definitely has scope but still smaller than I’d expect. When I was an intern at yelp, I was migrating us off internal credit card management to Braintree/stripe. No one would have imagined outsourcing that in early 2000s. There’s a long tail of stuff that you can’t sent to a 3rd party; but for most use cases it’ll suffice. For c, true; but this is expensive for everyone, including the Chinese model companies that are trying to undercut Open Ai / anthropic. In the nth degree, i think the field will bring the cost down to the place where it’s manageable (see the existence of the cheap Chinese models). The real question is the $$ spent on pushing the research forward at scale.
- iainctduncan 13d agoThe thing is, for big companies (or even small ones owned by PE, which is MOST of them), it's not just about cost. The big thing is risk. In my experience as a tech diligence assessor for PE firms for the last 7 years, investors really, really don't like companies being beholded to single entities that they don't control. Anthropic and OpenAI have demonstrated that they are not trustworthy, or predicatable, or finanically safe, or even capable of hitting three fucking nines. Investors know they need companies to be on the AI train, but they really don't like vendor lockin to the big AI companies. Every diligence I get asked "how easily can they change models?" I think when open models reach 80% or 90% capability (or maybe even less!) a whole lot of companies are going to say "almost as good with way less risk is a better deal".
- MrDresden 12d agoHow does this compare to being beholden to a single cloud providers like AWS or Google Cloud? Has there been a sea change in how investors view these things in general, or is it only AI?
- solid_fuel 12d agoWell the big thing that stands out in the reliability front is that AWS and Google cloud are typically stable, certainly more than 3 nines. Meanwhile openai seems to roll dice on every request to see if they’re going to return a 500 or not.
- ozozozd 12d agoI can’t personally tolerate the AWS/GCP interfaces and clickops. But in terms of reliability - uptime, product, legal - they are in a different league. There were exactly 0 instances waking up to a product decision at AWS completely breaking your product or workflows.
- iainctduncan 11d agoOur clients do indeed see multi-cloud as a nice-to-have, but it costs so much more to do, and the big clouds are so trusted, that it's rare that I see it. On the other hand, nobody trusts the big AI companies not to pull shenanigans or dramatically raise prices... or even be in business in five years.
- swader999 13d agoAt our small company we are hooked on individual subs. But yeah a larger dev shop can't really pull that off and I get how they'd be dying by the token cost.
- falaki 13d agoThey couldn't pick a more sinister headline for such an awesome technological development.
- dominotw 13d agohow much of performance comes from inference time tricks like scaling, topn ect . maybe models providers are also in position to run their models vs running os models by a generic providerc
- hintymad 13d agoI think using open-source AI is no longer about API cost but about company survival. Take Anthropic for an example. Anthropic has successfully destroyed customer trust, at least for me. DHH in a recent interview mentioned that Claude refused to translate an article about immigration. Not summarize. Not editorialize. Translate! I think this reveals an unacceptable level of paternalism: Anthropic fundamentally believes that it possesses a moral authority superior to the people actually paying for the API. If such basic and mechanical translation is already too sensitive to touch, the goalposts have moved from safety into outright censorship. What prevents them from quietly deciding tomorrow that your proprietary business logic, financial data, or legal documents cross their invisible moral line? Let alone how Anthropic treats Cursor and Figma - not that they are wrong as companies are free to compete legally, but nonetheless it shows that companies can't outsource their intelligence to a potential competitor.
- eli 13d agoI get what you're saying and it's concerning how much power these big labs have amassed and how little transparency there is in what they do with it... But I doubt this a major factor in the trend. I just don't think it's something most corporate users run into. My understanding is these guardrails are negotiable for enterprise customers anyway. And, not for nothing, but if I owned a human-powered translation company I would've refused to translate it too.
- thatmf 13d agoAh, yes, I'm sure the article that moral paragon DHH wished to translate was not at all harmful, and that this was a good-faith effort on his part /s While I agree that Claude can be overly paternalistic at times, how should it respond to a request to translate, say, bomb-making instructions? It's reasonable to me that it might refuse this.
- jimbokun 13d agoHow do you know what was in the article? Also, you see zero distinction between hearing opinions on political topics you might find objectionable, and building a bomb to kill people?
- biophysboy 13d agoI have recently come to the conclusion that thinking for 2 seconds and using a cheap model with a slightly more detailed prompt works just as well as zero-shotting an idea with a fancy model. I work in science, and instead of asking the model “write a topic extraction algorithm”, I just say “hey look at this matrix factorization script I found in a repo, now make it use plotly and duckdb”. Have others come to the same conclusion here? It makes me skeptical that the flagship companies are sustainable. Every company is going to maximize “fuel efficiency” to save time and money. Then again, maybe the cheaper models have more markup for them, in which case they are probably happy w this arrangement. I’d be curious to know how the money making varies by model.
- woah 13d agoReally? It's worth it to you to spend 10 minutes thinking about how to prompt a dumber model to save $0.05? (not that open source models are dumber any more)
- biophysboy 13d agoYes? The difference is often 10X or 100X with very little time lost. I’m learning how to give it enough info pretty quickly. Edit: I also have to read the methods anyway for scientific accountability/integrity anyways, so I may as well play that role at the outset.
- mylies43 13d agoWell its def not $0.05, I just started using claude sonnet 5 and Ive found most simple questions might be 0.05 cents, a unit test is something like 0.10 -> 0.20 and small features and classes get into the individual dollars. Sure its a lot faster but at the end of the day its not cheap. Plus there is something to say about being in the drivers seat, youll have a much better idea of how it works instead of needing to talk to claude and hope its correct. Since most LLMs also not very good at ideas even in my experience with better models its better to think for 10mins, youll get a much high quality result
- 13d ago
- manyatoms 13d agoMy long term guess: A&OAI will move away from being interference providers to just training models and then licencing the models for local use
- overfeed 13d ago> licencing the models for local use Who's going to be the new Bill Gates, with a vision for "a GPU cluster in every home?"
- anon373839 12d agoJensen Huang?
- simonw 13d ago> AT&T turned to artificial intelligence models from Anthropic and OpenAI in recent years to help with customer service, call transcription and coding. [...] > By May, open models accounted for 20 percent of AT&T’s A.I. use. That has since risen to 40 percent and may jump to 60 percent in the coming months, Mr. Markus said in an interview. This is missing a crucial detail. We know they "help with customer service, call transcription and coding", but which of those have been upgrade to open models? Call transcription is trivial to do with open models. I can run Whisper or Parakeet on a low-spec laptop. "Customer service" could mean a lot of things, but it sounds feasible for open models too. "Coding" - they might go to open models for that, but I expect the costs involved in paying for closed models for software developers within AT&T are a fraction of the costs involved in transcribing all of their calls or handling aspects of custom service for millions of customers. From later in the story: > AT&T researches Chinese models but is not using them, Mr. Markus said. Instead, it is working with popular alternatives made by American companies such as the Gemma A.I. model from Google and the Llama A.I. models from Meta. Gemma 4 is great, but really, Llama, in 2026?
- mewse-hn 12d ago> Gemma 4 is great, but really, Llama, in 2026? I'd assume the author is just getting confused because of ollama and llama.cpp and all the other ecosystem "llama" that are still in use. Llama really did kick off the open models thing
- kittikitti 13d agoSome of the most insidious parts of AI infrastructure includes the embedding model. Corporations have already spent an outstanding amount of time and money creating embedding vectors that are closed source and not reproducible. This means that all their data is locked into whatever embedding model they chose initially. I highly recommend utilizing an open sourced embedding model instead of paying for a closed source one. It's vastly more reasonable to run an open sourced embedding model as a first step. They're much, much smaller and, due to the overhead of network latency, and running it locally has almost the same speed as through an API even on slow computers. I would even go so far as to say that closed source embedding models have a high risk of data hostage. If a team doesn't have access to the embedding model, the embeddings become useless. A corporation like OpenAI could, say, hike the prices to that model by 1000x and everyone would have to pay up or forfeit any utility of the data. I envision a future where open source embedding models are shipped with relevant technologies and implemented by currently under-utilized chips like NPU's. A startup developing cheap microprocessors that can run them is an idea I would pay cash for. Or perhaps they will be bundled with security tokens. While it might be impractical for all corporate teams to run language models, it is very realistic for everyone to operate an open sourced embedding model, at least in their private cloud. Better yet, utilize transfer learning on an open sourced one to train your own, that way the embedding vector is more secure against competitors and trade secrets.
- Zambyte 13d agoWhat are people even using embeddings for these days? It certainly seems like giving an agent grep covers most of the use cases. Dare I say: grep is all you need.
- maxrev17 13d agoMaybe the bubble doesn’t come for all of us maybe it comes for Anthropic and OpenAI.
- _superposition_ 13d agoIt's just Linux all over again. Except nowadays open source isn't a "cancer", so it will happen faster.
- shevy-java 13d agoMeanwhile the rest of the world tries to un-hook itself from corporate America. Too many problems coming from the USA lately - it is not worth it to help sustain this anylonger. Canadians have realised this - others are realising this as well right now. Mr. Trump "no more forever wars", starting another forever war.
- sriniwasx 13d ago[dead]
- Kuyawa 13d agoThe innovative edge markup already faded and the race is to the bottom, more features, more reach, less cost. It's going to be extremely hard to recoup those giant investments. No, the bubble won't pop, it already popped and morphed at the speed of AI that we didn't even notice, money just realigned, llms keep pushing the frontier, and peripherals are gaining momentum The race is still on
- Balgair 13d agoAnecdata: I use opensource models at work because my work is too cheap to spring for a $20/mo account for me. Since HuggingFace models can be run on my laptop now (still very slow though), nothing is leaving the 'secure environment' and so I can actually get work done (instead of the 'old' version of coding and writing - google).
- pllbnk 12d agoYou still need the models to be able to perform web searches, don't you? In which case the data goes in and out of your machine and there is risk for prompt injection attacks. I think it's needed at least for documentation purposes.
- Balgair 10d agoOh, I ask for help with code, yes, but the actual data I'm working on doesn't ever leave the environment.
- siliconc0w 12d agoMy hot take is that open models don't really save you money and introduce more router complexity and security risk (because you're now sending your company data through more less trustworthy providers). Look at cost per task not cost per token and the pareto curve is largely owned by closed models. Just use Fable 5.1/Opus max for the hardest problems, GPT Sol high as your workhorse, and maybe terra for async batch stuff you don't really care about. Gemini 3.8 High also looks pretty good and is quite fast if you're already a GCP shop. You can basically benefit from open models without using them because they force the frontier models to be cheaper.
- bnchrch 12d agoI don't think the "don't really save you money" hot take holds water in every case. Coding, maybe. But for operationalized/repeatable tasks it definitely does. For example I have a workflow that I was running in April that effectively would cost $30k in token spend for each full run. However now, with GLM 5.3-flash, we've brought the cost down to $7k-9k with our evals showing we've had no loss in recall, precision etc..
- FuriouslyAdrift 12d agoWe run Gemini fast for random end user queries for general staff. We have our own on-premise inference server (quad MI300A) that runs Kimi 2.8 extremely well and we transitioned all heavy work to it since it's basically instantaneous for the whole team. It's a good enough solution and we will hit break even before the end of the year already. Not everyone needs frontier models and availability is frequently much more important than a lot of companies realize.
- Animats 12d agoThe amount of downtime the big AI companies are showing doesn't help. Do you want your company's sales and customer service to shut down every time OpenAI or Anthropic goes down?
- Quibblingeek 12d agoThis is a prescient concern and it all came about due to the initial export controls placed on Fable on a Friday afternoon with zero warning. It was partly Anthropic’s fault though. Engaging in criti-hype by going on an advertising blitz explaining how their new model is some kind of genius super hacker didn’t help their case. Nevertheless, it happened. The problem is that businesses demand stability. Building an application, service, or business process around an API that can be turned off whenever a government demands represents an unacceptable risk. It’s not a mental exercise when it has already happened once before and likely will happen again. If you control the weights and invest in your own hardware to run them (or rent it from a cloud provider), that risk can be mitigated.
- happycube 12d agoThose export controls might have saved the flow of open-weight models from China. There was talk about Qwen moving away from releases... until Xi himself started promoting open weight models!
- mudil 12d agoIt's possible that LLMs will be a commodity in the future. Just like airline industry, AI will be tremendously important for society, but AI companies will not be making lots of money. It will be Nvidia, Micron, Dell and others shovel makers making money.
- c7b 12d agoThe article clearly distinguishes between open source and open weights models, and the headline and parts of the article state that it's open source models that are having a moment. But it doesn't list any examples. The only model families explicitly named are open weights only. Could someone clarify whether there are actual open source models that are competitive with the likes of Gemma (mentioned in the article), or is the headline just wrong?
- timterim 12d agoContrary to what is mentioned in the thread there are no seats anymore, it is all raw token usage nowadays.
- happycube 12d ago... and practically nobody's got the money for Fable at API prices. with GLM 5.3 Flash being a Reasonable Opus 4.x Substitute, Anthropic's moat is effectively gone.
- krupan 12d agoOpen source does not apply to AI and we should discourage anyone from using that term. All the models are opaque and proprietary. You cannot go into any source code and fix bugs, or add features, or study it to learn more. It's not the same thing at all as open source software.
- hashstring 12d agoTrue, openweight is the correct term.
- ms_menardi 12d agohowever, you can test the model, by asking it things. What are you concerned is hiding inside of that big block of numbers? Order 66?
- NullPrefix 12d agothis also applies to proprietary black box software. You can still test it, interact with it
- ms_menardi 10d agothe difference is, that a model is literally just numbers. proprietary software is data and code, and while it's true that code is data, the difference is that some data is intended to be for control flow, while in a model it is all the same stuff. You can point to any single part of an LLM's model and say "this here is a weight" but if you point to any single part of a binary file you will have no idea what you're looking at. And yes, you could decompile the binary, but that still doesn't give you the entire picture, and all proprietary software does the shady shit on servers these days anyway so you're not even going to find anything interesting.
- 8note 10d agoit isn't really just numbers though. what if i want to change out the middle part of the training, and then still run the rest as before? or if i want to cull a bunch of the initial training set? a binary is also just numbers, but we know there's other parts to it
- subarctic 12d agoGotta love this article's use of both the terms OpenAI and open A.I. which are pronounce the same but mean very different things
- wewewedxfgdf 12d agoUsing models that are created as a psy-op by foreign countries will end badly.
- solid_fuel 12d agoUsing cloud hosted models that are created as a psy-op by domestic billionaires and crypto-fascists will end badly. At least with self-hosted models the people running them get full control and don’t need to worry about the model or guardrails changing under their feet.
- lvl155 12d agoThis is not a viable strategy. You're effectively paying for hardware and/or cloud. Why do this when OpenAI and Anthropic are both significantly subsidizing costs to win the market? Talk to “AI” executives at large firms and 95% of them are clueless sales types that crawled their way to the top. Then again, it is basically a repeat of IBM, Microsoft, Oracle, etc. Same dumb executives making decision to not get fired and enjoy their place at corp.
- DEF14A 12d agoThe "AI" executives at large firms are mostly product folks with engineering backgrounds, not sales, which should make us even more worried.
- ctkhn 12d agoBeen wishing my company was doing this. We are locked in with chatgpt for chat and claude for coding. I have a 36gb m4 max laptop but I'm not allowed to run anything on it even though it would be totally free. Absolute pain.
- therealdrag0 12d agoWhy do you care for company stuff? Local models would be slower and weaker.
- taf2 12d agoSpent most of today reworking a rack and rig of gpus for all of our internal ai work… our big server is 8 rtx 6000 pro and 3 psu, I definitely feel I made a mistake not upgrading our wall power to 240v but so far we have multiple 30amp 120v and with 3 PSU uninterrupted power we have been very stable. My big upgrade will be moving to epyc motherboard from threadripper so we get gen5x8 with bifurcation instead of what we are stuck with today gen5x4 due to bios limitations . What has been so encouraging though is first deepseek v4 flash at 200+ t/s for single user and much more in aggregate- now on qwen 3.8 flash next for image support and eyes on glm5.3 flash for some testing … next is realistically considering co location and quiet a sizable loan to scale this to real hardware instead of miner rip vibes
- MisterMunchkin 12d agoWhat kind of cost are you looking at and how many people could use it? I’d be interested to know what the payback period is like, because Claude code is getting ridiculously expensive.
- taf2 12d agoBought the rtx 6000 pros one per month starting in January- since they doubled in price I wish I just used a line of credit back in Jan to buy all of them. For me it’s about keeping internal company content internal. Slack channels etc with tools for teams to use. Triage tools for Zendesk etc.
- bfrog 12d agoI definitely see a future race of who can accelerate open models and their usage locally with realistic desktop hardware. I fully await my ai in a usb box. The models are plateauing and some clever company is secretly working on this already I’m sure of it. Imagine baking in a model weight set in ROM with an analog computer doing what otherwise takes way too much power in digital form.
- ungreased0675 12d agoA lot of corporate use doesn’t require SOTA models. Things like summaries or writing comments in Slack are light work. For those reasons, local AI makes a lot of sense for some companies.
- unrented7977 12d agoThe financials are incredible. If you spend one engineer's salary on hardware, you get a system running local AI that can multiply the efforts of an entire (small) team of engineers. It's a very "you can't afford not to" situation. Even considering the hardware prices today.
- spopejoy 11d agoI'm puzzled why so many comments here are about local AI? Article is clearly about open models on openrouter etc.
- Supermancho 11d agoCopilot has enterprise controls. Its not open source. This is the easiest integration and large companies are eating it up as the path of least friction, using API cost calls. I call BS.