5 ms·

How does it compare to Opus 4.7? I've been immersed in 4.7 all week participating in the Anthropic Opus 4.7 hackathon and it's pretty impressive even if it's ra

How does it compare to Opus 4.7? I've been immersed in 4.7 all week participating in the Anthropic Opus 4.7 hackathon and it's pretty impressive even if it's ravenous from a token perspective compared to 4.6

greenknight 5mo ago
The thing is, it doesnt need to beat 4.7. it just needs to do somewhat well against it. This is free... as in you can download it, run it on your systems and finetune it to be the way you want it to be.
- radlad 5mo ago
  ... if you have 800 GB of VRAM free.
  - inventor7777 5mo ago
    I remember reading about some new frameworks have been coming out to allow Macs to stream weights of huge models live from fast SSDs and produce quality output, albeit slowly. Apart from that...good luck finding that much available VRAM haha
- p1esk 5mo ago
  Do you think a lot of people have “systems” to run a 1.6T model?
  - deleted 5mo ago
    [deleted]
  - applfanboysbgon 5mo ago
    No, but businesses do. Being able to run quality LLMs without your business, or business's private information, being held at the mercy of another corp has a lot of value.
    - choldstare 5mo ago
      Not really - on prem llm hosting is extremely labor and capital intensive
      - applfanboysbgon 5mo ago
        But can be, and is, done. I work for a bootstrapped startup that hosts a DeepSeek v3 retrain on our own GPUs. We are highly profitable. We're certainly not the only ones in the space, as I'm personally aware of several other startups hosting their own GLM or DeepSeek models.
        wuschel 5mo ago
        Why a retrain? What are you using the model for?
    - forrestthewoods 5mo ago
      What type of system is needed to self host this? How much would it cost?
      - p1esk 5mo ago
        Depends on fast you want it to be. I’m guessing a couple of $10k mac studio boxes could run it, but probably not fast enough to enjoy using it.
      - disiplus 5mo ago
        Depends how many users you have and what is "production grade" for you but like 500k gets you a 8x B200 machine.
      - fragmede 5mo ago
        One GB200 NVL72 from Nvidia would do it. $2-3 million, or so. If you're a corporation, say Walmart or PayPal, that's not out of the question. If you want to go budget corporate, 7 x H200 is just barely going to run it, but all in, $300k ought to do it.
  - CJefferson 5mo ago
    To me, the important thing isn't that I can run it, it's that I can pay someone else to run it. I'm finding Opus 4.7 seems to be weirdly broken compared to 4.6, it just doesn't understand my code, breaks it whenever I ask it to do anything. Now, at the moment, i can still use 4.6 but eventually Anthropic are going to remove it, and when it's gone it will be gone forever. I'm planning on trying Deepseek v4, because even if it's not quite as good, I know that it will be available forever, I'll always be able to find someone to run it.
    - muyuu 5mo ago
      Yep, it's wild how little emphasis is there on control and replicability in these posts. Already these models are useful for a myriad of use cases. It's really not that important if a model can 1-shot a particular problem or draw a cuter pelican on a bike. Past a degree of quality, process and reliability are so much more important for anything other than complete hands-off usage, which in business it's not something you're really going to do. The fact that my tool may be gone tomorrow, and this actually has happened before, with no guarantees of a proper substitute... that's a lot more of a concern than a point extra in some benchmark.
- kelseyfrog 5mo ago
  What's the hardware cost to running it?
  - slashdave 5mo ago
    "if you have to ask..."
  - redox99 5mo ago
    Probably like 100 USD/hour
  - bbor 5mo ago
    I was curious, and some [intrepid soul](https://wavespeed.ai/blog/posts/deepseek-v4-gpu-vram-requirements/ https://wavespeed.ai/blog/posts/deepseek-v4-gpu-vram-require...) did an analysis. Assuming you do everything perfectly and take full advantage of the model's MoE sparsity, it would take: - To run at full precision: "16–24 H100s", giving us ~$400-600k upfront, or $8-12/h from [us-east-1](https://intuitionlabs.ai/articles/h100-rental-prices-cloud-comparison https://intuitionlabs.ai/articles/h100-rental-prices-cloud-c...). - To run with "heavy quantization" (16 bits -> 8): "8xH100", giving us $200K upfront and $4/h. - To run truly "locally"--i.e. in a house instead of a data center--you'd need four 4090s, one of the most powerful consumer GPUs available. Even that would clock in around $15k for the cards alone and ~$0.22/h for the electricity (in the US). Truly an insane industry. This is a good reminder of why datacenter capex from since 2023 has eclipsed the Manhattan Project, the Apollo program, and the US interstate system combined...
    - zargon 5mo ago
      That article is a total hallucination. "671B total / 37B active" "Full precision (BF16)" And they claim they ran this non-existent model on vLLM and SGLang over a month and a half ago. It's clickbait keyword slop filled in with V3 specs. Most of the web is slop like this now. Sigh.
    - oceanplexian 5mo ago
      All these number are peanuts to a mid sized company. A place I worked at used to spend a couple million just for a support contract on a Netapp. 10 years from now that hardware will be on eBay for any geek with a couple thousand dollars and enough power to run it.
- onchainintel 5mo ago
  Completely agree, not suggesting it needs ot just genuinely curious. Love that it can be run locally though. Open source LLMs punching back pretty hard against proprietary ones in the cloud lately in terms of performance.
- libraryofbabel 5mo ago
  > you can download it, run it on your systems In theory, sure, but as other have pointed out you need to spend half a million on GPUs just to get enough VRAM to fit a single instance of the model. And you’d better make sure your use case makes full 24/7 use of all that rapidly-depreciating hardware you just spent all your money on, otherwise your actual cost per token will be much higher than you think. In practice you will get better value from just buying tokens from a third party whose business is hosting open weight models as efficiently as possible and who make full use of their hardware. Even with the small margin they charge on top you will still come out ahead.
  - hsbauauvhabzb 5mo ago
    Sure, but that’s an incredibly short term viewpoint.
  - oceanplexian 5mo ago
    There are a lot of companies who would gladly drop half a million on a GPU to have private inference that Anthropic or OpenAI can’t use to steal their data. And that GPU wouldn’t run one instance, the models are highly parallelizable. It would likely support 10-15 users at once, if a company oversubscribed 10:1 that GPU supports ~100 seats. Amortized over a couple years the costs are competitive.
    - libraryofbabel 5mo ago
      > There are a lot of companies who would gladly drop half a million on a GPU to have private inference that Anthropic or OpenAI can’t use to steal their data. Obviously, and certainly companies do run their own models because they place some value on data sovereignty for regulatory or compliance or other reasons. (Although the framing that Anthropic or OpenAI might "steal their data" is a bit alarmist - plenty of companies, including some with _highly_ sensitive data, have contracts with Anthropic or OpenAI that say they can't train future models on the data they send them and are perfectly happy to send data to Claude. You may think they're stupid to do that, but that's just your opinion.) > the models are highly parallelizable. It would likely support 10-15 users at once. Yes, I know that; I understand LLM internals pretty well. One instance of the model in the sense of one set of weights loaded across X number of GPUs; of course you can then run batch inference on those weights, up to the limits of GPU bandwidth and compute. But are those 100 users you have on your own GPUs usings the GPUs evenly across the 24 hours of the day, or are they only using them during 9-5 in some timezone? If so, you're leaving your expensive hardware idle for 2/3 of the day and the third party providers hosting open weight models will still beat you on costs, even without getting into other factors like they bought their GPUs cheaper than you did. Do the math if you don't believe me.
rvz 5mo ago
It is more than good enough and has effectively caught up with Opus 4.6 and GPT 5.4 according to the benchmarks. It's about 2 months behind GPT 5.5 and Opus 4.7. As long as it is cheap to run for the hosting providers and it is frontier level, it is a very competitive model and impressive against the others. I give it 2 years maximum for consumer hardware to run models that are 500B - 800B quantized on their machines. It should be obvious now why Anthropic really doesn't want you to run local models on your machine.
- colordrops 5mo ago
  What's going to change in 2 years that would allow users to run 500B-800B parameter models on consumer hardware?
  - DiscourseFan 5mo ago
    I think its just an estimate
    - indigodaddy 5mo ago
      But the question remains
- snovv_crash 5mo ago
  With the ability of the Qwen3.6 27B, I think in 2 years consumers will be running models of this capability on current hardware.
- deaux 5mo ago
  Vibes > Benchmarks. And it's all so task-specific. Gemini 3 has scored very well in benchmarks for very long but is poor at agentic usecases. A lot of people prefering Opus 4.6 to 4.7 for coding despite benchmarks, much more than I've seen before (4.5->4.6, 4->4.5). Doesn't mean Deepseek v4 isn't great, just benchmarks alone aren't enough to tell.
spaceman_2020 5mo ago
Tbh I was more productive with 4.6 than ever before and if AI progress locks in permanently at 4.6 tier, I’d be pretty happy

greenknight 5mo ago

The thing is, it doesnt need to beat 4.7. it just needs to do somewhat well against it. This is free... as in you can download it, run it on your systems and finetune it to be the way you want it to be.

radlad 5mo ago
... if you have 800 GB of VRAM free.
- inventor7777 5mo ago
  I remember reading about some new frameworks have been coming out to allow Macs to stream weights of huge models live from fast SSDs and produce quality output, albeit slowly. Apart from that...good luck finding that much available VRAM haha
p1esk 5mo ago
Do you think a lot of people have “systems” to run a 1.6T model?
- deleted 5mo ago
  [deleted]
- applfanboysbgon 5mo ago
  No, but businesses do. Being able to run quality LLMs without your business, or business's private information, being held at the mercy of another corp has a lot of value.
  - choldstare 5mo ago
    Not really - on prem llm hosting is extremely labor and capital intensive
    - applfanboysbgon 5mo ago
      But can be, and is, done. I work for a bootstrapped startup that hosts a DeepSeek v3 retrain on our own GPUs. We are highly profitable. We're certainly not the only ones in the space, as I'm personally aware of several other startups hosting their own GLM or DeepSeek models.
      - wuschel 5mo ago
        Why a retrain? What are you using the model for?
  - forrestthewoods 5mo ago
    What type of system is needed to self host this? How much would it cost?
    - p1esk 5mo ago
      Depends on fast you want it to be. I’m guessing a couple of $10k mac studio boxes could run it, but probably not fast enough to enjoy using it.
    - disiplus 5mo ago
      Depends how many users you have and what is "production grade" for you but like 500k gets you a 8x B200 machine.
    - fragmede 5mo ago
      One GB200 NVL72 from Nvidia would do it. $2-3 million, or so. If you're a corporation, say Walmart or PayPal, that's not out of the question. If you want to go budget corporate, 7 x H200 is just barely going to run it, but all in, $300k ought to do it.
- CJefferson 5mo ago
  To me, the important thing isn't that I can run it, it's that I can pay someone else to run it. I'm finding Opus 4.7 seems to be weirdly broken compared to 4.6, it just doesn't understand my code, breaks it whenever I ask it to do anything. Now, at the moment, i can still use 4.6 but eventually Anthropic are going to remove it, and when it's gone it will be gone forever. I'm planning on trying Deepseek v4, because even if it's not quite as good, I know that it will be available forever, I'll always be able to find someone to run it.
  - muyuu 5mo ago
    Yep, it's wild how little emphasis is there on control and replicability in these posts. Already these models are useful for a myriad of use cases. It's really not that important if a model can 1-shot a particular problem or draw a cuter pelican on a bike. Past a degree of quality, process and reliability are so much more important for anything other than complete hands-off usage, which in business it's not something you're really going to do. The fact that my tool may be gone tomorrow, and this actually has happened before, with no guarantees of a proper substitute... that's a lot more of a concern than a point extra in some benchmark.
kelseyfrog 5mo ago
What's the hardware cost to running it?
- slashdave 5mo ago
  "if you have to ask..."
- redox99 5mo ago
  Probably like 100 USD/hour
- bbor 5mo ago
  I was curious, and some [intrepid soul](https://wavespeed.ai/blog/posts/deepseek-v4-gpu-vram-requirements/ https://wavespeed.ai/blog/posts/deepseek-v4-gpu-vram-require...) did an analysis. Assuming you do everything perfectly and take full advantage of the model's MoE sparsity, it would take: - To run at full precision: "16–24 H100s", giving us ~$400-600k upfront, or $8-12/h from [us-east-1](https://intuitionlabs.ai/articles/h100-rental-prices-cloud-comparison https://intuitionlabs.ai/articles/h100-rental-prices-cloud-c...). - To run with "heavy quantization" (16 bits -> 8): "8xH100", giving us $200K upfront and $4/h. - To run truly "locally"--i.e. in a house instead of a data center--you'd need four 4090s, one of the most powerful consumer GPUs available. Even that would clock in around $15k for the cards alone and ~$0.22/h for the electricity (in the US). Truly an insane industry. This is a good reminder of why datacenter capex from since 2023 has eclipsed the Manhattan Project, the Apollo program, and the US interstate system combined...
  - zargon 5mo ago
    That article is a total hallucination. "671B total / 37B active" "Full precision (BF16)" And they claim they ran this non-existent model on vLLM and SGLang over a month and a half ago. It's clickbait keyword slop filled in with V3 specs. Most of the web is slop like this now. Sigh.
  - oceanplexian 5mo ago
    All these number are peanuts to a mid sized company. A place I worked at used to spend a couple million just for a support contract on a Netapp. 10 years from now that hardware will be on eBay for any geek with a couple thousand dollars and enough power to run it.
onchainintel 5mo ago
Completely agree, not suggesting it needs ot just genuinely curious. Love that it can be run locally though. Open source LLMs punching back pretty hard against proprietary ones in the cloud lately in terms of performance.
libraryofbabel 5mo ago
> you can download it, run it on your systems In theory, sure, but as other have pointed out you need to spend half a million on GPUs just to get enough VRAM to fit a single instance of the model. And you’d better make sure your use case makes full 24/7 use of all that rapidly-depreciating hardware you just spent all your money on, otherwise your actual cost per token will be much higher than you think. In practice you will get better value from just buying tokens from a third party whose business is hosting open weight models as efficiently as possible and who make full use of their hardware. Even with the small margin they charge on top you will still come out ahead.
- hsbauauvhabzb 5mo ago
  Sure, but that’s an incredibly short term viewpoint.
- oceanplexian 5mo ago
  There are a lot of companies who would gladly drop half a million on a GPU to have private inference that Anthropic or OpenAI can’t use to steal their data. And that GPU wouldn’t run one instance, the models are highly parallelizable. It would likely support 10-15 users at once, if a company oversubscribed 10:1 that GPU supports ~100 seats. Amortized over a couple years the costs are competitive.
  - libraryofbabel 5mo ago
    > There are a lot of companies who would gladly drop half a million on a GPU to have private inference that Anthropic or OpenAI can’t use to steal their data. Obviously, and certainly companies do run their own models because they place some value on data sovereignty for regulatory or compliance or other reasons. (Although the framing that Anthropic or OpenAI might "steal their data" is a bit alarmist - plenty of companies, including some with _highly_ sensitive data, have contracts with Anthropic or OpenAI that say they can't train future models on the data they send them and are perfectly happy to send data to Claude. You may think they're stupid to do that, but that's just your opinion.) > the models are highly parallelizable. It would likely support 10-15 users at once. Yes, I know that; I understand LLM internals pretty well. One instance of the model in the sense of one set of weights loaded across X number of GPUs; of course you can then run batch inference on those weights, up to the limits of GPU bandwidth and compute. But are those 100 users you have on your own GPUs usings the GPUs evenly across the 24 hours of the day, or are they only using them during 9-5 in some timezone? If so, you're leaving your expensive hardware idle for 2/3 of the day and the third party providers hosting open weight models will still beat you on costs, even without getting into other factors like they bought their GPUs cheaper than you did. Do the math if you don't believe me.

rvz 5mo ago

It is more than good enough and has effectively caught up with Opus 4.6 and GPT 5.4 according to the benchmarks. It's about 2 months behind GPT 5.5 and Opus 4.7. As long as it is cheap to run for the hosting providers and it is frontier level, it is a very competitive model and impressive against the others. I give it 2 years maximum for consumer hardware to run models that are 500B - 800B quantized on their machines. It should be obvious now why Anthropic really doesn't want you to run local models on your machine.

colordrops 5mo ago
What's going to change in 2 years that would allow users to run 500B-800B parameter models on consumer hardware?
- DiscourseFan 5mo ago
  I think its just an estimate
  - indigodaddy 5mo ago
    But the question remains
snovv_crash 5mo ago
With the ability of the Qwen3.6 27B, I think in 2 years consumers will be running models of this capability on current hardware.
deaux 5mo ago
Vibes > Benchmarks. And it's all so task-specific. Gemini 3 has scored very well in benchmarks for very long but is poor at agentic usecases. A lot of people prefering Opus 4.6 to 4.7 for coding despite benchmarks, much more than I've seen before (4.5->4.6, 4->4.5). Doesn't mean Deepseek v4 isn't great, just benchmarks alone aren't enough to tell.

spaceman_2020 5mo ago

Tbh I was more productive with 4.6 than ever before and if AI progress locks in permanently at 4.6 tier, I’d be pretty happy