9 ms·
Accelerating GPT-5.6 Sol Ultrafast
- _345 1mo agoHope to be a part of this someday...
- dzonga 1mo agoare there Chinese companies doing the same as Cerebras is doing ? having large scale wafers designed for inference ?
- huflungdung 1mo ago[dead]
- HawtAds 1mo agoTheir dinner plate chips are impressive.
- crazysim 1mo agoGPT 5.6 Luna Ultrafast when?
- GodelNumbering 1mo agoThe corresponding OpenAI post https://openai.com/index/previewing-ultrafast/ https://openai.com/index/previewing-ultrafast/ There is no pricing info, which could mean it's "if you have to ask..." territory or they are simply gauging interest before deciding
- rirze 1mo agoThey're expanding access to companies that apply for the program and explain their use cases. So it's very real but limited imo.
- WarmWash 1mo agoThe stake in the side of cerebras has always been that the economics are pretty poor. Who knows if they will subsidizes it to mitigate sticker shock, but it's a safe assumption that it will be scarily expensive. However if you are in a "cost is no obstacle, speed is god" position, it will likely be pure magic.
- fcarraldo 1mo agoCan anyone explain why Cerberus needs to be _fast_ instead of _cheap_? I don't think I understand why they aren't leveraging the increased speed to do batching to serve more customers at a "normal" tok/s. Is the limitation, even on cerberus, still that the cache can only serve so many concurrent sessions over time? Is there no scaling advantage? I genuinely do not understand how any of this works.
- jaggederest 1mo agoThey're cache limited, almost certainly, so more slower sessions doesn't solve the problem - you still have to load and unload the whole cache hierarchy at some level and that's a network bandwidth and memory bandwidth problem between the external systems and the waferscale chip. Also worth looking into how they do cooling for it, because that's kind of absurd and awesome as well.
- petu 1mo agoI have no idea, but I've assumed that batching can't work on Cerebras. Batching works because of severe memory bottleneck, but Cerebras whole thing is serving models out of "L1 cache" (?).
- philipportner 1mo agoYou'd need hundreds of GB alone for the KV cache of each user. For something like LLama 3 405B you need ~67GB at ~130k tokens. A single CS-3 has 44GB on-chip sram. So, afaik, Cerebras are optimizing for ultra-low latency batch=1 inference. https://newsletter.semianalysis.com/p/cerebras-faster-tokens-please https://newsletter.semianalysis.com/p/cerebras-faster-tokens... goes quite in-depth.
- 2001zhaozhao 1mo agoThey're nearly certainly going to use it internally to speed up research that is serially bottlenecked. I would bet this is why they're interested in the Cerebras partnership more than everything else
- coder-3 1mo agoYeah, the main value of this for OAI is probaby to speedup RSI
- desterothx 1mo agoi would assume internally they have even better tps
- wxw 1mo ago> Compared with output speeds reported by Artificial Analysis GPT-5.6 Sol on Ultrafast mode runs 11x faster than Fable 5, and 5x faster than Opus 4.8 on Fast mode. Awesome work. I'm personally very excited for faster models/inference. I think speed is underrated to some degree in the current conversation. For a while, I was using Cursor's Composer quite a lot, even over frontier models, just because of how darn fast it was.
- kilroy123 1mo agoI've been using DeepSeek flash a lot this week to try it out. Now, I deeply want the smart frontier models to be just as fast.
- arw0n 1mo agoWhat do you need speed for? That's a genuine question, I feel like the limiting factor already is my creativity, attention span and budget. And I'm not even yet optimizing cost by batching things like review to slow local models over night, or schedule tasks to take full advantage of my subscriptions.
- black_knight 1mo agoI just finished waiting almost four hours for Fable to write 700 lines of code, based on my three paragraph prompt. Some speed on these harder tasks would definitely be welcome! It also spent almost 800k tokens on these lines…
- UltraSane 1mo agoI'm very curios what the code is doing.
- black_knight 1mo agoIt is a normalisation procedure for inductive and coinductive data types. Basically, the code is a function which takes in an expression where you can use generic data structures as variables and then some specific data structures, and it plugs them in for the variables. It then computes the structure of the resulting data type. So, admittedly not a trivial task – hence the choice of Fable as the model. Also, this would have taken me few days to do by hand! So, we are living in the future! But one could always wish for more speed and more intelligence.
- poly2it 1mo agoI guess Gemini 3.7 Flash is no longer at the pareto frontier of speed to intelligence.
- odo1242 1mo agoWell, there’s still price
- thraway3837 1mo agoThis is really cool. Someone here commented about similarity between this and hardware advancements for AV encode/decode. I think it's only a matter of time before miniaturization can have a thumbnail sized user-replaceable accessory that contains the LLM built onto the hardware. I admit I don't know how any of that works, but would be amazing to experience. Fully local, fully offline, ultra fast local inference better than any personal computing product.
- christkv 1mo agohttps://chatjimmy.ai/ https://chatjimmy.ai/ Is that. Company behind it just got acquired by AMD
- datadrivenangel 1mo agoIn 5-10 years nice smartphones will be able to run ChatGPT (~gpt3-4) class models. A memory rich laptop (highend mac/framework) can run GPT-OSS:120b or full Gemma4 at very interactive speeds. High end phones can already run the smaller models at enough speed to be probably useful, especially for background/overnight photo tagging and curation and things like that.
- raylad 1mo agoIn 5 years the models will probably be so much better and more compact that phones will be running models equivalent at least to Opus 4.6 if not Fable, at least within the areas they are tuned for (which probably won't include coding).
- thraway3837 1mo agoI'm really excited for this future. Both what you said and datadrivenangel. The fact that Apple shipped a more than capable laptop for most of the population using a last generation iPhone chip is just mind blowing. Silicon advancements are going to allow this, and I think the global majority will catch up and make their own chips that compete or exceed western performance. Especially when the US is scared of science, rapidly divesting and defunding it.
- iamcoder18 1mo agoI've been waiting so long for something amazing to come out of the OpenAI and Cerebras collaboration. > In our evaluations, GPT-5.6 Sol on Ultrafast mode answered all 2,500 HLE questions in 11 hours and 11 minutes. Claude Fable 5 needed 78 hours and 27 minutes, more than three days of continuous compute, to arrive at the same conclusions. In other words, Ultrafast worked through the frontier of human knowledge in a single working day, achieving comparable accuracy nearly 7× faster. This is actually insane. Hopefully the release ultrafast of Terra and Luna too.
- piyh 1mo agoFeels like the 90's again where single threaded speed is improving fast. ASICs and wafer scale rather than node shrinks, but end result to me the consumer feels the same.
- sscaryterry 1mo agoYep, https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law https://en.wikipedia.org/wiki/Andy_and_Bill%27s_law
- beng-nl 1mo agoI wish, but I don’t think this qualifies as single thread performance. Cerebras is whole wafer computation though right? So you get the result sooner, but the same amount of computation has occurred, just with huge parallelism.
- wrsh07 1mo agoSeems like they will do Sol first while capacity constrained? I can't imagine the margins they'll be charging
- sscaryterry 1mo agoI think this is what they need to actually have a margin :)
- zozbot234 1mo ago
- storus 1mo agoWow, that's even faster than diffusion LLMs but with the Fable-level quality! Congrats!
- scotty79 1mo agoI swear that now frontier AI stuff comes out few times a week.
- applfanboysbgon 1mo agoThis kills the crab. Compilation time will be a genuine bottleneck for slop coding if this becomes the standard generation rate over the next few years. Go, Zig or even C99 with TCC for dev builds, any language that can get you systems-level performance (or close to it) in a dev environment where you can iterate in ms rather than minutes is going to be immensely more appealing than generating a potential prototype in 10 seconds and waiting 15 minutes for it to compile.
- yetihehe 1mo agoMaybe then LLM's will switch to outputting raw machine code?
- applfanboysbgon 1mo agoNo. This conveys a fundamental misunderstanding of how anything pertaining to programming works. This will never happen, ever. For example, take... printf("Hello, world"); vs. a plausible illustration of how it might be compiled down to machine code... 48 65 6C 6C 6F 2C 20 77 6F 72 6C 64 48 83 EC 28 48 8D 0D F5 0F 00 00 E8 F0 00 00 00 33 C0 48 83 C4 28 C3 The latter now takes up 10x as many tokens (= 10x the cost/time, + context penalties), and is now architecture-specific, impossible to apply non-brittle program-wide optimizations to, etc. There is absolutely zero reason to ever have the LLM act as a compiler no matter how fast it is. Even if you believe LLMs will reach a state where they can actually generate good code at this level, you would be better off having them generate the compiler they would use.
- bakies 1mo agoI mean you're right, but what about a different tokenizer and some guardrails... trained on compiled code... I just learned that gpt-5.6 will talk to you in base64 without tool calls so maybe it's possible.
- Topfi 1mo agoStill, what'd be the advantage? There is a lot, a massive delta, a chasm still to improve before there is even a theoretical potential re. performance. GPT-5.6-Sol still writes JS in inefficient and roundabout ways that leave a lot on the table and even if we got to a point were models provide the highest performance output possible, there are hundreds of languages far more performant before machine code should be considered. Rust, C, Assembly if need be (at which point performance vs binary is, unless I am mistaken, pretty much moot). All machine code would gain us is lock in and no auditability.
- behnamoh 1mo agoFast mode is already 1.5 times faster and 2x more expensive in the Codex subscription plan. If this thing is 14 times faster, then I can imagine running out of my quota in one session.
- paxys 1mo agoThere is zero chance this will be offered to subscription users.
- cactusplant7374 1mo agoIt will eventually. Right now everyone is stuck on the equivalent of dialup.
- drcode 1mo agoI'm guessing a new $1000/month tier
- fg137 1mo ago> allowing Sol Ultrafast to accelerate your most time-sensitive, mission-critical work Curious, what are some of the use cases?
- bpbp-mango 1mo agoAgents would be excellent 'on call' team members. able to log in to production and diagnose issues, fix or raise issues, post outages etc.
- Aurornis 1mo agoThe example they give about debugging a service outage is good. For large companies, a very large burst expenditure on fast tokens to debug the outage as quickly as possible would be a good investment.
- ricardobeat 1mo agoThe omission of Mimo v2.5-Pro Ultraspeed, released in June, which can achieve 1000tok/s is an interesting flaw in the comparison graphs. It is a bit outdated (scores ± 40% lower), but smart enough for a lot of coding tasks, and can cost under 1/10th of Sol. https://mimo.mi.com/models/en-US/mimo-v2.5-pro-ultraspeed https://mimo.mi.com/models/en-US/mimo-v2.5-pro-ultraspeed
- anthonypasq 1mo agoI'd just like to point out that the largest model Cerebras has ever served is Kimi K2.6 which is 1T parameters, so that either means that theyve had a breakthrough on the hardware engineering side of things, or GPT-5.6 Sol is likely a lot smaller than people think. If it truly is only ~1-2T parameters, then this kinda kills 2 narratives for me. 1. all the handwringing about open source catching up via Kimi K3 (3T params) is complete nonsense. All that matters imo for determining which labs are leading is intelligence per parameter. Anyone with a enough compute can train a giant model, but being able to squeeze capabilities into smaller models gives you a massive inference and training edge. 2. Inference margins are clearly insane, and this explains why OpenAI was able to lower the price of Luna by 80%. Id guess that thing is probably 120b params based on the TPS they are serving it at.
- manmal 1mo agoIsn’t the fact Fable is more expensive than Sol-Max by multiples already an indication that Sol is way smaller?
- anthonypasq 1mo agono, because closed sourced model pricing has no relationship to its size. Thats what im saying. the inference margins are crazy, but people think the fonrtiner models must be 10T params or something because theyre expensive
- energy123 1mo ago> closed sourced model pricing has no relationship to its size. That's too strong. Only in an actual monopoly for a product with no substitutes that has price inelastic demand can pricing fully disconnect from costs. Frontier model serving is only maybe a soft version of that, where costs and moat both contribute to pricing.
- anthonypasq 1mo agosure, but my point is that supply and demand are what determines pricing, not cost to serve. its like thinking that because something costs $1 to make, its not possible for a company to sell it for $10, or that becuase they are selling something for $10, it must cost them $9 to make.
- owentbrown 1mo agoWhoa. This looks both powerful and expensive. My prediction is that, this time next year, top developers outside ai labs will be spending 50k USD+ on inference. Within labs, I've heard spend is already far beyond this per developer.
- jaggederest 1mo agoI mean I don't think $50k is the ceiling, unless you're talking about actual cash out. Claude code subscriptions right now can easily clear you $25-35k a year in nominal value for $2400 out of pocket cost. Given sufficient budget and scope, I could certainly productively burn a half million dollars in tokens a year or more. I think that's where we're headed anyway, buying a 2nd or 5th claude max subscription feels slightly excessive for personal usage, but at a corporate level...
- datadrivenangel 1mo agoat the corporate level you pay full price. And a moderately heavy user dev can easily spend a few $K a month, so yeah. Not impossible, but a high cost, and the diminishing returns definitely kick in
- vb-8448 1mo ago50k per month?! If someone subsidize maybe, but if the companies need to pay no way, unless there is hard evidence of the return.
- Squarex 1mo agoProbably per year, still a crazy amount.
- andriy_koval 1mo agonot sure how "top developers" are defined here, but there is huge diminishing return curve starts kicking in after $200/month price point for typical eng work.
- Topfi 1mo agoUnless I have read over it, besides the animation in the intelligence vs speed graph which only mentions internal data and not whether they truly reran the AA suite, there is no actually solid statement on the important aspect of performance. Neither the Cerebras or OpenAI post [0] outright state that this performs exactly the same as regular 5.6 Sol. I feel if this was 1:1 just Sol but much faster, they'd (rightfully) scream that off the rooftops. A line such as "this is the same performance, just faster, with no downsides" would go a long way in clarity and communication. Along with no pricing information, I'll hold out on further information. [0] https://openai.com/index/previewing-ultrafast/ https://openai.com/index/previewing-ultrafast/
- conception 1mo agoThis is what Cerebras does- take other people's models and run them very very fast.
- Scaevolus 1mo ago"delivering up to 750 output tokens per second and without any quality compromise" seems pretty definitive.
- Topfi 1mo agoNo quality compromise/degradation is something I have had this industry, including especially OpenAI, claim multiple times in the past and I have more than once been able to verify that it was in fact not the case. Examples being gpt-3.5-turbo vs text-davinci-003, GPT-4-Turbo and all the other post training checkpoints they had under one name (which was a major bug bear for me back then witnessing degradations with no naming change, industry got better in transparent checkpoint naming since), Opus 4.6 Fast Mode (which just was faster by skipping much of the required work), etc. Same for massive performance differences in the way providers like Cerebras, Groq, etc. have deployed models including K2.6 on Cereberas specifically. Massive deltas in tool call and overall quality despite there being far more clarity in open weight vs proprietary model deployment. The AA suite graph with that animation is the only time in either post that absolute parity is being asserted and I'd be amazed if that was the case, but am doubtful why their phrasing is so cagey. Why not assert full parity in writing? It "performs the same (within run-to-run variance) across all evals that Sol has been tested with" is very different to "no quality compromise/degradation", the later allowing for a lot more wiggle room and interpretation in what evals you use to assess that, what quality truly means, etc., the former meaning identical in all situations. Could also be a language barrier here in fairness, maybe this phrasing is more iron clad than I give them credit, but especially with OpenAI, I have seen enough checkpoints asserted as unchanged in "quality" to where I am skeptical. Ironically, I never saw that with Anthropic (which has gotten far more heat for degradation accusations) while a model was deployed with one exception in mid-late April this year. Pure speculation, but believe it wasn't noticed much before "agentic coding" became more popular, because chat output is far more subjective without a rating framework vs code passing which can be an objective metric with more potential for frustration.
- pingou 1mo agoMeanwhile they are down 12,68% today because of disappointing earnings.
- stephencoyner 1mo ago[dead]
- buybackoff 1mo agoThis is something I'm ready to pay for. Not more per token, but I will be happy to burn through 20x Pro subscription as fast as I consume my Plus weekly limit now, with 10x more tokens per unit of time. I've learned how to deal with and steer Sol medium quite efficiently, but at the same time I realize it's so slow for the small tasks it can do well, and still so unreliable for open-ended tasks.
- stillpointlab 1mo agoI haven't wrapped my head around what level of reasoning this involves. Is it equivalent to max? I didn't like Sol initially but it is growing on me the more I use it. Its personality is a bit flat and I caught it taking shortcuts a few times. But once I learned how to interact with it, I'm genuinely warming up to it. I find that it writes code that has fewer bugs even than Fable (although, to be fair I reach for Fable when the task is less well defined). If this has similar performance to Sol at max reasoning level, this would be a compelling reason to shift even more of my work (maybe the majority) to this model.
- ilaksh 1mo agoDid Cerebras get rid of their like $1500 per month plans for open models?
- Marciplan 1mo ago“our stock price went down today, here’s something to feed it”
- tristanMatthias 1mo ago> GPT-5.6 Sol on Ultrafast mode, delivering up to 750 output tokens per second https://taalas.com/products/ https://taalas.com/products/ > delivering 17k tokens per second per user on Llama 3.1 8B model. Obviously this is a much smaller model, but I really can't wait for ASICs to take over the LLM space. Imagine running a model like Sol/Fable (even half the size with 60-70% of it's intelligence) on your own ASIC hardware.
- auspiv 1mo agoI'd take qwen3.6 (3.8 as of tomorrow) 27B running at 17k per second first on the way to Sol/Fable! And then dsv4-flash-0731!
- tristanMatthias 1mo agoRight! Not sure that will happen since Talaas was just bought by AMD, so more likely a US model. Still, we can dream.
- sscaryterry 1mo agoIndeed, you could literally brute-force it that way :)
- mNovak 1mo agoASIC makes it sound like it's a single chip, but in reality serving trillion-param models on Cerebras requires a full cluster (as in multiple racks, MW of power). Some interesting twitter analysis here: https://x.com/bleysg/status/2073937651150029084 https://x.com/bleysg/status/2073937651150029084
- aenis 1mo agoGood news for Intel and AMD. Rught now on large scale codebases the bottleneck is both claude/codex inference, as well as time it takes to run tens of thousands of tests. We put those workloads on dedicated epyc 9005 build machines - but it still takes minutes per run. Those who can afford the fast tokens will be in the market for faster CPU that money can buy today.
- zuzululu 1mo agowhy intel and amd ? these are cerebras wafers? i know people are joking about the sol ultrafast prices (its unlikely to be accessible for average joes) but this shows scaling wafer cores works for inference boost which makes me very excited, sol ultrafast will be as slow as it will get if that makes sense. at these token speeds , we will see a much deeper economic impact.
- _diyar 1mo agoRight now the bottle neck is not the CPU, so people aren‘t spending big $ on them. But with this ultra fast mode, CPU becomes a bigger part of the bottleneck and thus Intel and AMD can charge more $$$.
- gpt5 1mo agoI noticed it myself. It's not just that the models are faster, but that by becoming more capable, they can take on larger tasks, which require larger compute.
- aenis 1mo agoI am settling on a way of working where during the day I spent the time iterating requirements, designs and low level implementation plans for, say, a dozen features, and then I park them for an unattended build during the night. It works well.
- aenis 1mo agoOn large codebases, literally anything I throw at Fable or Opus takes at least 20 minutes, 10 of those minutes is spent on running test suites. With this way of working, I put way more emphasis on test coverage, integration tests, behavioral tests, in-browser tests -- and I run them on every change, before any commit. Faster CPU solves this, while faster inference solves the other 10 minutes. I can't imagine how cool it would be to work with this ultrafast mode, without those mental-context-purging pauses.
- sashank_1509 1mo agoI don’t know if this is that useful for coding. In some autonomous world, where no one check the code and the agent can just spend 10X more time checking its work and leading to better results, yes maybe it is useful. But if humans need to check its work, then 10X speed doesn’t really matter I guess.
- logicchains 1mo ago>But if humans need to check its work, then 10X speed doesn’t really matter I guess. A human could have an agent run 10x more correction checks. If even after that they still need to check manually for issues, then they really need to work on their specification skills.
- fallingbananna 1mo agoI can imagine that all the tool calls that agents regularly make during regular software dev workloads would become the new bottleneck and so tasks overall won't be completed anywhere near 10x faster. However, I wouldn't diss the "ultraspeed" options untill I try them. Having agent thinking become near instant could change the way I (or you) use agents.
- navorad772 1mo agoWorking with agents right now is like working on huge legacy C++ codebase (feel free to insert your nightmares). You change 10 lines of code and then wait 20 minutes before it compiles and runs tests (or write prompt and wait 20 minues for code). External distractions and huge waiting times breaks your flow. Working long time in the flow is what makes mythical 10x programmer. IMHO 10x programmers are not superior in skill but in the ability to stay focused. Making agents faster is more important for people who care about the code than for vibecoders.
- sashank_1509 1mo agoStrong disagree, there’s nothing better about an agent generating a 1000 lines in 1 second or 1 minute. I can’t read it that quickly anyway.
- lostmsu 1mo agoStill no KV caching?
- johnfn 1mo agoThis does look pretty incredible, but don't forget that incredible token thoroughput can only necessarily solve certain bottlenecks. If your e2e tests take an hour, they'll still take an hour after Ultracode. If the agent runs a 10 minute typecheck after a change, that will still take 10 minutes. grep over a massive codebase is still just as slow, etc. I say this not to take away from this accomplishment but just to ensure everyone here keeps a clear head about what it means - 14x faster tokens does not mean it completes every task 14x faster. I suspect Humanity's Last Exam is without tool-calls, making it kind of the perfect benchmark to highlight how fast Ultrafast is, but not really the same as the everyday work you or I do.
- deleted 1mo ago[deleted]
- fireant 1mo agoI've been measuring waiting for tool calls/waiting for model response in my OMP with Sol 5.6 and usually it's 85%-95% of time spent waiting for model to respond, so 14x speedup in model perf would still be very significant. YMMV but speeding up tests and improving DX is somewhat well understood.
- csallen 1mo agoPeople underestimate the importance of speed on quality of thought, because people underestimate just how much quality is a result of simple iteration. When an LLM thinks, it typically just makes one pass. It outputs tokens from top to bottom, beginning to end, and then it's done. But when people think, especially strong thinkers, we typically iterate and revise our thoughts on the fly. We do numerous passes. We stop and restart, we reconsider, we review, we reevaluate. Sometimes we do this so quickly and automatically that we don't even realize we're doing it. I think a lot of what separates a highly intelligent or effective person from others has less to do with the quality of their first pass and more to do with just how many additional passes they're able to do in the same amount of time, and of course what kind of criteria they're habituated to consider during their review passes. Introspecting about this is difficult, but experimenting with LLMs is easy. First, simply ask an LLM to do something complex. For example, to come up with a new business idea, or to plan the next month of your life, etc. After it finishes, tell it: "Review what you just wrote, according to some appropriate list of evaluation criteria that you come up with first. And then, based on the results, iterate and generate a better response if warranted." It's insane how much better the next answer will usually to be. Often it'll catch and erase tons of hallucinations, logical errors, and inefficiencies. And you can simply copy-paste this again and again until you begin to hit diminishing returns. Or, in a harness like Claude Code, for example, I might shortcut this whole process by saying, "Use sub-agents to iteratively review and iterate on your work until convergence." The reason why most people don't prompt LLMs to do this (besides simply not thinking of it) is that it takes time. But what if it didn't? What if the LLM's response came back in milliseconds rather than minutes? Then there would be almost no reason NOT to do this. In fact, one could almost imagine it baked into the assistant/harness -- a massive step change in practical quality, enabled by nothing more than speed.
- loufe 1mo agoGoing back between two different company's AI tools when facing a tricky architecture question often surfaces holes in an approach I'd been building. Similarly, if I ever get a bit too vibey and don't carefully review code changes myself, the blast radius is generally significantly resolved by a carefully tuned "did you consider x, y, and z" skill after a first draft partnered with a "deploy an adversarial review agent for the worktree".
- damsta 1mo agoSo if Fast mode is 1.5x faster at 2x the price, will Ultrafast cost 20x as much? $100/$900 per 1M tokens?
- dewarrn1 1mo agoIn light of recent news, it is hard not to think about the 4.5-day hack on Hugging Face's systems happening ~10 times faster and be slightly concerned.
- Jr23_xd 1mo ago[flagged]
- andrethegiant 1mo agoWhy did OpenAI partner with Cerebras when they've already built their own chip, Jalapeño?
- minraws 1mo agoI will happily pay 2x luna pricing for this speed with Luna :P
- huey77 1mo agoI think token output speed is going to be one of the biggest fundamental shifts for AI this year. In my experience, models figure out problems after enough turns (or in agent swarms if its a lower tier model). Compressing that time horizon could take days of agentic coding into minutes. How ever will my monkey brain keep up?
- equinumerous 1mo agoThis is an amazing result. Can't wait until they release this to the general public, and I hope it's only a matter of time before other models are accelerated. I long for the day that regular consumers can run such models locally on specialized hardware.
- yieldcrv 1mo agoI'm loving this competition (now that I can see how workflows keep us employed) All the frontier labs go seemingly dormant for a month or two, while another one has its flurry of press releases, and people start to question whether the other lab is doing anything and then boom, the other lab finishes baking its next thing and releases its flurry of press releases
- Nevin1901 1mo agoWould gladly switch over to OpenAI and pay them 2x what I'm paying Claude if this becomes generally available
- drcode 1mo agoI'm guessing a new $1000/month plan if you want access to this
- z_rho_one 1mo agoWonder how many X usage this would consume when it becomes available for everyone. Fast mode already consumes 1.5x usage for 2.5x speed. Hopefully, this does not mean 8x usage for 14x speed, but rather something more reasonable such as 4x usage.
- johnnyApplePRNG 1mo agoNow I can blast through my weekly 20x pro codex credit in like an hour, great! The amount of usage you receive on Codex these days is dismal compared to what it was a few months ago, FYI. And they charge more for going faster. As a Codex customer, I am not impressed with their shenanigans over the past few months and I have resolved to master the art of Pi Coding Harness creation and loving it. Thanks for all the fish, Sam!
- tandr 1mo agoWait, you have noticed it too? I burned through weekly limit in 25 hours after last reset on Thursday, without changing what I was doing before that. Interestingly enough, previous subscription to Plus gave me about 2-3 days of coding. So I switched to Pro Light now, gave me about the same amount for a week or so (G-d bless these quota resets of theirs!). Now, with the last reset I only gathered 25 hours before I hit my weekly limit. Now I am buying credits, I switched to Luna to execute very narrow sets of patches, and offload things to "free" Spark model, and it is blowing through tokens less actively, but noticeable quickly too. I am not sure if it is something with how tokens are counted, or how they are counted depending where you are in a subscription cycle. Could it be that one person's "limit" is not the same as another, trying to push you to buy token credits?
- ttul 1mo agoWith this level of intelligence offered at this level of speed, new real-time applications become possible, such as providing expert advice during a phone call or court hearing. Current SOTA models are too slow in many cases to provide the kind of insights that we expect to receive from an intelligent human colleague, such as a sales coach or lawyer handing us a note or writing a Slack message during a difficult call. For these real-time applications, even a 10x increase in per-token cost would often be tolerable.
- exabrial 1mo agoRight now, the "economic model" of AI is "who has the best model", or really weights. That'll go away eventually, just like operating systems eventually became free. Instead, it's going to come down to selling inference hardware. We'll likely see the "apple" model where a custom OS runs on their hardware, but we'll probably also see more things like Cerebras become commodity hardware instead of kilowatt-class datacenter only hardware.
- bigyabai 1mo ago> we'll probably also see more things like Cerebras become commodity hardware instead of kilowatt-class datacenter only hardware. Cerebras uses a unreal amount of SRAM to make these dies feasible. If AI weights become small enough to fit into commodity-scale Cerebras chips like that, you might as well load it into unified memory instead and run inference on a GPGPU-capable SOC instead. CUDA-style acceleration makes much more sense at that scale, especially if your use-case is just realtime conversational AI on a smartphone.
- fenestella 1mo ago[dead]
- deleted 1mo ago[deleted]
- madhu_ghalame 1mo ago[dead]
- jdthedisciple 1mo agonow let's have this thing iterate away nonstop 24/7/365 solving humanity's major challenges and see how far we get, shall we?
- davidmurdoch 1mo agoI don't even have "Priority" (2.5x speed) in my Pro account, only Standard and Fast (1.5, speed). I actually didn't even know their was a level above Fast till now. I don't imagine non enterprise users will get the "UltraFast" mode option why time soon.