24 ms·
For comparison, openrouter says opus 4.8 is ~55 tokens/s and fast mode is ~102. 750 tokens/s for their largest model is going to be nuts
by sberens 3mo ago
For comparison, openrouter says opus 4.8 is ~55 tokens/s and fast mode is ~102.
750 tokens/s for their largest model is going to be nuts
- gandreani 3mo agoUsing gpt-5.4-mini in off-peak hours already feels like super-speed to me. That's probably no more than 100-150 tk/s. I can't imagine 750! I've always eyed Cerebras but never had a use for it that would justify paying for the API directly. Although now that I think about it, trying out the API would probably cost less than a subscription for a month...
- kegs_ 3mo agoI have a pretty good use case for gpt-oss. The amount of time savings has actually been wild. Definitely worth a try. Just to be clear, it gets like 2000tok/s
- embedding-shape 3mo agoThe ChatGPT subscription gives you access to the -spark model(s) in Codex which are blazing fast (but pretty dumb) which I think runs on Cerebras hardware too.
- rrvsh 3mo agois this specifically in codex? have been trying to use the models for months on opencode then pi but it says chatgpt subscriptions don't have access to it - i was under the assumption that OpenAI doesn't lock down their models based on harness a la Claude Code
- cactusplant7374 3mo agoWhat plan are you on? It is only available to Pro users.
- rrvsh 3mo agoPlus. No wonder - i suspected this but i couldnt find any docs. Side note, how are you liking Pro? I have really been considering getting Pro recently, but not sure if its more worth to just switch to openrouter. I feel like my usage currently barely outstrips the plus usage limits and Pro would be too much, and using Openrouter by default would mean I would have a lot more leeway to run more random lighter workloads without worrying about using up my limit, but I'll really miss GPT 5
- cactusplant7374 3mo agoI find it to be excellent. I have three Pro subscriptions so I can build stuff 24/7. I only use GPT 5.5 xhigh. Before 5.5 I wasted a lot of time with bugs. I want to make sure that doesn't happen again -- if I can help it.
- jasonjmcghee 3mo agoTry gpt-5.3-codex-spark - it's 1000 TPS and from my experience more capable than 5.4 mini. If you have a subscription it's a different pool of usage.
- small_model 3mo agoUsed it, very fast but tiny context window and doesn't have good reasoning. (good for quick simple code changes)
- beering 3mo agoAgreed, 1000tok/s just fills up the context window (which is big by 2004 standards) super fast. But seems like 5.3-spark was just a taste of what’s to come.
- taneq 3mo ago2004 standards? O.o
- partsch 3mo ago1904
- bogeholm 3mo agoBack when we were kids, we would get 0 tokens/sec _if we were lucky_
- mlinsey 3mo agoIn 2004, I took a class where we trained "language models" that were bigram word models, on an archive of a couple years of the Wall Street Journal. I remember someone who literally announced they were dropping the class to the whole room at the end of a lecture, saying "This isn't AI!!!"
- trollbridge 3mo ago
- order-matters 3mo agothe more advanced models also utilize a lot more tokens, and a lot of these extra tokens may go towards safeguards at a higher rate than prior models as well. not to say a speed boost isnt there but if they didnt increase tokens / s at all youd likely see things slow down a lot with the new model compared to current
- beering 3mo agoI think regular users will still have the old speed, so should be easy to tell whether it is more thinkier than 5.5.
- windexh8er 3mo agoWhat about 15k tokens per second? [0] I remember looking at this earlier in the year and it being so fast that it feels fake. And, yes, this model is old - but still awesome for what it is. [0] https://chatjimmy.ai/ https://chatjimmy.ai/
- ehsankia 3mo agoI just tried it, and the answer is non-sense. I asked it something simple, list some good indie puzzle games, and half the answers are games that don't exist. Imo quality > speed.
- Kirby64 3mo agoIt’s not just old, it’s also tiny and quantized. It’s llama 3.1 8b at 3/6-bit quant. This is the type of thing you can run on almost any device…
- windexh8er 3mo agoI get that, but not at 15k tokens/s.
- Kirby64 3mo agoBut it’s irrelevant. 750 tokens/s on a full frontier model is useful. 15000 poor quality tokens is much less useful no matter how much scaffolding you put around it.
- windexh8er 3mo agoI think you missed the point and don't understand / aren't considerate of SLM utility.
- Kirby64 3mo agoBut I’m not missing the point. If you can run one frontier model at 750t/s, then you can probably run many many instances of an SLM in parallel at a rate that exceeds 15k/s. That’s kinda the point of the flash or ultrafast variants. And they’re on something much more modern than llama3.1.
- comboy 3mo agoBut it seems that there is some queuing/load balancing on their side, I mean when opus is actually outputting this 55t/s it feles fast, but apart from it's internal reasoning I think there's sometimes just waiting.
- fragmede 3mo agoOh wait yeah good point. At 750 tokens a second and the same amount of human patients they can set it to think for the same amount of time but four or five times the amount of thinking tokens, which may improve the quality of the eventual output.
- xnx 3mo ago"up to 750 tokens per second" Emphasis on "up to". Imagine whatever limited situation (e.g. pre-cached query) and that will probably be the only time it hits 750 tokens/sec.