6 ms·
I think speed is actually going to be a bigger factor than cost. Even projects where “money is no object” often hit a wall with LLM response times. Sure you ca
by jbotdev 29d ago
I think speed is actually going to be a bigger factor than cost. Even projects where “money is no object” often hit a wall with LLM response times.
Sure you can speed things up with parallel work under subagents, but as with parallelizing traditional computational tasks, there are diminishing gains.
I keep hearing people saying just change the way you work to trust long-running agents and multi-task more, because they’re too slow to work with interactively for many use cases. I think that’s painful in a world where we expect humans to still heavily guide and interact with agents for their day-to-day work.
- perching_aix 29d agoGiven the 750 tok/sec GPT 5.6 Sol Ultrafast (via Cerebras), the many-1000 tok/sec Chinese models, and the 15000 tok/sec Taalas HC1, I think we're well on the way towards seeing that solved too. Combine the two, and yeah, wild ride incoming. What's especially bewildering to me is that translated back to raw bandwidth, even 15000 tok/sec is just like what, 75 KB/s? Extremely meager amounts of data, moving mountains. It's already kinda funny seeing LLMs throw out effort estimates in wall time terms. It's always some "hours, days, weeks" tier thing, when in reality, it's gone and done in minutes.
- cactusplant7374 29d agoOne of my projects has an estimate of 3,000+ hours. It seems accurate. It has spent months working on it.
- perching_aix 29d agoAround the clock?
- cactusplant7374 29d agoYes, with a few exceptions. It's much easier now that the five hour limit has been removed from codex.
- wyre 29d agoCan I ask what you are working on?
- cactusplant7374 28d agoI'm working on emulators with AOT and JIT.
- newAccount2025 28d agoHaha. Yeah. I somehow find Claude Code’s utter lack of time reasoning to be very fun. “The work we did two weeks ago…” oh friend that was this morning…
- jbstack 29d ago> they’re too slow to work with interactively for many use cases This just demonstrates how much we already take for granted the LLMs that we have now. If you compare it to what we had before (hand the task off to a junior dev and wait for them to complete the work) then it doesn't seem slow at all.