7 ms·

The abrupt swing in many non-technology company IT departments from "hey developer, you aren't using enough tokens" to this is just too funny. And I'm seeing a

• surgical_fire 4mo ago
  I've never seen self-awareness from leaders. They always lead on vibes. Understanding this was one of the most important things in my career.
• qoez 4mo ago
  I feel like most successful businesses have such a moat of required capital to compete with them that even tho in theory poor decisions like this is supposed to give opportunities for entreprenuers to hit when the big dogs make a wrong move, it doesn't end up happening.
• onlyrealcuzzo 4mo ago
  The actual cost is going to drop 99% in ~4 years. How much that makes it into enterprise pricing is TBD, since none of the hyper scalers are making money yet of selling AI inference. Almost all businesses are ahead of the gun. For most of their use cases, AI is either not yet good enough on its own, or good enough but too expensive. No one wants to get left behind, so everyone's trying to get onto it now, even though it's not ready for what most enterprises want to do with it. It's easy for them to look at a small startup without billions of lines of legacy business logic debt and see them having success and wonder why they can't have just as much - or more - why they're bigger so they should have better and more success, right??? Wrong... But when it gets ~99% cheaper for local inference over the next 4 years, at the same time the price per watt improve 4x -> a lot of those cases will start to pencil out.
  • datakan 4mo ago
    What makes you think prices will drop? Everyone I’ve spoken to believes they will only skyrocket. Genuinely curious
    • onlyrealcuzzo 4mo ago
      The technology already exists now on the algorithmic front for the next 10x drop between everyone adopting DeepSeek's MLA, MoE (mostly already done), Medusa (a better version of Google's speculative decoding), Kimi's Attn Residuals, and Mimo's Sliding Window Attn, and (possibly) Microsoft's 1.58b (this may be a nothing burger). Historic trends, every 18 months, performance for the same level of quality has gone down 90%. See: https://www.reddit.com/r/LocalLLaMA/comments/1gpr2p4/llms_co https://www.reddit.com/r/LocalLLaMA/comments/1gpr2p4/llms_co... And Chart 13 here: https://www.rdworldonline.com/ais-great-compression-20-chart https://www.rdworldonline.com/ais-great-compression-20-chart... And here: https://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends Historically, algorithmic gains are only ~30% of the pie, but there's enough out there to get to 10x, with just what's available already. The other ~70% of the pie is better training data (often synthetic) and distilling frontier knowledge. There's no sign we are tapped out on that front. Additionally, GRAM (from ~10 days ago) is likely to be a 5-10x on its own (if not substantially more for smaller models). It's unlikely within 4 years LeCun's JEPA ideas and similar ideas like GRAM applied to LLMs have ZERO impact. The preliminary results are absolutely astounding (5000x better reasoning - this is not peanuts). Further, that's not even counting that cost per watt is still dropping ~2x every 2 years on its own on the hardware front. If you look at the "cost" of inference. People think it's electricity - but it's currently almost ~80% hardware amortization. The memory shortage is not going to last, nor are Nvidia's ~80-90% margins. The human brain is still 8-10 orders of magnitude more efficient than the best LLMs of today. With ~1/10th of global capex riding on AI, if you don't think they're going to knock of 2 orders of magnitude more, when it's this obvious and easy... I don't know what to tell you... Sure, it might take 6 years instead of 4. My crystal ball isn't perfect.
      • datakan 4mo ago
        This is great food for thought, thank you
  • packetlost 4mo ago
    I don't see how this is even remotely true. Unless there's some super breakthrough into a fundamentally different architecture, there's not really a path to a 50% reduction in price, much less a 99% reduction.
    • onlyrealcuzzo 4mo ago
      And yet 90% drops for the same level of quality every 18 months have happened like clockwork... And the technology already exists on the algorithmic front TODAY to lock in another 10x gain -> when, typically, algorithmic gains only account for ~30% of that drop and the other ~70% comes from better data (often synthetic) and knowledge distilation from frontier models. Just look at DeepSeek's pricing...
    • kilroy123 4mo ago
      In fairness, I think _current_ capabilities will be cheaper. So the models of today will be run drastically cheaper in 4 years.
  • krona 4mo ago
    > The actual cost is going to drop 99% Do you mean the marginal cost by the producer, or the cost on the consumer? I can't see the price of electricity falling much, and the demand curve is apparently exponential if the hype is to be believed.
    • trollbridge 4mo ago
      DeepSeep V4 Pro is 99% cheaper than similarly performing models were 2 years ago (if such a model even existed). Computing has always been about how to wring out more efficiency. The ENIAC was 150,000 watts, with 3 phase 240 volt power, and cost about $500,000. My day to day laptop (a year old) is 35 watts, with 1 phase 20 volt power, and cost $1,000, so that's 99.98% less power consumption, 99.8% cheaper, and it has about 10 orders of magnitude more computing power, all on a time span of 80 years.
      • cratermoon 4mo ago
        Moore’s law is dead.
        • HappMacDonald 4mo ago
          It died before AI came around and today's coding agents are somewhere upwards of twice as competent as whatever the state of the art of automatic coding was in 2020. 8I
          • mrandish 4mo ago
            A good chunk of that was one-time gains from shifting GPU and memory architectures to better match what LLMs need at scale as well as some algorithmic improvements. Most of the low-hanging architecture optimization has already been harvested. We'll certainly have more algorithmic gains but the consensus is they'll generally be smaller and less frequent. There's always a chance we'll have some dramatic gains far larger than DeepSeek's optimizations a year ago, but it hasn't happened again yet at even that scale. It would be nice but I certainly wouldn't count on it.
  • bakugo 4mo ago
    Prices have been very obviously trending up, not down. Even open weights models are becoming more expensive with every release. Computer hardware is ballooning in price.
    • abalashov 4mo ago
      Just wait for the next model and the next model architecture. Just wait for it, bro.
      • onlyrealcuzzo 4mo ago
        Gemini 3.5 flash is 25% cheaper than 3.1 pro, and outperforms it on almost every benchmark, most by a pretty wide margin...
        • abalashov 4mo ago
          Cool.
        • bigstrat2003 4mo ago
          There has never yet been a new model which actually improved over the previous ones. They suck just as much, and in the same ways, as the models of 3 years ago.
        • Rebelgecko 4mo ago
          It's still 5x more expensive than 2.5 flash
    • onlyrealcuzzo 4mo ago
      Prices are going up for BETTER quality -> not for the SAME level of quality. People are willing to pay more for BETTER quality. You obviously haven't seen DeepSeek v4 Pro's pricing if you think pricing only goes up...
      • bakugo 4mo ago
        Maybe so, but that becomes irrelevant when you consider that the new, better quality instantly becomes the expected baseline. So the price of the "baseline" quality is going up regardless. Let's look at GPU prices as an example. Around 12 years ago, I bought a GTX 970 for around $350. That was considered a very good GPU at the time. Today, the "equivalent" GPU model (RTX 5070) now costs almost double. Of course, the newer GPU is much more powerful (more than double, in fact), but all the things you'd use a GPU for have also advanced and now expect an entirely new level of performance as a baseline, such that the older GPU is fairly worthless today. So most people agree that GPUs in general have become more expensive. Regarding DeepSeek's price: it's obviously subsidized, and unlikely to match the actual inference cost right now.
  • BearOso 4mo ago
    Going from Opus 4.5 to 4.7 secretly required 6x more compute to run. 4.8 is apparently 30% more on top. I haven't seen any optimizations lately aside from distillation. Nobody's optimizing, they're just scaling up.
    • rescbr 4mo ago
      > Nobody's optimizing The Chinese, since they lack computing hardware due to US export controls, are.
      • trollbridge 4mo ago
        And our export controls are going to turn China into a winner in the AI arms race if we're not careful.
        • rented_mule 4mo ago
          I retired a few years ago, but I still write a fair bit of code. I was using Copilot's code completion before I retired, but coding agents hadn't come around yet. I've been wanting to try them, but I kept putting it off, and now the price increases make it hard to justify. So I just started trying CodeWhale (https://github.com/Hmbown/CodeWhale https://github.com/Hmbown/CodeWhale) with DeepSeek V4. I expected to be impressed by the abilities (which still require plenty of oversight). I didn't expect to be completely shocked by how cheep it is. After most of a week of using it 4-8 hours a day, which would amount to a full week of coding in many jobs after you account for non-coding activities, I'm about to hit $3 in total usage. So we're talking $10-20 per month for single-agent use by a full time software developer? And I'm sure some of my usage is waste as I'm still getting my head around things like compaction. If I take a break for a few weeks, I pay nothing because there is no subscription. If DeepSeek and Xiaomi MiMo stay within a few months of the US-based models in terms of capabilities and US companies don't figure out how to drastically cut prices, I can't see how China hasn't already won. Protectionism would be one reason, but that might be ceding 50-90% of the total addressable market, and bring us closer to moving knowledge work out of the US the same way we did with manufacturing because it's too expensive in the US.
  • AllegedAlec 4mo ago
    > The actual cost is going to drop 99% in ~4 years. And fusion power is just 2 decades into the future!
    • jjav 4mo ago
      Full self driving guaranteed here before the end of the year (every year).
  • mrandish 4mo ago
    > The actual cost is going to drop 99% in ~4 years. We have little visibility into current frontier model costs at mass scale. As a broad historical trend, tech costs tend to fall over longer time periods but your claim far exceeds Moore's Law rates in its heyday - and that heyday is long gone. In 2021 TSMC announced it was increasing it's price per gate for new nodes for the first time in its history. In the past five years cutting edge nodes have delivered ~8-15% real-world performance gains on average at costs at least 10-20% more than the last node. If you're positing a string of unprecedented efficiency breakthroughs in LLM algorithms - such extraordinary claims require extraordinary evidence.
• datakan 4mo ago
  The closer people live to the consequences of their decisions the more rational they become. Until leaders(and I use that term loosely) are held accountable, the insanity will continue.
  • greesil 4mo ago
    Their only accountability is to the stock price. The insanity will continue.
    • dfedbeef 4mo ago
      As long as our stock price continues to... Continues to rise... Which... Hmm... I'm just now reading our balance sheet. Is this number right? Great, thanks. As I was saying, you're all fired.
    • Henchman21 4mo ago
      I’m willing to bet that most of us here are capable of acquiring pitchforks and torches. I predict that will be their comeuppance; it will begin a new era in history.
      • cindyllm 4mo ago
        [dead]
      • deleted 4mo ago
        [deleted]
  • deleted 4mo ago
    [deleted]
  • oofbey 4mo ago
    I’m sorry you are used to working with out of touch leadership. Not all companies are like that. Even big ones can have smart, empathetic leaders. Although very often money gets in the way of empathy.
    • 4mo ago
• sdeframond 4mo ago
  Groups resist to change - the bigger the group, the most resistance there is. As a leader, pushing for rapid change cannot really be nuanced lest the push dissipates into the organization's entropy.
  • HarHarVeryFunny 4mo ago
    Perhaps, but the change you get (if any) is most likely to be what you push for and reward/punish. It's irrational to push for tokenmaxxing (literally "please increase our AI spending") and not expect that this is the result you are going to get. You won't get productivity increase, since that is not what you are pushing for - you will get token usage maximization (engineers running inane agentic tasks against your code base to increase usage, using company paid AI for their side projects, etc, etc).
    • SpicyLemonZest 4mo ago
      I'm not sure the leaders would disagree with what you're saying. They tokenmaxxed to understand what it looks like when AI gets into every corner of the business; now they feel they've gotten enough info (or at least that more info wouldn't be worth the cost), so they're adding in cost controls. As the article says, this is not great for AI model providers trying to predict what their future revenue is going to be, but it's not obvious that there's any mistake here for AI users.
      • HarHarVeryFunny 4mo ago
        > They tokenmaxxed to understand what it looks like when AI gets into every corner of the business Perhaps that is what they were trying to do, but the reality is that all they will have got is a large token bill. The decision makers may have hoped that tokens would be used in most productive fashion possible so they could evaluate if the cost was worth it, but what they will have actually got is what they asked for and measured, high token usage (applied to whatever people needed to do to get their usage stats up, regardless of productivity). The other business-as-usual factor is that there will be false reporting up the chain, so if the company understands the CEO want to see high AI usage and productivity gains, then s/he will see high AI usage (a large token bill) and will be fed success reports of corresponding productivity gains. In a typical corporate environment, if all your peers are reporting success, achieving what the CEO wants, do you want to be the only one reporting failure? So - everyone reports high AI usage (easy for the employees to make happen), and most everyone also reports productivity gains if they understand this is the expectation.
• steve1977 4mo ago
  That's nothing new though. It's just very obvious this time.
• vasco 4mo ago
  During ZIRP they discovered that the way to lead companies nowadays is to become a maxxer of whatever current fad is, and the more you maxx the better. And then when things change and you're wrong, you'll be a strong leader and, in ZIRPs case fire everyone you over-hired, with AI will be similar. Why be a normal guy that waits to see what happens and is measured and pragmatic when you can get attention basically through the whole cycle by being the earliest adopter, adopt it to the maxx, then also be the loudest big brain when the tide changes and be praised for "taking hard decisions" when you revert everything you said so far? The fakemaxxing economy.
  • janussunaj 4mo ago
    A special case of the more general cringe economy we're in. The dumbest, most outrageous ideas win, amplified by social media. Say stupid sh*t loudly, be wrong, profit.
• im3w1l 4mo ago
  Having studied control theory I think it makes perfect sense. When trying to make a system target a new level it's quite natural for there to be overshoot that needs to be reigned in. It's also natural for the correction to go too far and need to be corrected in turn. This is not indicative of stupidity it's completely normal. It would only be laughable if they waited way too long to reverse course, but I don't think that's the case.
  • RJIb8RBYxzAMX9u 4mo ago
    Suppose I'm driving at 20 kph, and I set my cruise control to 40 kph. My car then goes WOT, overshoots my target speed and hits 120 kph, at which point it slams on the brakes[0], dropping my speed to 15 kph. It repeats until it finally settles at my target speed. (Rhetorical question) would that be considered "completely normal"? Over/undershoots and corrections are of course unavoidable and normal; the absurdity is at the magnitude and rate of change. Furthermore, this is giving it the benefit of the doubt, that measuring AI spend is a good indicator; that's arguably also in dispute. To stretch my car analogy a bit more: it would be like the cruse control system has to hit the target speed, but it only has data from the O2 sensors. [0] I know that the "classic" cruise control system cannot apply the brakes, but hey no analogy's perfect.
  • adammarples 4mo ago
    It's not like they accidentally overshot, they were telling people to tokenmax, they didn't even know you could overshoot they thought it was exponential gains all the way. Subtle ideas like balance were not on their minds.
    • im3w1l 4mo ago
      Intentionally overshooting can be a legitimate strategy.
• bunderbunder 4mo ago
  I've been enjoying journalist Ed Zitron's recent diatribes about how impossible it is to find a business leader who had a plan for measuring their ROI from adopting AI coding. What he says he's consistently hearing from them mirrors what I saw at my own employer: they thought they had ROI metrics, but they actually only had usage metrics such as "lines of code committed" or "number of pull requests". The only way those could possibly work as an ROI measure is if your business charges customers by the line of code.
  • conception 4mo ago
    What they really means is they previously had no valid metric to measure productivity of developers before either. AI or not.
    • no-name-here 4mo ago
      But at least pre AI, most managers presumably subjectively measured devs on relevant performance. Using systems where employees who burn the most tokens ($) per week ‘win’ is crazy - just ask the AI to spin up a subagents to implement every conceivable approach to a task, then spin up n agent judge to pick the winner, and repeat. You've immediately got 50x or whatever your previous usage from that alone.
      • canyp 4mo ago
        I had cynically done this sort of tokenmaxxing for a while as a burnt offering to the token-hungry non-leadership. Eventually I got tired of it and got back to work.
    • bunderbunder 4mo ago
      Measuring productivity of developers isn’t really in line with what needs to happen, either. A team can be incredibly productive and still generate negative 100% ROI if what they are building so industriously is stuff that nobody wants to buy. Which reflects another thing I’ve seen at work. A lot of what AI coding has enabled is diving headfirst into quagmires. Our costs have spiked - not just because of the token spend, also because we gotta pay the cloud platform to run all these new services, operators to operate them, marketers to market them, etc. - but revenue hasn’t budged.
• morgan814 4mo ago
  > leaders Don’t play their game and call them leaders. They are management, bosses, executives. > They are making decisions about things that they just don't understand. And are completely unworried about it. Clowns, even. > Just blindly following whatever the news cycle is about AI. But followers might be most apt. —— This is such a huge pet peeve of mine. Describing management goofs using their language that makes them sound all-so-brilliant. We constantly watch these people do the dumbest shit and then they go around describing themselves as “thought leaders” and “servant leaders”. When, really, most are just clowns with fragile egos. And, while I’m rambling, they’ve tried to take away the fact we are workers by calling us individual contributors. Using language to attempt and hide the hierarchy and power dynamic at play. It just…bothers me so much.
  • joquarky 4mo ago
    I don't hear them refer to themselves as "job creators" much these days. And many of them still claim they are "risk takers", but have effectively insulated themselves from risk by socializing losses.
  • danaris 4mo ago
    > Don’t play their game and call them leaders. They are management, bosses, executives. You're falling into a common trap here: the ambiguity of the English language. "Leader" means multiple different things. Yes, it means someone who has leadership qualities—who genuinely inspires those around them to do better, or who boldly marches into the unknown and gets people to follow them. But it also means "someone in charge of a thing." Now it's certainly true that many people in charge of things who are also really bad at actually inspiring or getting people to follow them (aside from with threats of destitution) also play on that ambiguity to try to convince people that because they're in charge of things, they must also be Good Leaders, and that's crappy...but yelling at others for using the term casually is very much an "old man yells at cloud" situation.
• schonfinkel 4mo ago
  The worst part is that techies can still work around the insanity if they keep their opinions private. For the serious average Joe the AI mandates must be feeling like hell on earth. I once worked in a company that had soviet-level efforts to push LLMs into everything, someone eventually made the classic "Natural Language -> SQL Query -> Magic Result in webpage" and got promoted, the tool got mandated for every non-tech employee as part of an AI-boosting effort (people pushing metrics up). One day I wake up with a product person in despair because the tool couldn't handle what looked like a very simple aggregation, I stopped what I was doing, crafted a 30-line SQL query over HORRIBLE TABLES, a couple CTEs and window functions here and there got him what he wanted. I found out later that single query that took 30 minutes to make saved him from inheriting a 6-month effort to create a microservice dedicated to patching said tool.