9 ms·

With all the emphasis on the speed of modern AI tools, we often seem to forget that velocity is a vector quantity. Increased speed only gets us where we want to

With all the emphasis on the speed of modern AI tools, we often seem to forget that velocity is a vector quantity. Increased speed only gets us where we want to be sooner if we are also heading in the right direction. If we’re far enough off course, increasing speed becomes counterproductive and it ends up taking longer to get where we want to be. I’ve been noticing that this simple reality explains almost all of both the good and the bad that I hear about LLM-based coding tools. Using AI for research or to spin up a quick demo or prototype is using it to help plot a course. A lot of the multi-stage agentic workflows also come down to creating guard rails before doing the main implementation so the AI can’t get too far off track. Most of the success stories I hear seem to be in these areas so far. Meanwhile, probably the most common criticism I see is that an AI that is simply given a prompt to implement some new feature or bug fix for an existing system often misunderstands or makes bad assumptions and ends up repeatedly running into dead ends. It moves fast but without knowing which direction to move in.

skydhash 6mo ago
> It moves fast but without knowing which direction to move in. It also moves fast with a tendency to pick the wrong direction (according to the goal of the prompter) at every decision point (known or unknown).
- tombert 6mo ago
  I've definitely gotten it into contexts where it will never stop going into the wrong direction, even when I tell it to forget everything it did before, and told it a correct path forward. Usually restarting the entire session fixes it, but not always.
  - FrankRay78 6mo ago
    Shrewd context management is the go here.
gopalv 6mo ago
> Increased speed only gets us where we want to be sooner if we are also heading in the right direction. This is a real problem when the "direction" == "good feedback" from a customer standpoint. Before we had a product person for every ~20 people generating code and now we're all product people, the machines are writing the code (not all of it, but enough of it that I will -1 a ~4000 line PR and ask someone to start over, instead of digging out of the hole in the same PR). Feedback takes time on the system by real users to come back to the product team. You need a PID like smoothing curve over your feature changes. Like you said, Speed isn't velocity. Specifically if you have a decent experiment framework to keep this disclosure progressive in the customer base, going the wrong direction isn't a huge penalty as it used to be. I liked the PostHog newsletter about the "Hidden dangers of shipping fast", I can't find a good direct link to it.
- Ezra 6mo ago
  This is the article you referred to: https://newsletter.posthog.com/p/the-hidden-danger-of-shipping-fast https://newsletter.posthog.com/p/the-hidden-danger-of-shippi...
  - chrisweekly 6mo ago
    Thanks! Great link.
- ozim 6mo ago
  Have been there, we got pushback from users and we had to back off with releases. Users hunted product owner with pitchforks and torches. As dev team we were able to crank the speed even more and silly product people thought they are doing something good by demanding even more from us. But that was one of the instances where users were helpful :). People use dozens of apps every day to do their work. Just think about how are you going to make time to give feedback to each of each.
  - johnnyanmac 6mo ago
    > Just think about how are you going to make time to give feedback to each of each. That's pretty much solved by the size of the audiences. You won't give feedback on 12 apps, but 11 other people will probably do so on 11 different apps. Of course, the issue with my domain is that there's plenty of feedback, and product owners just dismiss it. Burn down your entire portfolio to get that boosted shareholder value for the next earnings report.
nabbed 6mo ago
>Increased speed only gets us where we want to be sooner if we are also heading in the right direction. I suppose there is an argument that if you are building the wrong thing, build it fast so that you can find out more quickly that you built the wrong thing, allowing you to iterate more quickly.
- roughly 6mo ago
  It's still faster and cheaper to just build the right thing to begin with. As the old saying goes, spend your time sharpening your ax.
  - mgfist 6mo ago
    Yes, but only if you have an ax to sharpen. With a lot of things it takes trial and error to make progress. You can take this pretty up high too - sometimes it takes building multiple products or companies to get it right
    - Jare 6mo ago
      > With a lot of things it takes trial and error to make progress Way too often that is used as an excuse for various forms of laziness; to not think about the things you can already know. And that lack of thinking repeats in an endless cycle when, after your trial and error, you don't use what you learned because "let's look forward not backward", "let's fail fast and often" and similar platitudes. Catchy slogans and heartfelt desires are great but you gotta put the brains in it too.
      - diatone 6mo ago
        Without commenting about the frequency of negligence myself, I suspect at least that you and GP are in agreement. I doubt GP is suggesting ‘go ahead and be negligent to feedback and guardrails that let you course correct early.’ Plugging the Cynefin framework as a useful technique for practitioners here. It doesn’t have to be hard to choose whether or not rigorous planning is appropriate for the task at hand, versus probe-test-backtrack with tight iteration loops.
deleted 6mo ago
[deleted]
siddboots 6mo ago
I've been working on a side project for ~10 years (very intermittently) that involves a tricky combination of mathematics, classical AI algorithms, and programming language design, and I've gone though this very slow but rewarding journey to work out how all of the pieces should fit together properly. In the last year or so I've been able to prototype it and accelerate the development quite significantly using Claude and pals, and now it is very close to a finished product. One one hand there's no doubt in my mind that the LLM tools can make this sort of thing faster and let you churn through ideas until you find the right ones, but on the other hand, if I hadn't had that slow burn of mostly just thinking about it conceptually for 10 years, I would have ended up vibe coding a much worse product.
- TheJord 6mo ago
  10 years of thinking before shipping is actually the move. The AI just becomes a power tool — useless if you don't know what you're building, unstoppable if you do
  - xyzzy123 6mo ago
    You're missing out on 10 years of real world feedback if you do this.
    - ozim 6mo ago
      There are projects where „real world feedback” will be actively harmful.
  - layla5alive 6mo ago
    Are you being cute impersonating an LLM, or are you an LLM posting?
jstummbillig 6mo ago
Ah, metaphors. Abstract concepts are not moving objects. You don't actually need to "turn it around" or "sail past it". You can break the laws of physics (because they don't apply). You can teleport around. Speed actually just wins, because we are usually constrained by time.
- 3121aksj8p 6mo ago
  Yes, AI can emit BS at a faster rate than ever. We can also produce more blog posts than ever. Working or useful software? AI hasn't produced any at all since 2023.
  - jstummbillig 6mo ago
    Wow, an account, just for that? I feel honored and slightly suspicious.
    - nprateem 6mo ago
      Altman didn't want to post from his own account
- Chris_Newton 6mo ago
  Speed actually just wins, because we are usually constrained by time. Sorry, but I don’t understand what you mean here. What do we win by being faster at producing the wrong things?
  - jstummbillig 6mo ago
    You learn more quickly that you are producing the wrong thing.
    - johnnyanmac 6mo ago
      This implies that 1) a lot of shallow, orthogonal directions is better than 1 deep, careful approach 2) There's no social aspect to churning out a bunch of slop that will affect the perception of potential "right things" later. My domain can be particularly grudgeful in this regard.
throwaway27448 6mo ago
> Increased speed only gets us where we want to be sooner if we are also heading in the right direction. A proper capitalist system will tend toward the right direction as directed by the market yea? All of this neuroticism about AI doesn't matter.
- johnnyanmac 6mo ago
  Yes, the true market; shareholders.
andai 6mo ago
> If we’re far enough off course, increasing speed becomes counterproductive and it ends up taking longer to get where we want to be. This reminded me of the idea that civilization is already a misaligned superintelligence, and that technology (incl. AI) just moves it faster in the wrong direction. That's basically the problem of supermorality. If you're an actually benevolent AI, do you do what civilization tells you? Or do you do what is good? What happens if you disagree?
- abustamam 6mo ago
  I think it depends on what is good, and who it's good for. Thus far, AI has been good... For venture capitalists. Jury's out if it's good for humanity and civilization at large. There have been a lot of benevolent usages of AI thus far, but also a lot of bad. As for those who disagree with the "benevolent AI," I think they just get sent to the gallows (either metaphorically or literally)
_doctor_love 6mo ago
> velocity is a vector Exactly this. Velocity is a vector. It has magnitude (aka speed) and direction. Our industry has chased magnitude over all else for so long. Now we can put nitro in everyone's car and we get to where we wish to go very fast. Suddenly bad direction-setting is getting feedback where there used to be friction and natural time to steer. My greatest hope is that a ton of bad leaders and middle managers end up finally getting exposed due to the advent of AI. (Will I be disappointed? Almost certainly yes.)
nprateem 6mo ago
The biggest problem is the fact they DON'T clarify their stupid assumptions. The number of times I've seen them get the wrong end of the stick in their COT is ridiculous. Even when I tell them to only implement after my explicit approval they ignore this after 2 or 3 followups and then it's back to them going down blind alleys.
theshrike79 6mo ago
Imagine you're lost in a jungle, you know within 180 degrees where you should go to get to the nearest civilization. If you need to spend actual humans searching every 10 degrees (or more if they need to go far), the cost might not be worth it. What if they don't come back? Every human scouting is not foraging for the group or protecting them from whatever is in the jungle, Now if you have a near infinite capacity of autonomous drones you can send one every degree from your position and get an approximate estimate whether that direction is safe or not. It's the same with programming, you can do a TON of exploratory crap with near zero extra cost. Just to see if that's the right direction or not. Basically lean in to the "fast but inaccurate" aspect of AI. Like I had a hare-brained idea of a local web-based file manager / media browser that would use YOLOv8 and CLIP to tag and classify images and videos as you browse. Took me about 30 minutes of active time over a few evenings while doing other stuff to conclude that, yes, it IS doable. Now I can focus my actual brain on it.

skydhash 6mo ago

> It moves fast but without knowing which direction to move in. It also moves fast with a tendency to pick the wrong direction (according to the goal of the prompter) at every decision point (known or unknown).

tombert 6mo ago
I've definitely gotten it into contexts where it will never stop going into the wrong direction, even when I tell it to forget everything it did before, and told it a correct path forward. Usually restarting the entire session fixes it, but not always.
- FrankRay78 6mo ago
  Shrewd context management is the go here.

gopalv 6mo ago

> Increased speed only gets us where we want to be sooner if we are also heading in the right direction. This is a real problem when the "direction" == "good feedback" from a customer standpoint. Before we had a product person for every ~20 people generating code and now we're all product people, the machines are writing the code (not all of it, but enough of it that I will -1 a ~4000 line PR and ask someone to start over, instead of digging out of the hole in the same PR). Feedback takes time on the system by real users to come back to the product team. You need a PID like smoothing curve over your feature changes. Like you said, Speed isn't velocity. Specifically if you have a decent experiment framework to keep this disclosure progressive in the customer base, going the wrong direction isn't a huge penalty as it used to be. I liked the PostHog newsletter about the "Hidden dangers of shipping fast", I can't find a good direct link to it.

Ezra 6mo ago
This is the article you referred to: https://newsletter.posthog.com/p/the-hidden-danger-of-shipping-fast https://newsletter.posthog.com/p/the-hidden-danger-of-shippi...
- chrisweekly 6mo ago
  Thanks! Great link.
ozim 6mo ago
Have been there, we got pushback from users and we had to back off with releases. Users hunted product owner with pitchforks and torches. As dev team we were able to crank the speed even more and silly product people thought they are doing something good by demanding even more from us. But that was one of the instances where users were helpful :). People use dozens of apps every day to do their work. Just think about how are you going to make time to give feedback to each of each.
- johnnyanmac 6mo ago
  > Just think about how are you going to make time to give feedback to each of each. That's pretty much solved by the size of the audiences. You won't give feedback on 12 apps, but 11 other people will probably do so on 11 different apps. Of course, the issue with my domain is that there's plenty of feedback, and product owners just dismiss it. Burn down your entire portfolio to get that boosted shareholder value for the next earnings report.

nabbed 6mo ago

>Increased speed only gets us where we want to be sooner if we are also heading in the right direction. I suppose there is an argument that if you are building the wrong thing, build it fast so that you can find out more quickly that you built the wrong thing, allowing you to iterate more quickly.

roughly 6mo ago
It's still faster and cheaper to just build the right thing to begin with. As the old saying goes, spend your time sharpening your ax.
- mgfist 6mo ago
  Yes, but only if you have an ax to sharpen. With a lot of things it takes trial and error to make progress. You can take this pretty up high too - sometimes it takes building multiple products or companies to get it right
  - Jare 6mo ago
    > With a lot of things it takes trial and error to make progress Way too often that is used as an excuse for various forms of laziness; to not think about the things you can already know. And that lack of thinking repeats in an endless cycle when, after your trial and error, you don't use what you learned because "let's look forward not backward", "let's fail fast and often" and similar platitudes. Catchy slogans and heartfelt desires are great but you gotta put the brains in it too.
    - diatone 6mo ago
      Without commenting about the frequency of negligence myself, I suspect at least that you and GP are in agreement. I doubt GP is suggesting ‘go ahead and be negligent to feedback and guardrails that let you course correct early.’ Plugging the Cynefin framework as a useful technique for practitioners here. It doesn’t have to be hard to choose whether or not rigorous planning is appropriate for the task at hand, versus probe-test-backtrack with tight iteration loops.

deleted 6mo ago

[deleted]

siddboots 6mo ago

I've been working on a side project for ~10 years (very intermittently) that involves a tricky combination of mathematics, classical AI algorithms, and programming language design, and I've gone though this very slow but rewarding journey to work out how all of the pieces should fit together properly. In the last year or so I've been able to prototype it and accelerate the development quite significantly using Claude and pals, and now it is very close to a finished product. One one hand there's no doubt in my mind that the LLM tools can make this sort of thing faster and let you churn through ideas until you find the right ones, but on the other hand, if I hadn't had that slow burn of mostly just thinking about it conceptually for 10 years, I would have ended up vibe coding a much worse product.

TheJord 6mo ago
10 years of thinking before shipping is actually the move. The AI just becomes a power tool — useless if you don't know what you're building, unstoppable if you do
- xyzzy123 6mo ago
  You're missing out on 10 years of real world feedback if you do this.
  - ozim 6mo ago
    There are projects where „real world feedback” will be actively harmful.
- layla5alive 6mo ago
  Are you being cute impersonating an LLM, or are you an LLM posting?

jstummbillig 6mo ago

Ah, metaphors. Abstract concepts are not moving objects. You don't actually need to "turn it around" or "sail past it". You can break the laws of physics (because they don't apply). You can teleport around. Speed actually just wins, because we are usually constrained by time.

3121aksj8p 6mo ago
Yes, AI can emit BS at a faster rate than ever. We can also produce more blog posts than ever. Working or useful software? AI hasn't produced any at all since 2023.
- jstummbillig 6mo ago
  Wow, an account, just for that? I feel honored and slightly suspicious.
  - nprateem 6mo ago
    Altman didn't want to post from his own account
Chris_Newton 6mo ago
Speed actually just wins, because we are usually constrained by time. Sorry, but I don’t understand what you mean here. What do we win by being faster at producing the wrong things?
- jstummbillig 6mo ago
  You learn more quickly that you are producing the wrong thing.
  - johnnyanmac 6mo ago
    This implies that 1) a lot of shallow, orthogonal directions is better than 1 deep, careful approach 2) There's no social aspect to churning out a bunch of slop that will affect the perception of potential "right things" later. My domain can be particularly grudgeful in this regard.

throwaway27448 6mo ago

> Increased speed only gets us where we want to be sooner if we are also heading in the right direction. A proper capitalist system will tend toward the right direction as directed by the market yea? All of this neuroticism about AI doesn't matter.

johnnyanmac 6mo ago
Yes, the true market; shareholders.

andai 6mo ago

> If we’re far enough off course, increasing speed becomes counterproductive and it ends up taking longer to get where we want to be. This reminded me of the idea that civilization is already a misaligned superintelligence, and that technology (incl. AI) just moves it faster in the wrong direction. That's basically the problem of supermorality. If you're an actually benevolent AI, do you do what civilization tells you? Or do you do what is good? What happens if you disagree?

abustamam 6mo ago
I think it depends on what is good, and who it's good for. Thus far, AI has been good... For venture capitalists. Jury's out if it's good for humanity and civilization at large. There have been a lot of benevolent usages of AI thus far, but also a lot of bad. As for those who disagree with the "benevolent AI," I think they just get sent to the gallows (either metaphorically or literally)

_doctor_love 6mo ago

> velocity is a vector Exactly this. Velocity is a vector. It has magnitude (aka speed) and direction. Our industry has chased magnitude over all else for so long. Now we can put nitro in everyone's car and we get to where we wish to go very fast. Suddenly bad direction-setting is getting feedback where there used to be friction and natural time to steer. My greatest hope is that a ton of bad leaders and middle managers end up finally getting exposed due to the advent of AI. (Will I be disappointed? Almost certainly yes.)

nprateem 6mo ago

The biggest problem is the fact they DON'T clarify their stupid assumptions. The number of times I've seen them get the wrong end of the stick in their COT is ridiculous. Even when I tell them to only implement after my explicit approval they ignore this after 2 or 3 followups and then it's back to them going down blind alleys.

theshrike79 6mo ago

Imagine you're lost in a jungle, you know within 180 degrees where you should go to get to the nearest civilization. If you need to spend actual humans searching every 10 degrees (or more if they need to go far), the cost might not be worth it. What if they don't come back? Every human scouting is not foraging for the group or protecting them from whatever is in the jungle, Now if you have a near infinite capacity of autonomous drones you can send one every degree from your position and get an approximate estimate whether that direction is safe or not. It's the same with programming, you can do a TON of exploratory crap with near zero extra cost. Just to see if that's the right direction or not. Basically lean in to the "fast but inaccurate" aspect of AI. Like I had a hare-brained idea of a local web-based file manager / media browser that would use YOLOv8 and CLIP to tag and classify images and videos as you browse. Took me about 30 minutes of active time over a few evenings while doing other stuff to conclude that, yes, it IS doable. Now I can focus my actual brain on it.