19 ms·

The cost of restructuring has also gone down. The cost of shoring up behavior with tests ahead of a restructure has gone down because of AI. The cost of imple

The cost of restructuring has also gone down. The cost of shoring up behavior with tests ahead of a restructure has gone down because of AI. The cost of implementing a zero downtime migration has gone down because of AI. A big part of the rust hype has been the low cost of restructuring within an application, even before AI. And now even more so. The opportunity cost of not being able to safely restructure has gone up substantially. This is the number one thing I optimize for now: the ability to quickly and safely change significant parts of the code and product.

cassianoleal 3mo ago
> This is the number one thing I optimize for now: the ability to quickly and safely change significant parts of the code and product. This was always a good thing. Its value has nothing to do with the advent of AI coding. > The opportunity cost of not being able to safely restructure has gone up substantially. This bit is contradictory with everything else you said. Prior to AI coding it would take a lot longer to perform restructures. If anything, the thing you're now optimising for has gone down in value. It's still valuable, but perhaps a little less.
- mohamedkoubaa 3mo ago
  Ironically, AI assisted/generated code is not trending in the direction of the ability to safely and quickly change. Especially when piloted poorly
  - cassianoleal 3mo ago
    I hear you! I also actually find it a lot more difficult to ensure proper guardrails are in place to keep the agents doing good engineering work.
  - joshka 3mo ago
    Citation needed. But in seriousness, this intuitively feels like something (as phrased) that would be easily influenced by loud noise and quantity rather than hard facts. The "piloted poorly" part is applicable to any tool use. AI is no different there other than its adoption rate.
    - majormajor 3mo ago
      The pattern I observe is: "write code, write test, make things green" This is different in two ways from the classic TDD red-green-refactor suggestion: 1) they don't start with the test first, so the tests that get implemented are after writing the code, and run the risk of the model attention being now more influenced by the just-written code than the original spec 2) they finish when everything is green and don't followup with the "refactor" step unless manually prompted (either directly or indirectly by your own scaffolding/rules/whatever). this results in frequent hyper-local non-ideal-longterm fixes for things that went wrong in the first shot at writing the code pre-test-writing. (As always, the only person who can ensure the code landing in your repo is good is you.)
      - joshka 3mo ago
        But here's the rub - if you want your clanker to do those steps, it's usually a simple matter of adding them to your AGENTS.md and then it always does them. I'm a big fan of the characterization step step being added. And it can be reasonable to add this before or even after the fact as a commit prior to your actual commit (assuming you're familiar with using tools where that's easy to do - e.g. jj or git with rebase). And the agents can do this - they just don't tend to without you saying to do so. A lot of engineering practice comes from choosing which elements are reasonable to use given the context of what you're building. Providing that is your job. When you do that poorly, you get poor results. But garbage in garbage out has always been a thing. Any advanced automation amplifies ambient assumptions
- phamilton 3mo ago
  I'm not talking about time. I'm talking about safety. The amount of times I've seen "I refactored it, but I'm not confident enough to take it to prod" is significant. Being able to go faster but still not ship it is the huge opportunity cost.
  - sublinear 3mo ago
    In the broader corporate world, that's not "opportunity cost". All changes are considered "risk". All deployments must be approved by an advisory board. All work must originate from a clear business need. Analysis of those needs is not concerned with implementation, least of which whether "AI" is used. What matters far more is that a contract requires work to be done by a deadline. Those deadlines are driven by policy. There will be no adjustment to policy unless tangible benefits are shown from more frequent deployments of code. I gotta tell you that's extremely unlikely if you're already shipping every other week at the end of the sprint. Most of that sprint is spent in meetings, not writing code. Nobody is doing big refactors because it wasn't built so fast to require them. There's some technical debt, but nothing so severe. Those meetings are preventing risk, not wasting time. The bottleneck is a feature, not a bug. If you think the future of dev work is to be a bureaucrat, you're right! It looks like the rest of the world outside of SV is ahead of the curve and living in the future.
    - phamilton 3mo ago
      That's not at all what I meant. I mean "We can't build X because our code structure makes that difficult" has an opportunity cost of the value of X. I don't think the future of dev work is being a bureaucrat. I've done more rigorous engineering the last two years than I did previously. I'm more confident in the things I shop and they were built in a fraction of the time. It's a bright future for software engineering.
      - sublinear 3mo ago
        [dead]
  - cassianoleal 3mo ago
- xboxnolifes 3mo ago
  >> This is the number one thing I optimize for now: the ability to quickly and safely change significant parts of the code and product. >This was always a good thing. Its value has nothing to do with the advent of AI coding. The value of type safe code did not go up, the cost of development speed has gone down.
samrus 3mo ago
Your increasing thrash betting that AI will fix it for you. The only thing your getting in return is not having to think that hard. It doesnt cost that much time or effort to think hard, so you will be outcompeted by people levergaing AI as much as you, but thinking enough to not have it be thrashing around
- 6510 3mo ago
  > It doesnt cost that much time or effort to think hard If you manage to avoid effort and thinking long enough it will get harder.
rednb 3mo ago
> the ability to quickly and safely change significant parts of the code and product. Hum, this reminds me something... "O: open to extension, closed to modifications". Old things are new again. From context efficiency with approaches such ad DDD and clean architectures, all the way to items such as this one, AI is not creating new tradeoff, it just acts as a multiplier, multiplying productivity for teams doing things right, and multiplying debt for teams having a low quality bar as far as design and architecture are concerned.
- rzmmm 3mo ago
  Over the last couple months I've seen the debt multiplication in some OSS projects. It's like premature aging of a codebase.
- phamilton 3mo ago
  > Old things are new again. This is exactly how I've felt. I've read some many old papers and books and found great techniques that are even more applicable than ever.
lmm 3mo ago
> The cost of restructuring has also gone down. > The cost of shoring up behavior with tests ahead of a restructure has gone down because of AI. Disagree. The growth in brittle AI-generated tests means restructuring is more costly than it was before. Pruning your test suite so that it tests the essence of the problem and not the incidental design decisions is something AIs aren't yet capable of.
- chowells 3mo ago
  Oh, it's worse than that. I've seen a rise in mutation testing, intended to ensure any change in implementation is caught by tests. Think about that for a moment. It's giving a fancy name to creating brittle tests than fail if any line of code is changed. And this is seen as a good thing, because LLMs are really bad at confining their changes appropriately. Testing is really in a dark place right now.
majormajor 3mo ago
> The cost of shoring up behavior with tests ahead of a restructure has gone down because of AI. Yes. But the ease of not doing that and instead just getting a brittle set of three-quarters-baked tests is extremely high! And many people seem happy to go from "a few human-written mediocre brittle tests" to "a bunch of AI-written mediocre brittle tests" because it is an objective improvement and the people who weren't avoiding speculative structure and looking for the write boundaries before are happy to also not do so no. So completely agree with the "take advantage of the tools this way" but I also wouldn't claim it's a reason to no longer worry about if you're building the wrong castles in the sky too early, because perfect refactor-proof testing contracts are still usually pretty hard to design.
- phamilton 3mo ago
  > perfect refactor-proof testing contracts are still usually pretty hard to design Not sure about hard, but definitely rare and we as an industry are under-skilled in these areas. We have decades of research and tools for testing and verification of software. Property tests, dependent types, formal verification and proof, etc. The paths have been there, we have just collectively prioritized other things. It requires an intentional shift in how we design and build. That shift is the harder part.
ffsm8 3mo ago
> The cost of implementing a zero downtime migration has gone down because of AI. You either don't know what that technical term means or you're just wrong. AI does not meaningfully move the needle on that. It only makes backwards compatible deployments easier insofar you're able to do the overhead for splitting the change with less effort then before.
- phamilton 3mo ago
  I've always known how to do zero downtime migrations. The question has always been "Is the engineering cost worth it?" > It only makes backwards compatible deployments easier insofar you're able to do the overhead for splitting the change with less effort then before. Yes. That reduces the cost of implementing them. To be clear, I'm not talking about "Split my db migration and my code that depends on the new table". I'm talking about things like "Set up dual writes between an old database schema and a new database schema with a thorough test suite and do shadow reads against both datasets in prod to do differential testing". That's nontrivial engineering effort that would definitely warrant a discussion in the past. Today we just do it. It's fast and lowers risk.

cassianoleal 3mo ago

> This is the number one thing I optimize for now: the ability to quickly and safely change significant parts of the code and product. This was always a good thing. Its value has nothing to do with the advent of AI coding. > The opportunity cost of not being able to safely restructure has gone up substantially. This bit is contradictory with everything else you said. Prior to AI coding it would take a lot longer to perform restructures. If anything, the thing you're now optimising for has gone down in value. It's still valuable, but perhaps a little less.

mohamedkoubaa 3mo ago
Ironically, AI assisted/generated code is not trending in the direction of the ability to safely and quickly change. Especially when piloted poorly
- cassianoleal 3mo ago
  I hear you! I also actually find it a lot more difficult to ensure proper guardrails are in place to keep the agents doing good engineering work.
- joshka 3mo ago
  Citation needed. But in seriousness, this intuitively feels like something (as phrased) that would be easily influenced by loud noise and quantity rather than hard facts. The "piloted poorly" part is applicable to any tool use. AI is no different there other than its adoption rate.
  - majormajor 3mo ago
    The pattern I observe is: "write code, write test, make things green" This is different in two ways from the classic TDD red-green-refactor suggestion: 1) they don't start with the test first, so the tests that get implemented are after writing the code, and run the risk of the model attention being now more influenced by the just-written code than the original spec 2) they finish when everything is green and don't followup with the "refactor" step unless manually prompted (either directly or indirectly by your own scaffolding/rules/whatever). this results in frequent hyper-local non-ideal-longterm fixes for things that went wrong in the first shot at writing the code pre-test-writing. (As always, the only person who can ensure the code landing in your repo is good is you.)
    - joshka 3mo ago
      But here's the rub - if you want your clanker to do those steps, it's usually a simple matter of adding them to your AGENTS.md and then it always does them. I'm a big fan of the characterization step step being added. And it can be reasonable to add this before or even after the fact as a commit prior to your actual commit (assuming you're familiar with using tools where that's easy to do - e.g. jj or git with rebase). And the agents can do this - they just don't tend to without you saying to do so. A lot of engineering practice comes from choosing which elements are reasonable to use given the context of what you're building. Providing that is your job. When you do that poorly, you get poor results. But garbage in garbage out has always been a thing. Any advanced automation amplifies ambient assumptions
phamilton 3mo ago
I'm not talking about time. I'm talking about safety. The amount of times I've seen "I refactored it, but I'm not confident enough to take it to prod" is significant. Being able to go faster but still not ship it is the huge opportunity cost.
- sublinear 3mo ago
  In the broader corporate world, that's not "opportunity cost". All changes are considered "risk". All deployments must be approved by an advisory board. All work must originate from a clear business need. Analysis of those needs is not concerned with implementation, least of which whether "AI" is used. What matters far more is that a contract requires work to be done by a deadline. Those deadlines are driven by policy. There will be no adjustment to policy unless tangible benefits are shown from more frequent deployments of code. I gotta tell you that's extremely unlikely if you're already shipping every other week at the end of the sprint. Most of that sprint is spent in meetings, not writing code. Nobody is doing big refactors because it wasn't built so fast to require them. There's some technical debt, but nothing so severe. Those meetings are preventing risk, not wasting time. The bottleneck is a feature, not a bug. If you think the future of dev work is to be a bureaucrat, you're right! It looks like the rest of the world outside of SV is ahead of the curve and living in the future.
  - phamilton 3mo ago
    That's not at all what I meant. I mean "We can't build X because our code structure makes that difficult" has an opportunity cost of the value of X. I don't think the future of dev work is being a bureaucrat. I've done more rigorous engineering the last two years than I did previously. I'm more confident in the things I shop and they were built in a fraction of the time. It's a bright future for software engineering.
    - sublinear 3mo ago
      [dead]
- cassianoleal 3mo ago
xboxnolifes 3mo ago
>> This is the number one thing I optimize for now: the ability to quickly and safely change significant parts of the code and product. >This was always a good thing. Its value has nothing to do with the advent of AI coding. The value of type safe code did not go up, the cost of development speed has gone down.

samrus 3mo ago

Your increasing thrash betting that AI will fix it for you. The only thing your getting in return is not having to think that hard. It doesnt cost that much time or effort to think hard, so you will be outcompeted by people levergaing AI as much as you, but thinking enough to not have it be thrashing around

6510 3mo ago
> It doesnt cost that much time or effort to think hard If you manage to avoid effort and thinking long enough it will get harder.

rednb 3mo ago

> the ability to quickly and safely change significant parts of the code and product. Hum, this reminds me something... "O: open to extension, closed to modifications". Old things are new again. From context efficiency with approaches such ad DDD and clean architectures, all the way to items such as this one, AI is not creating new tradeoff, it just acts as a multiplier, multiplying productivity for teams doing things right, and multiplying debt for teams having a low quality bar as far as design and architecture are concerned.

rzmmm 3mo ago
Over the last couple months I've seen the debt multiplication in some OSS projects. It's like premature aging of a codebase.
phamilton 3mo ago
> Old things are new again. This is exactly how I've felt. I've read some many old papers and books and found great techniques that are even more applicable than ever.

lmm 3mo ago

> The cost of restructuring has also gone down. > The cost of shoring up behavior with tests ahead of a restructure has gone down because of AI. Disagree. The growth in brittle AI-generated tests means restructuring is more costly than it was before. Pruning your test suite so that it tests the essence of the problem and not the incidental design decisions is something AIs aren't yet capable of.

chowells 3mo ago
Oh, it's worse than that. I've seen a rise in mutation testing, intended to ensure any change in implementation is caught by tests. Think about that for a moment. It's giving a fancy name to creating brittle tests than fail if any line of code is changed. And this is seen as a good thing, because LLMs are really bad at confining their changes appropriately. Testing is really in a dark place right now.

majormajor 3mo ago

> The cost of shoring up behavior with tests ahead of a restructure has gone down because of AI. Yes. But the ease of not doing that and instead just getting a brittle set of three-quarters-baked tests is extremely high! And many people seem happy to go from "a few human-written mediocre brittle tests" to "a bunch of AI-written mediocre brittle tests" because it is an objective improvement and the people who weren't avoiding speculative structure and looking for the write boundaries before are happy to also not do so no. So completely agree with the "take advantage of the tools this way" but I also wouldn't claim it's a reason to no longer worry about if you're building the wrong castles in the sky too early, because perfect refactor-proof testing contracts are still usually pretty hard to design.

phamilton 3mo ago
> perfect refactor-proof testing contracts are still usually pretty hard to design Not sure about hard, but definitely rare and we as an industry are under-skilled in these areas. We have decades of research and tools for testing and verification of software. Property tests, dependent types, formal verification and proof, etc. The paths have been there, we have just collectively prioritized other things. It requires an intentional shift in how we design and build. That shift is the harder part.

ffsm8 3mo ago

> The cost of implementing a zero downtime migration has gone down because of AI. You either don't know what that technical term means or you're just wrong. AI does not meaningfully move the needle on that. It only makes backwards compatible deployments easier insofar you're able to do the overhead for splitting the change with less effort then before.

phamilton 3mo ago
I've always known how to do zero downtime migrations. The question has always been "Is the engineering cost worth it?" > It only makes backwards compatible deployments easier insofar you're able to do the overhead for splitting the change with less effort then before. Yes. That reduces the cost of implementing them. To be clear, I'm not talking about "Split my db migration and my code that depends on the new table". I'm talking about things like "Set up dual writes between an old database schema and a new database schema with a thorough test suite and do shadow reads against both datasets in prod to do differential testing". That's nontrivial engineering effort that would definitely warrant a discussion in the past. Today we just do it. It's fast and lowers risk.