4 ms·

I feel like there is an unspoken assumption of long term maintainability when it comes to LLM generated software. We are still very early in this, so I don't wa

I feel like there is an unspoken assumption of long term maintainability when it comes to LLM generated software. We are still very early in this, so I don't want to make assumptions. In principle, it shouldn't be impossible to both write and maintain stable software, purely with agents. At least, I'm not aware of any actual reasons, backed by a proper theory. On the other hand. I've noticed some persistent issues with code generated by agents, especially poorly supervised agents. If engineers become less vigilant, agents never get to the point of not needing supervision and juniors never pick up required skills, this could lead to real trouble.

jader201 1mo ago
I agree with everything you said. I’m simply playing devil’s advocate, because engineers can believe one thing, but until employers believe it, it doesn’t matter much. I’m also not convinced that the reality of LLMs will never catch up with what employers think they can do. It may never happen, but it very well could. Either way, I feel the days of employers being ok with handwriting code are limited.
- SoftTalker 1mo ago
  The complexity of code is combinatorial. Code is harder to understand than it is to create. The LLMs will be creating code that they will not (nor will any human) be able to understand later, or it won't be the same understanding that was used to create it. We'll find that systems that were created with LLM code will be impossible for LLMs to contextualize, or will be cost-prohibitive to do so.
iwontberude 1mo ago
It's a moot point. In Mag7 companies internal code gets thrown away within 2 years on average because of contract negotiations, new opensource projects that obviate need for the internal codebase, license changes, layoffs, etc, etc. We don't need long term maintainability because its already on its way to the garbage bin. Frankly it's always been this way.
- chubot 1mo ago
  Its' the exact opposite ... Google, nVidia, Amazon, Apple, etc. all have a deep bench of code that is a moat. They have lots of throwaway code too, but those companies are precisely the ones that pay attention to code quality, and comprehensibility by experts A rough proxy for this would be if they say contribute to the C++ standards process, which Google Microsoft nVidia do, and basically zero "normal" companies (say SaaS) do. It means they are investing in multi-decade maintenance of their codebase
  - SoftTalker 1mo ago
    "Contribute" is an interesting word, since those companies are so big they can just dictate the standards they prefer. Also they are so big and have so much cash pouring in that they can afford to pay employees with nothing better to do than spend time drafting C++ standards. The average normal SaaS company is hoping they can make payroll next month.
  - tumdum_ 1mo ago
    Is google still a big contributor to the c++ standard? I thought they are far less involved than a decade ago.
  - iwontberude 1mo ago
    I’m talking back of house, you’ve sort of mistakenly missed my point and made a rebuttal to some completely different question and I agree with you.
swat535 1mo ago
> long term maintainability Did business ever cared about this? Like ever?.. I've seen too many code bases with awful code by humans, now AI simply exacerbated the issue but this is nothing new in our field. I just don't see how this is going to be a battle engineering departments can win. As long as AI is brining money to the business, they are not going to listen.
- mythrwy 1mo ago
  LLMs aren't perfect and they have problems, but often they write better code than humans. At least this human. It seems they run into problems with larger concepts and general organization and need guidance at the moment but for a single code file they often do better then I would have.
  - bigstrat2003 1mo ago
    If an LLM writes better code than you, that says more about you than anything else (either your ability to write code or your ability to evaluate LLMs). The code they write is garbage.
    - bonzini 1mo ago
      They write garbage if you turn off your brain. They write code as good as yours if you work together.
      - noisy_boy 1mo ago
        I broadly agree with you about improving quality through working together. But even with tons of discussion, they still write code that seems fine from a distance but is weird when you actually get down to actually reading it and thinking about it, line by line. It is not junk but sure as hell I can write tighter and cleaner code and I am a very mediocre programmer. The abundance and frequency of "what, why?!" and "why not just..." reactions I have at LLM code throughout the day has made that much clear to me.
    - 1mo ago
axus 1mo ago
The wages for the LLM will increase as the codebase gets larger, the only inhibitor is how fast customers are willing to pay for new features. I don't think that cost ever goes higher than human wages for maintainable software. In theory (never happens) the software will become feature complete and you only pay the LLM for security updates.

jader201 1mo ago

I agree with everything you said. I’m simply playing devil’s advocate, because engineers can believe one thing, but until employers believe it, it doesn’t matter much. I’m also not convinced that the reality of LLMs will never catch up with what employers think they can do. It may never happen, but it very well could. Either way, I feel the days of employers being ok with handwriting code are limited.

SoftTalker 1mo ago
The complexity of code is combinatorial. Code is harder to understand than it is to create. The LLMs will be creating code that they will not (nor will any human) be able to understand later, or it won't be the same understanding that was used to create it. We'll find that systems that were created with LLM code will be impossible for LLMs to contextualize, or will be cost-prohibitive to do so.

iwontberude 1mo ago

It's a moot point. In Mag7 companies internal code gets thrown away within 2 years on average because of contract negotiations, new opensource projects that obviate need for the internal codebase, license changes, layoffs, etc, etc. We don't need long term maintainability because its already on its way to the garbage bin. Frankly it's always been this way.

chubot 1mo ago
Its' the exact opposite ... Google, nVidia, Amazon, Apple, etc. all have a deep bench of code that is a moat. They have lots of throwaway code too, but those companies are precisely the ones that pay attention to code quality, and comprehensibility by experts A rough proxy for this would be if they say contribute to the C++ standards process, which Google Microsoft nVidia do, and basically zero "normal" companies (say SaaS) do. It means they are investing in multi-decade maintenance of their codebase
- SoftTalker 1mo ago
  "Contribute" is an interesting word, since those companies are so big they can just dictate the standards they prefer. Also they are so big and have so much cash pouring in that they can afford to pay employees with nothing better to do than spend time drafting C++ standards. The average normal SaaS company is hoping they can make payroll next month.
- tumdum_ 1mo ago
  Is google still a big contributor to the c++ standard? I thought they are far less involved than a decade ago.
- iwontberude 1mo ago
  I’m talking back of house, you’ve sort of mistakenly missed my point and made a rebuttal to some completely different question and I agree with you.

swat535 1mo ago

> long term maintainability Did business ever cared about this? Like ever?.. I've seen too many code bases with awful code by humans, now AI simply exacerbated the issue but this is nothing new in our field. I just don't see how this is going to be a battle engineering departments can win. As long as AI is brining money to the business, they are not going to listen.

mythrwy 1mo ago
LLMs aren't perfect and they have problems, but often they write better code than humans. At least this human. It seems they run into problems with larger concepts and general organization and need guidance at the moment but for a single code file they often do better then I would have.
- bigstrat2003 1mo ago
  If an LLM writes better code than you, that says more about you than anything else (either your ability to write code or your ability to evaluate LLMs). The code they write is garbage.
  - bonzini 1mo ago
    They write garbage if you turn off your brain. They write code as good as yours if you work together.
    - noisy_boy 1mo ago
      I broadly agree with you about improving quality through working together. But even with tons of discussion, they still write code that seems fine from a distance but is weird when you actually get down to actually reading it and thinking about it, line by line. It is not junk but sure as hell I can write tighter and cleaner code and I am a very mediocre programmer. The abundance and frequency of "what, why?!" and "why not just..." reactions I have at LLM code throughout the day has made that much clear to me.
  - 1mo ago

axus 1mo ago

The wages for the LLM will increase as the codebase gets larger, the only inhibitor is how fast customers are willing to pay for new features. I don't think that cost ever goes higher than human wages for maintainable software. In theory (never happens) the software will become feature complete and you only pay the LLM for security updates.