5 ms·

What's the difference between using llms now vs the first half of 2025 among the best users?

simonw 9mo ago
Coding agents and much better models. Claude Code or Codex CLI plus Claude Opus 4.5 or GPT 5.2 Codex. The latest models and harnesses can crunch on difficult problems for hours at a time and get to working solutions. Nothing could do that back in ~March. I shared some examples in this comment: https://news.ycombinator.com/item?id=46436885 https://news.ycombinator.com/item?id=46436885
- drbojingle 9mo ago
  Are there techniques though? Tech pairing? Something we know now that we didn't then? Or just better models?
  - simonw 9mo ago
    Lots of technique stuff. A common observation among LLM nerds is that if the models stopped being improved and froze in time for a year we could still spend all twelve months discovering new capabilities and use-cases for the models we already have.
    - drbojingle 9mo ago
      Any specifics you'd recommend?
- William_BB 9mo ago
  Ok I will bite. Every single example you gave is in a hobby project territory. Relatively self-contained, maintainable by 3-4 devs max, within 1k-10k lines of code. I've been successfully using coding agents to create such projects for the past year and it's great, I love it. However, lots of us here work on codebases that are 100x, 1000x the size of these projects you and Karpathy are talking about. Years of domain specific code. From personal experience, coding agents simply don't work at that scale the same way they do for hobby projects. Over the past year or two, I did not see any significant improvement from any of the newest models. Building a slightly bigger hobby project is not even close to making these agents work at industrial scale.
  - deleted 9mo ago
    [deleted]
  - simonw 9mo ago
    Most of the stuff I'm talking about here came out in November. There hasn't been much time for professional teams to build new things with it yet, especially given the holidays!
    - qweiopqweiop 9mo ago
      For what it's worth, I'm working with it on a huge professional monorepo, and the difference was also stark.
  - baq 9mo ago
    That’s right, but it also hints at a solution: split big code bases into parts that are roughly the size of a big hobby project. You’ll need to write some docs to be effective at it, which also helps agents. CICD means continuous integration continuous documentation now.
    - majormajor 9mo ago
      I wonder if anyone has tried this thing before, like... micro-projects or such... ;)
    - bccdee 9mo ago
      Splitting one big codebase into 100 microservices always seems tempting, except that big codebases already exist in modules and that doesn't stop one module's concerns from polluting the other modules' code. What you've got now is 100 different repositories that all depend on each other, get deployed separately, and can only be tested with some awful docker-compose setup. Frankly, given the impedance of hopping back and forth between repos separated by APIs, I'd expect an LLM to do far worse in a microservice ecosystem than in an equivalent monolith.
    - rjzzleep 9mo ago
      It's not the size that's the issue, it's the domain that is. It's tempting to say that adding drivers to Linux is hard because Linux is big, but that's not the issue.
- mkozlows 9mo ago
  I was going back and looking at timelines, and was shocked to realize that Claude Code and Cursor's default-to-agentic-mode changes both came out in late February. Essentially the entire history of "mainstream" agentic coding is ten months old. (This helps me understand better the people who are confused/annoyed/dismissive about it, because I remember how dismissive people were about Node, about Docker, about Postgres, about Linux when those things were new too. So many arguments where people would passionately talk about all those things were irredeemably stupid and only suitable for toy/hobby projects.)
  - HarHarVeryFunny 9mo ago
    The entire history of RL-trained "reasoning models" from o1 to DeepSeek_R1 is basically just a year old!
- epolanski 9mo ago
  Cool, but most developers do mundane stuff like glueing APIs and implementing business logic, which require oversight and review. Those crunching hard problems will still review what's produced in search of issues.
  - generic92034 9mo ago
    What is (in general) mundane about business logic? This can be highly complex, with deep process integration all over your modules.
    - epolanski 9mo ago
      Which is why it requires detailed oversight.
deleted 9mo ago
[deleted]

Coding agents and much better models. Claude Code or Codex CLI plus Claude Opus 4.5 or GPT 5.2 Codex. The latest models and harnesses can crunch on difficult problems for hours at a time and get to working solutions. Nothing could do that back in ~March. I shared some examples in this comment: https://news.ycombinator.com/item?id=46436885 https://news.ycombinator.com/item?id=46436885

drbojingle 9mo ago
Are there techniques though? Tech pairing? Something we know now that we didn't then? Or just better models?
- simonw 9mo ago
  Lots of technique stuff. A common observation among LLM nerds is that if the models stopped being improved and froze in time for a year we could still spend all twelve months discovering new capabilities and use-cases for the models we already have.
  - drbojingle 9mo ago
    Any specifics you'd recommend?
William_BB 9mo ago
Ok I will bite. Every single example you gave is in a hobby project territory. Relatively self-contained, maintainable by 3-4 devs max, within 1k-10k lines of code. I've been successfully using coding agents to create such projects for the past year and it's great, I love it. However, lots of us here work on codebases that are 100x, 1000x the size of these projects you and Karpathy are talking about. Years of domain specific code. From personal experience, coding agents simply don't work at that scale the same way they do for hobby projects. Over the past year or two, I did not see any significant improvement from any of the newest models. Building a slightly bigger hobby project is not even close to making these agents work at industrial scale.
- deleted 9mo ago
  [deleted]
- simonw 9mo ago
  Most of the stuff I'm talking about here came out in November. There hasn't been much time for professional teams to build new things with it yet, especially given the holidays!
  - qweiopqweiop 9mo ago
    For what it's worth, I'm working with it on a huge professional monorepo, and the difference was also stark.
- baq 9mo ago
  That’s right, but it also hints at a solution: split big code bases into parts that are roughly the size of a big hobby project. You’ll need to write some docs to be effective at it, which also helps agents. CICD means continuous integration continuous documentation now.
  - majormajor 9mo ago
    I wonder if anyone has tried this thing before, like... micro-projects or such... ;)
  - bccdee 9mo ago
    Splitting one big codebase into 100 microservices always seems tempting, except that big codebases already exist in modules and that doesn't stop one module's concerns from polluting the other modules' code. What you've got now is 100 different repositories that all depend on each other, get deployed separately, and can only be tested with some awful docker-compose setup. Frankly, given the impedance of hopping back and forth between repos separated by APIs, I'd expect an LLM to do far worse in a microservice ecosystem than in an equivalent monolith.
  - rjzzleep 9mo ago
    It's not the size that's the issue, it's the domain that is. It's tempting to say that adding drivers to Linux is hard because Linux is big, but that's not the issue.
mkozlows 9mo ago
I was going back and looking at timelines, and was shocked to realize that Claude Code and Cursor's default-to-agentic-mode changes both came out in late February. Essentially the entire history of "mainstream" agentic coding is ten months old. (This helps me understand better the people who are confused/annoyed/dismissive about it, because I remember how dismissive people were about Node, about Docker, about Postgres, about Linux when those things were new too. So many arguments where people would passionately talk about all those things were irredeemably stupid and only suitable for toy/hobby projects.)
- HarHarVeryFunny 9mo ago
  The entire history of RL-trained "reasoning models" from o1 to DeepSeek_R1 is basically just a year old!
epolanski 9mo ago
Cool, but most developers do mundane stuff like glueing APIs and implementing business logic, which require oversight and review. Those crunching hard problems will still review what's produced in search of issues.
- generic92034 9mo ago
  What is (in general) mundane about business logic? This can be highly complex, with deep process integration all over your modules.
  - epolanski 9mo ago
    Which is why it requires detailed oversight.

deleted 9mo ago

[deleted]