6 ms·

Claude Code fan here... Codex is very good. Sometimes better. The killer feature is price. After 6+ months of exclusive Claude Code usage, I was begrudgingly f

by aroman 2mo ago

Claude Code fan here... Codex is very good. Sometimes better. The killer feature is price. After 6+ months of exclusive Claude Code usage, I was begrudgingly forced to try Codex once Anthropic rejiggered their limits such that I kept maxing out my $200/mo plan in just a few days. These days I pay both $200/mo plans, and it's just about enough to get me through a week's work (small game studio - infinite code to write!)

ValentineC 2mo ago
> (small game studio - infinite code to write!) Curious: what multiplier do you think your productivity has increased by, from before AI?
- aroman 2mo ago
  In terms of ability to ship? Easily tenfold. We literally ship 10 times more than before AI. This does not, however, translate into a tenfold increase in actual business success, of course :)
  - winrid 2mo ago
    Yes, because now your competitors do the same. The winners: inference providers
    - shwaj 2mo ago
      Inference providers, sure, but wouldn't we expect customers to win also?
      - winrid 2mo ago
        oh for sure! I think in some ways LLMs have raised the bar in terms of what you an expect from software (once we get past the hurdle of increased bugs, but that seems to be getting better)
    - ValentineC 2mo ago
      I think at some point, AI development becomes a competition for who has the best taste [1], because with seemingly unlimited/cheap tokens, AI ends up being more about throwing shit at a wall and hoping something sticks. [1] https://www.reddit.com/r/ProgrammerHumor/comments/10ek380/conversation_between_a_junior_engineer_and_the/ https://www.reddit.com/r/ProgrammerHumor/comments/10ek380/co...
ryan_n 2mo ago
Genuine question/not a critique-are you actually reviewing all that code or just sending it and hoping for the best? I just can't imagine someone is reading/reviewing that much code every day, but maybe I'm wrong?
- aroman 2mo ago
  Like before AI, the scrutiny varies with the sensitivity of the area being edited. Simple UI change? I do an AI review, but otherwise neither read nor write the code. The models are good enough they write better UI code than me, 9 out of 10 times. Not always the more idiomatic, but usually safer and more correct. Change to our core data plane? I might spend 2-3 times more effort reviewing it than before AI. Yes, I go more slowly than pre-AI. Many more reviews, many more angles considered, including both human and (lots of) AI review cycles. Most code is not that critical, and AI is also scarily good at writing tests. We also spend considerably more time paying down tech debt and testing thanks to AI, now that the cost is near-zero. Net: I spend 10-25X less time on low-risk changes. I often direct (or at least approve) the implementation approach, but I rarely read this code. I spend 2-3X more time on high-risk changes. In both cases, I never write code "by hand". Since about November, I've had no reason to actually edit code in a code editor (perhaps maybe except .env files, which we don't allow agents to edit for obvious reasons). AI is a tool. You can use it to go fast recklessly, or you can use it to go slow with confidence. Just like before AI... the skill and art of engineering is knowing when to do which.
  - William_BB 2mo ago
    > AI is also scarily good at writing tests :-) I hope you read those tests before claiming it's "scary good"
    - aroman 2mo ago
      Indeed, much of the scariness is how fearlessly and confidently it writes them with little regard to their actual usefulness or value. When I find it adding a lot of tests, I often say something like: "audit each test carefully, and consider whether the test is testing a meaningful boundary or is more ceremonial. delete low-value tests and add new tests to cover meaningful boundaries not exercised by the gaps you identify". Without fail, this always produces some decent results. Having said that, in truth, I almost never read the unit tests. Before AI, we had almost none (see: several person game studio) so the tradeoff is not "AI-generated tests" vs "human written ones", it's whether we have tests at all. So, I take them for what they're worth - not much - but if it catches an extra regression before it ships every now and then, it was worth it for the price (~free).

by aroman 2mo ago

ValentineC 2mo ago

> (small game studio - infinite code to write!) Curious: what multiplier do you think your productivity has increased by, from before AI?

aroman 2mo ago
In terms of ability to ship? Easily tenfold. We literally ship 10 times more than before AI. This does not, however, translate into a tenfold increase in actual business success, of course :)
- winrid 2mo ago
  Yes, because now your competitors do the same. The winners: inference providers
  - shwaj 2mo ago
    Inference providers, sure, but wouldn't we expect customers to win also?
    - winrid 2mo ago
      oh for sure! I think in some ways LLMs have raised the bar in terms of what you an expect from software (once we get past the hurdle of increased bugs, but that seems to be getting better)
  - ValentineC 2mo ago
    I think at some point, AI development becomes a competition for who has the best taste [1], because with seemingly unlimited/cheap tokens, AI ends up being more about throwing shit at a wall and hoping something sticks. [1] https://www.reddit.com/r/ProgrammerHumor/comments/10ek380/conversation_between_a_junior_engineer_and_the/ https://www.reddit.com/r/ProgrammerHumor/comments/10ek380/co...

ryan_n 2mo ago

Genuine question/not a critique-are you actually reviewing all that code or just sending it and hoping for the best? I just can't imagine someone is reading/reviewing that much code every day, but maybe I'm wrong?

aroman 2mo ago
Like before AI, the scrutiny varies with the sensitivity of the area being edited. Simple UI change? I do an AI review, but otherwise neither read nor write the code. The models are good enough they write better UI code than me, 9 out of 10 times. Not always the more idiomatic, but usually safer and more correct. Change to our core data plane? I might spend 2-3 times more effort reviewing it than before AI. Yes, I go more slowly than pre-AI. Many more reviews, many more angles considered, including both human and (lots of) AI review cycles. Most code is not that critical, and AI is also scarily good at writing tests. We also spend considerably more time paying down tech debt and testing thanks to AI, now that the cost is near-zero. Net: I spend 10-25X less time on low-risk changes. I often direct (or at least approve) the implementation approach, but I rarely read this code. I spend 2-3X more time on high-risk changes. In both cases, I never write code "by hand". Since about November, I've had no reason to actually edit code in a code editor (perhaps maybe except .env files, which we don't allow agents to edit for obvious reasons). AI is a tool. You can use it to go fast recklessly, or you can use it to go slow with confidence. Just like before AI... the skill and art of engineering is knowing when to do which.
- William_BB 2mo ago
  > AI is also scarily good at writing tests :-) I hope you read those tests before claiming it's "scary good"
  - aroman 2mo ago
    Indeed, much of the scariness is how fearlessly and confidently it writes them with little regard to their actual usefulness or value. When I find it adding a lot of tests, I often say something like: "audit each test carefully, and consider whether the test is testing a meaningful boundary or is more ceremonial. delete low-value tests and add new tests to cover meaningful boundaries not exercised by the gaps you identify". Without fail, this always produces some decent results. Having said that, in truth, I almost never read the unit tests. Before AI, we had almost none (see: several person game studio) so the tradeoff is not "AI-generated tests" vs "human written ones", it's whether we have tests at all. So, I take them for what they're worth - not much - but if it catches an extra regression before it ships every now and then, it was worth it for the price (~free).