10 ms·

Interview with Boris Cherny [video]

74 points by knighthacker 2mo ago

lostmsu 2mo ago
[flagged]
ventana 2mo ago
The TL;DR of the fragment at around 6:57 in the video, for those who don't want to watch, is that Boris recommends to delete all the customizations, CLAUDE.md, skills, and other stuff you had for older models, and try using the new models without all that, because the new models can surprise you.
- ares623 2mo ago
  CHANGELOG: surprise me
- georgemcbay 2mo ago
  Even for older models I was (and am) of the opinion that a lot of the context-overstuffing cruft people have been wrapping around their LLM usage is more of an RNG-impacting (but not always for the better) lucky charm rather than it being universally helpful. And a lot of people latched on to it as a form of self-soothing. "I may not write much code anymore, but I can still be an expert prompt engineer!"
  - troupo 2mo ago
    > is more of an RNG-impacting (but not always for the better) lucky charm rather than it being universally helpful. I wrote this last year, it's still true: --- start quote --- https://dmitriid.com/prompting-llms-is-not-engineering https://dmitriid.com/prompting-llms-is-not-engineering In reality these are just shamanic rituals with outcomes based on faith, fear, or excitement. Engineering it is not. --- end quote --- There are some prompts useful for the user like brainstorming [1] but on the whole it's nothing but lucky charms [1] brainstorming from superpowers: https://github.com/obra/superpowers https://github.com/obra/superpowers
  - CuriouslyC 2mo ago
    Not really. The big change in newer models is the amount of RL relative to pretraining. RL makes the models stable by collapsing distributions around desired outcomes. The old tweaks like giving the agent a role ("You are an expert XYZ engineer") or stuffing the context with engineering idioms have effectively been RL'd into negligence, but they still waste tokens. You can see this visually in older image generation models. Stable Diffusion 1.5 would produce wildly different images based on slight variations in prompt and seed, but the latest image gen models are nearly seed indifferent and can tolerate a decent amount of prompt tweaking while staying "consistent"
- m_ke 2mo ago
  All of these harnesses should support pinning config files and tools to specific models or families of models. It's really tiring to have to tweak everything with each model release and then watch those changes mess up cheaper models in the process.
  - swader999 2mo ago
    Yeah or config older than x days...
- rvz 2mo ago
  Salesman "recommends" you to continue wasting more tokens and on top of that, he "recommends" you to use only recursive loops with the latest and greatest models. Finally he also "recommends" that you do not look at the code, or even understand it. His "recommendations" are designed to get you to spend even more tokens and get you hooked on the Opus / Fable slot machine in order to extract as much money as possible from your wallets. News at 10.
  - hvb2 2mo ago
    I'm waiting for the next headline that's about a codebase that existed for 10+ years and then the agents came. Now, the codebase is managed by agents and, while the company has the source, it's as if they bought it from a vendor and pay the vendor for changes. The vendor of their own system is the AI company selling them tokens.
  - Diogenesian 2mo ago
    To be fair I think Cherny is high on his own supply, not cynically trying to sell us something. He is absolutely stoked that Claude Code is an overengineered piece of junk he doesn't understand.
    - KludgeShySir 2mo ago
      Boris Cherny is most definitely the kind of guy who sniffs his own farts. He can be informative to listen to, as long as you keep that in mind.
- troupo 2mo ago
  > because the new models can surprise you. Yes, they keep surprising me that they still keep doing all of this: https://news.ycombinator.com/item?id=48962703 https://news.ycombinator.com/item?id=48962703 with no improvement despite all the marketing assurances that "hey you don't need to read code anymore"
  - slopinthebag 2mo ago
    Yeah, I'm going over a bunch of components a frontier model generated and while they "work" the code is quite shocking and will create significant maintenance burden. Not to mention they don't use any of our shared utilities and duplicate so much code.
  - darepublic 2mo ago
    I do have to still remind codex to fetch large amounts of db records in batches otherwise it will always make a first false step of a naive select all style approach. Context wise naive can be ok for small datasets though so codex didn't know the naive approach was not viable. Context management the new code plumbing these days
    - troupo 2mo ago
      Context management doesn't help. I had this requirement explicitly in my AGENTS.md file. Just a minute or so into a coding session it (and Claude) would ignore/forget half of AGENTS.md (because the advertised 1M token context is nothing but marketing bullshit)
- throwaway219450 2mo ago
  Steering/corrective instructions are now supposed to go into memories - which are non-portable? How do we manage these documents for team members without everyone repeating themselves? And if you remove all these documents, how do you work with other models which might use them (nevermind that claude ignored AGENTS.md for the longest time). I am on board with not putting stuff like "write clean code" into an agent file, or using plugins for tools that are now built into the harness. I don't see enough evidence to support models being significantly better at figuring out intent, or getting the assumption correct. I've always gotten better results (as ever) with very constrained instructions, vs "fix the install".
  - anon373839 2mo ago
    > Steering/corrective instructions are now supposed to go into memories - which are non-portable? I’m sure this just intended to steer you to vendor lock-in. Remember, these are the same people who are so insecure/petty about their product that they won’t make it recognize the .agents/AGENTS.md standard.
- consumer451 2mo ago
  That is the interesting part to me. Also, was anyone else familiar with the LLM-app terms "ablation" and "product overhang," prior to this? I was not. If you were, please share the knowledge with us. Where can I learn more?
  - iambenm 2mo ago
    Ablation is a medical term, he's just using it generically to describe removing existing config.
    - consumer451 2mo ago
      I only knew ablation from spaceflight re-entry. I should have been more clear. I had never heard those words in terms of software products.
- mupuff1234 2mo ago
  I was using Opus 4.6 until 2 days ago with no CLAUDE.md or anything and it was great. Tried out Opus 5 and it's been a super annoying experience out of the box.
- knighthacker 2mo ago
  Yup. That's pretty much the most interesting part.
- sivanmz 2mo ago
  There’s really not much to specialize in and differentiate yourself. All this harness customization is fleeting. Remember prompt engineering?
  - aiisjustanif 2mo ago
    To be fair prompts is just English language intent abstracted for code. It is still am very much a thing. Why do you feel that harness customization is fleeting, enterprises care about them a lot for many reasons from compliance to security.
    - sivanmz 2mo ago
      From the point of view of a developer looking to differentiate themselves. The instinct is to look for generalizations, abstractions, composability, but as Boris says, these just get subsumed into the latest model RL. So what expertise is left for the developer to invest in?
- joebates 2mo ago
  The ability to start a Claude Code session in a temporary "config free" mode, similar to how incognito works for chrome, would be a nice feature to make this easy to try out, especially since model releases are so frequent.
  - matltc 2mo ago
    You can do this but you have to pass a million long options like --disable-system-prompt --allowed-tools "" etc I have some wrappers but the only one I really use is a variant of that pinned to haiku for quick questions.
    - kxxx 2mo ago
      `claude --bare --system-prompt ""` is a good starting point!
      - matltc 2mo ago
        Yep! Only (potential) downside to --bare is it runs in print mode; can't kick off an interactive session with it. The flags I listed above are a close approximation. Anthropic had something about API-only in the description a couple months back, but they've been waking all those back for months since the fable access rollback fallout
- ahmetson 2mo ago
  What if I don't use customizations at all, bare prompt per feature or function only? Do those customizations help?
iooi 2mo ago
I don't get why Anthropic keeps saying how cool it is that they were able to rewrite bun in rust, I feel like if the model is really that good you wouldn't need a rewrite, you'd be able to create the runtime from scratch.
- jdross 2mo ago
  I like this, it will be a really exciting benchmark to refer back to in a few years
- __alexs 2mo ago
  Productivity theatre is when activity becomes celebrated over outcomes. It's absolutely everywhere in US tech culture.
  - deleted 2mo ago
    [deleted]
- chromakode 2mo ago
  Code is a fantastic way to capture requirements, particularly iterative product discoveries over time. It's much easier to start with a manifestation of the requirements (and tests of invariants) than starting from base principles.
- schainks 2mo ago
  Rust is a great tool for what they did with Bun. Why make a whole new tool when existing off the shelf tools can do the job?
- sushid 2mo ago
  I mean if the model were really that good they'd be able to fix various TUI issues that have plagued claude code since last year.
  - nnx 2mo ago
    is flickering fully fixed yet?
firasd 2mo ago
I feel like the Big Insight with Claude Code was 'give the AI access to tools, in fact to your actual laptop'. And of course 1.5 years later 'a harness to enable an agent to use my computer' has turned into the flagship product we expect from frontier labs However. I don't think all the other 'Ideas' these harness-building guys have are all that universally applicable. Heck they are naysaying their own gearhead stack of skills.md and claude.md lately including in this video. But also I'm suspicious of this whole thing of spawning agents. For example "find every function in this codebase" is probably better done deterministically using a script that extracts function names rather than by spawning 20 agents to 'read' chunks of the code in token space. But it certainly racks up the token usage which is good for the person selling you inference... And there is a lot missing in the actual harnesses frankly that isn't about more parallel agent ninjutsu. Like why isn't there a 'move this function from this file to this file' tool (copy/paste char range) and we have to see Claude/Codex/etc flail around rewriting huge chunks of code in token space
- rescripting 2mo ago
  > Like why isn't there a 'move this function from this file to this file' tool (copy/paste char range) There is, you can install LSPs for a given language which act as just another tool the model can use to more efficiently manipulate code.
  - firasd 2mo ago
    LSPs don't really do this. They're like a better way to grep for something (so they do address the previous thing I said about listing all function names etc.) But they often don't say (depending on the particular LSP setup etc) this function ends at this char and they definitely don't then provide the text editing tool
    - throw478239 2mo ago
      This is not accurate. Most LSPs do provide text editing tools, which the Language Server Protocol calls "code actions". These actions do things like: reorganizing imports, expanding macros, extracting a selection into a function, or converting one type of control flow into another (e.g. if to match). Common LSPs, like rust-analyzer and Ruff (Python) support code actions. Editors also expose code actions: Zed, for instance, exposes actions under the default keymap with the binding Command + . source: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_codeAction https://microsoft.github.io/language-server-protocol/specifi... and https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_executeCommand https://microsoft.github.io/language-server-protocol/specifi...
mfallon 2mo ago
I've spent the last year building an app with Claude Code without knowing how to code. I'm still surprised at how much it gets wrong, even on the newest models. What makes it work for me is describing exactly what I want and then actually checking everything it gives back.
- EagleEdge 2mo ago
  For me, I just don't trust Claude Code to write first-class professional code. I have used all frontier claude models and it always does sloppy job that needs to be fixed by codex. I have claude/codex setup and any execution must go to codex and all reviews as well.
  - wilj 2mo ago
    Similar here. I use Claude Code and Codex for developing specs, but then have Deepseek v4 Pro implement because it just seems to actually do what it's told, given enough tries. And Reasonix + Deepseek is mostly cache hits when I let it grind on it for a long time, and I can have a huge fleet of agents with very specific focus for code review and rule enforcement mostly sharing cache.
jke_kang 2mo ago
2 weeks rebuilding in Swift is absolutely crazy. Wonder if they're using production Opus 5 for this or their internal Mythos (if they even use it internally)
- kev009 2mo ago
  If it's on the house (and they can probably turn up the token speed) why use anything other than the most powerful models?
  - blovescoffee 2mo ago
    it's still more expensive and slower to run bigger models though. resource usage is not free
jake_and_fatman 2mo ago
30:08 "This means engineers can talk to users... stuff that's actually fun". Wow, he totally gets us. /s 31:09 He flubs the questioning. His first says "Raise your hand if 100% of your code uses agents." Then he says, "What about more than 50%". You can see the same audience members who raised the first time shrug and also raise the second time. He then says "Slightly less hands," which just means he sees what he wants to see. Don't believe this man.
albert_e 2mo ago
So maybe SKILL.md and CLAUDE.md should have model version pinned? Somewhat like AWS CloudFormation templates start with something like TemplateVersion: 2017-10-01 But maybe that's old school thinking still.
- knighthacker 2mo ago
  I'm thinking a lot about this recently as I'm building AQ.dev (Multiplayer coding harness). My instinct is to keep it simple and not trying to do too many shenanigans with these files and let engineers maintain control.
- verdverm 2mo ago
  this assumes everyone uses the same model, but where I'm at, we use a wide variety of models we keep the AGENTS.md simple and monorepo quirk focussed, no instructions for specific agents or models
- crazylogger 2mo ago
  They should just be symlinks to CONTRIBUTING.md and other docs humans are meant to read. The whole idea behind LLM is they use the same language and tools as us, so we literally don't design anything specifically for them.
knighthacker 2mo ago
The most interesting part is Boris' recommendation to delete your Claude.md every 6 months to see what the model is capable of. The link in the post sends you directly to that moment.
- discordance 2mo ago
  It would have been nice to hear about how to be more efficient, task/token wise, especially with Opus 5. Most users are on a subscription, and aren’t able to run dynamic workflows, spinning up 1000 agents or leaving a bun rewrite going for weeks. Don’t get me wrong… I like cake.
  - deleted 2mo ago
    [deleted]
- bmitc 2mo ago
  Seems like something they should work on, not customers. Reinventing my workflow every six months because a company/product is lazy is not high on my list.
  - lorey 2mo ago
    What I read from that is that there's a chance claude gets worse using a claude.md and that's a usability issue on their side.
    - bmitc 2mo ago
      I've seen this particular in the post and others from Anthropic make similar claims of "oh, just do this or that", and I've always been of the opinion: "isn't this what everyone is paying Anthropic to do?".
Taka108 2mo ago
This gives you more raw behind-the-scenes. I think the video from Startup School is too polished. https://www.youtube.com/watch?v=PQU9o_5rHC4 https://www.youtube.com/watch?v=PQU9o_5rHC4
nojs 2mo ago
> From what I'm hearing, you for every model release, you basically delete all of the codebase, delete all of the prompt and start from scratch every time. That in the old world would have been not something Startup would have done for the product. It's like press delete every six months for everything. > That's right. So to be fair, we don't delete the entire codebase.
linxy97 2mo ago
still a long way to go!
DennisL123 2mo ago
There’s something missing in the discussion. Building software is about achieving outcomes, not about generating output, ie tokens. Yet, the ai business case is selling output, obviously. Not sure how to reconcile the two. Ideas?
- zazuke 2mo ago
  Yes this 100%. I'd say: don't overrate how you do it; it matters more what you do, especially with LLM models. It's just a tool.
- yetihehe 2mo ago
  Building bridges is about achieving outcomes and utility, but steel and cement suppliers are selling by ton.
  - miyoji 2mo ago
    To extend this analogy, vibe coding can be defined as dumping steel and cement out of a helicopter into the water and calling the result a bridge because you can technically walk on it.

74 points by knighthacker 2mo ago

[flagged]

The TL;DR of the fragment at around 6:57 in the video, for those who don't want to watch, is that Boris recommends to delete all the customizations, CLAUDE.md, skills, and other stuff you had for older models, and try using the new models without all that, because the new models can surprise you.

ares623 2mo ago
CHANGELOG: surprise me
georgemcbay 2mo ago
Even for older models I was (and am) of the opinion that a lot of the context-overstuffing cruft people have been wrapping around their LLM usage is more of an RNG-impacting (but not always for the better) lucky charm rather than it being universally helpful. And a lot of people latched on to it as a form of self-soothing. "I may not write much code anymore, but I can still be an expert prompt engineer!"
- troupo 2mo ago
  > is more of an RNG-impacting (but not always for the better) lucky charm rather than it being universally helpful. I wrote this last year, it's still true: --- start quote --- https://dmitriid.com/prompting-llms-is-not-engineering https://dmitriid.com/prompting-llms-is-not-engineering In reality these are just shamanic rituals with outcomes based on faith, fear, or excitement. Engineering it is not. --- end quote --- There are some prompts useful for the user like brainstorming [1] but on the whole it's nothing but lucky charms [1] brainstorming from superpowers: https://github.com/obra/superpowers https://github.com/obra/superpowers
- CuriouslyC 2mo ago
  Not really. The big change in newer models is the amount of RL relative to pretraining. RL makes the models stable by collapsing distributions around desired outcomes. The old tweaks like giving the agent a role ("You are an expert XYZ engineer") or stuffing the context with engineering idioms have effectively been RL'd into negligence, but they still waste tokens. You can see this visually in older image generation models. Stable Diffusion 1.5 would produce wildly different images based on slight variations in prompt and seed, but the latest image gen models are nearly seed indifferent and can tolerate a decent amount of prompt tweaking while staying "consistent"
m_ke 2mo ago
All of these harnesses should support pinning config files and tools to specific models or families of models. It's really tiring to have to tweak everything with each model release and then watch those changes mess up cheaper models in the process.
- swader999 2mo ago
  Yeah or config older than x days...
rvz 2mo ago
Salesman "recommends" you to continue wasting more tokens and on top of that, he "recommends" you to use only recursive loops with the latest and greatest models. Finally he also "recommends" that you do not look at the code, or even understand it. His "recommendations" are designed to get you to spend even more tokens and get you hooked on the Opus / Fable slot machine in order to extract as much money as possible from your wallets. News at 10.
- hvb2 2mo ago
  I'm waiting for the next headline that's about a codebase that existed for 10+ years and then the agents came. Now, the codebase is managed by agents and, while the company has the source, it's as if they bought it from a vendor and pay the vendor for changes. The vendor of their own system is the AI company selling them tokens.
- Diogenesian 2mo ago
  To be fair I think Cherny is high on his own supply, not cynically trying to sell us something. He is absolutely stoked that Claude Code is an overengineered piece of junk he doesn't understand.
  - KludgeShySir 2mo ago
    Boris Cherny is most definitely the kind of guy who sniffs his own farts. He can be informative to listen to, as long as you keep that in mind.
troupo 2mo ago
> because the new models can surprise you. Yes, they keep surprising me that they still keep doing all of this: https://news.ycombinator.com/item?id=48962703 https://news.ycombinator.com/item?id=48962703 with no improvement despite all the marketing assurances that "hey you don't need to read code anymore"
- slopinthebag 2mo ago
  Yeah, I'm going over a bunch of components a frontier model generated and while they "work" the code is quite shocking and will create significant maintenance burden. Not to mention they don't use any of our shared utilities and duplicate so much code.
- darepublic 2mo ago
  I do have to still remind codex to fetch large amounts of db records in batches otherwise it will always make a first false step of a naive select all style approach. Context wise naive can be ok for small datasets though so codex didn't know the naive approach was not viable. Context management the new code plumbing these days
  - troupo 2mo ago
    Context management doesn't help. I had this requirement explicitly in my AGENTS.md file. Just a minute or so into a coding session it (and Claude) would ignore/forget half of AGENTS.md (because the advertised 1M token context is nothing but marketing bullshit)
throwaway219450 2mo ago
Steering/corrective instructions are now supposed to go into memories - which are non-portable? How do we manage these documents for team members without everyone repeating themselves? And if you remove all these documents, how do you work with other models which might use them (nevermind that claude ignored AGENTS.md for the longest time). I am on board with not putting stuff like "write clean code" into an agent file, or using plugins for tools that are now built into the harness. I don't see enough evidence to support models being significantly better at figuring out intent, or getting the assumption correct. I've always gotten better results (as ever) with very constrained instructions, vs "fix the install".
- anon373839 2mo ago
  > Steering/corrective instructions are now supposed to go into memories - which are non-portable? I’m sure this just intended to steer you to vendor lock-in. Remember, these are the same people who are so insecure/petty about their product that they won’t make it recognize the .agents/AGENTS.md standard.
consumer451 2mo ago
That is the interesting part to me. Also, was anyone else familiar with the LLM-app terms "ablation" and "product overhang," prior to this? I was not. If you were, please share the knowledge with us. Where can I learn more?
- iambenm 2mo ago
  Ablation is a medical term, he's just using it generically to describe removing existing config.
  - consumer451 2mo ago
    I only knew ablation from spaceflight re-entry. I should have been more clear. I had never heard those words in terms of software products.
mupuff1234 2mo ago
I was using Opus 4.6 until 2 days ago with no CLAUDE.md or anything and it was great. Tried out Opus 5 and it's been a super annoying experience out of the box.
knighthacker 2mo ago
Yup. That's pretty much the most interesting part.
sivanmz 2mo ago
There’s really not much to specialize in and differentiate yourself. All this harness customization is fleeting. Remember prompt engineering?
- aiisjustanif 2mo ago
  To be fair prompts is just English language intent abstracted for code. It is still am very much a thing. Why do you feel that harness customization is fleeting, enterprises care about them a lot for many reasons from compliance to security.
  - sivanmz 2mo ago
    From the point of view of a developer looking to differentiate themselves. The instinct is to look for generalizations, abstractions, composability, but as Boris says, these just get subsumed into the latest model RL. So what expertise is left for the developer to invest in?
joebates 2mo ago
The ability to start a Claude Code session in a temporary "config free" mode, similar to how incognito works for chrome, would be a nice feature to make this easy to try out, especially since model releases are so frequent.
- matltc 2mo ago
  You can do this but you have to pass a million long options like --disable-system-prompt --allowed-tools "" etc I have some wrappers but the only one I really use is a variant of that pinned to haiku for quick questions.
  - kxxx 2mo ago
    `claude --bare --system-prompt ""` is a good starting point!
    - matltc 2mo ago
      Yep! Only (potential) downside to --bare is it runs in print mode; can't kick off an interactive session with it. The flags I listed above are a close approximation. Anthropic had something about API-only in the description a couple months back, but they've been waking all those back for months since the fable access rollback fallout
ahmetson 2mo ago
What if I don't use customizations at all, bare prompt per feature or function only? Do those customizations help?

iooi 2mo ago

I don't get why Anthropic keeps saying how cool it is that they were able to rewrite bun in rust, I feel like if the model is really that good you wouldn't need a rewrite, you'd be able to create the runtime from scratch.

jdross 2mo ago
I like this, it will be a really exciting benchmark to refer back to in a few years
__alexs 2mo ago
Productivity theatre is when activity becomes celebrated over outcomes. It's absolutely everywhere in US tech culture.
- deleted 2mo ago
  [deleted]
chromakode 2mo ago
Code is a fantastic way to capture requirements, particularly iterative product discoveries over time. It's much easier to start with a manifestation of the requirements (and tests of invariants) than starting from base principles.
schainks 2mo ago
Rust is a great tool for what they did with Bun. Why make a whole new tool when existing off the shelf tools can do the job?
sushid 2mo ago
I mean if the model were really that good they'd be able to fix various TUI issues that have plagued claude code since last year.
- nnx 2mo ago
  is flickering fully fixed yet?

firasd 2mo ago

I feel like the Big Insight with Claude Code was 'give the AI access to tools, in fact to your actual laptop'. And of course 1.5 years later 'a harness to enable an agent to use my computer' has turned into the flagship product we expect from frontier labs However. I don't think all the other 'Ideas' these harness-building guys have are all that universally applicable. Heck they are naysaying their own gearhead stack of skills.md and claude.md lately including in this video. But also I'm suspicious of this whole thing of spawning agents. For example "find every function in this codebase" is probably better done deterministically using a script that extracts function names rather than by spawning 20 agents to 'read' chunks of the code in token space. But it certainly racks up the token usage which is good for the person selling you inference... And there is a lot missing in the actual harnesses frankly that isn't about more parallel agent ninjutsu. Like why isn't there a 'move this function from this file to this file' tool (copy/paste char range) and we have to see Claude/Codex/etc flail around rewriting huge chunks of code in token space

rescripting 2mo ago
> Like why isn't there a 'move this function from this file to this file' tool (copy/paste char range) There is, you can install LSPs for a given language which act as just another tool the model can use to more efficiently manipulate code.
- firasd 2mo ago
  LSPs don't really do this. They're like a better way to grep for something (so they do address the previous thing I said about listing all function names etc.) But they often don't say (depending on the particular LSP setup etc) this function ends at this char and they definitely don't then provide the text editing tool
  - throw478239 2mo ago
    This is not accurate. Most LSPs do provide text editing tools, which the Language Server Protocol calls "code actions". These actions do things like: reorganizing imports, expanding macros, extracting a selection into a function, or converting one type of control flow into another (e.g. if to match). Common LSPs, like rust-analyzer and Ruff (Python) support code actions. Editors also expose code actions: Zed, for instance, exposes actions under the default keymap with the binding Command + . source: https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_codeAction https://microsoft.github.io/language-server-protocol/specifi... and https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#workspace_executeCommand https://microsoft.github.io/language-server-protocol/specifi...

mfallon 2mo ago

I've spent the last year building an app with Claude Code without knowing how to code. I'm still surprised at how much it gets wrong, even on the newest models. What makes it work for me is describing exactly what I want and then actually checking everything it gives back.

EagleEdge 2mo ago
For me, I just don't trust Claude Code to write first-class professional code. I have used all frontier claude models and it always does sloppy job that needs to be fixed by codex. I have claude/codex setup and any execution must go to codex and all reviews as well.
- wilj 2mo ago
  Similar here. I use Claude Code and Codex for developing specs, but then have Deepseek v4 Pro implement because it just seems to actually do what it's told, given enough tries. And Reasonix + Deepseek is mostly cache hits when I let it grind on it for a long time, and I can have a huge fleet of agents with very specific focus for code review and rule enforcement mostly sharing cache.

jke_kang 2mo ago

2 weeks rebuilding in Swift is absolutely crazy. Wonder if they're using production Opus 5 for this or their internal Mythos (if they even use it internally)

kev009 2mo ago
If it's on the house (and they can probably turn up the token speed) why use anything other than the most powerful models?
- blovescoffee 2mo ago
  it's still more expensive and slower to run bigger models though. resource usage is not free

jake_and_fatman 2mo ago

30:08 "This means engineers can talk to users... stuff that's actually fun". Wow, he totally gets us. /s 31:09 He flubs the questioning. His first says "Raise your hand if 100% of your code uses agents." Then he says, "What about more than 50%". You can see the same audience members who raised the first time shrug and also raise the second time. He then says "Slightly less hands," which just means he sees what he wants to see. Don't believe this man.

albert_e 2mo ago

So maybe SKILL.md and CLAUDE.md should have model version pinned? Somewhat like AWS CloudFormation templates start with something like TemplateVersion: 2017-10-01 But maybe that's old school thinking still.

knighthacker 2mo ago
I'm thinking a lot about this recently as I'm building AQ.dev (Multiplayer coding harness). My instinct is to keep it simple and not trying to do too many shenanigans with these files and let engineers maintain control.
verdverm 2mo ago
this assumes everyone uses the same model, but where I'm at, we use a wide variety of models we keep the AGENTS.md simple and monorepo quirk focussed, no instructions for specific agents or models
crazylogger 2mo ago
They should just be symlinks to CONTRIBUTING.md and other docs humans are meant to read. The whole idea behind LLM is they use the same language and tools as us, so we literally don't design anything specifically for them.

knighthacker 2mo ago

The most interesting part is Boris' recommendation to delete your Claude.md every 6 months to see what the model is capable of. The link in the post sends you directly to that moment.

discordance 2mo ago
It would have been nice to hear about how to be more efficient, task/token wise, especially with Opus 5. Most users are on a subscription, and aren’t able to run dynamic workflows, spinning up 1000 agents or leaving a bun rewrite going for weeks. Don’t get me wrong… I like cake.
- deleted 2mo ago
  [deleted]
bmitc 2mo ago
Seems like something they should work on, not customers. Reinventing my workflow every six months because a company/product is lazy is not high on my list.
- lorey 2mo ago
  What I read from that is that there's a chance claude gets worse using a claude.md and that's a usability issue on their side.
  - bmitc 2mo ago
    I've seen this particular in the post and others from Anthropic make similar claims of "oh, just do this or that", and I've always been of the opinion: "isn't this what everyone is paying Anthropic to do?".

Taka108 2mo ago

This gives you more raw behind-the-scenes. I think the video from Startup School is too polished. https://www.youtube.com/watch?v=PQU9o_5rHC4 https://www.youtube.com/watch?v=PQU9o_5rHC4

nojs 2mo ago

> From what I'm hearing, you for every model release, you basically delete all of the codebase, delete all of the prompt and start from scratch every time. That in the old world would have been not something Startup would have done for the product. It's like press delete every six months for everything. > That's right. So to be fair, we don't delete the entire codebase.

linxy97 2mo ago

still a long way to go!

DennisL123 2mo ago

There’s something missing in the discussion. Building software is about achieving outcomes, not about generating output, ie tokens. Yet, the ai business case is selling output, obviously. Not sure how to reconcile the two. Ideas?

zazuke 2mo ago
Yes this 100%. I'd say: don't overrate how you do it; it matters more what you do, especially with LLM models. It's just a tool.
yetihehe 2mo ago
Building bridges is about achieving outcomes and utility, but steel and cement suppliers are selling by ton.
- miyoji 2mo ago
  To extend this analogy, vibe coding can be defined as dumping steel and cement out of a helicopter into the water and calling the result a bridge because you can technically walk on it.