7 ms·

Why has public comms been so poor on this issue? There's been lots of Github issues posted in the Claude Code repo with lots of new comments each day screaming

Why has public comms been so poor on this issue? There's been lots of Github issues posted in the Claude Code repo with lots of new comments each day screaming into the void, but radio silence from Anthropic since the revert in December. It's clearly causing a lot of frustration for users leading to clever workarounds like this. It was obviously a complex issue (I appreciate that and your work!). But I think there's a lot to be improved on with communication. This issue in particular seems like it has lost a lot of user trust - not because it was hard to solve and took awhile - but because the comms and progress around it was so limited. Eg issues: * https://github.com/anthropics/claude-code/issues/1913 https://github.com/anthropics/claude-code/issues/1913 * https://github.com/anthropics/claude-code/issues/826 https://github.com/anthropics/claude-code/issues/826 * https://github.com/anthropics/claude-code/issues/3648 https://github.com/anthropics/claude-code/issues/3648

chrislloyd 8mo ago
The communication is definitely on me! There honestly wasn't much new to say -I've been slowly ramping since early Jan just to be extra sure there's no regressions. The main two perf. issues were: 1. Since we no longer have <Static> components the app re-renders much more frequently with larger component trees. We were seeing unusual GC pauses because of having too much JSX... Better memoization has largely solved that. 2. The new renderer double buffers and blits similar cells between the front and back buffer to reduce memory pressure. However, we were still seeing large GC pauses from that so I ended up converting the screen buffer to packed TypedArrays.
- catlifeonmars 8mo ago
  I’m really surprised that GC is an issue at the bits/sec throughput a TUI would be pushing. At the risk of making an obvious observation: your render loop is doing way too much work for what it is producing.
  - moltar 8mo ago
    I think this is the main issue. When I would get into flickering mode, it appeared that the entire TUI was re-rendering on every key press. I don’t know if it’s maybe just the limitation of Ink or even terminals.
    - catlifeonmars 8mo ago
      Well vim doesn’t flicker so it’s definitely not a limitation of terminals, but you’re probably right about the Ink/React stack.
      - nevertoolate 8mo ago
        So why are you stuck with ink/react stack?
        catlifeonmars 8mo ago
        I don’t use React/Ink for anything, what do you mean?
  - chrislloyd 8mo ago
    Most people's mental model of CC is that "it's just a TUI" but it should really be closer to "a small game engine". For each frame our pipeline constructs a scene graph with React -> layout elements -> rasterize them to a 2d screen -> diff that against the previous screen -> _finally_ use the diff to generate ANSI sequences to draw. We have a ~16ms frame budget so we have roughly ~5ms to go from the React scene graph to ANSI written. You're right that in theory we shouldn't have to do much work, but in practice that's required optimizations at every step. For the GC pauses specifically, what mattered is predictable performance. More allocations == more GC == more frames where the VM is locked up seemingly doing nothing. On slower machines we were seeing this be in the order of seconds, not ms and when somebody is typing all they feel is the 1 character that's stuttering. Honestly, I was surprised about this too as GC in JS is often not something that's too impactful.
    - brazukadev 8mo ago
      Maybe the problem is using React for that.
    - catlifeonmars 8mo ago
      Thanks for the in depth explanation! I think the comparison to a game engine makes a lot of sense. Is the diff just part of the react rendering engine, or is it something you intentionally introduce as a performance optimization? Mostly I’m wondering how much the diff saves on rendering performance if you’ve already generated a 2D raster. In the browser, this saves on API calls to the DOM but at that point you haven’t rendered anything resembling an image. Is this what helps to resolve flickering, perhaps?
    - vintagedave 8mo ago
      Can I ask why you used JavaScript at all for CC? Or even React for a simple UI? It seems misaligned with the app’s nature. This bug, GC pauses, everything else you mention… This isn’t criticism, because I believe people make good judgements and you and Anthropic will have good reasons! It’s just puzzlement, in that I don’t understand what the balance judgement was that led you here, and having experienced all the issues it led to I would love to know what the reasons were. Thankyou for any insights you can share :)
pqtyw 8mo ago
Presumably they had no clue how to fix it and just ignored it and pretended everything works fine because they didn't want to admit it?

chrislloyd 8mo ago

The communication is definitely on me! There honestly wasn't much new to say -I've been slowly ramping since early Jan just to be extra sure there's no regressions. The main two perf. issues were: 1. Since we no longer have <Static> components the app re-renders much more frequently with larger component trees. We were seeing unusual GC pauses because of having too much JSX... Better memoization has largely solved that. 2. The new renderer double buffers and blits similar cells between the front and back buffer to reduce memory pressure. However, we were still seeing large GC pauses from that so I ended up converting the screen buffer to packed TypedArrays.

catlifeonmars 8mo ago
I’m really surprised that GC is an issue at the bits/sec throughput a TUI would be pushing. At the risk of making an obvious observation: your render loop is doing way too much work for what it is producing.
- moltar 8mo ago
  I think this is the main issue. When I would get into flickering mode, it appeared that the entire TUI was re-rendering on every key press. I don’t know if it’s maybe just the limitation of Ink or even terminals.
  - catlifeonmars 8mo ago
    Well vim doesn’t flicker so it’s definitely not a limitation of terminals, but you’re probably right about the Ink/React stack.
    - nevertoolate 8mo ago
      So why are you stuck with ink/react stack?
      - catlifeonmars 8mo ago
        I don’t use React/Ink for anything, what do you mean?
- chrislloyd 8mo ago
  Most people's mental model of CC is that "it's just a TUI" but it should really be closer to "a small game engine". For each frame our pipeline constructs a scene graph with React -> layout elements -> rasterize them to a 2d screen -> diff that against the previous screen -> _finally_ use the diff to generate ANSI sequences to draw. We have a ~16ms frame budget so we have roughly ~5ms to go from the React scene graph to ANSI written. You're right that in theory we shouldn't have to do much work, but in practice that's required optimizations at every step. For the GC pauses specifically, what mattered is predictable performance. More allocations == more GC == more frames where the VM is locked up seemingly doing nothing. On slower machines we were seeing this be in the order of seconds, not ms and when somebody is typing all they feel is the 1 character that's stuttering. Honestly, I was surprised about this too as GC in JS is often not something that's too impactful.
  - brazukadev 8mo ago
    Maybe the problem is using React for that.
  - catlifeonmars 8mo ago
    Thanks for the in depth explanation! I think the comparison to a game engine makes a lot of sense. Is the diff just part of the react rendering engine, or is it something you intentionally introduce as a performance optimization? Mostly I’m wondering how much the diff saves on rendering performance if you’ve already generated a 2D raster. In the browser, this saves on API calls to the DOM but at that point you haven’t rendered anything resembling an image. Is this what helps to resolve flickering, perhaps?
  - vintagedave 8mo ago
    Can I ask why you used JavaScript at all for CC? Or even React for a simple UI? It seems misaligned with the app’s nature. This bug, GC pauses, everything else you mention… This isn’t criticism, because I believe people make good judgements and you and Anthropic will have good reasons! It’s just puzzlement, in that I don’t understand what the balance judgement was that led you here, and having experienced all the issues it led to I would love to know what the reasons were. Thankyou for any insights you can share :)

pqtyw 8mo ago

Presumably they had no clue how to fix it and just ignored it and pretended everything works fine because they didn't want to admit it?