> If models keep lasting ~year timescales could we ever see people going with ROM chips for the weights instead of memory? Before ROM, there's a step where HBM

> If models keep lasting ~year timescales could we ever see people going with ROM chips for the weights instead of memory? Before ROM, there's a step where HBM for weights is replaced with Flash or Optane (but still high bandwidth, on top of the chip) and KV cache lives in SRAM - for small batch sizes, that would actually be decently cheap. In this case, even if weights change weekly, it's not a big deal at all.