8 ms·
It's not quadratic attention, you get that curve from the input tokens going up linearly, since the graph is measuring cumulative cost at each token count. Basi
by conradkay 2mo ago
It's not quadratic attention, you get that curve from the input tokens going up linearly, since the graph is measuring cumulative cost at each token count. Basically for y=5 it's 5+4+3+2+1, or f(x) = x(x+1)/2
https://pbs.twimg.com/media/HNFc4Dma8AA76FW.jpg?name=orig https://pbs.twimg.com/media/HNFc4Dma8AA76FW.jpg?name=orig
- salamo 2mo agoOn the one hand, compacting at 300k saves OpenAI 40%. That's great. On the other hand, $10 for 1M tokens still seems really high? It's not too hard to blow through that in an hour or two.