thebeas

6 ms·

1. 1.▲
   by thebeas 6mo ago

   The "infinite context soon" concern comes up a lot — but even at 1M+ tokens, agents still hit limits on long enough tasks, and cost scales linearly with context size. The compression models are the product, not the proxy. The gate
2. 2.▲
   by thebeas 6mo ago

   We provide the model with a tool, we call expand() that allows the model to get access to more context if needed by using it. We state this directly appended into the outputs so the model knows exactly where the lines were removed from.
3. 3.▲
   by thebeas 6mo ago

   We do both: We compress tool outputs at each step, so the cache isn't broken during the run. Once we hit the 85% context-window limit, we preemptively trigger a summarization step and load that when the context-window fills up.
4. 4.▲
   by thebeas 6mo ago

   That's why give the chance to the model to call expand() in case if it needs more context. We know it's counterintuitive, so we will add the benchmarks to the repo soon. Given our observations, the performance depends on the task