ivzak

9 ms·

1.
by ivzak 6mo ago
Speaking from experience - serving good context compression is not trivial.
2.
by ivzak 6mo ago
Thanks, checking it out!
3.
by ivzak 6mo ago
We keep preserved pieces of context unchanged = compression removes some pieces of the input while keeping the others verbatim. Let us shortly share a concrete example
4.
by ivzak 6mo ago
For auto-compact, we do essentially the same Anthropic does, but at 85% filled context window. Then, when the window is 100% filled, we pull this precompaction + append accumulated 15%. This allows to run compaction instantly
5.
by ivzak 6mo ago
It seems to be the hit rate of a very straightforward (literal matching) retrieval. Just checked the benchmark description ( https://huggingface.co/datasets/openai/mrcr ), here it is: "The task is as follows: T
6.
by ivzak 6mo ago
Probably LLM-generated, but that's a fair point :D Well, the proxy is open source, maybe someone will even implement this before we do :) Talking about the features proxy unlocks - we have already added some monitoring, such as a dashb
7.
by ivzak 6mo ago
Claude code still has /compact taking ages - and it is a relatively easy fix. Doing proactive compression the right way is much tougher. For now, they seem to bet on subagents solving that, which is essentially summarization with Haiku
8.
by ivzak 6mo ago
I think we should draw distinction between two compression "stages" 1. Tool output compression: vanilla claude code doesn't do it at all and just dumps the entire tool outputs, bloating the context. We add <0.5s in compres
9.
by ivzak 6mo ago
Subagents do summarization - usually with the cheaper models like Haiku. Summarizing tool outputs doesn't work well because of the information loss: https://arxiv.org/pdf/2508.21433 . Compression is different becau
10.
by ivzak 6mo ago
I doubt Anthropic would single-handedly cut their API revenue in half by rolling out compression. Zero incentive.
11.
by ivzak 6mo ago
You’re right - poor compression can cause that. But skipping compression altogether is also risky: once context gets too large, models can fail to use it properly even if the needed information is there. So the way to go is to compress with
12.
by ivzak 6mo ago
Your comment reminded me of this old simulacra paper ( https://arxiv.org/pdf/2304.03442 ) :) iirc, they compressed the "memory roll" of the agents every once in a while
13.
by ivzak 6mo ago
We don't sell data to model distillers.
14.
by ivzak 6mo ago
There is a nice JetBrains paper showing that summarization "works" as well as observation masking: https://arxiv.org/pdf/2508.21433 . In other words, summarization doesn't work well. On top of that, they
15.
Show HN: Context Gateway – Compress agent context before it hits the LLM(github.com)
97 points by ivzak 6mo ago|64 comments