Well, with proper compaction algorithm (like in Codex), it could indeed be useful for majority of tasks even in the future. The context length in Codex is just

Well, with proper compaction algorithm (like in Codex), it could indeed be useful for majority of tasks even in the future. The context length in Codex is just 272k, but it can reason well about much larger codebases due to good exploration and compaction algorithm. I have yet to saturate the 1M context of Gemini, for example.