8 ms·
OpenAI reduces Codex Model Context Size from 372k to 272k
- dannyw 2mo agoThis was tweeted about when it happened, with some explanation from Tibo here: https://x.com/thsottiaux/status/2076543065045795309 https://x.com/thsottiaux/status/2076543065045795309
- mkl 2mo agoTo see replies: https://xcancel.com/thsottiaux/status/2076543065045795309 https://xcancel.com/thsottiaux/status/2076543065045795309 The linked tweet is an unofficial reply to Tibo's official info and Tibo makes a correction in a reply.
- deleted 2mo ago[deleted]
- youre-wrong3 2mo ago[flagged]
- Bolwin 2mo agoX.com is not publicly accessible. I wish people would stop using it as a source
- youre-wrong3 2mo ago[flagged]
- youre-wrong3 2mo ago[flagged]
- imgyuri 2mo agoHow can the overall trajectory length be the same across reasoning efforts? I don't see how this is possible even if reasoning is not included in the trajectory length calculation.
- chaos_emergent 2mo agoI think Tibo was just keeping all else fixed and it’s an illustrative example rather than a perfect real-world trajectory.
- causal 2mo agoAm I dumb or does this chart make no sense? Or why does the line only go up even with compaction? Or maybe "overall trajectory size" is hiding some meaning I don't understand?
- ph4rsikal 2mo agoThe blue line (200K context) is lower than the red line (300K context). Indicating that it's cheaper to run blue rather than red over longer cycles.
- chaos_emergent 2mo agoThe chart makes sense and is describing the cumulative cost of a trajectory. Cache tokens are created when a trajectory’s prefix is used more than once. A larger pre-compaction context window means that a greater number of cache tokens are used per turn, and a larger number of turns are completed before compaction runs. So you get a cumulative cost that grows quadratically until the compaction event.
- crazylogger 2mo agoThe Y axis is total cost in USD. For it to go down would mean OpenAI refunding you money.
- trilogic 2mo agoHugstonOne increased coding context size, from 1 to 4 Million ctx
- cyb3ralbert 2mo agoContext size cuts like this are usually a cost/latency tradeoff rather than a capability one - serving a smaller window is cheaper and keeps latency in check, and most sessions probably don't need anywhere near 372k tokens anyway. Curious if this affects people who were actually relying on the larger window for big codebases.
- mikhailt 2mo agoWe'll have to wait because Tibo (OpenAI) said this was a temporary measure and it'd go back to 372k soon. The issue was more specific to higher token burn rates, not latency.
- bel8 2mo agoThat's quite small for my workloads. I try to keep it under 200k but my DeepSeek and MiMo sessions can sometimes grow to 350k tokens when I try to squeeze one last iteration I compact. Can't OpenAI copy DeepSeek K/V cache tech (from published papers) to make it super cheap?
- SwellJoe 2mo agoNobody does caching as well as DeepSeek, so I guess it's a big enough difference in the implementation to make it difficult. If you use Reasonix with DeepSeek it gets silly, as it is append-only to work with how caching works. It gets something like 97-98% cached tokens in a long session. It makes an already cheap model even cheaper.
- cyanydeez 2mo agousing open models and local AI via llamacpp, I'm compacting (via agent directed) between 55k-85k; going up to 120k is rare unless the agent is actually following a non trivial log tracing or some other thing that absolutely requires keeping large context. I've also tailored an agent system prompt with llamacpp reasoning budget and message that asks the agent to spawn subagents then compress to reduce context bloat. It's in opencode using the dynamic context pruning, so it's mostly keeping a vector&direction without having to bloat up. It mostly works to allow iteration into multiple sub components.
- d4rkp4ttern 2mo agoThe other day I was trying to find out exactly how much context size 5.6 has in codex CLI (via subscription) and I didn’t find that documented in any of the obvious places. Sort of ridiculous that we have to find this out in X or Reddit, e.g: https://www.reddit.com/r/codex/s/hVv29obfFD https://www.reddit.com/r/codex/s/hVv29obfFD
- tekacs 2mo agoYou can run ’codex debug models' into jq! I forget whether it's documented or not, and it is kind of annoying that that's how you find it, but it does tell you. Maybe useful for the future if nothing else.
- spongebobstoes 2mo agoit's in the GitHub repo
- davidkuennen 2mo agoI never felt that context size was any issue at all in codex. I don't know how their compaction works, but it just keeps on going as if it has no context limit at all. At least in my experience.
- embedding-shape 2mo ago> I never felt that context size was any issue at all in codex. I'm guessing you just started using codex, it suffered greatly from "model context size exceeded" errors early on, where compaction couldn't even recover from it, these errors just stopped happening maybe some month(s) ago. Nowadays it's a lot better though and I've don't get stuck in those anymore. Although I don't like the idea that they don't surface what goes into the "concise summary" afterwards, make it really hard to know if it actually got everything important or not. In general, codex seems to be moving into the way of hiding as much from the user as possible, and it wouldn't surprise me if they eventually start encrypting the entire session logs just like they did with agent>sub-agent prompts recently. Sad though as it's easily the best harness+model combo available today out of the ones I've tried.
- cmrdporcupine 2mo agoClaude also had issues with this kind of thing for months early on, too, where it simply couldn't compact its own emissions. But definitely earlier GPT models suffered greatly as context got large, and the compaction itself in Codex was really crappy. That changed around January.
- mikhailt 2mo agoThat has nothing to do with context window or the models at all; it's the compute capacity. Pretty much every single AI firm had the same issue when the new model came out, it's the result of everyone jumping onboard and consuming all available compute capacity to break it for days. Claude had the same issues when they released their newer models.
- embedding-shape 2mo ago
- nttylock 2mo ago[flagged]
- simonw 2mo agoAnother interesting change in that commit is the addition of this section to the system prompt: Before taking a destructive action: - Make sure the action is clearly within the user's request. - Resolve the exact targets with read-only checks when necessary. - Do not use `$HOME`, `~`, `/`, a workspace root, or another broad directory as the target of a recursive or destructive command Looks like the fix for this bug where Codex would occasionally accidentally delete your entire home directory: https://twitter.com/thsottiaux/status/2077630111499882637 https://twitter.com/thsottiaux/status/2077630111499882637
- AmazingTurtle 2mo agoThis happened to me one time. On windows though, deleting C:/ lmao. The direction I'm heading now is a better harness, i.e. isolating codex at a container level with dedicated workspaces / mounts etc. I'm building something at the moment that serves my needs.
- Razengan 2mo agoThat's ok, if AI deletes my home directory I can just ask AI to regenerate my home directory.
- tekacs 2mo agohttps://github.com/tekacs/fast-rm https://github.com/tekacs/fast-rm I've overridden my rm with this, which I threw together for fast-deletes of things like Rust target/ directories, and after seeing the GPT horror story, I taught it to flatly reject deletions directly under `/` and under home directories, with a message printing the path that it's trying to delete. Not exactly a perfect mitigation, but given that the stated risk was the model mistakenly using the wrong $HOME, it seems like a reasonable safety. I should probably make it use an even scarier rejection notice, though. I also... have backups.
- skerit 2mo agoNo matter how good compaction is, on some big projects it needs to read a lot of files. In my experience the first 200.000 tokens go FAST, but after that it slows down. Most of my Fable sessions don't go over 500.000 tokens, I don't need to compact once. But when I use Codex a single session has to compact over and over again.
- HDThoreaun 2mo agoIn my experience the only reason it needs to read a lot of files is if your agents.md is deficient. It should only need to read the file it's actually working on, maybe a few others. Everything else should be explained in documentation
- ashu1461 2mo agoI am wondering why the codex repo is open source ? Can they afford it to be open source in the longer run ?
- simonw 2mo agoNot much point making it closed source at this point, coding agent harnesses are trivial to reverse-engineer. Being open source was also a useful community lever they could pull when they were trying to catch up with Claude Code.
- CamperBob2 2mo agoNot much point making anything closed source at this point, if the client has access to the binary.
- killix 2mo ago[flagged]
- Topfi 2mo agoWhat do you mean “afford”?
- Topfi 2mo agoGood move given some experienced issues and compaction across the 5.6 range is closer to 5.4 than 5.5, i.e solid and reliable. Will say that 5.6-Sol is a minor bump in my benchmarks in most areas vs 5.5 but a severe regression in a few specific task focused on rearranging trees, addressing merge conflicts, etc. where the model to accomplish the task does not properly adhere to prompts in a way GPT-5 originally managed, not retaining parts of history in the way prompted despite specific instructions not to as that made the final completion easier… I am of the conservative and cautious opinion that no model should be able to run destructive tasks at all, I have seen every model do things that make me concerned enough to maintain that opinion and know my evals can’t catch everything. But for 5.6-Sol specifically, I’d caution everyone to reevaluate how you run the model, maybe take a few more precautions you tend to forgo. It is extremely capable as a reviewer and for extensive tasks, though for the later, the safety net I feel is required to be comfortable limits the utility. The code 5.6-Sol provides also still is a bit harder to parse in reviews. Release strategy wise, feel it’s have been smarter to release only Luna and Sol now, then Terra a few weeks of posttraining later, I simply cannot see a purpose for it in the current form given how well both Luna and Sol scale up and down respectively with reasoning. Two models from a lab at a time is also the limit I feel one can properly assess at a time.
- damsta 2mo agoNot a fan of their context compaction and I feel like 1M-token context should minimum today. Each day I see how GPT 5.5 and 5.6 struggle a bit after each compaction before they get to the full speed, sometimes focusing too much on some older steering message that made it into the compacted context.
- iugtmkbdfil834 2mo agoFull agreement. I sincerely hope that reducing token is mostly a cost saving measure and not some roundabout way to get more token use. It still does not help. In my corp, bean counters rendered initially useful internal llm largely pointless as they severely limited context. I swear there is a executive convention out there sharing worst practices.
- Sammi 2mo agoJust use md files for working memory. No need for large context. LLMs get dumber as you put more in context, as it stretches their attention. Keeping context small is better for quality.
- jubilanti 2mo agoThe fact there is no way to disable auto-compaction like every other coding harness makes me scream. It fires randomly at 10-20%, so really you now only have guaranteed context of 272k * 80%. So many times I've had it work on a large codebase, it is almost done with a task, it needs maybe a 2000 token response, but it drops below 20%, it churns and churns and churns and then I see the dreaded ``Context compacted`` and scream. You can't go back in the history to the state of the conversation before the compaction. So then it has to investigate the codebase again gets to 20% and then auto-compacts. So it then has to read more, and then auto-compacts again, and then I'm out of tokens.
- n4te 2mo agoUse pi. Can't trust OAI or Anthropic to make good tools.
- 2mo ago
- amelius 2mo ago"272k ought to be enough for anybody"
- trilogic 2mo agoThats not even enough to read a simple codebase, how is that enough?
- deleted 2mo ago[deleted]
- sokoloff 2mo agoAlmost certainly an homage to “640kB ought to be enough for anybody.” https://quoteinvestigator.com/2011/09/08/640k-enough/ https://quoteinvestigator.com/2011/09/08/640k-enough/
- trilogic 2mo agoThat is a good point actually (this is how we at Hugston measure tokens, in bytes). 270k tokens should be around ~1100kb or 1 MB, so not really enough for a serious project.
- tekacs 2mo agoI know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately. Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine? The lack of long context is the main reason that I still end up using Anthropic. The worst is when you need it to hold for example a number of papers in its head, or large and complex materials that it needs full resolution on and your context window ends up being perennially at 16%. You have about five minutes of conversation and it compacts and then you have to wait for it to read that again, get to 16%... and repeat. 372 was not perfect, but it was so much better and a godsend. It turned that 12 to 20% into more like 40%.
- planckscnst 2mo agoI agree. Compaction sucks, so I made tools that let the LLM selectively delete (and recall if needed) chunks of its context. You might want to try context bonsai if you're routinely hitting the auto-compaction wall. https://github.com/Vibecodelicious/context-bonsai-agents https://github.com/Vibecodelicious/context-bonsai-agents
- tekacs 2mo agoI've implemented a similar approach – although I'm surprised not to see mention of cache prefix busting in there!
- jeremyjh 2mo agoYes the idea is cool but this could really hammer usage, especially just leaving it up to the agent to decide when to do it. I'm not surprised though, considering the github account is named "Vibecodelicious" and became active in December. Poking around in the repo the whole implementation is an unsupervised LLM fever-dream.
- planckscnst 2mo agoIt depends on what you mean by "unsupervised" - I've been strictly working through the agent, through specs the entire time, but it's been very supervised, I just leave the mistakes in-place and have it work from there. However at this point it can completely maintain itself. When a new version of Claude or OpenCode is released, it updates itself to work on the latest version. It can also add new implementations for harnesses pretty reliably. It's actually pretty fun to watch it at this point. "Make this work on Hermes agent and message me when you're done" and an hour later or so, I can go play with it in Hermes.
- cmrdporcupine 2mo agoCodex compaction is really quite good. Smaller context doesn't really harm me. I had a /goal running last night for 9.5 hours straight while I slept. When I woke up in the morning it was fully on task and focused. Write up a detailed design doc. Build a decent AGENTS.md, and write up a good prompt or /goal. Long context can be more of a curse than a benefit sometimes anyways.
- maherbeg 2mo agoYeah, agreed. It makes everything else look pretty bad. I still like to manage my work in a more structured form, but Codex can just rip on a goal end to end in the same thread.
- weeksie 2mo agoYou just need good compaction and a memory system with decent project layout and documentation. If you don't have that 1M tokens won't really save you either.
- Topology1 2mo agoLooks like we will be waiting quite a while to see a 1M context window from OpenAI
- miranaproarrow 2mo agoclaudes/codex performance suffers after 250k anyway and my workflow reset after every 200 so this is a non-issue
- fandorin 2mo agoDoes the context size really matter that much if a “dumb zone” starts around 120k-150k anyway?
- conradkay 2mo agoThings change fast! For Fable 5 it definitely feels past at least 272k
- fandorin 2mo agohmm, what about other models?
- wonnage 2mo agoNot sure if this is still up to date (2023), but https://arxiv.org/abs/2307.03172 https://arxiv.org/abs/2307.03172 shows that performance degrades mostly in the middle of the context. Anecdotally I've been stuck in that situation of being at 400-500k tokens and "just one more prompt bro" will get the task done, and I appreciate not having to wait through a compaction. If anything, keeping the bloated context helps with accuracy at the expense of speed in these cases.
- aurbxyajwur 2mo ago5.6 is a mess
- gurjeet 2mo agoPlease elaborate. > Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. Sincerely, HN Guidelines Police :-) https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- robkam 2mo ago[flagged]
- greenoracle9 2mo ago[dead]
- hakanderyal 2mo agoI have rule files that guides the agent towards my coding standards, code style, house rules etc. They alone cost 60-80k tokens, and they are the backbone of my system that prevents slop. Pre 1M context, I had to build complicated tooling to re-include the relevant docs to the context upon compaction, which relied on unstable transcription file format, which was a pain to maintain. With 1M context I deleted all of those. Nowadays most of my sessions uses 300-450k context. Another thing that's preventing me from trying Codex. (the other is @ referencing files not auto including them to the context) 1M should be table stakes for frontier models at this point for programming.
- yearolinuxdsktp 2mo agoTry running your rule files through an LLM for optimization. 60k-80k tokens is massive. Funnily enough, most anti-slop skills I found are both way too verbose and miss some common slop constructs. I also reduced many rules from “When doing X, don’t do Y, but do Z.” Instead, the rule is “When doing X, do Z.” Fewer tokens and often works better. I had one critical rule I was maintaining about searching the codebase using a structural index/graph and not grep. Every time the agent missed it, I asked it how to improve the rules. Eventually, I asked the AI to review that rule file and it rewrote it to be 30% smaller, but, crucially, structured to be more understandable by the LLM. Another helpful thing was to ask AI to review my rules for things it can load on-demand when it works in that area.
- hakanderyal 2mo agoIt’s optimized to death. Every word, every comma. I have a rule file for writing rule files. Problem stems from using my own abstractions instead of common frameworks. For a full stack session, backend + front end + docs about the part we are working on adds to that amount. I’ve accepted it because it results in code exactly the style I would’ve written. It’s a good tradeoff to cut off the slop.
- yearolinuxdsktp 2mo agoFascinating! I am glad you’re getting good results with that. Maintaining abstractions is critical and it’s very frustrating when the LLM ignores them.
- brandoncarl 2mo agoWith purely quadratic attention, the cost of the token at 372K is 87% more than the token at 272K. There are attention mechanisms that help to mitigate this, but you can clearly see in the chart that Tibo posted that the attention mechanisms they are using are still scaling quadratically. This becomes an optimization problem for token t: min f[cost, quality loss] = compaction cost[t] + compaction quality loss[t] + token cost[t]
- conradkay 2mo agoIt's not quadratic attention, you get that curve from the input tokens going up linearly, since the graph is measuring cumulative cost at each token count. Basically for y=5 it's 5+4+3+2+1, or f(x) = x(x+1)/2 https://pbs.twimg.com/media/HNFc4Dma8AA76FW.jpg?name=orig https://pbs.twimg.com/media/HNFc4Dma8AA76FW.jpg?name=orig
- salamo 2mo agoOn the one hand, compacting at 300k saves OpenAI 40%. That's great. On the other hand, $10 for 1M tokens still seems really high? It's not too hard to blow through that in an hour or two.
- monster_truck 2mo agoIf ChatGPT 5.6 is so good why do they continue to have the same motherfucking cache and usage issues that they were struggling with when 5.2 is new? Could not pay me to use this shit. Enjoy your reset bingo
- quotemstr 2mo agoCodex is open source. You can build it yourself. You don't have to use upstream's parameter choices, compaction strategies, subagent heuristics, or whatever. Strange how people act like tweaks to these parameters are mandates when they're really just suggestions relayed to you via git. For example: upstream recently changed the effort level hotkeys so M-. would stop at xhigh, not max. If you want max, upstream, you go to the /model menu. I didn't like this change, so I undid it locally. Easy peasy.
- noname120 2mo agoNot true. Try to increase the context window and you will be greeted with an error when you reach the real limit enforced server-side.
- quotemstr 2mo agoPerhaps not this for this parameter, but for others? Client side.
- noname120 2mo agoApart from the local scaffolding you can configure very few parameters, and the ones you can configure are the ones that don’t matter
- searealist 2mo agoThe models themselves support up to 1M. You are just charged more for all context over 400k. The 272k limit is just a client limitation to ensure you can never go over 400k since the maximum output size of the model is 128k.
- noname120 2mo agoNot true. This is the case for the API, not for subscriptions: they are hard capped at 400k.
- KronisLV 2mo agoOn long sessions even with workflows and sub-agents I typically hit around 500-750k tokens in context (say 4-16 hours of work in a session, across multiple 5-hour limits) with Claude Code. I was going to explore paying for Codex since OpenAI seems to be a bit more generous with rate limits, but I'm now not sure - for the stuff I do limited context size would be a dealbreaker - basically lots and lots of documentation and guideline references, code review loops, documentation references and web searches, multi-repo exploration, lots of tool calls and so on. I might just upgrade to Anthropic's more expensive Max subscription or something to get more subsidized tokens. Even with plan files and the plan mode, it's like a slot machine after compacting the context, sometimes steps or other details just evaporate in thin air, the less of that I need to do the better. It's not that it can't work, it just doesn't work reliably enough not to be annoying. Luckily DeepSeek V4 Pro, GLM 5.2 and Kimi K3 don't seem to have those limits either - though DS is around Sonnet, GLM 5.2 feels a bit above Sonnet and only K3 only really is in the proper Opus ballpark that's good enough for me to work undisturbed, even if it seems to be slower.
- killix 2mo ago[flagged]
- throwatdem12311 2mo agoI primarily use Opus for my day to day and I /clear often. 1M context sound great but it quickly degrades once you start hitting the 50% mark. I usually just /clear once I hit between 30 and 40 percent context and get much better results from the model. Compaction doesn’t really help much in my experience, starting fresh and having the model front load its context from scratch works much better IME. I have multiple skills with a pile of markdown docs organized by feature to assist in that first load by telling it where to find relevant information for the task. Works great.
- rotariuvladimir 2mo ago[dead]
- onetrickwolf 2mo agoI don't know if this is why they did it, but I think it is generally a mistake to go beyond this context size anyway. I don't think people realize how much dumber the models get at larger contexts and how much more the token cost is. I never let claude get about 300k and in general I don't compact, I just divide up the work in chunks where I can fit it into 300k and I try to keep really clean and slim docs and modular code bases. It's still kind of annoying, sometimes the larger context is useful for one time tasks, but I think if you are regularly going beyond 300k you are losing a lot and probably have a poorly designed code base.
- enraged_camel 2mo ago>> I don't think people realize how much dumber the models get at larger contexts and how much more the token cost is. It does not match my experience that the model gets significantly dumber. It does get slower and more expensive, yes, but that's a sacrifice that needs to be made when working on anything complex. My process involves having the main agent use subagents to explore what is needed for the given task. Then it writes a plan. Then it has the plan adversarially reviewed by more subagents and hardens it. After all is said and done, the 1M token window is 30-40% full. This flow would never work with 272k context, and in fact I've had to tone it down significantly for 5.6 Sol. Which, now that I think about it, probably explains why the results I get with it are inferior.
- andix 2mo agoExactly my experience. I would even draw the border at 100-150k. Long contexts would be nice, but they just don't work well, even if the models support them.
- Havoc 2mo agoYeah I also try to compact/restart at 250k. I noticed though that it tracks with how big the project is so I guess the people needing more are just working on bigger projects
- smcleod 2mo agoI haven't used Codex in some time, but in Claude Code I force my context size down to 280k on purpose as in my experience your best work is done within the 30-250k~ range regardless of a models maximum context size.
- coderenegade 2mo agoThis is a shame, and hopefully it reverts soon. I switched from Claude to Codex just before the Fable release because Claude doesn't follow instructions and goes off the ranch too frequently. I really don't want to have to switch back to what I consider to be a fundamentally worse model.
- kccqzy 2mo agoEven Fable doesn't fully follow instructions.
- andix 2mo agoI didn't try long coding sessions for a while. I had really noticeable performance degradation with all models above 100-150k and always try to stay below those numbers.
- hendersoon 2mo agoThis was done because they were charging the 100%/50% premium for long-context over 272k to Codex users, even though they set 372k as the default. It cost people a LOT of money.
- CurbStomper 2mo ago[dead]
- greatsage_sh 2mo ago[flagged]
- Fr4nZ82 2mo agoBig context windows made everyone lazy about deciding what actually deserves to be in there... compaction makes it worse, it's lossy compression applied to everything at once so the details you actually needed probably get lost. I always thought that retransmitting the full chat every turn is a foundamental problem... in my experience with conversational agents I clear the context every turn and reinject all the relevant info with a memory/context plugin I built myself, so small context stopped being a problem cos the model reads a few k of curated state instead of 200k of scrollback. Coding agents are harder and I haven't cracked that part, but in my opinion the real fix to the context problem is in this direction: a policy that keeps what's needed to finish the job or to do the next job and drops the rest. This can be achieved with a separate LLM specialized for exactly that.
- nullsanity 2mo ago[dead]
- paytonjjones 2mo ago> I always thought that retransmitting the full chat every turn is a foundamental problem So did many NLP experts who worked on LSTM, GRU, etc. But transformers empirically won out and here we are. It will be interesting to see if future model architectures need to take this into account again. If humans are any indication, one thing we're still missing is efficient movement of information from short term to long term memory. Fine tuning in principle does something akin to this, but not efficiently.
- yalogin 2mo agoI have over the last two months switched to OpenAI from Claude since it’s been vastly better for my cases. Wonder if I see any difference in the quality of the output with this change
- feiz45607 2mo ago[flagged]
- majestic5762 2mo agoJust use GPT-5.6 Model Router from the Codex plugins
- deleted 2mo ago[deleted]