7 ms·
API page lists the knowledge cutoff as Dec 01, 2025 but when prompting the model it says June 2024. Knowledge cutoff: 2024-06 Current date: 2026-04-24
by czk 5mo ago
API page lists the knowledge cutoff as Dec 01, 2025 but when prompting the model it says June 2024.
Knowledge cutoff: 2024-06
Current date: 2026-04-24
You are an AI assistant accessed via an API.
- htrp 5mo agoCan you really believe things that the model says? (A lot of prior model api pages say knowledge cutoffs of June 2024, maybe the model picks that up?)
- czk 5mo agoyou cant but its pretty reproducible across api and codex and other agents so i just thought it was odd. full text it gives: Knowledge cutoff: 2024-06 Current date: 2026-04-24 You are an AI assistant accessed via an API. # Desired oververbosity for the final answer (not analysis): 5 An oververbosity of 1 means the model should respond using only the minimal content necessary to satisfy the request, using concise phrasing and avoiding extra detail or explanation." An oververbosity of 10 means the model should provide maximally detailed, thorough responses with context, explanations, and possibly multiple examples." The desired oververbosity should be treated only as a *default*. Defer to any user or developer requirements regarding response length, if present.
- deleted 5mo ago[deleted]
- swyx 5mo agocan u test it on say who won the 2024 US election
- ghurtado 5mo agoI can't really think of a less reliable test for anything at all than making a random guess as to something that had about 50/50 odds to begin with Easiest Turing test ever...
- himata4113 5mo agoask it 10 times.
- pixel_popping 5mo agoMASSIVE ADVERSARIAL x50
- czk 5mo agowith thinking off and tools disabled: Donald Trump won the 2024 U.S. presidential election.
- WarmWash 5mo agoUsually the labs do some kind of post training on major events so the model isn't totally lost. A better test is something like "what is the latest version of NumPy?"
- bakugo 5mo agoThat sort of test isn't super reliable either, in my experience. You're probably better off asking something like "what are the most notable changes in version X of NumPy?" and repeating until you find the version at which it says "I don't know" or hallucinates.
- redsocksfan45 5mo agoI thought that one specifically was placed in the default system prompts of basically all providers.
- BeetleB 5mo agoI don't know why this keeps coming up. This has always been the least reliable way to know the cutoff date (and indeed, it may well have been trained on sites with comments like these!) Just ask it about an event that happened shortly before Dec 1, 2025. Sporting event, preferably.
- bakugo 5mo agoModels don't know what their cutoff dates are unless told via a system prompt. The proper way to figure out the real cutoff date is to ask the model about things that did not exist or did not happen before the date in question. A few quick tests suggest 5.5's general knowledge cutoff is still around early 2025.
- czk 5mo agoi wonder if they put an older cutoff date into the prompt intentionally so that when asked on more current events it leans towards tool calls / web searches for tuning
- ssl-3 5mo agoI wonder if the cutoff date is the result of so many people posting about the date over time and poisoning the data. "Dead cutoff date theory," perhaps. Whatever it is, the cutoff date reporting discrepancy isn't new. Back when Musk was making headlines about buying/not buying Twitter, I was able to find recent-ish related news that was published well after the bot's stated cutoff date. ChatGPT was not yet browsing/searching/using the web at that point. That tool didn't come for another year or so.
- soco 5mo agoStupid question: wouldn't it then search the web for that event?
- bakugo 5mo agoIf you have web search enabled, sure. But if you're testing on the API, you can just not enable it.
- MallocVoidstar 5mo agoOpenAI does tell the model the current date via API, so it's odd for them not to also tell the model its cutoff
- jumploops 5mo ago[dead]