6 ms·

It is suspicious that OpenAI decided to generate 300 billion output tokens from a model still in training, right after learning there was a credible chance that

by fwlr 7d ago

It is suspicious that OpenAI decided to generate 300 billion output tokens from a model still in training, right after learning there was a credible chance that a major math proof was in that model’s training data. Obviously there are reasonably plausible explanations for each step, but it does sort of feel like parallel construction.

cbarrick 6d ago
I think people are focusing on the training data issue too much. If the data was contaminated, I can still blame that on negligence. But, at least with the Navier-Stokes solution, it's clear [^1] that they learned that Alpöge and Buckmaster were getting close to a solution and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt. What makes this worse to me is the intention. They intentionally threw $15 million in compute at the problem in order to scoop the result. They intentionally left Buckmaster and Alpöge out of the citations. Data contamination should be enough to disqualify them from the prize, but I can believe it to be accidental. On the other hand, someone made an intentional decision to scoop the result by throwing money at the problem. That's so much worse. [^1]: That's the timeline claimed by Buckmaster, and no one from OAI has disputed it.
- unified101 6d ago
  > the secret So such thing existed. In fact, what they learnt was some progress existed, not what the specific progress was.
- fwlr 6d ago
  I think you’re overlooking what I’m implying here. It’s not that they knew contamination was possible but they went ahead anyway. To spell it out just a little bit more: learning the answer might be in model X’s training data made them believe that model X specifically might be able to solve the question, and they were able to very quickly find enough certainty about the former to commit millions of dollars to the latter.
- square_usual 6d ago
  > and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt. Do you have any evidence of this? They don't dispute the timeline, but they never said they knew what Levant/Buckmaster were doing.
  - robotpepi 6d ago
    It's in OpenAI's first announcement that they had solved the problem.
    - derangedHorse 6d ago
      > Only after learning the secret to cracking the problem did they send the first prompt. Which quote in the announcement post provides evidence for the above quote?
      - OneManyNone 6d ago
        “ On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems.” - https://openai.com/index/navier-stokes-solution/ https://openai.com/index/navier-stokes-solution/ They do not explicitly admit to knowing about NS specifically, but are extremely explicit that they tried to scoop some potential millennium prize winners.
        randomblock1 6d ago
        So then they DIDN'T "learn the secret to cracking the problem". They simply knew that part of the problem was solved. Knowing a problem can be solved and knowing the solution are not the same thing.
        freejazz 6d ago
- freejazz 6d ago
  > but I can believe it to be accidental What accident is it when the system is designed to function that way?
  - Lerc 6d ago
    Their claim is that training on their solution is "unlikely but possible". Consider this scenario. Has a google crawler read my new novel, which I may or may not have posted on my blog, page by page, as I wrote it? Can you, without knowledge of what I have actually done, claim that the google crawler has not seen the novel? Without any evidence that I have posted the novel online, it might be tempting to say that the crawler has not seen the novel, but what if I were in an adversarial position against Google on this topic and were challenging them to make that claim. You would wonder if I were hoping Google to overreach by making a definitive claim without taking into account some action that they had no knowledge of. It becomes difficult to use the scientific expression "There is no evidence for this" when there is an accusation of malfeasance because it can be so easily be conflated as "You can't prove we did it". It seems like the best you could say would be 'Unlikely, but possible'
    - freejazz 6d ago
      I'm not taking them at their word, sorry. Genuinely, there is no reason to.
- lnrd 6d ago
  > They intentionally threw $15 million in compute at the problem what? really?
  - abathologist 6d ago
    Yes. Maybe much more: > Such intensive use of AI doesn't come cheap. In a post on X, LisanBench, an LLM benchmark evaluator, estimated that the output tokens alone would cost about $6.5 million at OpenAI's average consumer price. Including the far larger volume of input tokens, the post estimated the total could reach $10 million to $40 million. https://www.businessinsider.com/openai-math-problem-solved-tokens-cost-altman-2026-9 https://www.businessinsider.com/openai-math-problem-solved-t...
    - hyperbovine 6d ago
      But think of all the IPO Monopoly money they just generated.
    - square_usual 6d ago
      That's their API pricing. There's no way they actually paid $15M in compute. I'd say much more likely it's in the order of $1M.
      - malfist 6d ago
        Who are you who is so wise in the ways of a private company's internal cost accounting
  - dekhn 6d ago
    When I worked at Google, we spent $100M in power on protein folding and drug discovery (this was long before AlphaFold). Never underestimate the willingness of smart rich people to invest in speculative science.
- nomel 6d ago
  > They intentionally left Buckmaster and Alpöge out of the citations. No, they asked if they could do a joint publish.
  - oefrha 6d ago
    No, they asked one guy to do a joint publish conditioned on leaving the other collaborator out, with veiled threats. The joint publish part smells awfully like admission of guilt given there’s absolutely no reason to do it if you believe you independently arrived at the result using only public prior work. The leaving out collaborator part is outright academic malpractice. Disclosure: I was an academic once.
    - golly_ned 6d ago
      To add: with a requirement that he rewrite the proof to credit OpenAI.
    - derangedHorse 5d ago
      This would be a new publication with OpenAI's novel solution. Buckmaster and Alpöge did not arrive at the NS solution, they used an approach that also happened to be used in OpenAI's proof. > there’s absolutely no reason to do it if you believe you independently arrived at the result using only public prior work Unless the goal is to assuage the existential pain felt by many mathematicians with respect to what may feel like increasingly inconsequential efforts. It also seemed like a way to build good will amongst knowledge workers dealing with similar issues, with the goal of showing they are dedicated to easing the transition for them as society parades into the future. This has clearly backfired. > The leaving out collaborator part is outright academic malpractice. They would not be leaving a collaborator out. Again, this would be a new publication. Here's the quote from Buckmaster's post [1]: "Two proposals were offered to me. The first was that we post our Euler result, and that OpenAI post its Navier-Stokes result the next day. The second was that, after posting Euler, I alone write a paper presenting the Navier-Stokes result, acknowledging that an internal OpenAI model had resolved it." [1] https://cims.nyu.edu/~tristanb/statement.pdf https://cims.nyu.edu/~tristanb/statement.pdf

by fwlr 7d ago

cbarrick 6d ago

I think people are focusing on the training data issue too much. If the data was contaminated, I can still blame that on negligence. But, at least with the Navier-Stokes solution, it's clear [^1] that they learned that Alpöge and Buckmaster were getting close to a solution and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt. What makes this worse to me is the intention. They intentionally threw $15 million in compute at the problem in order to scoop the result. They intentionally left Buckmaster and Alpöge out of the citations. Data contamination should be enough to disqualify them from the prize, but I can believe it to be accidental. On the other hand, someone made an intentional decision to scoop the result by throwing money at the problem. That's so much worse. [^1]: That's the timeline claimed by Buckmaster, and no one from OAI has disputed it.

unified101 6d ago
> the secret So such thing existed. In fact, what they learnt was some progress existed, not what the specific progress was.
fwlr 6d ago
I think you’re overlooking what I’m implying here. It’s not that they knew contamination was possible but they went ahead anyway. To spell it out just a little bit more: learning the answer might be in model X’s training data made them believe that model X specifically might be able to solve the question, and they were able to very quickly find enough certainty about the former to commit millions of dollars to the latter.
square_usual 6d ago
> and learned of the general approach they were taking. Only after learning the secret to cracking the problem did they send the first prompt. Do you have any evidence of this? They don't dispute the timeline, but they never said they knew what Levant/Buckmaster were doing.
- robotpepi 6d ago
  It's in OpenAI's first announcement that they had solved the problem.
  - derangedHorse 6d ago
    > Only after learning the secret to cracking the problem did they send the first prompt. Which quote in the announcement post provides evidence for the above quote?
    - OneManyNone 6d ago
      “ On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model, we launched an effort to evaluate it on all open Millennium Prize problems and a few other high-impact problems.” - https://openai.com/index/navier-stokes-solution/ https://openai.com/index/navier-stokes-solution/ They do not explicitly admit to knowing about NS specifically, but are extremely explicit that they tried to scoop some potential millennium prize winners.
      - randomblock1 6d ago
        So then they DIDN'T "learn the secret to cracking the problem". They simply knew that part of the problem was solved. Knowing a problem can be solved and knowing the solution are not the same thing.
        freejazz 6d ago
freejazz 6d ago
> but I can believe it to be accidental What accident is it when the system is designed to function that way?
- Lerc 6d ago
  Their claim is that training on their solution is "unlikely but possible". Consider this scenario. Has a google crawler read my new novel, which I may or may not have posted on my blog, page by page, as I wrote it? Can you, without knowledge of what I have actually done, claim that the google crawler has not seen the novel? Without any evidence that I have posted the novel online, it might be tempting to say that the crawler has not seen the novel, but what if I were in an adversarial position against Google on this topic and were challenging them to make that claim. You would wonder if I were hoping Google to overreach by making a definitive claim without taking into account some action that they had no knowledge of. It becomes difficult to use the scientific expression "There is no evidence for this" when there is an accusation of malfeasance because it can be so easily be conflated as "You can't prove we did it". It seems like the best you could say would be 'Unlikely, but possible'
  - freejazz 6d ago
    I'm not taking them at their word, sorry. Genuinely, there is no reason to.
lnrd 6d ago
> They intentionally threw $15 million in compute at the problem what? really?
- abathologist 6d ago
  Yes. Maybe much more: > Such intensive use of AI doesn't come cheap. In a post on X, LisanBench, an LLM benchmark evaluator, estimated that the output tokens alone would cost about $6.5 million at OpenAI's average consumer price. Including the far larger volume of input tokens, the post estimated the total could reach $10 million to $40 million. https://www.businessinsider.com/openai-math-problem-solved-tokens-cost-altman-2026-9 https://www.businessinsider.com/openai-math-problem-solved-t...
  - hyperbovine 6d ago
    But think of all the IPO Monopoly money they just generated.
  - square_usual 6d ago
    That's their API pricing. There's no way they actually paid $15M in compute. I'd say much more likely it's in the order of $1M.
    - malfist 6d ago
      Who are you who is so wise in the ways of a private company's internal cost accounting
- dekhn 6d ago
  When I worked at Google, we spent $100M in power on protein folding and drug discovery (this was long before AlphaFold). Never underestimate the willingness of smart rich people to invest in speculative science.
nomel 6d ago
> They intentionally left Buckmaster and Alpöge out of the citations. No, they asked if they could do a joint publish.
- oefrha 6d ago
  No, they asked one guy to do a joint publish conditioned on leaving the other collaborator out, with veiled threats. The joint publish part smells awfully like admission of guilt given there’s absolutely no reason to do it if you believe you independently arrived at the result using only public prior work. The leaving out collaborator part is outright academic malpractice. Disclosure: I was an academic once.
  - golly_ned 6d ago
    To add: with a requirement that he rewrite the proof to credit OpenAI.
  - derangedHorse 5d ago
    This would be a new publication with OpenAI's novel solution. Buckmaster and Alpöge did not arrive at the NS solution, they used an approach that also happened to be used in OpenAI's proof. > there’s absolutely no reason to do it if you believe you independently arrived at the result using only public prior work Unless the goal is to assuage the existential pain felt by many mathematicians with respect to what may feel like increasingly inconsequential efforts. It also seemed like a way to build good will amongst knowledge workers dealing with similar issues, with the goal of showing they are dedicated to easing the transition for them as society parades into the future. This has clearly backfired. > The leaving out collaborator part is outright academic malpractice. They would not be leaving a collaborator out. Again, this would be a new publication. Here's the quote from Buckmaster's post [1]: "Two proposals were offered to me. The first was that we post our Euler result, and that OpenAI post its Navier-Stokes result the next day. The second was that, after posting Euler, I alone write a paper presenting the Navier-Stokes result, acknowledging that an internal OpenAI model had resolved it." [1] https://cims.nyu.edu/~tristanb/statement.pdf https://cims.nyu.edu/~tristanb/statement.pdf