5 ms·
OpenAI fought dirty on career-making math problem
- nairboon 10d agoStatement from Tristan: https://news.ycombinator.com/item?id=49605915 https://news.ycombinator.com/item?id=49605915 Statement from OpenAI: https://news.ycombinator.com/item?id=49613262 https://news.ycombinator.com/item?id=49613262
- cwillu 10d agoWho on earth reads 2 pages of an all-text article, and decides at that point “you know what, I want to watch a video; oh good, here is the article in video form, just when I needed it!”?
- esseph 10d agoMe? I can read the 2 pages in ~30s or so. How long is the video? 15m? Nah. Edit: video is 37 minutes
- ahahahahah 10d agoYou couldn't even read and correctly comprehend the very short comment you replied to.
- esseph 10d agoahahahahah
- cmiles8 10d agoOpenAI has a lot of explaining to do here. The “least worst” representation of the course of events is that their researchers are just jerks, and not behaving in a manner that’s considered acceptable in the research community. Other alleged scenarios just go downhill from there.
- jdm2212 10d agoIt is worth actually reading OpenAI's response, which is basically that they were trying to see if their models could do what Anthropic had already done, and were surprised to discover that (a) Anthropic had not done it at all, and (b) in fact no one else had done it yet. And then they didn't want to give an Anthropic employee coauthor credit for work that OpenAI had done, which idk seems pretty fair?
- deleted 10d ago[deleted]
- johnnyApplePRNG 10d ago>were surprised to discover that (a) Anthropic had not done it at all They were surprised another competitor had not wastefully thrown $20 million+ worth of compute at a problem they had no business solving in the first place? That's reaching, imho.
- gorgolo 10d agoWhy is it wasteful to solve one of the most famous problems in modern mathematics? And why would one particular company have no business solving it in the first place?
- johnnyApplePRNG 10d agoThey brute forced it, that proves they had no business solving it. They didn't know what they were doing and they pumped a bunch more carbon into our atmosphere because hubris basically. Shame on them.
- babelfish 10d agoEvery sentence in this comment is incorrect
- 9d ago
- dumberquestions 10d agoThe most sympathetic interpretation possible of the events for OAI is that they learned two mathematicians were closing in on a solution and decided to throw all of their weight behind getting there first, which honestly still doesn’t paint them in a particularly positive light.
- jdm2212 10d agoThe most sympathetic interpretation is just what OpenAI actually claims: they thought the other guys already got there, wanted to see if OpenAI could do it too, and were surprised to discover the other guys hadn't gotten there yet.
- DrewADesign 10d agoI don’t have a dog in this fight, but it sounds a bit “I was just punching the air— it’s not my fault someone was in the way.” It’s not like it’s impossible, but it doesn’t immediately pass my smell test. I’ll wait until someone close enough to be knowledgeable but has a lesser stake weighs in.
- jdm2212 10d agoWhy would it even be bad for OpenAI to try to beat Anthropic to cracking one of the 6 biggest open problems in math? Like who is getting punched?
- DrewADesign 10d agoI mean it sounds like a retroactively contrived explanation of events. If they released a statement on it, they clearly feel the need to give an explanation of their behavior. People criticize OpenAI every day and they rarely address it. They addressed this. That right there says that, at the very least, they’re concerned about the optics.
- jdm2212 10d ago
- spockz 10d agoSee also https://x.com/roelof_vandijk/status/2097219223470629038 https://x.com/roelof_vandijk/status/2097219223470629038 So sad that the thread on OpenAI claiming to have solved it first gets so much attention while the posts of Tristan get snowed under.
- EA-3167 10d agoShocking, I really expected more from the "totally legitimate startup" planning a $2 trillion IPO with their bottomless money pit. I genuinely assumed that the company credibly accused of stealing from Apple in the most ham-fised way possible would have some kind of guiding ethical principles. At the very least the paragon of decency that is Sam Altman would have stopped this. Get ready, bag-holders on index-tracking funds... you're about to lose your shirts.
- 9864325789976 10d agoWhat index tracking funds will cause investors to lose their shirts? I can't wait to see your short position that will make you rich enough to retire.
- EA-3167 10d agoYou don't remember SpaceX fighting to get fast-tracked on the S&P? Well you are only a day old I suppose.
- sublinear 10d ago> It is not the direction one arrives at in a few days by giving a model the problem statement. Such a satisfying quote.
- machina_ex_deus 10d agoBeware everyone working on ground breaking research, keep your research secret from openAI or they might spend 22$ million worth of tokens just to beat you to the finish line, while possibly abusing your user data for training.
- incognition 10d agoThe guy is working with an anthropic researcher and they're not using Claude. It just doesn't make sense.
- tmsh 10d agoEven through the lack of ethics here and there and mistakes - I think the major headline is that talented people are collaborating with AI to achieve impressive results. It’s shrouded in competitive mistakes of judgment. But it’s an existence proof for collaborating with AI and achieving incredible things.
- dist-epoch 10d agoOpenAI says they just pointed the AI at the problem: > Regarding the level of human involvement on our end: although a group of people was involved in our efforts, we collectively had no research-level expertise in fluid dynamics and the Navier-Stokes problem, and therefore were unable to meaningfully contribute to the mathematical content. https://x.com/SebastienBubeck/status/2097379415747342689 https://x.com/SebastienBubeck/status/2097379415747342689
- tmsh 10d agoThey needed the general direction and idea it might work from a few folks tinkering for a year with the same models that we all have access to. The majority of credit goes to them. The idea of where to look with AI is key. Models didn't do that by themselves. Trust: https://x.com/__alpoge__ https://x.com/__alpoge__ "since on my side things were mostly me and claude having a good time yoloing random stuff in the corner rather than anything institutional" Not: https://x.com/SebastienBubeck https://x.com/SebastienBubeck "If you don't want me to be nice, then I don't have to be nice." [when clearly there's not enough deference paid to where this line of research originated]
- tmsh 9d agoAlso: https://cims.nyu.edu/~tristanb/statement.pdf https://cims.nyu.edu/~tristanb/statement.pdf I had planned to say on announcing our work that the results are not the important thing. Rather the important thing is instead the significance that a mathematician and an LLM model can now do all this work in a month
- deleted 10d ago[deleted]
- biophysboy 10d agohttps://mathstodon.xyz/@tao/117237320796901560 https://mathstodon.xyz/@tao/117237320796901560 Terrence Tao recently published an interesting take that zooms out from the details of the Navier Stokes drama. Its an interesting observation he makes, because it is not dissimilar from the relatively common phenomenon of one academic lab getting scooped by another lab (usually by coincidence).
- moregrist 10d ago> one academic lab getting scooped by another lab (usually by coincidence). I was with you up to “usually by coincidence”. There’s a long and sordid history in areas of chemistry and areas of biology of holding up a competing paper in review so you can scoop them. I’m sure it exists in physics as well. Certainly biophysics, but probably most subfields. Often it’s a famous labs that can steamroll review or even just dump the work into PNAS as a “member contribution.” At least one author of a famous inorganic chemistry textbook was rumored to do this routinely. And I know of at least one National Academy member who swore off arxiv prepublication after getting scooped. None of this makes it all right. But plagiarism and academic theft is old and definitely not always accidental.
- biophysboy 10d agoI know this happens, but my impression during my phd was that many labs use popular methods to test popular questions, leading to a lot of simultaneous work. You see this in history as well.
- evilturnip 10d agoThis is interesting. He seems to imply that the AI will just provide the solution. He's concerned that the process of getting to that solution is the important bit. But I can see two versions of "process". A.) The actual steps of the proof, which I assume the AI would provide. B.) People, while working toward a solution, finding novel properties/methods along the way that open new avenues of research + new open problems. Does B.) actually happen? Would knowing the solution to a problem stymie the process of finding new open questions? I would assume finding a solution may unlock other problems too. So maybe on balance it's not really bad? I guess in the end, I'm just making the obvious case "the future is uncertain in the face of AI".
- hatthew 10d agoI feel like this whole thing hinges on one point. OAI says[0]: > However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced). How true is this? If the implied statement is true (i.e. OAI couldn't have stolen results because the results were so different anyways), then I feel like it's pretty clear that OAI solved the problem on their own, and offering some amount of credit to Tristan and Levent is generous. Though on the other hand it's dirty to have even attempted a scoop in the first place. If the implied statement is false, and Tristan/Levent's results are a substantial portion of the solution to the millennium problem, then it's probably unintentional but clear plagiarism. I think it's plausible to assume that OAI has trained their models on Tristan's codex conversations, and so regardless of legal ownership, the academic ownership definitely includes Tristan and Levent. The rest of the drama (individual statements and wordings, e.g. by Sebastien) seems like a bit of a red herring. Worth noting, but not worth basing conclusive judgements on regarding academic misconduct. Regardless, OAI does not seem like the good guys. [0]: https://openai.com/index/navier-stokes-solution/ https://openai.com/index/navier-stokes-solution/
- Hansenq 10d agoAgree! We have both published solutions now; someone (AI?) should be able to analyze their methods and see how similar they are
- ygt2 10d ago[dead]
- quicklywilliam 10d agoAgree with this framing, but we may find out that the answer is somewhere in the middle. Personally I think it is extremely unlikely that this is straight plagiarism in the sense of the model simply regurgitating training data from prior work by Tristan, but it is very plausible that his work (and others’) was foundational to the breakthrough. What is unfortunate is that the stakes are so high (and no I don’t mean $1M) and the timeline is so compressed. I expect a lot more of this kind of drama in the near future.
- qwerty_clicks 10d agoOpen ai will do everything it can to convince us it’s unlikable and fulling an optional worse version of AI that we don’t want the world to be. Altman is so Zuckey
- golly_ned 10d agoWho is "Bubeck"? The article doesn't introduce him. Or give his name. Same with "Luis" and "Diego". I am supposing it is https://en.wikipedia.org/wiki/S%C3%A9bastien_Bubeck https://en.wikipedia.org/wiki/S%C3%A9bastien_Bubeck This is terrible: When Buckmaster pushed to make the dispute public, he says that Bubeck replied: “Why would you ruin your career?” Buckmaster says that when he pushed back, Bubeck followed up with: “If you don’t want me to be nice, then I don’t have to be nice.”
- gcr 10d agoThe report pdf confirms Sebastian was the Bubeck in question
- DetroitThrow 10d agoGiven that he has other former collaborators corroborating this horrific behavior, it seems like this a career spanning pattern, and it's interesting to see just how much @sama is willing to lend his support to someone like Bubeck. Stains an important moment in the history of AI progress for me. The future seems bleak with people like this at the reins. https://x.com/dheeraj_nagaraj/status/2097266146445774924 https://x.com/dheeraj_nagaraj/status/2097266146445774924
- enraged_camel 10d agoI think the context is really important. OpenAI has been behind in the AI race since last November. They have been playing catch-up. They recently released Astra and declared it is AGI. Now they are desperate for anything they can use as evidence for that claim. In other words: they had clear motive to do anything they could to steal the glory from prominent mathematicians who worked hard on this problem and solved it. And they also cannot prove that said mathematician's data was not accessed by either the model or the OAI users prompting the model.
- mortar 10d agohttps://archive.is/9Gv11 https://archive.is/9Gv11
- skepticATX 10d agoThe saddest part of this is no one actually cares about the proof itself. Does it prove what it claims to prove? Were new mathematics invented? Can this be applied to other areas? I don’t have a problem with labs solving hard problems if they can, but they could at least pretend to care more about the problems and less about the marketing opportunity.
- pinkmuffinere 10d agoSetting aside the disagreement, I was very interested to see the net pricing of the discovery: > All told, the week-long effort consumed 300 billion output tokens — $22.5 million worth of compute, if charged at current Astra rates. > The Navier-Stokes existence and smoothness problem is one of the seven Millennium Prize problems — a set of major unsolved math problems, each carrying a $1 million bounty I know openAI isn't solving these problems in order to make profit, but it's interesting to guess how close we are to these things becoming profitable. Eg, if you think their public pricing for Astra is ~2x as expensive as their internal price, then they lost ~10M on net for this proof. That's not profitable, but it is much better than I would have expected, which is exciting for the other Millennium prize problems! Of course the fundamental approach (which they may have plagiarized from Buckmaster and Alpoge) might have added cost to that as well. Nonetheless, I wouldn't be too surprised if they're all solved within the next 3 years!
- qznc 10d agoI would assume they spent similar amounts on the other Millennium problems too. Also, they probably tried it before with older models.
- timschmidt 10d ago> 300 billion output tokens — $22.5 million worth of compute, if charged at current Astra rates. This makes the 100 billion tokens (total in/out) I've spent on my project on the $200/mo plan seem like a deal. Wow.
- pinkmuffinere 10d agoWow I wonder how much of that differential is them operating at an extreme loss, and how much is the relative cost of their new models vs old. If a significant portion of that is net losses, I'm very concerned for their business model :|
- skepticATX 10d agoThis discounts the human breakthroughs needed to get to this point, the cost of the employees working in it, previous attempts, and more.
- addandsubtract 10d agoI was told their models were SAFE and they focus on SAFETY. Why would they backstab us like this? If not even OpenAI can be safe, who then?! I think it's time we ban open models so this doesn't happen to anyone else.
- 627467 10d agoI for one am excited to see the return of secret and hermetical societies to counter this idea of information scrapping