8 ms·

> Their goals and their methods of achieving them are not aligned with those of mathematicians. This is exactly the assumption that Tao is smuggling in with "m

> Their goals and their methods of achieving them are not aligned with those of mathematicians. This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is the issue that AI companies are willfully refusing to provide mathematical insight that they could provide (because they have diverging underlying "goals" to those of human mathematicians) or are they merely working under a capability gap, where current AIs can awkwardly settle major open questions but are not smart enough to provide the kind of understanding and insight that the community of human mathematicians relies on? These are two very different problems and by foregrounding the word "misalignment" in his letter so openly (as opposed to talking about AI capability to provide valued insight), Tao is picking the more adversarial reading with zero proof or motivation.

omnicognate 5d ago
His view is the "capability gap" one, and you can just read the the statement and Tao's other writings for far more eloquent explanations of the difference between a raw LLM proof dump and real mathematical insight, and what it takes to get from the former to the latter, than I can provide. Again, nobody is accusing anyone of deliberately trying to harm mathematics. It's about misaligned objectives. Eg. Tao wrote the following before the Navier-Stokes announcement (referring to exactly the project OpenAI was undertaking): > At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task. But such an exercise does not particularly hold my interest; I am far more interested in digesting the proof methods and extracting out the key new insights uncovered by this approach. And then went further to say that such activity could be actively harmful to the field. (All this can be read on Mathstodon: https://mathstodon.xyz/@tao https://mathstodon.xyz/@tao). The misalignment is that this activity that he and 24 other Fields medalists think is actively harmful to their field is deemed by OpenAI and others to be worth ploughing vast financial, human and compute resources into.
- zozbot234 4d ago
  > His view is the "capability gap" one That's the far more sensible reading, so thanks for confirming I guess. But then the misalignment talk is pretty clearly a distraction. > ...And then went further to say that such activity could be actively harmful to the field. If true (and there is as of yet insufficient evidence of this), that's merely a contingent fact about very real institutional misalignment within the human mathematical community, not about AI itself or even AI frontier labs. There's simply zero inherent reason why providing a bare truth value or a completely inscrutable proof about the status of some open conjecture should make that entire subfield of math "contaminated" for the foreseeable future when it comes to extracting further human-relevant insight. That's the misalignment we should be caring about.
  - SpicyLemonZest 4d ago
    The declaration is about that institutional misalignment. They want to change the norms of the mathematical community, so that producing inscrutable proofs is a low-value activity nobody cares much about rather than a high-value activity that AI frontier labs can make headlines by performing. Their ask of the frontier labs is to please be aware of the problems with the old norms and not exploit them during the transition period; they agree this doesn't have much to do with AI itself, which they acknowledge is a powerful technology that will improve and accelerate mathematical research.
    - zozbot234 4d ago
      Broadly agreed, with a key proviso: producing inscrutable proofs has negligible value as a mathematician's finished output but that doesn't make it a "low-value activity" in and of itself. Ultimately, the status of these proof-like objects as a raw input into mathematical practice will probably be comparable to any other sort of computationally-driven https://en.wikipedia.org/wiki/Experimental_mathematics https://en.wikipedia.org/wiki/Experimental_mathematics . These are not new problems: "computer" used to be a job description for humans before it was the name of a machine, but we now view raw computations as a trivial matter that's not worthy of any human credit.
      - 4d ago
freehorse 4d ago
It is really not complicated at all. And if tao's post is a bit vague, the letter signed by many mathematicians is imo very clear. AI can be used to advance/deepen understanding, or it can be used to superficially go settle a whole bunch of open problems in a field without helping really in understanding them. It all depends on who uses the AI and why. Essentially, it is exactly the same concept as using the AI as a course tutor vs having it do your homework. Or using the AI to write millions of lines of code that nobody can actually read, vs keeping overview of what is going on. In math it is probably worse because there is no objective function to maximise. Some people here think that the objective function of mathematics is to prove things, which is actually wrong. Mathematicians are not only maximising an objective function, they are also defining the objective function they need to maximise (they are defining which problems to study). The problem with AI/ML is that it can be pretty good when the goal is to maximise a set objective function, but not to set intentions and goals themselves. I would not call that a "capability gap" because we can actually get to have very useful and smart AI systems without ever reaching that point.
deleted 4d ago
[deleted]

omnicognate 5d ago

His view is the "capability gap" one, and you can just read the the statement and Tao's other writings for far more eloquent explanations of the difference between a raw LLM proof dump and real mathematical insight, and what it takes to get from the former to the latter, than I can provide. Again, nobody is accusing anyone of deliberately trying to harm mathematics. It's about misaligned objectives. Eg. Tao wrote the following before the Navier-Stokes announcement (referring to exactly the project OpenAI was undertaking): > At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task. But such an exercise does not particularly hold my interest; I am far more interested in digesting the proof methods and extracting out the key new insights uncovered by this approach. And then went further to say that such activity could be actively harmful to the field. (All this can be read on Mathstodon: https://mathstodon.xyz/@tao https://mathstodon.xyz/@tao). The misalignment is that this activity that he and 24 other Fields medalists think is actively harmful to their field is deemed by OpenAI and others to be worth ploughing vast financial, human and compute resources into.

zozbot234 4d ago
> His view is the "capability gap" one That's the far more sensible reading, so thanks for confirming I guess. But then the misalignment talk is pretty clearly a distraction. > ...And then went further to say that such activity could be actively harmful to the field. If true (and there is as of yet insufficient evidence of this), that's merely a contingent fact about very real institutional misalignment within the human mathematical community, not about AI itself or even AI frontier labs. There's simply zero inherent reason why providing a bare truth value or a completely inscrutable proof about the status of some open conjecture should make that entire subfield of math "contaminated" for the foreseeable future when it comes to extracting further human-relevant insight. That's the misalignment we should be caring about.
- SpicyLemonZest 4d ago
  The declaration is about that institutional misalignment. They want to change the norms of the mathematical community, so that producing inscrutable proofs is a low-value activity nobody cares much about rather than a high-value activity that AI frontier labs can make headlines by performing. Their ask of the frontier labs is to please be aware of the problems with the old norms and not exploit them during the transition period; they agree this doesn't have much to do with AI itself, which they acknowledge is a powerful technology that will improve and accelerate mathematical research.
  - zozbot234 4d ago
    Broadly agreed, with a key proviso: producing inscrutable proofs has negligible value as a mathematician's finished output but that doesn't make it a "low-value activity" in and of itself. Ultimately, the status of these proof-like objects as a raw input into mathematical practice will probably be comparable to any other sort of computationally-driven https://en.wikipedia.org/wiki/Experimental_mathematics https://en.wikipedia.org/wiki/Experimental_mathematics . These are not new problems: "computer" used to be a job description for humans before it was the name of a machine, but we now view raw computations as a trivial matter that's not worthy of any human credit.
    - 4d ago

freehorse 4d ago

It is really not complicated at all. And if tao's post is a bit vague, the letter signed by many mathematicians is imo very clear. AI can be used to advance/deepen understanding, or it can be used to superficially go settle a whole bunch of open problems in a field without helping really in understanding them. It all depends on who uses the AI and why. Essentially, it is exactly the same concept as using the AI as a course tutor vs having it do your homework. Or using the AI to write millions of lines of code that nobody can actually read, vs keeping overview of what is going on. In math it is probably worse because there is no objective function to maximise. Some people here think that the objective function of mathematics is to prove things, which is actually wrong. Mathematicians are not only maximising an objective function, they are also defining the objective function they need to maximise (they are defining which problems to study). The problem with AI/ML is that it can be pretty good when the goal is to maximise a set objective function, but not to set intentions and goals themselves. I would not call that a "capability gap" because we can actually get to have very useful and smart AI systems without ever reaching that point.

deleted 4d ago

[deleted]