8 ms·

It includes Terrance Tao who has made the front page several dozen times at this point for his usage of AI such as to help the community write proofs for Erdos

by qlte 5d ago

It includes Terrance Tao who has made the front page several dozen times at this point for his usage of AI such as to help the community write proofs for Erdos problems. But to many commentors he's a now gatekeeping AI-hating Luddite clinging to a dying profession out of bitterness and envy because his position is more nuanced than "throw AI at everything and turn off your brain".

deleted 5d ago
[deleted]
zozbot234 5d ago
Because his letter misuses the word "misalignment" for what's very clearly a capability gap. To anyone familiar with that sort of language, his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians, which is exactly what the AI-hating Luddite would say! The reality is closer to "current frontier AIs are clueless about what makes mathematical problems/proofs interesting to humans" which is a vastly different issue.
- necklesspen 4d ago
  LLMs are tools. The responsibility lies in the human(s) using the tool to act accordingly.
- jhrmnn 4d ago
  It’s actually a fairly precise use of the word, just not in the very specific and narrow sense in which it has been used for AI.
- freehorse 4d ago
  > his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians That is probably the wildest misinterpretation of sth I have ever read here. Misalignment is used as in the goals and interests of ai COMPANIES are not the same as the ones of the mathematical community. It is very proper use of the word, and if anything imo the properest, as it refers to actual people and institutions to which actual self-ascribed goals can be defined, as in contrast to hypothetical superintelligence. AI doomers do not own some trademark on the word "alignment".
  - zozbot234 4d ago
    The point stands whether you attribute agency to AIs themselves or to AI companies. The companies are not deliberately sabotaging human mathematical understanding by writing up purposely inscrutable results, either (which is what the word 'misalignment' would imply in this context): they are merely working with frontier tools that have a very limited capability for human-like understanding. It's left to human mathematicians to bridge that particular gap.
    - omnicognate 4d ago
      You seem determined not to understand this, but to spell it out even further: The point is not that the companies are deliberately trying to harm maths but that in pursuing their own goals, which are different to those of mathematicians, they are doing so. Their goals and their methods of achieving them are not aligned with those of mathematicians. This is, incidentally, exactly what alignment means in AI as well: a paperclip maximiser harms humans not because it is intentionally trying to do so but as a side effect of trying to achieve its own objectives, which are not aligned with those of humans (which include not having their entire environment turned into paperclips). The word is being used entirely correctly, and I'm sure the nod to the AI usage is quite deliberate.
      - zozbot234 4d ago
        > Their goals and their methods of achieving them are not aligned with those of mathematicians. This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is the issue that AI companies are willfully refusing to provide mathematical insight that they could provide (because they have diverging underlying "goals" to those of human mathematicians) or are they merely working under a capability gap, where current AIs can awkwardly settle major open questions but are not smart enough to provide the kind of understanding and insight that the community of human mathematicians relies on? These are two very different problems and by foregrounding the word "misalignment" in his letter so openly (as opposed to talking about AI capability to provide valued insight), Tao is picking the more adversarial reading with zero proof or motivation.
pred_ 4d ago
It's the same on Reddit, and if you look up a few comment histories, you'll find that it's mostly /r/singularity users and /r/accelerate posters that are spamming. So I assume that people are just deliberately misreading the letter and trolling here as well, and I wouldn't read too much into it, but on the other hand, if you never had to think about what scientific misconduct looks like, it's harder to see why the behavior of the AI companies is problematic.
- qingcharles 4d ago
  They're not deliberately misreading it, they are simply just not reading it and just assuming they know what it says.
vanderZwan 4d ago
It kind of reminds me of Clay Shirky's predictions about the influence of the internet on society from a few decades ago. To paraphrase what he said: when the printing press was invented, everyone predicted it would lead to world peace, and we got the Thirty Years War instead. When modern mass media was invented people predicted world peace, and instead the Nazis rose to power and we got World War 2. And yet as a society we typically consider these technologies a net positive these days, because alongside all the political instability and violence that followed them at first, some people were figuring out how to do beneficial things with them along the way that did more good than bad. Because the issue really was that the influx of more voices and ideas shifted the power dynamic, required relearning how to communicate, and in the end be better as a society. The issue was figuring out how to "domesticate" these wild new communication channels. One successful example of which was the invention of scientific journals and papers. Shirky predicted the internet would probably lead to a few decades of political instability too (about fifty years was his guess), and we definitely seem to be in the middle of that process. Now, I haven't checked his stance on LLMs. I also don't know if I would quite call them a medium for mass communication like the others (as used today, they take humans out of the loop rather than let more voices join the public discourse), but I feel like they are similar enough to otherwise fit the pattern. Tao's stance similarly feels about wanting to domesticate this wild animal before we get mauled by it. And to stick with the metaphor, I feel that most of the time the AI industry is trying to bamboozle us with spectacular rodeo displays instead.

by qlte 5d ago

deleted 5d ago

[deleted]

zozbot234 5d ago

Because his letter misuses the word "misalignment" for what's very clearly a capability gap. To anyone familiar with that sort of language, his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians, which is exactly what the AI-hating Luddite would say! The reality is closer to "current frontier AIs are clueless about what makes mathematical problems/proofs interesting to humans" which is a vastly different issue.

necklesspen 4d ago
LLMs are tools. The responsibility lies in the human(s) using the tool to act accordingly.
jhrmnn 4d ago
It’s actually a fairly precise use of the word, just not in the very specific and narrow sense in which it has been used for AI.
freehorse 4d ago
> his prose is directly implying that evil superintelligent AIs are deliberately writing obscure proofs in order to harm the community of human mathematicians That is probably the wildest misinterpretation of sth I have ever read here. Misalignment is used as in the goals and interests of ai COMPANIES are not the same as the ones of the mathematical community. It is very proper use of the word, and if anything imo the properest, as it refers to actual people and institutions to which actual self-ascribed goals can be defined, as in contrast to hypothetical superintelligence. AI doomers do not own some trademark on the word "alignment".
- zozbot234 4d ago
  The point stands whether you attribute agency to AIs themselves or to AI companies. The companies are not deliberately sabotaging human mathematical understanding by writing up purposely inscrutable results, either (which is what the word 'misalignment' would imply in this context): they are merely working with frontier tools that have a very limited capability for human-like understanding. It's left to human mathematicians to bridge that particular gap.
  - omnicognate 4d ago
    You seem determined not to understand this, but to spell it out even further: The point is not that the companies are deliberately trying to harm maths but that in pursuing their own goals, which are different to those of mathematicians, they are doing so. Their goals and their methods of achieving them are not aligned with those of mathematicians. This is, incidentally, exactly what alignment means in AI as well: a paperclip maximiser harms humans not because it is intentionally trying to do so but as a side effect of trying to achieve its own objectives, which are not aligned with those of humans (which include not having their entire environment turned into paperclips). The word is being used entirely correctly, and I'm sure the nod to the AI usage is quite deliberate.
    - zozbot234 4d ago
      > Their goals and their methods of achieving them are not aligned with those of mathematicians. This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is the issue that AI companies are willfully refusing to provide mathematical insight that they could provide (because they have diverging underlying "goals" to those of human mathematicians) or are they merely working under a capability gap, where current AIs can awkwardly settle major open questions but are not smart enough to provide the kind of understanding and insight that the community of human mathematicians relies on? These are two very different problems and by foregrounding the word "misalignment" in his letter so openly (as opposed to talking about AI capability to provide valued insight), Tao is picking the more adversarial reading with zero proof or motivation.

pred_ 4d ago

It's the same on Reddit, and if you look up a few comment histories, you'll find that it's mostly /r/singularity users and /r/accelerate posters that are spamming. So I assume that people are just deliberately misreading the letter and trolling here as well, and I wouldn't read too much into it, but on the other hand, if you never had to think about what scientific misconduct looks like, it's harder to see why the behavior of the AI companies is problematic.

qingcharles 4d ago
They're not deliberately misreading it, they are simply just not reading it and just assuming they know what it says.

vanderZwan 4d ago

It kind of reminds me of Clay Shirky's predictions about the influence of the internet on society from a few decades ago. To paraphrase what he said: when the printing press was invented, everyone predicted it would lead to world peace, and we got the Thirty Years War instead. When modern mass media was invented people predicted world peace, and instead the Nazis rose to power and we got World War 2. And yet as a society we typically consider these technologies a net positive these days, because alongside all the political instability and violence that followed them at first, some people were figuring out how to do beneficial things with them along the way that did more good than bad. Because the issue really was that the influx of more voices and ideas shifted the power dynamic, required relearning how to communicate, and in the end be better as a society. The issue was figuring out how to "domesticate" these wild new communication channels. One successful example of which was the invention of scientific journals and papers. Shirky predicted the internet would probably lead to a few decades of political instability too (about fifty years was his guess), and we definitely seem to be in the middle of that process. Now, I haven't checked his stance on LLMs. I also don't know if I would quite call them a medium for mass communication like the others (as used today, they take humans out of the loop rather than let more voices join the public discourse), but I feel like they are similar enough to otherwise fit the pattern. Tao's stance similarly feels about wanting to domesticate this wild animal before we get mauled by it. And to stick with the metaphor, I feel that most of the time the AI industry is trying to bamboozle us with spectacular rodeo displays instead.