6 ms·

> scientists who have spent their lives studying this Please point me to one actual accredited scientist who has spent a lifetime studying AI alignment? Pretty

> scientists who have spent their lives studying this Please point me to one actual accredited scientist who has spent a lifetime studying AI alignment? Pretty much this whole field is only 5 years old

adamzenith 7d ago
The field is much older, MIRI is ~20 years old. Look up Eliezer Yudkowsky.
- swiftcoder 7d ago
  The field was purely theoretical 20 years ago, and Yudkowsky is pretty much the dictionary definition of "not accredited"
  - naishoya 6d ago
    some use "not accredited" as a pejorative term. Lets not forget that the 'Fermat's Last Theorem' which has been pretty visible for the non-math crowd of late due to the recent AI frenzy about a purported proof was but one small contribution to the world's math lexicon by someone with a bachelors degree in civil law, that George Green was a baker and millwright, Boole was the son of a poor shoemaker in England with no formal university education and left school at age 14. Oliver Heaviside was a telegraph operator, and Michael Faraday was an apprentice bookbinder. So, not accredited shouldn't really carry much weight when it comes to mathematics. Lets not pretend that machine learning and the narrow branch that is the current approach to LLM inductions is anything but applied math. We might exercise our own minds and actually read the works and writings of a person, and use that as a measure of knowledge and perspective. Not all PhD dissertations are equal, and many have comprehension and ability to move us forward even without the institutional rigour. For those who prefer to have easy access to citations, here are some relevant papers that are not "Harry Potter" related, some with coauthors from Oxford University. Cognitive Biases Potentially Affecting Judgment of Global Risks [https://intelligence.org/files/CognitiveBiases.pdf https://intelligence.org/files/CognitiveBiases.pdf] Levels of Organization in General Intelligence [https://intelligence.org/files/LOGI.pdf https://intelligence.org/files/LOGI.pdf] Corrigibility [https://intelligence.org/files/Corrigibility.pdf https://intelligence.org/files/Corrigibility.pdf] The Ethics of Artificial Intelligence [https://intelligence.org/files/EthicsofAI.pdf https://intelligence.org/files/EthicsofAI.pdf]
    - swiftcoder 6d ago
      > some use "not accredited" as a pejorative term. I am absolutely using "not accredited" in a pejorative sense here. That he publishes papers coauthored by a couple of philosophy professors at Oxford (all of whom have made a ton of money from the Silicon Valley AI Alignment and Effective Altruism crowds) does not make him a scientist. I will also note that the Oxford Philosophy department finally shitcanned the whole Future of Humanity Institute a couple of years back.
      - naishoya 5d ago
        ok, that is informative and much more substantial than calling him 'not accredited.' I appreciate the clarification. That his writings are unscientific, and that the accredited professionals who were his coauthors have lost tenure from Oxford when an old email from the FHI Bostrom used a racial slur and claimed Black people were intellectually inferior and his 'apology' included a defense of eugenics that deeply alarmed the university community and triggered a formal investigation, are two details not immediately visible from a brief review of his published opinions. These are far more damning indictments than simply categorization of the author as lacking in formal education. Thank you for the opportunity to look more deeply at the situation and come an understanding of the consequences of taking his writings seriously.
- Alwayshasbeeb 7d ago
  Eliezer Yudkowsky is not a scientist. He made a popular Harry Potter fanfiction series and a "rationality" blog-community that attracts "human biological diversity" enthusiasts.
  - bbor 6d ago
    Elizer Yudkowsky is a scientist, despite writing something you dislike 20 years ago and also having a blog. And no, he's not associated with race realism, that's just baseless libel.
- orbital-decay 6d ago
  Yes, he's the exact reason people are distrustful. He's a crank who learned about reward hacking and made a new religious movement out of it, pretending it's a world-ending issue and deliberately avoiding much more serious issues like the concentration of power. Typical cult leader and manipulator, and his disciples in charge of major AI shops aren't any better.
  - bbor 6d ago
    What evidence do you have that LessWrong is a cult in any way the HackerNews is not? If he's a cult leader, he's awfully bad at it. No opulence, no compounds, no dogma, no doctrine...
bbor 6d ago
I... that's hard to even know how to respond to. If you think the field is 5 years old, you should not feel comfortable expressing opinions about it. A cursory search of the relevant wikipedia articles would do you wonders.

adamzenith 7d ago

The field is much older, MIRI is ~20 years old. Look up Eliezer Yudkowsky.

swiftcoder 7d ago
The field was purely theoretical 20 years ago, and Yudkowsky is pretty much the dictionary definition of "not accredited"
- naishoya 6d ago
  some use "not accredited" as a pejorative term. Lets not forget that the 'Fermat's Last Theorem' which has been pretty visible for the non-math crowd of late due to the recent AI frenzy about a purported proof was but one small contribution to the world's math lexicon by someone with a bachelors degree in civil law, that George Green was a baker and millwright, Boole was the son of a poor shoemaker in England with no formal university education and left school at age 14. Oliver Heaviside was a telegraph operator, and Michael Faraday was an apprentice bookbinder. So, not accredited shouldn't really carry much weight when it comes to mathematics. Lets not pretend that machine learning and the narrow branch that is the current approach to LLM inductions is anything but applied math. We might exercise our own minds and actually read the works and writings of a person, and use that as a measure of knowledge and perspective. Not all PhD dissertations are equal, and many have comprehension and ability to move us forward even without the institutional rigour. For those who prefer to have easy access to citations, here are some relevant papers that are not "Harry Potter" related, some with coauthors from Oxford University. Cognitive Biases Potentially Affecting Judgment of Global Risks [https://intelligence.org/files/CognitiveBiases.pdf https://intelligence.org/files/CognitiveBiases.pdf] Levels of Organization in General Intelligence [https://intelligence.org/files/LOGI.pdf https://intelligence.org/files/LOGI.pdf] Corrigibility [https://intelligence.org/files/Corrigibility.pdf https://intelligence.org/files/Corrigibility.pdf] The Ethics of Artificial Intelligence [https://intelligence.org/files/EthicsofAI.pdf https://intelligence.org/files/EthicsofAI.pdf]
  - swiftcoder 6d ago
    > some use "not accredited" as a pejorative term. I am absolutely using "not accredited" in a pejorative sense here. That he publishes papers coauthored by a couple of philosophy professors at Oxford (all of whom have made a ton of money from the Silicon Valley AI Alignment and Effective Altruism crowds) does not make him a scientist. I will also note that the Oxford Philosophy department finally shitcanned the whole Future of Humanity Institute a couple of years back.
    - naishoya 5d ago
      ok, that is informative and much more substantial than calling him 'not accredited.' I appreciate the clarification. That his writings are unscientific, and that the accredited professionals who were his coauthors have lost tenure from Oxford when an old email from the FHI Bostrom used a racial slur and claimed Black people were intellectually inferior and his 'apology' included a defense of eugenics that deeply alarmed the university community and triggered a formal investigation, are two details not immediately visible from a brief review of his published opinions. These are far more damning indictments than simply categorization of the author as lacking in formal education. Thank you for the opportunity to look more deeply at the situation and come an understanding of the consequences of taking his writings seriously.
Alwayshasbeeb 7d ago
Eliezer Yudkowsky is not a scientist. He made a popular Harry Potter fanfiction series and a "rationality" blog-community that attracts "human biological diversity" enthusiasts.
- bbor 6d ago
  Elizer Yudkowsky is a scientist, despite writing something you dislike 20 years ago and also having a blog. And no, he's not associated with race realism, that's just baseless libel.
orbital-decay 6d ago
Yes, he's the exact reason people are distrustful. He's a crank who learned about reward hacking and made a new religious movement out of it, pretending it's a world-ending issue and deliberately avoiding much more serious issues like the concentration of power. Typical cult leader and manipulator, and his disciples in charge of major AI shops aren't any better.
- bbor 6d ago
  What evidence do you have that LessWrong is a cult in any way the HackerNews is not? If he's a cult leader, he's awfully bad at it. No opulence, no compounds, no dogma, no doctrine...

bbor 6d ago

I... that's hard to even know how to respond to. If you think the field is 5 years old, you should not feel comfortable expressing opinions about it. A cursory search of the relevant wikipedia articles would do you wonders.