9 ms·

He coined the concept 'singularity' in the sense of machines becoming smarter than humans what a time for him to die with all the advancements we're seeing in a

He coined the concept 'singularity' in the sense of machines becoming smarter than humans what a time for him to die with all the advancements we're seeing in artificial intelligence. I wonder what he thought about it all. >The concept and the term "singularity" were popularized by Vernor Vinge first in 1983 in an article that claimed that once humans create intelligences greater than their own, there will be a technological and social transition similar in some sense to "the knotted space-time at the center of a black hole",[8] and later in his 1993 essay The Coming Technological Singularity,[4][7] in which he wrote that it would signal the end of the human era, as the new superintelligence would continue to upgrade itself and would advance technologically at an incomprehensible rate. He wrote that he would be surprised if it occurred before 2005 or after 2030. Looks like he was spot on.

trenchgun 2y ago
He popularized and advanced the concept, but originally it was by von Neumann.
- nabla9 2y ago
  The concept predates von Neuman. First known person to present the idea was mathematician and philosopher Nicolas de Condorcet in the late 1700s. Not surprising, because he also laid out most ideals and values of modern liberal democracy as they are now. Amazing philosopher. He basically invented the idea of ensemble learning (known as boosting in machine learning). Nicolas de Condorcet and the First Intelligence Explosion Hypothesis https://onlinelibrary.wiley.com/doi/10.1609/aimag.v40i1.2855 https://onlinelibrary.wiley.com/doi/10.1609/aimag.v40i1.2855
  - n4r9 2y ago
    That kind of niche knowledge is what I come to HN for!
    - protomolecule 2y ago
      Also "Darwin among the Machines"[0] written by Samuel Butler in 1863, that's 4 years after Darwin's "On the Origin of Species". Butlerian jihad[1] is the war against machines in the Dune universe. [0] https://en.wikipedia.org/wiki/Darwin_among_the_Machines https://en.wikipedia.org/wiki/Darwin_among_the_Machines [1] https://dune.fandom.com/wiki/Butlerian_Jihad https://dune.fandom.com/wiki/Butlerian_Jihad
      - jhbadger 2y ago
        Butler also expanded this idea in his 1872 novel Erewhon, where he described a seemingly primitive island civilization that turned out to once had greater technology than the West, including mechanical AI, but they abandoned it when they began to fear its consequences. A lot of 20th century SF tropes in the Victorian period. https://en.wikipedia.org/wiki/Erewhon https://en.wikipedia.org/wiki/Erewhon
  - jart 2y ago
    That essay is written by a political scientist. His arguments aren't very persuasive. Even if they were, he doesn't actually cite the person he's writing about, so I have no way to check the primary materials. It's not like this is uncommon either. Everyone who's smart since 1760 has extrapolated the industrial revolution and imagined something similar to the singularity. Malthus would be a bad example and Nietzsche would be a good example. But John von Neumann was a million times smarter than all of them, he named it the singularity, and that's why he gets the credit.
    - tim333 2y ago
      There are some quotes but they guy seems to be talking about improving humans rather than anything AI like: "...natural [human] faculties themselves and this [human body] organisation could also be improved?"
    - nabla9 2y ago
      Check out "Sketch for a Historical Picture of the Progress of the Human Mind", by Marquis de Condorcet, 1794. The last chapter, The Tenth epoch/The future progress of the human mind. There he lays out unlimited advance of knowledge, unlimited lifespan for humans, improvement of physical faculties, and then finally improvement of the intellectual and moral faculties. And this was not some obscure author, but leading figure in the French Enlightenment. Thomas Malthus wrote his essay on population as counterargument.
gcr 2y ago
with respect, we don’t know if he was spot on. Companies shoehorning language models into their products is a far cry from the transformative societal change he describes will happen. nothing like a singularity has yet happened at the scale he describes, and might not happen without more fundamental shifts/breakthroughs in AI research.
- mnsc 2y ago
  Imagine the first llm to suggest an improvement to itself that no human has considered. Then imagine what happens next.
  - deleted 2y ago
    [deleted]
  - dsr_ 2y ago
    OK. I'm imagining a correlation engine that looks through code as a series of prompts that are used to generate more code from the corpus that is statistically likely to follow. And now I'm transforming that through the concept of taking a photograph and applying the clone tool via a light airbrush. Repeat enough times, and you get uncompilable mud. LLMs are not going to generate improvements.
    - ben_w 2y ago
      Saying they definitely won't or they definitely will are equally over-broad and premature. I currently expect we'll need another architectural breakthrough; but also, back in 2009 I expected no-steering-wheel-included self driving cars no later than 2018, and that the LLM output we actually saw in 2023 would be the final problem to be solved in the path to AGI. Prediction is hard, especially about the future.
      - jart 2y ago
        GPT4 does inference at 560 teraflops. Human brain goes 10,000 teraflops. NVIDIA just unveiled their latest Blackwell chip yesterday which goes 20,000 teraflops. If you buy an NVL72 rack of the things, it goes 1,400,000 teraflops. That's what Jensen Huang's GPT runs on I bet.
gumby 2y ago
Just to clarify, the “singularity” conjectures a slightly different and more interesting phenomenon, one driven by technological advances, true, but its definition was not those advances. It was more the second derivative of future shock: technologies and culture that enabled and encouraged faster and faster change until the curve bent essentially vertical…asymptotimg to a mathematical singularity. An example my he spoke of was that, close to the singularity, someone might found a corporation, develop a technology, make a profit from it, and then have it be obsolete by noon. And because you can’t see the shape of the curve on the other side of such a singularity, people living on the other side of it would be incomprehensible to people on this side. Ray Lafferty’s 1965 story “Slow Tuesday Night” explored this phenomenon years before Toffler wrote “Future Shock”
- ethagnawl 2y ago
  Here's a link to the full text of _Slow Tuesday Night_: https://web.archive.org/web/20060719184509/www.scifi.com/scifiction/classics/classics_archive/lafferty5/lafferty51.html https://web.archive.org/web/20060719184509/www.scifi.com/sci...
- PaulHoule 2y ago
  Note that the "Singularity" turns up in the novel https://en.wikipedia.org/wiki/Marooned_in_Realtime https://en.wikipedia.org/wiki/Marooned_in_Realtime where people can use a "Bobble" to freeze themselves in a stasis field and travel in time... forward. The singularity is some mysterious event that causes all of unbobbled humanity to disappear leaving the survivors wondering, even 10s of millions of years later, what happened. As such it is one of the best pretenses ever in sci-fi. (I am left wondering though if the best cultural comparison is "The Rapture" some Christians believe in making this more of a religiously motivated concept as opposed to sound futurism.) I've long been fascinated by this differential equation dx -- = x^2 dt which has solutions that look like x = 1/(t₀-t) which notably blows up at time t₀. It's a model of an "intelligence explosion" where improving technology speeds up the rate of technological process but the very low growth when t ≪ t₀ could also be a model for why it is hard to bootstrap a two-sided market, why some settlements fail, etc. About 20 years ago I was very interested in ecological accounting and wondering if we could outrace resource depletion and related problems and did a literature search for people developing models like this further and was pretty disappointed not to find much also it did appear as a footnote in the ecology literature here and there. Even papers like https://agi-conf.org/2010/wp-content/uploads/2009/06/agi10singmodels2.pdf https://agi-conf.org/2010/wp-content/uploads/2009/06/agi10si... seem to miss it. (Surprised the lesswrong folks haven't picked it up but they don't seem too mathematically inclined) --- Note I don't believe in the intelligence explosion because what we've seen in "Moore's law" recently is that each generation of chips is getting much more difficult and expensive to develop whereas the benefits of shrinks are shrinking and in fact we might be rudely surprised that the state of the art chips of the new future (and possibly 2024) burn up pretty quickly. It's not so clear that chipmakers would have continued to invest in a new generation if governments weren't piling huge money into a "great powers" competition... That is, already we might be past the point of economic returns.
0xdeadbeefbabe 2y ago
> He wrote that he would be surprised if it occurred before 2005 or after 2030. Being surprised is also an exciting outcome. Was he thinking about that too?
holoduke 2y ago
"We should destroy all AI out there before its taking over"

trenchgun 2y ago

He popularized and advanced the concept, but originally it was by von Neumann.

nabla9 2y ago
The concept predates von Neuman. First known person to present the idea was mathematician and philosopher Nicolas de Condorcet in the late 1700s. Not surprising, because he also laid out most ideals and values of modern liberal democracy as they are now. Amazing philosopher. He basically invented the idea of ensemble learning (known as boosting in machine learning). Nicolas de Condorcet and the First Intelligence Explosion Hypothesis https://onlinelibrary.wiley.com/doi/10.1609/aimag.v40i1.2855 https://onlinelibrary.wiley.com/doi/10.1609/aimag.v40i1.2855
- n4r9 2y ago
  That kind of niche knowledge is what I come to HN for!
  - protomolecule 2y ago
    Also "Darwin among the Machines"[0] written by Samuel Butler in 1863, that's 4 years after Darwin's "On the Origin of Species". Butlerian jihad[1] is the war against machines in the Dune universe. [0] https://en.wikipedia.org/wiki/Darwin_among_the_Machines https://en.wikipedia.org/wiki/Darwin_among_the_Machines [1] https://dune.fandom.com/wiki/Butlerian_Jihad https://dune.fandom.com/wiki/Butlerian_Jihad
    - jhbadger 2y ago
      Butler also expanded this idea in his 1872 novel Erewhon, where he described a seemingly primitive island civilization that turned out to once had greater technology than the West, including mechanical AI, but they abandoned it when they began to fear its consequences. A lot of 20th century SF tropes in the Victorian period. https://en.wikipedia.org/wiki/Erewhon https://en.wikipedia.org/wiki/Erewhon
- jart 2y ago
  That essay is written by a political scientist. His arguments aren't very persuasive. Even if they were, he doesn't actually cite the person he's writing about, so I have no way to check the primary materials. It's not like this is uncommon either. Everyone who's smart since 1760 has extrapolated the industrial revolution and imagined something similar to the singularity. Malthus would be a bad example and Nietzsche would be a good example. But John von Neumann was a million times smarter than all of them, he named it the singularity, and that's why he gets the credit.
  - tim333 2y ago
    There are some quotes but they guy seems to be talking about improving humans rather than anything AI like: "...natural [human] faculties themselves and this [human body] organisation could also be improved?"
  - nabla9 2y ago
    Check out "Sketch for a Historical Picture of the Progress of the Human Mind", by Marquis de Condorcet, 1794. The last chapter, The Tenth epoch/The future progress of the human mind. There he lays out unlimited advance of knowledge, unlimited lifespan for humans, improvement of physical faculties, and then finally improvement of the intellectual and moral faculties. And this was not some obscure author, but leading figure in the French Enlightenment. Thomas Malthus wrote his essay on population as counterargument.

gcr 2y ago

with respect, we don’t know if he was spot on. Companies shoehorning language models into their products is a far cry from the transformative societal change he describes will happen. nothing like a singularity has yet happened at the scale he describes, and might not happen without more fundamental shifts/breakthroughs in AI research.

mnsc 2y ago
Imagine the first llm to suggest an improvement to itself that no human has considered. Then imagine what happens next.
- deleted 2y ago
  [deleted]
- dsr_ 2y ago
  OK. I'm imagining a correlation engine that looks through code as a series of prompts that are used to generate more code from the corpus that is statistically likely to follow. And now I'm transforming that through the concept of taking a photograph and applying the clone tool via a light airbrush. Repeat enough times, and you get uncompilable mud. LLMs are not going to generate improvements.
  - ben_w 2y ago
    Saying they definitely won't or they definitely will are equally over-broad and premature. I currently expect we'll need another architectural breakthrough; but also, back in 2009 I expected no-steering-wheel-included self driving cars no later than 2018, and that the LLM output we actually saw in 2023 would be the final problem to be solved in the path to AGI. Prediction is hard, especially about the future.
    - jart 2y ago
      GPT4 does inference at 560 teraflops. Human brain goes 10,000 teraflops. NVIDIA just unveiled their latest Blackwell chip yesterday which goes 20,000 teraflops. If you buy an NVL72 rack of the things, it goes 1,400,000 teraflops. That's what Jensen Huang's GPT runs on I bet.

gumby 2y ago

Just to clarify, the “singularity” conjectures a slightly different and more interesting phenomenon, one driven by technological advances, true, but its definition was not those advances. It was more the second derivative of future shock: technologies and culture that enabled and encouraged faster and faster change until the curve bent essentially vertical…asymptotimg to a mathematical singularity. An example my he spoke of was that, close to the singularity, someone might found a corporation, develop a technology, make a profit from it, and then have it be obsolete by noon. And because you can’t see the shape of the curve on the other side of such a singularity, people living on the other side of it would be incomprehensible to people on this side. Ray Lafferty’s 1965 story “Slow Tuesday Night” explored this phenomenon years before Toffler wrote “Future Shock”

ethagnawl 2y ago
Here's a link to the full text of _Slow Tuesday Night_: https://web.archive.org/web/20060719184509/www.scifi.com/scifiction/classics/classics_archive/lafferty5/lafferty51.html https://web.archive.org/web/20060719184509/www.scifi.com/sci...
PaulHoule 2y ago
Note that the "Singularity" turns up in the novel https://en.wikipedia.org/wiki/Marooned_in_Realtime https://en.wikipedia.org/wiki/Marooned_in_Realtime where people can use a "Bobble" to freeze themselves in a stasis field and travel in time... forward. The singularity is some mysterious event that causes all of unbobbled humanity to disappear leaving the survivors wondering, even 10s of millions of years later, what happened. As such it is one of the best pretenses ever in sci-fi. (I am left wondering though if the best cultural comparison is "The Rapture" some Christians believe in making this more of a religiously motivated concept as opposed to sound futurism.) I've long been fascinated by this differential equation dx -- = x^2 dt which has solutions that look like x = 1/(t₀-t) which notably blows up at time t₀. It's a model of an "intelligence explosion" where improving technology speeds up the rate of technological process but the very low growth when t ≪ t₀ could also be a model for why it is hard to bootstrap a two-sided market, why some settlements fail, etc. About 20 years ago I was very interested in ecological accounting and wondering if we could outrace resource depletion and related problems and did a literature search for people developing models like this further and was pretty disappointed not to find much also it did appear as a footnote in the ecology literature here and there. Even papers like https://agi-conf.org/2010/wp-content/uploads/2009/06/agi10singmodels2.pdf https://agi-conf.org/2010/wp-content/uploads/2009/06/agi10si... seem to miss it. (Surprised the lesswrong folks haven't picked it up but they don't seem too mathematically inclined) --- Note I don't believe in the intelligence explosion because what we've seen in "Moore's law" recently is that each generation of chips is getting much more difficult and expensive to develop whereas the benefits of shrinks are shrinking and in fact we might be rudely surprised that the state of the art chips of the new future (and possibly 2024) burn up pretty quickly. It's not so clear that chipmakers would have continued to invest in a new generation if governments weren't piling huge money into a "great powers" competition... That is, already we might be past the point of economic returns.

0xdeadbeefbabe 2y ago

> He wrote that he would be surprised if it occurred before 2005 or after 2030. Being surprised is also an exciting outcome. Was he thinking about that too?

holoduke 2y ago

"We should destroy all AI out there before its taking over"