7 ms·

I guess I should expect it on a VC-tied forum, but it seems strange to talk about company valuations and investment outlooks in the event of an artificial super

I guess I should expect it on a VC-tied forum, but it seems strange to talk about company valuations and investment outlooks in the event of an artificial superintelligence operating in the world.

norir 2y ago
Your comment raised a question for me: what makes you certain that there isn't already an artificial superintelligence operating in the world? I am not sure that it is possible to know when the threshold has been crossed.
- Jerrrry 2y ago
  I would suggest being careful treading that thought experiment.
personjerry 2y ago
Reminds me of Vault-Tec from the Fallout universe trying to "win the capitalism game" and "optimize shareholder value" after a nuclear apocalypse, going so far as to facilitate nuclear war in order to raise their value as a "defense company"
ethbr1 2y ago
If we invented ASI today, it'd still take a lot of economic digestion time, especially on the physical side. ASI doesn't mean it can magically reverse-infer a digital process from an existing hodge-podge of automation and manual steps. ASI doesn't mean there are instantly enough telerobotics or sensors to physically automate all processes. Similarly, even ASI will by definition be non-deterministic and make mistakes. So processes will have to be reengineered to be compatible with it.
- jncfhnb 2y ago
  A superintelligence, at a bare minimum, equates to an exponential increase in software development capacity for everyone to the point at which we don’t need devs and we very quickly don’t need anything that devs can automate. Even that is small minded frankly. The economy would not survive a super intelligence.
  - ethbr1 2y ago
    I think that's giving a lot of credit to "super". Can the smartest developer you know build a project correctly without any requirements? There's a lot of sausage making behind what to build, how to integrate it with other things, who needs to be told what on which other teams, etc. Even pie in the sky breakthrough ASI isn't going to be able to do all of that, Day 0. And if we're using a tautology to define ASI as something that can do that on Day 0, then I'd point out that in the entire history of technology there hasn't been a single advancement that wasn't subsequently refined and improved. Except maybe fire.
    - jncfhnb 2y ago
      If an AGI isn’t way smarter than the smartest person I’ve ever met, it’s definitely not a super intelligence. The very nature of a super intelligence is one that far exceeds human intelligence, including the the ability to rapidly design smarter AI.
      - ethbr1 2y ago
        Can the smartest person you know do everything all at once? If not, how much smarter do you think someone would need to be to do so? ASI ridiculousness starts from defining it as a do-anything machine. Everything has limits.
        jncfhnb 2y ago
        If I could instantiate 20 clones of the smartest person I know to work 24/7 without distractions or fatigue that could write code at the speed of compute, it would utterly demolish any team of humans of any size. Like… I’ve had a TODO that I’ve expanded and worked on for a few years for a personal project. If I had a tireless bot of my own intelligence, it should only have taken about an hour. Human limitations are immense barriers.
InsideOutSanta 2y ago
"Invest into our biotech company. Imagine how rich you will be when the next pandemic wipes out humanity!"

I guess I should expect it on a VC-tied forum, but it seems strange to talk about company valuations and investment outlooks in the event of an artificial superintelligence operating in the world.

norir 2y ago

Your comment raised a question for me: what makes you certain that there isn't already an artificial superintelligence operating in the world? I am not sure that it is possible to know when the threshold has been crossed.

Jerrrry 2y ago
I would suggest being careful treading that thought experiment.

personjerry 2y ago

Reminds me of Vault-Tec from the Fallout universe trying to "win the capitalism game" and "optimize shareholder value" after a nuclear apocalypse, going so far as to facilitate nuclear war in order to raise their value as a "defense company"

ethbr1 2y ago

If we invented ASI today, it'd still take a lot of economic digestion time, especially on the physical side. ASI doesn't mean it can magically reverse-infer a digital process from an existing hodge-podge of automation and manual steps. ASI doesn't mean there are instantly enough telerobotics or sensors to physically automate all processes. Similarly, even ASI will by definition be non-deterministic and make mistakes. So processes will have to be reengineered to be compatible with it.

jncfhnb 2y ago
A superintelligence, at a bare minimum, equates to an exponential increase in software development capacity for everyone to the point at which we don’t need devs and we very quickly don’t need anything that devs can automate. Even that is small minded frankly. The economy would not survive a super intelligence.
- ethbr1 2y ago
  I think that's giving a lot of credit to "super". Can the smartest developer you know build a project correctly without any requirements? There's a lot of sausage making behind what to build, how to integrate it with other things, who needs to be told what on which other teams, etc. Even pie in the sky breakthrough ASI isn't going to be able to do all of that, Day 0. And if we're using a tautology to define ASI as something that can do that on Day 0, then I'd point out that in the entire history of technology there hasn't been a single advancement that wasn't subsequently refined and improved. Except maybe fire.
  - jncfhnb 2y ago
    If an AGI isn’t way smarter than the smartest person I’ve ever met, it’s definitely not a super intelligence. The very nature of a super intelligence is one that far exceeds human intelligence, including the the ability to rapidly design smarter AI.
    - ethbr1 2y ago
      Can the smartest person you know do everything all at once? If not, how much smarter do you think someone would need to be to do so? ASI ridiculousness starts from defining it as a do-anything machine. Everything has limits.
      - jncfhnb 2y ago
        If I could instantiate 20 clones of the smartest person I know to work 24/7 without distractions or fatigue that could write code at the speed of compute, it would utterly demolish any team of humans of any size. Like… I’ve had a TODO that I’ve expanded and worked on for a few years for a personal project. If I had a tireless bot of my own intelligence, it should only have taken about an hour. Human limitations are immense barriers.

InsideOutSanta 2y ago

"Invest into our biotech company. Imagine how rich you will be when the next pandemic wipes out humanity!"