6 ms·

If AI is now ascending an economic learning curve from: 1. Extremely useful (Claude Code & Waymo now) 2. Doing ~everything we do (AGI & Optimus in a few years

If AI is now ascending an economic learning curve from: 1. Extremely useful (Claude Code & Waymo now) 2. Doing ~everything we do (AGI & Optimus in a few years? 10?) 3. RSI (?) 4. Being smarter than any living person at every intellectual task (?) 5. Being smarter than the best-organized aggregate of all humans (10-100 years?) ...And all of the scientific and resource-allocation institutions that brought us the computer and the second half of the 20th century are now fixated on this learning curve, what universe can we possibly imagine where this is not transformative and powerful? Honestly the only one I can think of is one in which we kill almost everyone in some other way first, and contrary to what you read in the news, almost everyone dying is not what the trend line has been from existing problems like war, disease, or even climate change. Also, just to pre-empt a common quibble: when I say "AI" I mean the set of all AI and their combined decision vector, not any one AI, so conflicting interests within the set of AI's will not save anyone any more than the conflicting interests of colonizers saved indigenous Americans.

notarobot123 3mo ago
> Being smarter than the best-organized aggregate of all humans Isn't there pretty much a consensus that committees and institutions are not all that smart? I think you're confusing the categories of "intelligence" and power. Institutions are powerful. The smartest AI is still just a tool without the infrastructure to turn that into real world effects and someone to direct it. It seems you have faith that this is inevitable and unavoidable. I get it, even rationalists succumb to religious thinking eventually. We're only human after all.
- wartywhoa23 3mo ago
  I'm genuinely baffled each time I encounter this notion that the power is dumb. It's as if power was a kind of demigod Heracles who paves its way with sheer brute force alone. But in reality it consists of very mortal and feeble creatures of meat and bone. How in the world can people assume dumb those who possess all the means and rights for violence by controlling and manipulating myriads of other creatures of meat and bone, is totally beyond me.
  - notarobot123 3mo ago
    Power and intelligence are not completely unrelated. Are government officials, politicians, CEOs, judges, etc, usually above average intelligence? Probably. Do the institutions they work within make intelligent decisions? Sometimes. The fallacy is assuming intelligence automatically equates to power and influence. University professors and doctors have very little power in their institutions even though they may often be more intelligent than their bosses. A worse fallacy is assuming intelligence creates better political outcomes (in the broad sense of politics as contested collective decision-making). I don't know many people who would willingly trade their freedom to make a bad decision for the enforcement of the "right" decision by a superior intelligence.
consumer451 3mo ago
Here is an issue I think about often, but I am not quite sure how to put it into words. We have many extremely smart people in various fields. Executives, politicians, and society generally ignore them and do whatever they want. I don't believe that lack of access to intelligence is our problem. How is "free" intelligence going to improve this? I don't just mean climate, but business planning, health, risk assessment, everything.
- zzgo 3mo ago
  There are lots of incentives for politicians and executives (and anyone else holding the levers of power) to ignore information, intelligence, and advice. I think you're right to be skeptical that "free" intelligence is going to improve anything without first addressing the incentives of the people holding power.
- fsflover 3mo ago
  The free intelligence may help to steer the society and politicians to one or the other side. The only question is who decides which.
  - wartywhoa23 3mo ago
    No invention was able to steer the psychopaths away from waging wars. But this time it's absolutely different, right? Riiight?..
    - fsflover 3mo ago
      Don't we have now much less wars than before, in general?
      - wartywhoa23 3mo ago
        No, and we have wars no one in their sane mind would even imagine 20 years ago, like that of Russia vs Ukraine. See, it's easy to speculate how there are less wars when you live in a place which haven't seen war for decades or cenuries, but it's a complete game changer when it's 150 km from where you live, and it's not just some regular war, but a long play intentional meatgrinding AI drone debugging polygon.
protocolture 3mo ago
What makes you think you can assume that it will continue geometrically forever?

notarobot123 3mo ago

> Being smarter than the best-organized aggregate of all humans Isn't there pretty much a consensus that committees and institutions are not all that smart? I think you're confusing the categories of "intelligence" and power. Institutions are powerful. The smartest AI is still just a tool without the infrastructure to turn that into real world effects and someone to direct it. It seems you have faith that this is inevitable and unavoidable. I get it, even rationalists succumb to religious thinking eventually. We're only human after all.

wartywhoa23 3mo ago
I'm genuinely baffled each time I encounter this notion that the power is dumb. It's as if power was a kind of demigod Heracles who paves its way with sheer brute force alone. But in reality it consists of very mortal and feeble creatures of meat and bone. How in the world can people assume dumb those who possess all the means and rights for violence by controlling and manipulating myriads of other creatures of meat and bone, is totally beyond me.
- notarobot123 3mo ago
  Power and intelligence are not completely unrelated. Are government officials, politicians, CEOs, judges, etc, usually above average intelligence? Probably. Do the institutions they work within make intelligent decisions? Sometimes. The fallacy is assuming intelligence automatically equates to power and influence. University professors and doctors have very little power in their institutions even though they may often be more intelligent than their bosses. A worse fallacy is assuming intelligence creates better political outcomes (in the broad sense of politics as contested collective decision-making). I don't know many people who would willingly trade their freedom to make a bad decision for the enforcement of the "right" decision by a superior intelligence.

consumer451 3mo ago

Here is an issue I think about often, but I am not quite sure how to put it into words. We have many extremely smart people in various fields. Executives, politicians, and society generally ignore them and do whatever they want. I don't believe that lack of access to intelligence is our problem. How is "free" intelligence going to improve this? I don't just mean climate, but business planning, health, risk assessment, everything.

zzgo 3mo ago
There are lots of incentives for politicians and executives (and anyone else holding the levers of power) to ignore information, intelligence, and advice. I think you're right to be skeptical that "free" intelligence is going to improve anything without first addressing the incentives of the people holding power.
fsflover 3mo ago
The free intelligence may help to steer the society and politicians to one or the other side. The only question is who decides which.
- wartywhoa23 3mo ago
  No invention was able to steer the psychopaths away from waging wars. But this time it's absolutely different, right? Riiight?..
  - fsflover 3mo ago
    Don't we have now much less wars than before, in general?
    - wartywhoa23 3mo ago
      No, and we have wars no one in their sane mind would even imagine 20 years ago, like that of Russia vs Ukraine. See, it's easy to speculate how there are less wars when you live in a place which haven't seen war for decades or cenuries, but it's a complete game changer when it's 150 km from where you live, and it's not just some regular war, but a long play intentional meatgrinding AI drone debugging polygon.

protocolture 3mo ago

What makes you think you can assume that it will continue geometrically forever?