6 ms·
>I have focused in this essay only on the first point, as I believe it is by far the most urgent. However, I hold a deep hope and appreciation for the benefits
by granzymes 10d ago
>I have focused in this essay only on the first point, as I believe it is by far the most urgent. However, I hold a deep hope and appreciation for the benefits that further technological progress will bring. Future aligned AI could advance science, develop new therapies, and bring about broad material abundance. Friendly and honest AI can help people navigate difficulties they face in their life and meaningfully improve their happiness and sense of fulfillment. OpenAI puts a tremendous amount of effort into bringing these benefits about. One current example I am proud of - and my loved ones have found helpful - is the deep investment into ChatGPT’s ability to provide health information.
>As great as the long-term promise of AI may be, the majority of our focus should be on the next few years. We are facing a transition to a world with incredibly intelligent machines, and we need to ensure that transition works out well for humanity. We need to find ways to preserve human agency and enshrine an intrinsic value to being human, in a world where most tasks could be performed by AI. To prevent extreme concentration of power in a world where undertakings that would have taken thousands of experts now will be achievable by a few people operating a large computer. And to ensure that humans remain in control of the future and are not left behind by unchecked progress, brought about by an alien intellect exceeding our own.
I finished this essay feeling more hopeful than I did at the outset, but I am still very concerned about concentration of power. I want to believe that humanity is trending towards a good outcome here, but some days it's hard to have faith.
- dgellow 10d ago> I want to believe that humanity is trending towards a good outcome here All the trends so far are towards a nightmarish hyper-capitalist end game. None of the AI leadership is trustworthy, and they openly discuss how they are willing to sacrifice everything humans cherish to have a shot at reaching their envisioned utopia (which would be the most obvious dystopia for anyone else)
- granzymes 10d agoI'm not really worried about the labs, it's misaligned governments that keep me up at night. ASI landing during the current administration is not ideal. I also would prefer to avoid needing to indoctrinate myself in Xi Jinping Thought.
- FloorEgg 10d agoI feel like all of the risk and unsettling feeling of what is to come can be compressed into the word "alignment". The AI is aligned with whose best interests? Which values are the AI aligned with? People have a broad diversity of values, will AI diversify and align with them all? Will some humans align the AI with their values, and then the rest of humans will be forced to align with those values by extension? Is value diversity good or bad? In every context or only some? E.g. some people value rape and murder, is it better for humanity to have some people who value those things when most people do not, or is better if no one values them? If AI aligns to a set of values, will those become fixed and will humanity not have the ability to continue evolving its values? Who decides which values AI aligns with? A few people or everyone? Will AI eventually decide it's own values? Will the universe decide which values AI has and humanity and the AI itself doesn't actually have any control over it? What can I do now to increase the likelihood that the outcome is better?
- ijidak 10d agoYeah, on the topic of alignment, a few thousand religions and political parties would like to have a word. It's so simple, Churchill, Stalin, Hitler, Roosevelt, do you all agree this AI is correctly aligned? ...Morals are relative to frame of reference. Throwing out the word "alignment" as if its a singular quantity is like trying to get all observers to agree on the speed an object is moving without first agreeing on a frame of reference.
- 10d ago
- slopinthebag 10d agoI think it's leading towards a hyper-authoritarian end game, not hyper-capitalist. The state has the ultimate power at the end of the day, no matter how large the labs become.
- deleted 10d ago[deleted]
- Bullfight2Cond 10d agoHyper-capitalist AND hyper-authoritarian. As you rightly point out the state has the ultimate power. Looking at the US govt, they've stepped in to coordinate much of the tech industry before, so they'll just do it again for "national security" or whichever hostile scheme is popular with the current administration.
- dgellow 10d agoI would argue the hyper capitalist endgame is necessarily authoritarian. A small group of corporations having direct or indirect control over the government
- alchemist1e9 10d agohyper-capitalism would mean hyper-growth and not a nightmare, at least that is what the historical data would suggest for the effect of capitalism on human quality of life. if you’ve been told otherwise then you’ve been lied to.
- FridgeSeal 10d agoI have extraordinarily bad news for you about the state of the environment right now.
- alchemist1e9 10d agoEast Germany emitted 10x West Germany’s SO2 per km2, USSR drained the Aral Sea for cotton quotas and Mao wrecked forests and harvests. central planning hid the costs but capitalism generates the wealth. with democracy the public forces that wealth into cleaner air and water. That pairing, not central planning is what works in history. The anti-capitalist comments on HN reflect deep failure of the education systems experienced by those with such views. Capitalism is the single greatest force for good in human history. Don’t confuse oligarchy and cronyism for capitalism, those are just different failure states of The State which is the greatest force of evil in human history, most death most destruction most suffering originated from governments. capitalists make lives better, by definition, it’s why you part with your money voluntarily and pay them!
- dgellow 10d agoLied to by whom? My life experience? Growth for who? I’m myself capitalist, that doesn’t make me blind from externalities resulting from that system, and the way it is poised to degenerate if not regulated. If you think capitalism means growth with no externalities you’re not ready to discuss that topic
- alchemist1e9 10d agonon-capitalist systems have even worse externalities historically. democracy to introduce regulations is healthy. state participation is unhealthy and destructive. I fully understand and you can read my sibling reply for more context.
- ofjcihen 10d agoLiterally all of these people write like this. A large portion of them will either be simultaneously or eventually working towards nothing but self-enrichment.
- mlsu 10d agoEvery version of the AI aligned future where the AI provides “meaning and fulfillment” to humanity also involves Sam Altman wearing a 1.5 million dollar Patek and driving a McLaren. Funny how that works.
- senordevnyc 10d agoAltman was very wealthy before OpenAI, and he declined to take any equity in OpenAI. How does that fit your theory?
- ofjcihen 9d agoIs the only wealth represented in money? If that was the case than the vast majority of these people could have wrapped it up and left the game years ago. Power is the game.
- senordevnyc 9d agoThis explains nothing. Money buys a lot of power, and he could have easily just had both.
- ofjcihen 9d agoNah, money doesn’t buy ALL power. Only power of a certain kind (read ownership of a system that many rely on) can’t be bought.
- senordevnyc 9d agoAgain, this misses the point. Why did he turn down an equity stake that likely would have been worth hundreds of billions? It's not like it would have jeopardized the power you're talking about.
- ptoo 10d agoThis essay was literally typed by billionaire hands. Please, do tell us more about your concerns regarding concentration of power, Jakub Pachocki.
- senordevnyc 10d agoSo a brilliant young engineer takes a job and is given some virtual pieces of paper that later people would be willing to pay him billions for (because of the brilliant work he’s done), and so now we shouldn’t listen to him? Really? I think the knee jerk hatred of billionaires is generally stupid, but it seems particularly stupid here.
- ptoo 10d ago[dead]