11 ms·

I'd really love to talk to someone that both really believes this to be true, and has a hands-on experience with building and using generative AI. The intersec

I'd really love to talk to someone that both really believes this to be true, and has a hands-on experience with building and using generative AI. The intersection of the two seems to be quite hard to find. At the state that we're in the AIs we're building are just really useful input/output devices that respond to a stimuli (e.g., a "prompt"). No stimuli, no output. This isn't a nuclear weapon. We're not going to accidentally create Skynet. The only thing it's going to go nuclear on is the market for jobs that are going to get automated in an economy that may not be ready for it. If anything, the "danger" here is that AGI is going to be a printing press. A cotton gin. A horseless carriage -- all at the same time and then some, into a world that may not be ready for it economically. Progress of technology should not be artitrarily held back to protect automateable jobs though. We need to adapt.

ev7 1y ago
alignmentforum.com
voidspark 1y ago
> Progress of technology should not be artitrarily held back to protect automateable jobs though. We need to adapt. So you don't mind if your economic value drops to zero, with all human labour replaced by machines? Dependent on UBI, existing in a basic pod, eating rations of slop.
- cik 1y ago
  > So you don't mind if your economic value drops to zero, with all human labour replaced by machines? This was the fear when the cotton gin was invented. It was the ear when cars were created. The same complaint happened with the introduction of electronic, automated, telephone switchboards. Jobs change. Societies change. Unemployment worldwide, is near the lowest it has ever been. Work will change. Society will eventually move to a currency based on energy production, or something equally futuristic. This doesn't mean that getting there will be without pain.
  - voidspark 1y ago
    The goal for AGI/ASI is to create machines that can do any job much faster, better, and cheaper than humans. That's the ultimate end point of this progress. The economic value of human labour will drop to zero. That would be an existential threat to our civilization.
  - achierius 1y ago
    Where did all the work-horses go? Why is there barely a fraction of the population there once was? Why did they not adapt and find niches where they had a competitive advantage over cars and machines?
    - deleted 1y ago
      [deleted]
    - wickedsight 1y ago
      The horses weren't the market the economy is selling to, the people are. Ford figured out that people having both time and money is best for the economy. We'll figure out that having all the production capabilities but none of the market benefits nobody.
- TobTobXX 1y ago
  Yes! Sounds like a dream. My value isn't determined by some economic system, but rather by myself. There is so much to do when you don't have to work. Of course, this assumes we actually get to UBI first, and it doesn't create widespread poverty. But even if humanity will have to go through widespread poverty, we'd porbably come out with UBI on the other side (minus a few hundred millions starved). There's so much to do, explore and learn. The prospect of AI stealing my job is only scary because my income depends on this job.
  - voidspark 1y ago
    > There's so much to do, explore and learn. Hobbies, hanging out with friends, reading, etc. That's basically it. Probably no international travel. It will be like a simple retirement on a low income, because in a socialist system the resources must be rationed. This will drive a lot of young ambitious people to insanity. Nothing meaningful for them to achieve. No purpose. Drug use, debauchery, depression, violence, degeneracy, gangs. It will be a true idiocracy. No Darwinian selection pressures, unless the system enforces eugenics and population control.
    - pixelbro 1y ago
      Wait, wait, wait. Our society's gonna fall apart due to a lack of Darwinian selection pressure? What do you think we're selecting for right now? Seems to me like our culture treats both survival and reproduction as an inalienable right. Most people would go so far as to say everyone deserves love, "there's a lid for every pot".
      - deleted 1y ago
        [deleted]
    - knowaveragejoe 1y ago
      > This will drive a lot of young ambitious people to insanity. Nothing meaningful for them to achieve. Maybe, if the only flavor of ambition you're aware of is that of SV types. Plenty of people have found achievement and meaning before and alongside the digital revolution world.
thurn 1y ago
Which of these statements do you disagree with? - Superintelligence poses an existential threat to humanity - Predicting the future is famously difficult - Given that uncertainty, we can't rule out the chance of our current AI approach leading to superintelligence - Even a 1-in-1000 existential threat would be extremely serious. If an asteroid had a 1-in-1000 chance of hitting Earth and obliterating humanity we should make serious contingency plans. Second question: how confident are you that you're correct? Are you 99.9% sure? Confident enough to gamble billions of lives on your beliefs? There are almost no statements about the future which I'd assign this level of confidence to.
- ZuFyf4Q6K4wjoS 1y ago
  [dead]
- geysersam 1y ago
  Isn't the question you're posing basically Pascals wager? I think the chance they're going to create a "superintelligence" is extremely small. That said I'm sure we're going to have a lot of useful intelligence. But nothing general or self-conscious or powerful enough to be threatening for many decades or even ever. > Predicting the future is famously difficult That's very true, but that fact unfortunately can never be used to motivate any particular action, because you can always say "what if the real threat comes from a different direction?" We can come up with hundreds of doomsday scenarios, most don't involve AI. Acting to minimize the risk of every doomsday scenario (no matter how implausible) is doomsday scenario no. 153.
  - polynomial 1y ago
    Yes, this is literally Pascal's wager / Pascal's mugging.
  - nearbuy 1y ago
    > I think the chance they're going to create a "superintelligence" is extremely small. I'd say the chance that we never create a superintelligence is extremely small. You either have to believe that for some reason the human brain achieved the maximum intelligence possible, or that progress on AI will just stop for some reason. Most forecasters on prediction markets are predicting AGI within a decade.
utbabya 1y ago
> At the state that we're in the AIs we're building are just really useful input/output devices that respond to a stimuli (e.g., a "prompt"). No stimuli, no output. It was true before we allowed them to access external systems, disregarding certain rule which I forgot the origin. The more general problem is a mix between the tradegy of the common; we have better understanding every passing day yet still don't understand exacly why LLM perform that well emergently instead of engineered that way; and future progress. Do you think you can find a way around access boundaries to masquerade your Create/Update requests as Read in the log system monitoring it, when you have super intelligence?
otabdeveloper4 1y ago
> are just really useful input/output devices that respond to a stimuli LLMs are huge pretrained models. The economic benefit here is that you don't have to train your own text classification model anymore. (The LLM was likely already trained on whatever training set you could think of.) That's a big time and effort saver, but no different from "AI" that we had decades prior. It's just more accessible to the normal person now.
digbybk 1y ago
> I'd really love to talk to someone that both really believes this to be true, and has a hands-on experience with building and using generative AI. Any of the signatories here match your criteria? https://safe.ai/work/statement-on-ai-risk#signatories https://safe.ai/work/statement-on-ai-risk#signatories Or if you’re talking more about everyday engineers working in the field, I suspect the people soldering vacuum tubes to the ENIAC would not necessarily have been the same people with the clearest vision for the future of the computer.
coryfklein 1y ago
Sounds a little too much like, "It's not AGI today ergo it will never become AGI" Does the current AI give productivity benefits to writing code? Probably. Do OpenAI engineers have exclusive access to more capable models that give them a greater productivity boost than others? Also probably. If one exclusive group gets the benefit of developing AI with a 20% productivity boost compared to others, and they develop a 2.0 that grants them a 25% boost, then a 3.0 with a 30% boost, etc... The question eventually becomes, "is AGI technically possible"; is there anything special about meat that cannot be reproduced on silicon? We will find AGI someday, and more than likely that discovery will be aided by the current technologies. It's the path here that matters, not the specific iteration of generative LLM tech we happen to be sitting on in May 2025.
- Retric 1y ago
  > Does the current AI give productivity benefits to writing code? Probably. > If one exclusive group gets the benefit of developing AI with a 20% productivity boost compared to others, and they develop a 2.0 that grants them a 25% boost, then a 3.0 with a 30% boost, etc... That’s a bit of a stretch, generative AI is least capable of helping with novel code such as needed to make AGI. If anything I’d expect companies working on generative AI to be at a significant disadvantage when trying to make AGI because they’re trying to leverage what they are already working on. That’s fine for incremental improvement, but companies rarely ride one wave of technology to the forefront of the next. Analog > digital photography, ICE > EV, coal mining > oil, etc.
  - coryfklein 1y ago
    The "novel AGI code" probably accounts for <5% of work by time spent. If they can reduce the remaining 95% of grunt work (wiring yet another DB query to a frontend, tweaking the build pipeline, automating GPU allocation scripts) then that means they can focus more on that 5%. Then it looks like Company A spends 90% of time on novel research work (while LLMs do all the busy work) and Company B spends 5% of time on novel research work.
    - 1y ago

ev7 1y ago

alignmentforum.com

voidspark 1y ago

> Progress of technology should not be artitrarily held back to protect automateable jobs though. We need to adapt. So you don't mind if your economic value drops to zero, with all human labour replaced by machines? Dependent on UBI, existing in a basic pod, eating rations of slop.

cik 1y ago
> So you don't mind if your economic value drops to zero, with all human labour replaced by machines? This was the fear when the cotton gin was invented. It was the ear when cars were created. The same complaint happened with the introduction of electronic, automated, telephone switchboards. Jobs change. Societies change. Unemployment worldwide, is near the lowest it has ever been. Work will change. Society will eventually move to a currency based on energy production, or something equally futuristic. This doesn't mean that getting there will be without pain.
- voidspark 1y ago
  The goal for AGI/ASI is to create machines that can do any job much faster, better, and cheaper than humans. That's the ultimate end point of this progress. The economic value of human labour will drop to zero. That would be an existential threat to our civilization.
- achierius 1y ago
  Where did all the work-horses go? Why is there barely a fraction of the population there once was? Why did they not adapt and find niches where they had a competitive advantage over cars and machines?
  - deleted 1y ago
    [deleted]
  - wickedsight 1y ago
    The horses weren't the market the economy is selling to, the people are. Ford figured out that people having both time and money is best for the economy. We'll figure out that having all the production capabilities but none of the market benefits nobody.
TobTobXX 1y ago
Yes! Sounds like a dream. My value isn't determined by some economic system, but rather by myself. There is so much to do when you don't have to work. Of course, this assumes we actually get to UBI first, and it doesn't create widespread poverty. But even if humanity will have to go through widespread poverty, we'd porbably come out with UBI on the other side (minus a few hundred millions starved). There's so much to do, explore and learn. The prospect of AI stealing my job is only scary because my income depends on this job.
- voidspark 1y ago
  > There's so much to do, explore and learn. Hobbies, hanging out with friends, reading, etc. That's basically it. Probably no international travel. It will be like a simple retirement on a low income, because in a socialist system the resources must be rationed. This will drive a lot of young ambitious people to insanity. Nothing meaningful for them to achieve. No purpose. Drug use, debauchery, depression, violence, degeneracy, gangs. It will be a true idiocracy. No Darwinian selection pressures, unless the system enforces eugenics and population control.
  - pixelbro 1y ago
    Wait, wait, wait. Our society's gonna fall apart due to a lack of Darwinian selection pressure? What do you think we're selecting for right now? Seems to me like our culture treats both survival and reproduction as an inalienable right. Most people would go so far as to say everyone deserves love, "there's a lid for every pot".
    - deleted 1y ago
      [deleted]
  - knowaveragejoe 1y ago
    > This will drive a lot of young ambitious people to insanity. Nothing meaningful for them to achieve. Maybe, if the only flavor of ambition you're aware of is that of SV types. Plenty of people have found achievement and meaning before and alongside the digital revolution world.

thurn 1y ago

Which of these statements do you disagree with? - Superintelligence poses an existential threat to humanity - Predicting the future is famously difficult - Given that uncertainty, we can't rule out the chance of our current AI approach leading to superintelligence - Even a 1-in-1000 existential threat would be extremely serious. If an asteroid had a 1-in-1000 chance of hitting Earth and obliterating humanity we should make serious contingency plans. Second question: how confident are you that you're correct? Are you 99.9% sure? Confident enough to gamble billions of lives on your beliefs? There are almost no statements about the future which I'd assign this level of confidence to.

ZuFyf4Q6K4wjoS 1y ago
[dead]
geysersam 1y ago
Isn't the question you're posing basically Pascals wager? I think the chance they're going to create a "superintelligence" is extremely small. That said I'm sure we're going to have a lot of useful intelligence. But nothing general or self-conscious or powerful enough to be threatening for many decades or even ever. > Predicting the future is famously difficult That's very true, but that fact unfortunately can never be used to motivate any particular action, because you can always say "what if the real threat comes from a different direction?" We can come up with hundreds of doomsday scenarios, most don't involve AI. Acting to minimize the risk of every doomsday scenario (no matter how implausible) is doomsday scenario no. 153.
- polynomial 1y ago
  Yes, this is literally Pascal's wager / Pascal's mugging.
- nearbuy 1y ago
  > I think the chance they're going to create a "superintelligence" is extremely small. I'd say the chance that we never create a superintelligence is extremely small. You either have to believe that for some reason the human brain achieved the maximum intelligence possible, or that progress on AI will just stop for some reason. Most forecasters on prediction markets are predicting AGI within a decade.

utbabya 1y ago

> At the state that we're in the AIs we're building are just really useful input/output devices that respond to a stimuli (e.g., a "prompt"). No stimuli, no output. It was true before we allowed them to access external systems, disregarding certain rule which I forgot the origin. The more general problem is a mix between the tradegy of the common; we have better understanding every passing day yet still don't understand exacly why LLM perform that well emergently instead of engineered that way; and future progress. Do you think you can find a way around access boundaries to masquerade your Create/Update requests as Read in the log system monitoring it, when you have super intelligence?

otabdeveloper4 1y ago

> are just really useful input/output devices that respond to a stimuli LLMs are huge pretrained models. The economic benefit here is that you don't have to train your own text classification model anymore. (The LLM was likely already trained on whatever training set you could think of.) That's a big time and effort saver, but no different from "AI" that we had decades prior. It's just more accessible to the normal person now.

digbybk 1y ago

> I'd really love to talk to someone that both really believes this to be true, and has a hands-on experience with building and using generative AI. Any of the signatories here match your criteria? https://safe.ai/work/statement-on-ai-risk#signatories https://safe.ai/work/statement-on-ai-risk#signatories Or if you’re talking more about everyday engineers working in the field, I suspect the people soldering vacuum tubes to the ENIAC would not necessarily have been the same people with the clearest vision for the future of the computer.

coryfklein 1y ago

Sounds a little too much like, "It's not AGI today ergo it will never become AGI" Does the current AI give productivity benefits to writing code? Probably. Do OpenAI engineers have exclusive access to more capable models that give them a greater productivity boost than others? Also probably. If one exclusive group gets the benefit of developing AI with a 20% productivity boost compared to others, and they develop a 2.0 that grants them a 25% boost, then a 3.0 with a 30% boost, etc... The question eventually becomes, "is AGI technically possible"; is there anything special about meat that cannot be reproduced on silicon? We will find AGI someday, and more than likely that discovery will be aided by the current technologies. It's the path here that matters, not the specific iteration of generative LLM tech we happen to be sitting on in May 2025.

Retric 1y ago
> Does the current AI give productivity benefits to writing code? Probably. > If one exclusive group gets the benefit of developing AI with a 20% productivity boost compared to others, and they develop a 2.0 that grants them a 25% boost, then a 3.0 with a 30% boost, etc... That’s a bit of a stretch, generative AI is least capable of helping with novel code such as needed to make AGI. If anything I’d expect companies working on generative AI to be at a significant disadvantage when trying to make AGI because they’re trying to leverage what they are already working on. That’s fine for incremental improvement, but companies rarely ride one wave of technology to the forefront of the next. Analog > digital photography, ICE > EV, coal mining > oil, etc.
- coryfklein 1y ago
  The "novel AGI code" probably accounts for <5% of work by time spent. If they can reduce the remaining 95% of grunt work (wiring yet another DB query to a frontend, tweaking the build pipeline, automating GPU allocation scripts) then that means they can focus more on that 5%. Then it looks like Company A spends 90% of time on novel research work (while LLMs do all the busy work) and Company B spends 5% of time on novel research work.
  - 1y ago