7 ms·

weird - even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing

weird - even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do. Unless you're of course, willing to YOLO the entire company on output you haven't actually checked yourself. for that reason alone humans will always need to be in the loop. of course you can debate how many people you need to the above activity, but given that AI isn't omniscient, nor omnipotent I expect that number to be quite high for the foreseeable future. one example - I've been vibe coding some stuff, and even though a pretty comprehensive set of tests are passing, I still end up reading all of the code. if I'm being honest some of the decisions the AI makes are a bit opaque to me so I end up spending a bunch of time asking it why (of course there's no real ego there, but bare with me...), re-reading the code, thinking about whether that actually makes sense. I personally prefer this activity/mode since the tests pass (which were written by the AI too), and I know anything I manually change can be tested, but it's not something I could just submit to prod right away. this is just a MVP. I can't imagine delegating if real money/customers were on the line without even more scrutiny.

w4yai 9mo ago
> even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do no no no you don't get it, you would have ANOTHER AI for that
- morkalork 9mo ago
  You're being sarcastic but if I "LLM as judge" one more time, I might jump off a bridge. Also, it does appear that there are companies willing to YOLO themselves off a cliff with AI
gradus_ad 9mo ago
It's not even about humans "needing" to be in the loop, but that humans "want" to be in the loop. AI is like a genius employee who has no ego and no desire to rise up the ranks, forever a peon while more willful colleagues surpass them in the hierarchy. Until AI gets ego and will of its own (probably the end of humanity) it will simply be a tool, regardless of how intelligent and capable it is.
- hnlmorg 9mo ago
  Humans need to be in the loop for the same reason other humans peer review humans pull requests: we all fuck up. And AI makes just as many mistakes as humans do. They just do so significantly quicker.
- undersuit 9mo ago
  Yes, "Mecha-hitler" has no aspirations. /s
- only-one1701 9mo ago
  This is the opposite of both what the article is saying, and reality
serf 9mo ago
>weird - even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do. one would hope that one ability of an 'omniscient and omnipotent' AI would be greater understanding. When speaking of the divine (the only typical example of the omniscient and omnipotent that comes to mind) we never consider what happens when God (or whoever) misunderstands our intent -- we just rely on the fact that an All-Being type thing would just know. I think the understanding of minute intent is one such trait an omniscient and omnipotent system must have. p.s. what a bar raise -- we used to just be happy with AGI!
- danenania 9mo ago
  That’s because gods are a mythical/supernatural invention. No technology can ever really be omniscient or omnipotent. It will always have limitations. In reality, even an ASI won’t know your intent unless you communicate it clearly and unambiguously.
  - consumer451 9mo ago
    > In reality, even an ASI won’t know your intent unless you communicate it clearly and unambiguously. I recently came to this realization as well, and it now seems so obvious. I feel dumb for not realizing it sooner. Is there any good writing or podcast on this topic?
  - bdangubic 9mo ago
    The communication I get from customers is seldom clear and never unambiguous but I’ve managed since the 90’s
    - array_key_first 9mo ago
      Right, but you have to do a lot of work, and really most of your work is in this area. Less on the actual building stuff. Figuring out what to build is 80% of the work, building it is maybe 20%. The 20% has never been the bottleneck. We make a lot of software, and most of it is not optimal and requires years if not decades of tweaking to meet the true requirements.
Mountain_Skies 9mo ago
Move fast and break things. When a black box can be blamed, why care about quality? What we need is EXTREMELY strict liability on harms done by AIs and other black box processes. If a company adopts a black box, that should be considered reckless behavior until proven otherwise. Taking humans out of the loop is a conscious decision they make therefore they should be fully responsible for any mistakes or harms that result.
- callc 9mo ago
  Shhhhh that’s a primary unspoken feature - lack of responsibility
65 9mo ago
I've always found it much quicker to just... do the work myself. AI slows me down more than anything.
- websiteapi 9mo ago
  fair. I used to think that too, but I find at least for golang, the sota models write tests way faster than I would be able to. tdd is actually really possible with ai imo. except of course you get the scaffolding implementation (I haven't figured out a way to get models to write tests in a way that ensures the tests actually do something useful without an implementation).
  - bediger4000 9mo ago
    Your final sentence is interesting. I'm not a strict doctrine adherent, but in TDD, don't you write some minimal test, then implement the system to pass the test?
    - websiteapi 9mo ago
      yes, but I find it hard to constrain it to a minimal implementation. what usually happens is it writes some tests, then an implementation, and then according to the thinking, makes some modification. it works with a relatively precise prompt, but starts to go a bit off the rails when you say things in broad terms ("write tests to ensure concurrency works, and the implementation to ensure said tests are correct")
nick486 9mo ago
>you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do. this sort of assumes that most humans actually know what they want to do. It is very untrue in my experience. Its like most complaints I hear about AI art. yes, it is generic and bland. just like 90% of what human artists produce.
veunes 9mo ago
The problem goes deeper: verification is harder than generation. When writing an answer yourself, you build the logic chain from scratch. When verifying AI, you have to deconstruct its logic, cross-reference facts, spot hidden hallucinations, and only then approve. For complex cases (which are exactly what the humans were left with), the time for quality verification approaches the time to write from scratch. If the time becomes roughly equal, the AI stops being an accelerator and becomes just a source of noise that yields no productivity gains

w4yai 9mo ago

> even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do no no no you don't get it, you would have ANOTHER AI for that

morkalork 9mo ago
You're being sarcastic but if I "LLM as judge" one more time, I might jump off a bridge. Also, it does appear that there are companies willing to YOLO themselves off a cliff with AI

gradus_ad 9mo ago

It's not even about humans "needing" to be in the loop, but that humans "want" to be in the loop. AI is like a genius employee who has no ego and no desire to rise up the ranks, forever a peon while more willful colleagues surpass them in the hierarchy. Until AI gets ego and will of its own (probably the end of humanity) it will simply be a tool, regardless of how intelligent and capable it is.

hnlmorg 9mo ago
Humans need to be in the loop for the same reason other humans peer review humans pull requests: we all fuck up. And AI makes just as many mistakes as humans do. They just do so significantly quicker.
undersuit 9mo ago
Yes, "Mecha-hitler" has no aspirations. /s
only-one1701 9mo ago
This is the opposite of both what the article is saying, and reality

serf 9mo ago

>weird - even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do. one would hope that one ability of an 'omniscient and omnipotent' AI would be greater understanding. When speaking of the divine (the only typical example of the omniscient and omnipotent that comes to mind) we never consider what happens when God (or whoever) misunderstands our intent -- we just rely on the fact that an All-Being type thing would just know. I think the understanding of minute intent is one such trait an omniscient and omnipotent system must have. p.s. what a bar raise -- we used to just be happy with AGI!

danenania 9mo ago
That’s because gods are a mythical/supernatural invention. No technology can ever really be omniscient or omnipotent. It will always have limitations. In reality, even an ASI won’t know your intent unless you communicate it clearly and unambiguously.
- consumer451 9mo ago
  > In reality, even an ASI won’t know your intent unless you communicate it clearly and unambiguously. I recently came to this realization as well, and it now seems so obvious. I feel dumb for not realizing it sooner. Is there any good writing or podcast on this topic?
- bdangubic 9mo ago
  The communication I get from customers is seldom clear and never unambiguous but I’ve managed since the 90’s
  - array_key_first 9mo ago
    Right, but you have to do a lot of work, and really most of your work is in this area. Less on the actual building stuff. Figuring out what to build is 80% of the work, building it is maybe 20%. The 20% has never been the bottleneck. We make a lot of software, and most of it is not optimal and requires years if not decades of tweaking to meet the true requirements.

Mountain_Skies 9mo ago

Move fast and break things. When a black box can be blamed, why care about quality? What we need is EXTREMELY strict liability on harms done by AIs and other black box processes. If a company adopts a black box, that should be considered reckless behavior until proven otherwise. Taking humans out of the loop is a conscious decision they make therefore they should be fully responsible for any mistakes or harms that result.

callc 9mo ago
Shhhhh that’s a primary unspoken feature - lack of responsibility

65 9mo ago

I've always found it much quicker to just... do the work myself. AI slows me down more than anything.

websiteapi 9mo ago
fair. I used to think that too, but I find at least for golang, the sota models write tests way faster than I would be able to. tdd is actually really possible with ai imo. except of course you get the scaffolding implementation (I haven't figured out a way to get models to write tests in a way that ensures the tests actually do something useful without an implementation).
- bediger4000 9mo ago
  Your final sentence is interesting. I'm not a strict doctrine adherent, but in TDD, don't you write some minimal test, then implement the system to pass the test?
  - websiteapi 9mo ago
    yes, but I find it hard to constrain it to a minimal implementation. what usually happens is it writes some tests, then an implementation, and then according to the thinking, makes some modification. it works with a relatively precise prompt, but starts to go a bit off the rails when you say things in broad terms ("write tests to ensure concurrency works, and the implementation to ensure said tests are correct")

nick486 9mo ago

>you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do. this sort of assumes that most humans actually know what they want to do. It is very untrue in my experience. Its like most complaints I hear about AI art. yes, it is generic and bland. just like 90% of what human artists produce.

veunes 9mo ago

The problem goes deeper: verification is harder than generation. When writing an answer yourself, you build the logic chain from scratch. When verifying AI, you have to deconstruct its logic, cross-reference facts, spot hidden hallucinations, and only then approve. For complex cases (which are exactly what the humans were left with), the time for quality verification approaches the time to write from scratch. If the time becomes roughly equal, the AI stops being an accelerator and becomes just a source of noise that yields no productivity gains