6 ms·

Sure, incentives can be gamed. The same oversight mechanism that applies to humans cannot correct the flaws of AI agents? What do you think is the catch? I am

Sure, incentives can be gamed. The same oversight mechanism that applies to humans cannot correct the flaws of AI agents? What do you think is the catch? I am not saying things are clearly defined in most settings. But my accounting agent ( real person) gets paid only when he files my tax returns.

free_bip 9mo ago
Right, it doesn't work the same for humans as it does AI agents. If you finetune a model and it starts misbehaving, what are you going to do to it exactly? PIP it? Fire it? Of course not. AIs cannot be managed the same ways as humans (and I would argue that's for the best). Best you can do is try using a different model, but you have no guarantee that whatever issue your model has is actually solved in the new one.
lbreakjai 9mo ago
Humans respect the rules because if they don't, then they lose their jobs, can't pay their mortgages, and become homeless. That's quite a powerful incentive not to fudge the numbers too much. There's no LLM equivalent.
- rajvarkala 9mo ago
  The agent builder loses contract .. Is this not force enough to make AI worthwhile?
  - lbreakjai 9mo ago
    Why would the AI care? The agent builder is still asking a non-deterministic black box with no skin in the game to behave a certain way, they have no guarantees.
    - rajvarkala 9mo ago
      If the AI is never going to be manageable, never trustable, then the whole idea of agentic systems is dumb. What is the point of an agent running and you don't trust it? That would be equivalent to calling this whole AI wave useless. May be it is, maybe it is not.
      - lbreakjai 9mo ago
        That's ... how they fundamentally work?
Neywiny 9mo ago
I think it gets more nebulous. For example, does he only get paid if the tax returns are accepted by the government? If they aren't, he's still put in the work. This becomes an extremely slippery slope. A better example is probably retail. In the US at least, places like Walmart and Amazon allow for returns, but they usually just throw it out. That's gotta be built into the price. Meaning, the cheapy no returns accepted online stores are cheaper because the cost for the purchaser isn't tied to satisfaction. Your accountant has to build in margin that you pay for for clients who stiff him on the bill or who he has to take to court to argue he did the service as described in the contract. If you didn't hold that threshold over his head, he would be able to charge less. Would he? Maybe not, I don't know the guy, but he could.
- rajvarkala 9mo ago
  Understood. So, a better way is to keep him on a retainer? Or let Amazon or Cheaper store do a cost-plus model? I think that is the core of the argument. It is the risk-sharing between buyer and seller. If sold on outcomes, seller carries all risk. If sold on work-put-in, buyer carries all risk. Add to that, in some scenarios, outcomes themselves are fuzzy.
  - Neywiny 9mo ago
    Cost plus I'm not sure on. Maybe if your work was in small enough chunks. But if, did example, just generating one response is too expensive, there's no plus, it's just somebody paid $ for some bits in GPU memory and that's likely not useful to anyone. Yes exactly. Your second paragraph hits the nail on the head. And I'm sure you agree that the AI companies aren't going to take on more risk for free.

free_bip 9mo ago

Right, it doesn't work the same for humans as it does AI agents. If you finetune a model and it starts misbehaving, what are you going to do to it exactly? PIP it? Fire it? Of course not. AIs cannot be managed the same ways as humans (and I would argue that's for the best). Best you can do is try using a different model, but you have no guarantee that whatever issue your model has is actually solved in the new one.

lbreakjai 9mo ago

Humans respect the rules because if they don't, then they lose their jobs, can't pay their mortgages, and become homeless. That's quite a powerful incentive not to fudge the numbers too much. There's no LLM equivalent.

rajvarkala 9mo ago
The agent builder loses contract .. Is this not force enough to make AI worthwhile?
- lbreakjai 9mo ago
  Why would the AI care? The agent builder is still asking a non-deterministic black box with no skin in the game to behave a certain way, they have no guarantees.
  - rajvarkala 9mo ago
    If the AI is never going to be manageable, never trustable, then the whole idea of agentic systems is dumb. What is the point of an agent running and you don't trust it? That would be equivalent to calling this whole AI wave useless. May be it is, maybe it is not.
    - lbreakjai 9mo ago
      That's ... how they fundamentally work?

Neywiny 9mo ago

I think it gets more nebulous. For example, does he only get paid if the tax returns are accepted by the government? If they aren't, he's still put in the work. This becomes an extremely slippery slope. A better example is probably retail. In the US at least, places like Walmart and Amazon allow for returns, but they usually just throw it out. That's gotta be built into the price. Meaning, the cheapy no returns accepted online stores are cheaper because the cost for the purchaser isn't tied to satisfaction. Your accountant has to build in margin that you pay for for clients who stiff him on the bill or who he has to take to court to argue he did the service as described in the contract. If you didn't hold that threshold over his head, he would be able to charge less. Would he? Maybe not, I don't know the guy, but he could.

rajvarkala 9mo ago
Understood. So, a better way is to keep him on a retainer? Or let Amazon or Cheaper store do a cost-plus model? I think that is the core of the argument. It is the risk-sharing between buyer and seller. If sold on outcomes, seller carries all risk. If sold on work-put-in, buyer carries all risk. Add to that, in some scenarios, outcomes themselves are fuzzy.
- Neywiny 9mo ago
  Cost plus I'm not sure on. Maybe if your work was in small enough chunks. But if, did example, just generating one response is too expensive, there's no plus, it's just somebody paid $ for some bits in GPU memory and that's likely not useful to anyone. Yes exactly. Your second paragraph hits the nail on the head. And I'm sure you agree that the AI companies aren't going to take on more risk for free.