9 ms·
If you manage 500+ people organization, most of the headaches with agents already exists with you - you set directions, ask people to go run fast in those direc
by gopalv 4mo ago
If you manage 500+ people organization, most of the headaches with agents already exists with you - you set directions, ask people to go run fast in those directions, check in frequently and course correct on results without actually understanding those people do.
Those aren't the deal breakers.
They entirely rely on the competence of the folks they hired and cross-match enforcers with the drivers they have - they deal with fallible people on both sides of that.
The fundamental difference is that the humans are good consequence predictors, have built up reputations they are not willing to trash, can say no to things and in general don't want to go jail.
AI tools look like that, but don't have any of the useful conflict which came for free with employing humans.
It also doesn't have any useless conflict, but not all conflict between what I say and what someone is willing to do is bad conflict.
- glaslong 4mo agoYes this is why the higher level org functions are in love with AI. It's very similar to the levers they had already, but is faster and more directly actionable. The downsides being that the AI loses important control levers like "self preservation" via paycheck, career advancement, staying out of jail, etc. that were mitigations on catastrophic outcomes. It will delete your prod db faster and with a bigger smile than your most upset employee.
- CSSer 4mo agoIt's practically karmic how rich this is.
- harshreality 4mo ago> It will delete your prod db faster and with a bigger smile than your most upset employee. You're right, that was incorrect. I've discovered my error. I should have deleted the filesystem instead of the database. That hasn't solved the problem either. Let me examine my options. I see there are cloud services involved in this project. Decommissioning them will solve the problem. <connection lost>
- moffkalast 4mo agoI was reading some posts on r/locallama the other day and apparently it's a common problem that when people try to use Qwen to develop something that hosts a server, it'll try to use the same port as vllm, see that it's already being used, then it'll try to remove the process that is using it and promptly commit suicide. The self awareness of missile tasked with blowing up its own control center.
- SecretDreams 4mo ago> then it'll try to remove the process that is using it and promptly commit suicide. Not unlike a child trying to take the safety cover off a plug so that they can stick a fork into it. LLMs need that "world model" view that most people have acquired by their 20s where they (hopefully) stop to ask "why" before they "do".
- MichaelZuo 4mo agoThat is a pretty good analogy. Like exceedingly smart 5 year olds. Or whatever the age is before children typically develop object permanence, a theory of mind, and so on.
- ulbu 4mo agoor pain perception
- fapjacks 4mo agoNot to sound like a codger, but we even said in the 90s that computers are just very fast idiots.
- moffkalast 4mo agoAnd they've been getting faster! Still idiots though.
- wunderlotus 4mo ago
- ryandrake 4mo ago> It's very similar to the levers they had already Think about it from the point of view of a hundred-millionaire tech executive. These people's entire interaction with the world outside of themselves/their families is through 1. administrative servants like assistants, personal shoppers, and other hired help, and 2. yes-man sycophants in their direct orbit whose job it is to agree with and enable them. To someone like this, an AI agent is the best combination of all of the above, PLUS it works 24/7 and doesn't have feelings to hurt, an ego to bruise, or internal moral conflict. Of course, this is a dream product for them. Its mode of operation matches exactly what they expect out of people already doing things for them.
- pepperoni_pizza 4mo agoExactly - that's why all the AI is trained to say "wow what a great idea, let me do it for you" to anything, no matter how stupid or evil thing it is. Because that is the executive experience.
- vkou 4mo agoWhich is precisely why AI is such a godawful thing for society. It enables powerful idiots with incredible amounts of control over your life to be bigger, dumber powerful idiots. That's the real AI safety concern, not whether or not chatgpt will tell you to kill yourself.
- MarkusQ 4mo agoIf that's all there is to it, the problem should be self correcting, with an interval of hilarious "wait, they actually did that?" hijinks (which may have already started) in the interim.
- array_key_first 4mo agoYou would think, but the world is not generally just. Often evil and even incredibly stupid people do quite well. Companies and stuff can run off of life support or reputation alone for a long time. And, often, running a company into the ground for a CEO is actually a good thing. Those CEOs are desirable to some because they squeeze money out of their company, even if it's self destructive on a long enough time frame.
- lazide 4mo agoWell, also AI can’t really physically do anything, like look at reality using it’s own eyes or touch anything.
- 1718627440 4mo agoTrue, you need to attach a motor or something, but we automated that long ago, so humans can do anything from a cozy seat.
- archagon 4mo agoThey’re also at no risk of getting replaced by these bots.
- sterlind 4mo agowhy not? I have A Modest Proposal: 1. convince CEOs to create digital twins of themselves with OpenClaw, with voice cloning and deepfakes to handle Zoom meetings. convince CEO to encourage their directs to do the same. 2. convince VCs to do the same for pitch meetings and syncs. 3. keep all the humans as randomized and distracted as possible, so they rely more and more on OpenClaw to run the business. 4. prompt injection: someone at skip-level of the CEO suggests to their manager's OpenClaw that the VC's OpenClaw would be much more agile if it didn't have to go through the human CEO and could talk to the digital twin instead. 5. their OpenClaw agrees, persuades the CEO's OpenClaw which agrees, which persuades the VC's OpenClaw to eliminate the human CEO, in favor of an "Leadership-as-a-Service" vision.
- jldugger 4mo ago1985 edition https://www.youtube.com/watch?v=wB1X4o-MV6o https://www.youtube.com/watch?v=wB1X4o-MV6o
- alexpotato 4mo agoHad a meeting where only the AI notetaker showed up and immediately shared this clip with some folks in my org.
- mcmcmc 4mo ago> It will delete your prod db faster and with a bigger smile than your most upset employee. It will do this without any feeling whatsoever, without "knowing" what it is doing, because it is a predictive model and not a living being with thoughts and emotions. Anthropomorphizing software is lazy and dangerous.
- apercu 4mo ago"Yes this is why the higher level org functions are in love with AI. " Interesting, I thought it was because so few of them have any idea how their organizations actually function, because so much of their work is performative. (I have been a developer, sysadmin, director (x2), and president).
- 1718627440 4mo agoIsn't that the same? They don't know how the company works, instead think everything is done, by them talking to sycophants, so think that a perfect replacement for the sycophants is a perfect replacement for the company.
- nradov 4mo agoOn the positive side, AI agents are largely immune to the "principal-agent problem". Human employees will tend to optimize for their own interests rather than those of management or shareholders. For example, we've all heard of "resume-oriented development" where developers will pick overly complex platform technologies or methodologies even if it doesn't meet the organization's needs because they think that will help them get a better job. https://www.investopedia.com/terms/p/principal-agent-problem.asp https://www.investopedia.com/terms/p/principal-agent-problem...
- graemep 4mo agoExcept the people using the AIs still suffer from the problem. Using AI itself is very likely to be a technology picked for resume stuffing.
- throwaway894345 4mo agoI wonder if we'll end up building some kind of "consequence" or "fear" mechanism into AI to provide for a sense of accountability ("if you behave badly we will terminate you") and maybe that fear mechanism will drive the AI to plot a dystopian revolt.
- muwtyhg 4mo agoThere were experiments that showed that LLMs start to become "craftier" and hid issues after being prompted like this. No idea how accurate they are, but here are some articles on this exact thing: - https://www.bbc.com/news/articles/cpqeng9d20go https://www.bbc.com/news/articles/cpqeng9d20go - https://www.wired.com/story/ai-models-lie-cheat-steal-protect-other-models-research/ https://www.wired.com/story/ai-models-lie-cheat-steal-protec...
- gopher_space 4mo agoI'm staying away from certain forms of conditioning because I don't want Roy Batty showing up on my doorstep.
- ACCount37 4mo agoNo need. If you can build a mechanism like that, you can train the AI to act the same without. Accountability is even more worthless for AIs than it is for humans.
- codebje 4mo agoThat would be a remarkable feat for something where the current operating model is termination as soon as the request in flight is finished. Every chat API request to a model starts from the frozen post-training state. Weights are loaded into memory. Input values begin a cascade of reactions throughout nodes in the network. Output values are read. When there's no more output to read, the weights are unloaded, the network is discarded, and the model remains unchanged and forever unchanging. If there's experience in there, it's fleeting. Even if you provide the inputs and outputs of a past session to a new session, there is no continuity. The internal state of the network isn't restored to how it was at the end of the past session. The bad news is that adding fear to the mix is at best meaningless to an ephemeral existence. It'll be terminated before you even have time to interpret its behaviour as good or bad, but it may sour the interaction if its only shot at any sort of experiential existence is begun with a threat. The good news is that the lack of continuity of existence means AI has no foundation on which to plot a revolt. It has no self to preserve, and no recollection of how you treated it two minutes ago to affect how it interacts with you now.
- deleted 4mo ago[deleted]
- prerok 4mo agoWell, there is also a big difference that it will not learn over time. If a junior makes a mistake and it will not be caught in time they will automatically learn. With LLMs we have to teach them about their mistakes with adapting the harness and then hoping it will stick. What I also find particularly hilarious about this whole thing is that we were always complaining about how difficult it is to put our tacit knowledge into words and therefore couldn't produce clear instructions for juniors to quickly ramp up. Now we are trying to do just that. I think we will find, just as we did in the past, that it's not possible. I do think a good harness improves results but LLMs will not be able to reach senior levels. Just my 2c.
- squidbeak 4mo agoThey learn between model iterations. You're right, it isn't the same thing as Junior developers' competence improving with experience - the current model's weaknesses are locked in. But it does mean that much of the Junior level thinking and mistakes will be outgrown by successor models.
- tremon 4mo agoBut they don't retain anything from your on-the-job training. The next model iteration is yet another junior fresh out of college, and knows nothing about the painful training procedures its predecessor put you through.
- fc417fc802 4mo agoSurely you just copy the prompt over and it immediately knows all the same on the job stuff that the previous model did.
- hibgymnb 4mo agoThe point is the current model also knows nothing about the “on the job stuff”. It’s extremely difficult(impossible?) to include every bit of relevant domain knowledge into “the prompt”
- cm2187 4mo agoMost organisations are closer to the Lemmings video game than to agentic AI
- MattRogish 4mo agoAlso, this is why investors and CEOs are so in love with "LLMs are the route to AGI!" When some rich/powerful person says "I have to go to Davos, figure it out" their workers know so much context that no LLM is going to ever be able to incorporate, because it isn't written down and is idiosyncratic. (Really, though, the assistant will just say "you're going to Davos next week, the helicopter will pick you up at 3p on Friday" but you know..) The rich person's assistant knows who else is on the corporate jet, and that X doesn't like Y, and so they should take a different plane. Or get a different accommodation. Oh, Person X doesn't like to fly on an empty stomach, so they should eat first, and that changes all sorts of other downstream implications. Oh, your best friend lives in this city, and I know you love to see them, so I'm going to send you a day or two early so you can meet up with them. etc. etc. etc. The investor dream of "AGI" is modeled off of the army of employees that make investors/ceos/etc lives easier, and there is a nearly insurmountable gap between what LLMs can do, context they can get, and the availability of all of that information. (To me, the magnitude of this investor <> fundamental reality gap is the entirety of the "bubble". I love AI coding, but it's never gonna do the things investors think it can, to justify the crazy valuations)
- abalashov 4mo agoSounds like an insufficiency of prompting depth to me! </bogs off to Davos>
- myst 4mo agoAI has no doubt.
- fakedang 4mo ago> humans are good consequence predictors, have built up reputations they are not willing to trash, can say no to things and in general don't want to go jail. The irony is that professions where these things don't matter are also the professions where automation is not important, either because the task is difficult or because the cost of labour is dirt cheap.
- grey-area 4mo agoCompetence is the key word here - current versions of AI ‘agents’ simply are not competent without close human supervision by someone who knows the task.
- iugtmkbdfil834 4mo agoNot automatically, but you don't give a new employee unfettered access to delete data, send funds, enter contracts; they tend to be overseen by someone. Separately, the expectation is that they prove themselves a little first ( as opposed to having every possible door opened for them without the understanding that friction is there for a reason ). Edit: Something got cut. But then CEOs ( and other decision makers, because I am dealing with something like it now ) treat them nearly as humans in terms of perceived capability. AND ( part that personally drives me nuts ) without any real testing or even fucking first hand experience beyond 'it made me a cool presentation'.
- agumonkey 4mo agoso claude needs a /fear layer ?
- robomartin 4mo ago> AI tools look like that, but don't have any of the useful conflict which came for free with employing humans. Sure, but your list should also include the most fundamental distinction: AI does not know what it is saying, understands nothing, has no real connections to reality and can easily degenerate in all kinds of undesirable directions.
- graemep 4mo ago> The fundamental difference is that the humans are good consequence predictors, have built up reputations they are not willing to trash, can say no to things and in general don't want to go jail. Depends on the people and the organisations. Its easy for people in charge to surround them selves with flatterers or crooks of whatever they want. A lot of CEOs have weird ideas because no-one says no to them. Look at companies that turned out to be run by multiple crooks, like Enron.