5 ms·
The framing they use is hilarious and their little graphic is perfect. The risk of harm doesn't go down, but the reward goes up, so the harm just becomes the co
by 6gvONxR4sf7o 4mo ago
The framing they use is hilarious and their little graphic is perfect. The risk of harm doesn't go down, but the reward goes up, so the harm just becomes the cost of doing business, justified by the reward. So as the reward gets higher and higher, the amount of harm they're willing to justify goes up. Feels like society in a nutshell.
- heisenbit 3mo agoLimited liability makes taking unlimited risks a rational choice. AI ‚only‘ scales this corporate model up and compresses the timeframe to the next disaster.
- ronsor 4mo agoThis is how humans weigh most decisions in practice.
- Maxious 4mo ago[dead]
- esikich 4mo agoSure. You start a PC repair business. At first, losing a stick of RAM or frying someone's motherboard is super costly when you are doing 10 a week. But once you're doing 1000, that's pretty damn good and easily covered. When you have more tools, velocity, and whatnot, the proportions change.
- altmanaltman 4mo agoWouldn't you lose multiple sticks or fry multiple motherboards as you scale and do 1000? If you're frying 1 at 10, that means you're frying 100 at 1000. Your costs etc will scale as well unless you actually lower the risk/reward ratio, no?
- kuboble 4mo agoI think the point is that at small scale a single accident poses a risk of ruin to your small operations.
- chrncirurp 4mo ago> I think the point is that at small scale a single accident poses a risk of ruin to your small operations. At big scale, a single big accident poses a risk to ruin your big operations.
- enraged_camel 3mo agoNo, it does not. Every large company eventually has a big accident. They survive because they have both the resources (e.g. to fight ensuing legal battles, or pay fines, or simply weather a hit to reputation and the resulting downturn in revenue) as well as redundancy, different types of insurance, and so on.
- zaphar 3mo agoThey also survive because they invest those resources in some amount of mitigation ahead of time. They don't survive when they don't scale their mitigations along with the business.
- daveshistory 3mo agoCompanies of all sizes should have insurance to cover such scenarios. You need to get tradesman's insurance on your repair work, or you need to ask yourself why the insurance companies won't insure you.
- truculent 3mo agoThe point is that if you have a 10% chance of frying motherboard, at 10 a week, you might expect 1 fried p/w, but it could easily be more which may be catastrophic. At 1000, the number of fried boards will be more predictable and therefore the risk to the business is lower, even if the long-run averages are the same.
- andai 4mo agoYeah I was thinking about Simon Wilson's "lethal trifecta"[0] in the context of OpenClaw style "general purpose" AI agents, where people just gave it access to their full hard drive, gmail account, etc. I was thinking you can't make the chance of catastrophic failure zero (we still hear about "Claude deleted my home folder"), but you can definitely limit the blast radius. You can't get the risk to zero. But the opportunity cost of not playing the game is rising. So you accept some level of risk. My personal take here is "why screw around with containers and virtualization when a used ThinkPad is $50". Just give it its own machine. Then it can blow it up all it wants. (Or a $3 VPS, as the case may be :) [0] The lethal trifecta for AI agents: private data, untrusted content, and external communication - https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
- koolba 4mo ago> Then it can blow it up all it wants. (Or a $3 VPS, as the case may be :) Just make sure it doesn’t have ssh access to any other machines!
- charcircuit 4mo agoAll of ecommerce is built on top of encryption with a non 0 chance of being cracked. The risk is much smaller than the benefit so people are willing to use it and then deal with whatever potential fraud comes from encryption being broken separately. Technically a merchant could require meeting in person to exchange a OTP to avoid this and make it 0 but it is not worth it and you will get out competed by other businesses willing to take on a marginally higher amount of risk to unlock a lot of utility for the user.
- chrisweekly 4mo agoIs a used Thinkpad really a viable part of your AI workflow? (And is that really a better solution than eg smolmachines microvms?)
- zaptheimpaler 4mo agoI tried the VPS briefly, it didn't really solve anything for me. The personal assistant agent is only as useful as the data & tools it has, that's where the real risk is. Separate box gives you isolated FS but docker also does that very easily.
- 7e 4mo agoThey don’t consider risk of ruin and that is where this calculus falls apart. The reward does not reduce the risk of ruin, which increases with blast radius. YOLO!
- xp84 4mo agoI’m a usual booster of AI (others have accused me of being completely in the bag for the clankers) and even I agree fully. These yahoos would clearly give Claude the nuclear launch codes or enough access to copy its full model into the wild if the supposed “reward” promised was large enough.
- daveshistory 3mo agoHardly a new hypothetical scenario, that Wargames movie is probably 40 years old now.
- keithnz 4mo agobut no matter what you do this is the tradeoff you are making. Different people have different tolerances for that balance, hence why I'm happy to watch people on youtube in wingsuits and not do it myself. Of course in this new AI world, quantifying the probability and scale of harm is hard/not fully known. We are trying to mitigate risks with AI, but who knows, could be one misstep away from plummeting off a cliff.
- deleted 4mo ago[deleted]
- solenoid0937 4mo agoThat's how decisions are made IRL. Risk/reward is a thing.
- vrganj 4mo agoThis is risk to us and reward for them though.
- alansaber 3mo agoExactly. Though with inference cost they're still only making money on enterprise use.
- daveshistory 3mo agoMany companies would say that's the best kind of risk-reward balance. For them, anyway.
- TeMPOraL 3mo agoBecause we're all paying for LLM access for shits and giggles, and not because we're getting actual value from it.
- soundworlds 4mo agoIf I understand this correctly, Anthropic's argument is now "yes this will blow up some of your infrastructure, but it will be worth it" The problem is that no one has been able to prove that it is actually worth the cost. That is a very fragile assumption.
- szundi 4mo ago[dead]
- alansaber 3mo agoThis has always been the premise. They can't fix the fundamental problems with LLMs but they can continue to optimise them for IE parsing large volumes of data quickly
- daveshistory 3mo agoIt's Shrek logic. "Some of you are going to die, and that is a sacrifice I am willing to make."
- TeMPOraL 3mo agoNo, it's the actual reasonable approach that sane people have to security. In the real world, security is always about costs and benefits, because you can always make something more secure than it is by spending more money, but it also doesn't make sense to spend more than you're getting from it. Normally, you secure things up to minimize (${cost of security measures} + ${expected damage from attacks that materialized}), writing off actual material damage with insurance wherever possible. You pick security measures based on their effectiveness, which usually translates to "how expensive will it make success for attackers", aiming to push that above the value the attackers can expect to gain. There are obvious exceptions to that, like risk to life and limb, as well as some other special situations where attackers may have unusual motivations and thus the economic logic of "make stealing treasure cost more than the treasure" stops applying. But those are exceptions. Almost everything you deal with in your life - from your bike shed to the corporation that owns your bank - follows the above logic in terms of security. -- I spell this out because I've noticed that tech industry circles have this weird, belief in security as some kind of binary, holy good, that you either have and are blessed, or don't and sin. This obsession starts with failing to even recognize, much less ask, the most important questions about security: why do you want to protect it, and who are you protecting it from?
- Frieren 4mo ago> the amount of harm they're willing to justify goes up. Feels like society in a nutshell. Neocon society. Socialism is not like that.
- pjc50 3mo agoRunning into the problem that Americans are very bad at defining "socialism" here, meaning anything from social democrat to full Communism, but: there is a strong utilitarian streak in socialist societies that is also vulnerable to "the pain (for you) will be worth it (for someone else)" reasoning.
- Frieren 3mo ago> there is a strong utilitarian streak in socialist societies that is also vulnerable to "the pain (for you) will be worth it (for someone else)" reasoning. Socialism is not perfect, it is just better than any other alternative.
- radu_floricica 3mo agoWell, yeah, which is why it's evil. Socialism I mean. How else would you call failing to do basic utility math while insisting you should govern and shape society? My answer to the trolley problem is that you're allowed to not kill... unless you're the railway manager. If you're in a position of authority you pull the shit out of that switch, and then drink yourself to sleep at night. This is what authority means, not choosing the "feel good, ignore the people that could have been saved" path.
- jon-wood 3mo agoEverything you do a risk/reward equation, you just don't usually see it drawn out quite so starkly. Getting out of bed in the morning carries a risk that you'll trip and crack your head on the floor. Crossing a road carries a risk of being hit by a bus. Eating food carries a risk of choking on it. The same is true in computer security. The only truly secure computer is one you don't turn on, and even that carries some risk of an attacker breaking in and stealing the storage from it. Whether you agree that the potential harms outweigh the benefits in this case or not those calculations are always happening, so yes, I guess you're right. That is society in a nutshell.
- vrganj 3mo agoBut if you eat food, I don't risk choking. They want us to take the risk for their reward.
- hombre_fatal 3mo agoWhat do you have in mind? You're paying for their services to collect reward for yourself, but also deciding your own risk/reward when choosing e.g. how much access to grant Claude for any given task. I guess there's the case where the more capable Claude is, the more someone else can use it to find vulns in your services while Anthropic collects their subscription money? But that is mitigable risk that you shipped regardless of what Anthropic is doing.
- pixl97 3mo agoBut if I drive a car, You do run the risk of getting ran over. We can come up with any number of analogies of varying rightness and wrongness here.
- daveshistory 3mo agoAnd then there's a whole truckload of case law about liability that comes into play. We haven't yet written those laws for "AI."
- 6gvONxR4sf7o 3mo ago