6 ms·

Or it’ll accelerate the push to the cloud even more making the client even thinner.

by Schnitz 29d ago

api 29d ago
Memory in the cloud isn't free either.
- flohofwoe 29d ago
  Easier to monetize though ;)
- deleted 29d ago
  [deleted]
- 8note 29d ago
  it isnt free but its easier to get 99.999% of the time usage out of it, vs having the client only sometimes have their machine on
  - deleted 29d ago
    [deleted]
- gosub100 28d ago
  it's cheaper considering they buy in bulk and can expense it against their taxes
stephen_cagle 29d ago
Concidently, I asked Claude yesterday whether we have crossed the point that there is more compute and memory in the cloud or in personal computers. It claims fp8 and fp16 has had higher throughput in the cloud since 2023. fp32 and general computing still has more compute than personal computers. And that memory is at the crossover point, but at this point only 10% of memory is going to personal computers, so the next few years will radically change the pc to cloud memory ratios. https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0af152549e32 https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0... Anyway, all of this made me a (tiny bit) sad. It is the gradual end of general purpose computing you own, and the start of everything just being cloud compute that someone else owns and you rent.
- api 29d ago
  I agree with the sentiment, but there is a devils' advocate argument: Most compute loads are bursty. Most PCs and phones sit idle almost all the time. It's an inefficient use of capital. In the cloud you can be idle and then suddenly command 1000X more power than any affordable PC could ever have... for 10 seconds... then be idle again. Those kinds of work loads aren't possible to realize on a PC without massively overbuying hardware. I'd like to see more research on secure private cloud computing. Homomorphic encryption has gotten better but is still very slow, but there's other approaches that can help quite a bit but don't require much of a performance hit. Look into secure multiparty computation, reversible anonymization, etc. I don't think personal computing will die, but I think it'll go back to being a hobbyist and pro thing like it was from its birth in the 1970s up until the middle 1990s. In the 1980s the average person did not own a computer, or if they did it was just to play some games. The average person was not loading BASIC programs off bootleg tapes and calling BBSes. BTW I've been thinking a lot lately about how AI could save personal computing by allowing us to create an off-ramp from the present rats nest of complexity to something saner and simpler. Just thoughts right now but it's interesting.
  - wizzwizz4 29d ago
    Generative AI is definitely creating political will for simpler, more purposely-designed computer systems, but it's certainly not going to directly create help them in any meaningful way. AI-generated codebases are the posterchild of rat's nests of complexity.
    - api 29d ago
      I'd say the code I see coming out of leading models today is okay. Not great, but not terrible. It's also easy to get LLMs to code in more efficient languages, and you can also goad them via prompts and CLAUDE.md type stuff into prioritizing performance and simplicity. It's sort of like what I say about self driving cars. Are they better than good human drivers? Not yet. Are they better than bad, drunk, or texting drivers? Yes. LLMs are already better than bad coders and honestly most coders are bad coders.
ratmeadow 29d ago
1. Make shit inefficient software for basic tasks 2. Say it's infeasible to have enough compute locally to do basic tasks 3. Force usage of centrally controlled, subscription-based cloud software. 4. Capitalism win???!
- 20after4 27d ago
  I think this is legitimately (at least part of) the motivation for cornering the supply and driving up the price.

Or it’ll accelerate the push to the cloud even more making the client even thinner.

by Schnitz 29d ago

Or it’ll accelerate the push to the cloud even more making the client even thinner.

api 29d ago

Memory in the cloud isn't free either.

flohofwoe 29d ago
Easier to monetize though ;)
deleted 29d ago
[deleted]
8note 29d ago
it isnt free but its easier to get 99.999% of the time usage out of it, vs having the client only sometimes have their machine on
- deleted 29d ago
  [deleted]
gosub100 28d ago
it's cheaper considering they buy in bulk and can expense it against their taxes

stephen_cagle 29d ago

Concidently, I asked Claude yesterday whether we have crossed the point that there is more compute and memory in the cloud or in personal computers. It claims fp8 and fp16 has had higher throughput in the cloud since 2023. fp32 and general computing still has more compute than personal computers. And that memory is at the crossover point, but at this point only 10% of memory is going to personal computers, so the next few years will radically change the pc to cloud memory ratios. https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0af152549e32 https://claude.ai/public/artifacts/8733239a-328b-4aaa-8d76-0... Anyway, all of this made me a (tiny bit) sad. It is the gradual end of general purpose computing you own, and the start of everything just being cloud compute that someone else owns and you rent.

api 29d ago
I agree with the sentiment, but there is a devils' advocate argument: Most compute loads are bursty. Most PCs and phones sit idle almost all the time. It's an inefficient use of capital. In the cloud you can be idle and then suddenly command 1000X more power than any affordable PC could ever have... for 10 seconds... then be idle again. Those kinds of work loads aren't possible to realize on a PC without massively overbuying hardware. I'd like to see more research on secure private cloud computing. Homomorphic encryption has gotten better but is still very slow, but there's other approaches that can help quite a bit but don't require much of a performance hit. Look into secure multiparty computation, reversible anonymization, etc. I don't think personal computing will die, but I think it'll go back to being a hobbyist and pro thing like it was from its birth in the 1970s up until the middle 1990s. In the 1980s the average person did not own a computer, or if they did it was just to play some games. The average person was not loading BASIC programs off bootleg tapes and calling BBSes. BTW I've been thinking a lot lately about how AI could save personal computing by allowing us to create an off-ramp from the present rats nest of complexity to something saner and simpler. Just thoughts right now but it's interesting.
- wizzwizz4 29d ago
  Generative AI is definitely creating political will for simpler, more purposely-designed computer systems, but it's certainly not going to directly create help them in any meaningful way. AI-generated codebases are the posterchild of rat's nests of complexity.
  - api 29d ago
    I'd say the code I see coming out of leading models today is okay. Not great, but not terrible. It's also easy to get LLMs to code in more efficient languages, and you can also goad them via prompts and CLAUDE.md type stuff into prioritizing performance and simplicity. It's sort of like what I say about self driving cars. Are they better than good human drivers? Not yet. Are they better than bad, drunk, or texting drivers? Yes. LLMs are already better than bad coders and honestly most coders are bad coders.

ratmeadow 29d ago

1. Make shit inefficient software for basic tasks 2. Say it's infeasible to have enough compute locally to do basic tasks 3. Force usage of centrally controlled, subscription-based cloud software. 4. Capitalism win???!

20after4 27d ago
I think this is legitimately (at least part of) the motivation for cornering the supply and driving up the price.