6 ms·
Yeah but energy is still a cost, and local inference without batch and multiplexing for many users (like an office would be) is even less optimal. I'm not sure
by alainrk 1mo ago
Yeah but energy is still a cost, and local inference without batch and multiplexing for many users (like an office would be) is even less optimal.
I'm not sure I think it's just pushing the problem forward
- deadbabe 1mo agoThen there will also be a push for cheaper, freer energy as well.
- yladiz 1mo agoYeah, with all the AI needs we’ll just have to transition to fusion energy.
- deadbabe 1mo agoAnd geothermal. Not even for the environmental aspects, just purely from energy demands, at any spot on the Earth where a data center could go.
- Geezus_42 1mo agoHahahahaha sorry, that's hilarious.