4 ms·
i thought my post was already too long to include this, but to your point, you can run AI inference in this setup and the performance can be pretty good.
by scottjg 8mo ago
i thought my post was already too long to include this, but to your point, you can run AI inference in this setup and the performance can be pretty good.
- colechristensen 8mo agoYeah it is obviously actually useful for AI inference over which the pcie speed isn't particularly important and a single board computer gets you a small system.
- geerlingguy 8mo agoThere are definitely some use cases where it works out, others where it doesn't; I spent a bit of time testing that side of things late last year: https://www.jeffgeerling.com/blog/2025/big-gpus-dont-need-big-pcs/ https://www.jeffgeerling.com/blog/2025/big-gpus-dont-need-bi...
- scottjg 8mo agoa great post that definitely inspired this one. i link to it in the first paragraph of my blog post.
- geerlingguy 8mo agoI feel like maybe by the end of this year someone with access to a bunch of RTX Pro 6000s will have them running on a Pi or RK3588 lol.
- scottjg 8mo agowe can only hope
- shmeeed 8mo agoI appreciate you making the post not about AI.