7 ms·
Has anyone considered a home server? Assuming mobility is not important if we pick components to match a similar hardware would it be more value for money?
by prasanthabr 3mo ago
Has anyone considered a home server? Assuming mobility is not important if we pick components to match a similar hardware would it be more value for money?
- LeBit 3mo agoWhich components are you thinking about?
- prasanthabr 3mo agoAm unsure - was hoping someone tried this and there is a tested component list of consumer grade pc parts that can do the trick
- drillsteps5 3mo agoA decent gaming machine perfectly doubles as your friendly local inference server. Just start llama-server with the model of your choosing and start chatting with it through its Web interface or connect any chat completion-compatible client (agentic or not) which will use REST to send requests and receive responses. From any device on your network. Voila.
- cpburns2009 3mo agoGenerally speaking a home server/workstation set up is going to provide better performance at lower cost. You don't sacrifice much mobility either so long as you have an internet connection and can either SSH tunnel or use Tailscale (never used, just know it's popular).
- Greenpants 3mo agoI specifically chose a Mac Studio 128GB as my home server that's also running LLMs to be always online, in part due to the minimal idle power consumption and mostly fan-less operation. It's definitely expensive, especially nowadays, but I can still recommend Mac Minis as a cheaper alternative for someone to just get started with an affordable, always-on home server that won't annoy any housemates. I think both are in some sweet spot in terms of value for money, depending on what you're looking for in a home server. If image or video generation is your thing, look further though, definitely look into a proper GPU then. Macs are quite slow at that. They're just great at MoE LLMs because it's mostly a matter of (V)RAM size.
- deleted 3mo ago[deleted]