7 ms·

Last thing I need is Kubernetes at home

by bauerd 1y ago

nodesocket 1y ago
I run Kubernetes on my homelab on 4x Raspberry Pi's then use Portainer to manage it. Works quite well, can use Helm charts when available, otherwise I use Portainer and deploy container apps manually. It's really not that bad.
- dailykoder 1y ago
  I'm sure you find some joy in it or just like to explore what's possible. But just a reminder: Pieter Levels is running his million dollar businesses on a single VPS (if that's still correct). But yeah, if you like it, why not. "Premature clustering is the source of all evil" - or something like that.
import 1y ago
Exactly. I am hosting 30+ services using docker compose and very happy. I don’t want to troubleshoot k8s in the early morning because home assistant is down and light dimmers are not working for some random k8s reason.
- johnisgood 1y ago
  I am quite happy even without Docker, but I can see the appeal in some cases.
- seba_dos1 1y ago
  All my "smart home" stuff needs is mosquitto on a OpenWrt router and bunch of cgi-bin scripts that can run anywhere. I already went through a phase of setting up tons of services which ended up being turned off when something changed in my life (moving, replacing equipment etc.) never to be resurrected as I couldn't be bothered to redo it without the novelty effect, so I learned from that.
- vbezhenar 1y ago
  Since I've migrated our company to Kubernetes, I almost stopped to worry about anything. It just works. Had much more troubles running it with spaghetti of docker containers and host-installed software on multiple servers, that setup definitely breaks like every week or every month. With Kubernetes I just press "update cluster" in some saturday evening once or twice a year and that's about it, pretty smooth sailing.
dailykoder 1y ago
I guess it can be comfortable for some people. But I just wanted to comment something similar. It's probably heavily dependend on how many services you self-host, but I have 6 services on my VPS and they are just simple podman containers that I just run. Some of them automatically, some of them manually. On top of that a very simple nginx configuration (mostly just subdomains with reverse proxy) and that's it. I don't need an extra container for my nginx, I think (or is there a very good security reason? I have "nothing to hide" and/or lose, but still). My lazy brain thinks as long as I keep nginx up to date with my package manager and my certbot running, ill be fine
mrweasel 1y ago
It is still my opinion that most businesses do not need Kubernetes, and neither should anyone self-hosting a service at home. I can see running something in a Docker container, and while I'd advise against containers what ships with EVERYTHING, I'd also advise against using Docker-compose to spin up an ungodly amount of containers for every service. You shouldn't be running multiple instances of Postgresql, or anything for that matter, at home. Find a service that can be installed using your operating systems package manager and set everything to auto-update. Whatever you're self-hosting, be it for yourself, family or a few friends, there's absolutely nothing wrong with SQLite, files on disk or using the passwd file as your authentication backend. If you are self hosting Kubernetes to learn Kubernetes, then by all means go ahead and do so. For actual use, stay away from anything more complex than a Unix around the year 2000.
- deleted 1y ago
  [deleted]
- dailykoder 1y ago
  I shared this sentiment. But since I just host some personal fun projects and I got really lazy when it comes to self-hosting, I found great pleasure in just creating the simplest possible docker containers. It just keeps the system super clean and easy to wipe and setup again. My databases are usually just mounted volumes which do reside on the host system
- vbezhenar 1y ago
  I'd love to use Kubernetes for my self hosting. The only problem is it's too expensive.
  - k8sToGo 1y ago
    How is it too expensive? If you want to use the eco system you can still use something like k3s
- ohgr 1y ago
  If it wasn't for Kubernetes we'd need 1/3rd of our operations team. We're keeping unemployment down!
deleted 1y ago
[deleted]
MortyWaves 1y ago
There is a fairly well known/popular blogger who’s blog I was following because of their self hosting/homelab/nix adventures. Then they decided to port everything to K8 because of overblown internet drama and I lost all interest. Total shame that a great resource for Nix became yet another K8 fest.
seba_dos1 1y ago
"apt-get install" tends to be enough once you stop chasing latest-and-greatest and start to appreciate things just running with low maintenance more.
- skydhash 1y ago
  I totally agree! Containers are nice when your installation is ephemeral, deploying and updating several time in a short period. But using the package manager is as easy as you get.
- alabastervlog 1y ago
  I only host 3rd party daemons (nothing custom) and only on my local network (plus Tailscale) so Docker’s great for handling package management and init, since I get up-to-date versions of a far broader set of services than Debian or ubuntu’s repos, clean isolation for easy management, and init/restarts are even all free. Plus it naturally documents what I need to back up (any “mounted” directories) Docker lets my OS be be boring (and lets me basically never touch it) while having up to date user-facing software. No “well, this new version fixes a bug that’s annoying me, but it’s not in Debian stable… do I risk a 3rd party back port repo screwing up my system or other services, or upgrade the whole OS just to get one newer package, which comes with similar risks?” I just use shell scripts to launch the services, one script per service. Run the script once, forget about it until I want to upgrade it. Modify the version in the script, take the container down and destroy it (easily automated as part of the scripts, but I haven’t bothered), run the script. Done, forget about it again until next time. Almost all the commands I run on my server are basic file management, docker stuff, or zfs commands. I could switch distros entirely and hardly even notice. Truly a boring OS.
  - seba_dos1 1y ago
    > do I risk a 3rd party back port repo screwing up my system or other services, or upgrade the whole OS just to get one newer package, which comes with similar risks? In these rare cases I usually just compile a newer deb package myself and let the package manager deal with it as usual. If there are too many dependencies to update or it's unusually complex, then it's container time indeed - but I didn't have to go there on my server so far. Not diverging from the distro packages lets me not worry about security updates; Debian handles that for me.
raphinou 1y ago
Exactly, my first reaction was "I should write a blog post about why I still use Docker Swarm". I deploy to single node swarms, and it's a zero boiler plate solution. I had to migrate services to another server recently, and it was really painless. Why oh why doesn't Docker Swarm get more love (from its owners/maintainers and users)?.... Edit: anyone actually interested in such a post?
- kiney 1y ago
  bugs in it's infancy is what killed swarm for users.
- mfashby 1y ago
  I moved us off docker swarm to GKE some years back. The multi node swarm was quite unstable, and none of the big cloud providers offered managed swarm in the same way they offer managed k8s. It's a shame I agree because it was nicely integrated with dockers own tooling. Plus I wouldn't have had to learn about k8s :)
- galbar 1y ago
  I just want to add that I also have a Docker Swarm running, with four small nodes for my personal stuff plus a couple of friends' companies. No issues whatsoever and it is so easy to manage. It just works!
- gmm1990 1y ago
  I'd be interested. Might be a strange question but I'll throw it out there, I seem to have a hard time finding a good way to define my self hosted infrastructure nodes and which containers can run on them, have you run into/have a solution for this? Like I want my database to run on my two beefier machines but some of the other services could run on the mini pcs.
  - raphinou 1y ago
    I am running one-node swarms, so everything I deploy is running on the same node. But from my understanding you can apply labels to the nodes, and limit the placement of containers. See here for an example (I am not affiliated to this site): https://www.sweharris.org/post/2017-07-30-docker-placement/ https://www.sweharris.org/post/2017-07-30-docker-placement/
jauntywundrkind 1y ago
K3s installs super fast. Writing your first yaml or two is scary & seems intimidating at first . But after that, everything is cut from the same cloth. Its an escape from the long dark age of every sysadmin forever cooking up whatever whimsy sort of served them at the time, escape from each service having very different management practices around it. And there's no other community anywhere like Kubernetes. Unbelievably many very good quality very smart helm charts out there, such as https://github.com/bitnami/charts/tree/main/bitnami https://github.com/bitnami/charts/tree/main/bitnami just ready to go. Really sweet home-ops setups like https://github.com/onedr0p/home-ops https://github.com/onedr0p/home-ops that show that once you have a platform under foot, adding more services is really easy, showing an amazing range of home-ops things you might be interested in. > Last thing I need is Kubernetes at home Last thing we need is incredibly shitty attitude. Fuck around and find out is the hacker spirit. Its actually not hard if you try, and actually having a base platform where things follow common patterns & practices & you can reuse existing skills & services is kind of great. Everything is amazing but the snivelling shitty whining without even making the tiniest little case for your unkind low-effort hating will surely continue. Low signal people will remain low signal, best avoid.
bitsandboots 1y ago
I clicked expecting a list of cool things to self host. Instead I got a list of ways I would never want to host. Mankind invented BSD jails so that I do not have to tie myself in a knot of container tooling and abstraction.
- Gud 1y ago
  Indeed. I run a setup as you mentioned, with the various daemons in their own jail. Super simple set up, easy to maintain. Lord knows why people overcomplicate things with docker/kubernetes/etc.
shepherdjerred 1y ago
I'm very happy with Kubernetes at home. Everything just works at this point, though it did take a fair bit of fiddling at first. I think it's a great way to learn Kubernetes if you're interested in that.

Last thing I need is Kubernetes at home

by bauerd 1y ago

Last thing I need is Kubernetes at home

I run Kubernetes on my homelab on 4x Raspberry Pi's then use Portainer to manage it. Works quite well, can use Helm charts when available, otherwise I use Portainer and deploy container apps manually. It's really not that bad.

dailykoder 1y ago
I'm sure you find some joy in it or just like to explore what's possible. But just a reminder: Pieter Levels is running his million dollar businesses on a single VPS (if that's still correct). But yeah, if you like it, why not. "Premature clustering is the source of all evil" - or something like that.

import 1y ago

Exactly. I am hosting 30+ services using docker compose and very happy. I don’t want to troubleshoot k8s in the early morning because home assistant is down and light dimmers are not working for some random k8s reason.

johnisgood 1y ago
I am quite happy even without Docker, but I can see the appeal in some cases.
seba_dos1 1y ago
All my "smart home" stuff needs is mosquitto on a OpenWrt router and bunch of cgi-bin scripts that can run anywhere. I already went through a phase of setting up tons of services which ended up being turned off when something changed in my life (moving, replacing equipment etc.) never to be resurrected as I couldn't be bothered to redo it without the novelty effect, so I learned from that.
vbezhenar 1y ago
Since I've migrated our company to Kubernetes, I almost stopped to worry about anything. It just works. Had much more troubles running it with spaghetti of docker containers and host-installed software on multiple servers, that setup definitely breaks like every week or every month. With Kubernetes I just press "update cluster" in some saturday evening once or twice a year and that's about it, pretty smooth sailing.

dailykoder 1y ago

I guess it can be comfortable for some people. But I just wanted to comment something similar. It's probably heavily dependend on how many services you self-host, but I have 6 services on my VPS and they are just simple podman containers that I just run. Some of them automatically, some of them manually. On top of that a very simple nginx configuration (mostly just subdomains with reverse proxy) and that's it. I don't need an extra container for my nginx, I think (or is there a very good security reason? I have "nothing to hide" and/or lose, but still). My lazy brain thinks as long as I keep nginx up to date with my package manager and my certbot running, ill be fine

mrweasel 1y ago

It is still my opinion that most businesses do not need Kubernetes, and neither should anyone self-hosting a service at home. I can see running something in a Docker container, and while I'd advise against containers what ships with EVERYTHING, I'd also advise against using Docker-compose to spin up an ungodly amount of containers for every service. You shouldn't be running multiple instances of Postgresql, or anything for that matter, at home. Find a service that can be installed using your operating systems package manager and set everything to auto-update. Whatever you're self-hosting, be it for yourself, family or a few friends, there's absolutely nothing wrong with SQLite, files on disk or using the passwd file as your authentication backend. If you are self hosting Kubernetes to learn Kubernetes, then by all means go ahead and do so. For actual use, stay away from anything more complex than a Unix around the year 2000.

deleted 1y ago
[deleted]
dailykoder 1y ago
I shared this sentiment. But since I just host some personal fun projects and I got really lazy when it comes to self-hosting, I found great pleasure in just creating the simplest possible docker containers. It just keeps the system super clean and easy to wipe and setup again. My databases are usually just mounted volumes which do reside on the host system
vbezhenar 1y ago
I'd love to use Kubernetes for my self hosting. The only problem is it's too expensive.
- k8sToGo 1y ago
  How is it too expensive? If you want to use the eco system you can still use something like k3s
ohgr 1y ago
If it wasn't for Kubernetes we'd need 1/3rd of our operations team. We're keeping unemployment down!

deleted 1y ago

[deleted]

MortyWaves 1y ago

There is a fairly well known/popular blogger who’s blog I was following because of their self hosting/homelab/nix adventures. Then they decided to port everything to K8 because of overblown internet drama and I lost all interest. Total shame that a great resource for Nix became yet another K8 fest.

seba_dos1 1y ago

"apt-get install" tends to be enough once you stop chasing latest-and-greatest and start to appreciate things just running with low maintenance more.

skydhash 1y ago
I totally agree! Containers are nice when your installation is ephemeral, deploying and updating several time in a short period. But using the package manager is as easy as you get.
alabastervlog 1y ago
I only host 3rd party daemons (nothing custom) and only on my local network (plus Tailscale) so Docker’s great for handling package management and init, since I get up-to-date versions of a far broader set of services than Debian or ubuntu’s repos, clean isolation for easy management, and init/restarts are even all free. Plus it naturally documents what I need to back up (any “mounted” directories) Docker lets my OS be be boring (and lets me basically never touch it) while having up to date user-facing software. No “well, this new version fixes a bug that’s annoying me, but it’s not in Debian stable… do I risk a 3rd party back port repo screwing up my system or other services, or upgrade the whole OS just to get one newer package, which comes with similar risks?” I just use shell scripts to launch the services, one script per service. Run the script once, forget about it until I want to upgrade it. Modify the version in the script, take the container down and destroy it (easily automated as part of the scripts, but I haven’t bothered), run the script. Done, forget about it again until next time. Almost all the commands I run on my server are basic file management, docker stuff, or zfs commands. I could switch distros entirely and hardly even notice. Truly a boring OS.
- seba_dos1 1y ago
  > do I risk a 3rd party back port repo screwing up my system or other services, or upgrade the whole OS just to get one newer package, which comes with similar risks? In these rare cases I usually just compile a newer deb package myself and let the package manager deal with it as usual. If there are too many dependencies to update or it's unusually complex, then it's container time indeed - but I didn't have to go there on my server so far. Not diverging from the distro packages lets me not worry about security updates; Debian handles that for me.

raphinou 1y ago

Exactly, my first reaction was "I should write a blog post about why I still use Docker Swarm". I deploy to single node swarms, and it's a zero boiler plate solution. I had to migrate services to another server recently, and it was really painless. Why oh why doesn't Docker Swarm get more love (from its owners/maintainers and users)?.... Edit: anyone actually interested in such a post?

kiney 1y ago
bugs in it's infancy is what killed swarm for users.
mfashby 1y ago
I moved us off docker swarm to GKE some years back. The multi node swarm was quite unstable, and none of the big cloud providers offered managed swarm in the same way they offer managed k8s. It's a shame I agree because it was nicely integrated with dockers own tooling. Plus I wouldn't have had to learn about k8s :)
galbar 1y ago
I just want to add that I also have a Docker Swarm running, with four small nodes for my personal stuff plus a couple of friends' companies. No issues whatsoever and it is so easy to manage. It just works!
gmm1990 1y ago
I'd be interested. Might be a strange question but I'll throw it out there, I seem to have a hard time finding a good way to define my self hosted infrastructure nodes and which containers can run on them, have you run into/have a solution for this? Like I want my database to run on my two beefier machines but some of the other services could run on the mini pcs.
- raphinou 1y ago
  I am running one-node swarms, so everything I deploy is running on the same node. But from my understanding you can apply labels to the nodes, and limit the placement of containers. See here for an example (I am not affiliated to this site): https://www.sweharris.org/post/2017-07-30-docker-placement/ https://www.sweharris.org/post/2017-07-30-docker-placement/

jauntywundrkind 1y ago

K3s installs super fast. Writing your first yaml or two is scary & seems intimidating at first . But after that, everything is cut from the same cloth. Its an escape from the long dark age of every sysadmin forever cooking up whatever whimsy sort of served them at the time, escape from each service having very different management practices around it. And there's no other community anywhere like Kubernetes. Unbelievably many very good quality very smart helm charts out there, such as https://github.com/bitnami/charts/tree/main/bitnami https://github.com/bitnami/charts/tree/main/bitnami just ready to go. Really sweet home-ops setups like https://github.com/onedr0p/home-ops https://github.com/onedr0p/home-ops that show that once you have a platform under foot, adding more services is really easy, showing an amazing range of home-ops things you might be interested in. > Last thing I need is Kubernetes at home Last thing we need is incredibly shitty attitude. Fuck around and find out is the hacker spirit. Its actually not hard if you try, and actually having a base platform where things follow common patterns & practices & you can reuse existing skills & services is kind of great. Everything is amazing but the snivelling shitty whining without even making the tiniest little case for your unkind low-effort hating will surely continue. Low signal people will remain low signal, best avoid.

bitsandboots 1y ago

I clicked expecting a list of cool things to self host. Instead I got a list of ways I would never want to host. Mankind invented BSD jails so that I do not have to tie myself in a knot of container tooling and abstraction.

Gud 1y ago
Indeed. I run a setup as you mentioned, with the various daemons in their own jail. Super simple set up, easy to maintain. Lord knows why people overcomplicate things with docker/kubernetes/etc.

shepherdjerred 1y ago

I'm very happy with Kubernetes at home. Everything just works at this point, though it did take a fair bit of fiddling at first. I think it's a great way to learn Kubernetes if you're interested in that.