11 ms·

I share the author's sentiment completely. At my day job, I manage multiple Kubernetes clusters running dozens of microservices with relative ease. However, for

I share the author's sentiment completely. At my day job, I manage multiple Kubernetes clusters running dozens of microservices with relative ease. However, for my hobby projects—which generate no revenue and thus have minimal budgets—I find myself in a frustrating position: desperately wanting to use Kubernetes but unable to due to its resource requirements. Kubernetes is simply too resource-intensive to run on a $10/month VPS with just 1 shared vCPU and 2GB of RAM. This limitation creates numerous headaches. Instead of Deployments, I'm stuck with manual docker compose up/down commands over SSH. Rather than using Ingress, I have to rely on Traefik's container discovery functionality. Recently, I even wrote a small script to manage crontab idempotently because I can't use CronJobs. I'm constantly reinventing solutions to problems that Kubernetes already solves—just less efficiently. What I really wish for is a lightweight alternative offering a Kubernetes-compatible API that runs well on inexpensive VPS instances. The gap between enterprise-grade container orchestration and affordable hobby hosting remains frustratingly wide.

thenewwazoo 1y ago
> I'm constantly reinventing solutions to problems that Kubernetes already solves—just less efficiently. But you've already said yourself that the cost of using K8s is too high. In one sense, you're solving those solutions more efficiently, it just depends on the axis you use to measure things.
- randallsquared 1y ago
  The original statement is ambiguous. I read it as "problems that k8s already solves -- but k8s is less efficient, so can't be used".
- AdrianB1 1y ago
  That picture with the almost-empty truck seems to be the situation that he describes. He wants the 18 wheeler truck, but it is too expensive for just a suitcase.
sweettea 1y ago
Have you seen k0s or k3s? Lots of stories about folks using these to great success on a tiny scale, e.g. https://news.ycombinator.com/item?id=43593269 https://news.ycombinator.com/item?id=43593269
- mikepurvis 1y ago
  Or microk8s. I'm curious what it is about k8s that is sucking up all these resources. Surely the control plane is mostly idle when you aren't doing things with it?
  - mdaniel 1y ago
    There are 3 components to "the control plane" and realistically only one of them is what you meant by idle. The Node-local kubelet (that reports in the state of affairs and asks if there is any work) is a constantly active thing, as one would expect from such a polling setup. The etcd, or it's replacement, is constantly(?) firing off watch notifications or reconciliation notifications based on the inputs from the aforementioned kubelet updates. Only the actual kube-apiserver is conceptually idle as I'm not aware of any compute that it, itself, does only in response to requests made of it Put another way, in my experience running clusters, in $(ps auwx) or its $(top) friend always show etcd or sqlite generating all of the "WHAT are you doing?!" and those also represent the actual risk to running kubernetes since the apiserver is mostly stateless[1] 1: but holy cow watch out for mTLS because cert expiry will ruin your day across all of the components
    - icedchai 1y ago
      I've noticed that etcd seems to do an awful lot of disk writes, even on an "idle" cluster. Nothing is changing. What is it actually doing with all those writes?
      - mdaniel 1y ago
        Almost certainly it's the propagation of the kubelet checkins rippling through etcd's accounting system[1]. Every time these discussions come up I'm always left wondering "I wonder if Valkey would behave the same?" or Consul (back when it was sanely licensed). But I am now convinced after 31 releases that the pluggable KV ship has sailed and they're just not interested. I, similarly, am not yet curious enough to pull a k0s and fork it just to find out 1: related, if you haven't ever tried to run a cluster bigger than about 450 Nodes that's actually the whole reason kube-apiserver --etcd-servers-overrides exists because the torrent of Node status updates will knock over the primary etcd so one has to offload /events into its own etcd
nvarsj 1y ago
Just do it like the olden days, use ansible or similar. I have a couple dedicated servers I fully manage with ansible. It's docker compose on steroids. Use traefik and labeling to handle reverse proxy and tls certs in a generic way, with authelia as simple auth provider. There's a lot of example projects on github. A weekend of setup and you have a pretty easy to manage system.
- nicce 1y ago
  What is the advantage of traefik over oldschool Nginx?
  - gonzo41 1y ago
    There's very little advantage IMO. I've used both. I always end up back at Nginx. Traefik was just another configuration layer that got in the way of things.
    - nvarsj 1y ago
      Traefik is waaay simpler - 0 config, just use docker container labels. There is absolutely no reason to use nginx these days. I should know, as I spent years building and maintaining a production ingress controller for nginx at scale, and I'd choose Traefik every day over that.
  - c0balt 1y ago
    Traefik has some nice labeling for docker that allows you to colocate your reverse proxy config with your container definition. It's slightly more convenient than NGINX for that usecase with compose. It effectively saves you a dedicated vietualhost conf by setting some labels. One can read more here: https://doc.traefik.io/traefik/routing/providers/docker/ https://doc.traefik.io/traefik/routing/providers/docker/ This obviously has some limits and becomes significantly less useful when one requires more complex proxy rules.
  - nvarsj 1y ago
    Basically what c0balt said. It's zero config and super easy to set everything up. Just run the traefik image, and add docker labels to your other containers. Traefik inspects the labels and configures reverse proxy for each. It even handles generating TLS certs for you using letsencrypt or zerossl.
404mm 1y ago
I found k3s to be a happy medium. It feels very lean and works well even on a Pi, and scales ok to a few node cluster if needed. You can even host the database on a remote mysql server, if local sqlite is too much IO.
hkon 1y ago
I've used caprover a bunch
Alupis 1y ago
> Kubernetes is simply too resource-intensive to run on a $10/month VPS with just 1 shared vCPU and 2GB of RAM I hate sounding like an Oracle shill, but Oracle Cloud's Free Tier is hands-down the most generous. It can support running quite a bit, including a small k8s cluster[1]. Their k8s backplane service is also free. They'll give you 4 x ARM64 cores and 24GB of ram for free. You can split this into 1-4 nodes, depending on what you want. [1] https://www.oracle.com/cloud/free/ https://www.oracle.com/cloud/free/
- lakomen 1y ago
  [dead]
- waveringana 1y ago
  the catch is: no commercial usage and half the time you try to spin up an instance itll tell you theres no room left
  - SOLAR_FIELDS 1y ago
    That limitation (spinning up an instance) only exists if you don't put a payment card in. If you put a payment card in, it goes away immediately. You don't have to actually pay anything, you can provision the always free resources, but obviously in this regard you have to ensure that you don't accidentally provision something with cost. I used terraform to make my little kube cluster on there and have not had a cost event at all in over 1.5 years. I think at one point I accidentally provisioned a volume or something and it cost me like one cent.
  - Alupis 1y ago
    > no commercial usage I think that's if you are literally on their free tier, vs. having a billable account which doesn't accumulate enough charges to be billed. Similar to the sibling comment - you add a credit card and set yourself up to be billed (which removes you from the "free tier"), but you are still granted the resources monthly for free. If you exceed your allocation, they bill the difference.
    - 1y ago
investa 1y ago
SSH up/down can be scripted. Or maybe look into Kamal? Or use Digital Ocean app service. Got integration, cheap, just run a container. But get your postgres from a cheaper VC funded shop :)
kartikarti 1y ago
Virtual Kubelet is one step forward towards Kubernetes as an API https://github.com/virtual-kubelet/virtual-kubelet https://github.com/virtual-kubelet/virtual-kubelet
czhu12 1y ago
This is exactly why I built https://canine.sh https://canine.sh -- basically for indie hackers to have the full experience of Heroku with the power and portability of Kubernetes. For single server setups, it uses k3s, which takes up ~200MB of memory on your host machine. Its not ideal, but the pain of trying to wrangle docker deployments, and the cheapness of hetzner made it worth it.
- satvikpendem 1y ago
  How does it compare to Coolify and Dokploy?
  - czhu12 1y ago
    Neither of those use kubernetes unfortunately, the tool has kind of a bad rap, but every company I’ve worked at has eventually migrated on to kubernetes
    - satvikpendem 1y ago
      Sure, I'm looking for more of a personal project use case where it doesn't much matter to me whether it uses Kubernetes or not, I'm more interested in concrete differences.
      - czhu12 1y ago
        Ah yeah then I’d say the biggest difference is the fact that it can use to helm to install basically anything in the world to your cluster
singron 1y ago
NixOS works really well for me. I used to write these kinds of idempotent scripts too but they are usually irrelevant in NixOS where that's the default behavior.
- lewo 1y ago
  And regarding this part of the article > Particularly with GitOps and Flux, making changes was a breeze. i'm writing comin [1] which is GitOps for NixOS machines: you Git push your changes and your machines fetch and deploy them automatically. [1] https://github.com/nlewo/comin https://github.com/nlewo/comin
vrosas 1y ago
Why not just use something like Cloud Run? If you're only running a microVM deploying it there will probably be at or near free.
pachevjoseph 1y ago
I’ve been using https://www.coolify.io/ https://www.coolify.io/ self hosted. It’s a good middle ground between full blown k8s and systemd services. I have a home lab where I host most of my hobby projects though. So take that into account. You can also use their cloud offering to connect to VPSs
nullpoint420 1y ago
Have you tried nixOS? I feel like it solves the functional aspect you're looking for.
osigurdson 1y ago
Podman is a fairly nice bridge. If you are familiar with Kubernetes yaml, it is relatively easy to do docker-compose like things except using more familiar (for me) K8s yaml. In terms of the cloud, I think Digital Ocean costs about $12 / month for their control plane + a small instance.
turtlebits 1y ago
I'm a cheapskate too, but at some point, the time you spend researching cheap hosting, signing up and getting deployed is not worth the hassle of paying a few more $ on bigger boxes.
CoolCold 1y ago
6$/m - will likely bring you peace of mind - Netcup hosting VPS 1000 ARM G11 6 vCore (ARM64) 8 GB RAM 256 GB NVMe
- Jipazgqmnm 1y ago
  They also have regular promotions that offer e.g. double the disk space. There you get 6 vCore (ARM64) 8 GB RAM 512 GB NVMe for 6 $ / m - traffic inclusive. You can choose between "6 vCore ARM64, 8 GB RAM" and "4 vCore x86, 8 GB ECC RAM" for the same price. And much more, of course. https://www.netcup.com/en/server/vps https://www.netcup.com/en/server/vps
daitangio 1y ago
I developed a tiny wrapper around docker compose which work on my use case: https://github.com/daitangio/misterio https://github.com/daitangio/misterio It can manage multiple machine with just ssh access and docker install.
alex5207 1y ago
> I'm stuck with manual docker compose up/down commands over SSH Out of curiosity, what is so bad about this for smaller projects?
eigengrau 1y ago
It’s been a couple of years since I’ve last used it, but if you want container orchestration with a relatively small footprint, maybe Hashicorp Nomad (perhaps in conjunction with Consul and Traefik) is still an option. These were all single binary tools. I did not personally run them on 2G mem VPSes, but it might still be worthwhile for you to take a look. It looks like Nomad has a driver to run software via isolated fork/exec, as well, in addition to Docker containers.
aequitas 1y ago
I've ran K3s on a couple of Raspberry Pi's as a homelab in the past. It's lightweight and ran nice for a few years, but even so, one Pi was always dedicated as controller, which seemed like a waste. Recently I switched my entire setup (few Pi's, NAS and VM's) to NixOS. With Colmena[0] I can manage/update all hosts from one directory with a single command. Kubernetes was a lot of fun, especially the declarative nature of it. But for small setups, where you are still managing the plumbing (OS, networking, firewall, hardening, etc) yourself, you still need some configuration management. Might as well put the rest of your stuff in there also. [0] https://colmena.cli.rs/unstable/ https://colmena.cli.rs/unstable/
rcarmo 1y ago
What about Portainer? I deploy my compose files via git using it.
BiteCode_dev 1y ago
The solution to this is to not solve all the problems a billion dollar tech does on a personnal project. Let it not be idempotent. Let it crash sometimes. We lived without kubs for years and the web was ok. Your users will survive.
- bigstrat2003 1y ago
  Yeah, unless you're doing k8s for the purpose of learning job skills, it's way overkill. Just run a container with docker, or a web server outside a container if it's a website. Way easier and it will work just fine.
melodyogonna 1y ago
For $10 you can buy VPS with a lot more resources than that on both Contabo and Ovh
rollcat 1y ago
I've been using Docker swarm for internal & lightweight production workloads for 5+ years with zero issues. FD: it's a single node cluster on a reasonably powerful machine, but if anything, it's over-specced for what it does. Which I guess makes it more than good enough for hobby stuff - I'm playing with a multi-node cluster in my homelab and it's also working fine.
- Taikonerd 1y ago
  I think Docker Swarm makes a lot of sense for situations where K8s is too heavyweight. "Heavyweight" either in resource consumption, or just being too complex for a simple use case.
  - psviderski 1y ago
    The only problem is Docker Swarm is essentially abandonware after Docker was acquired by Mirantis in 2019. Core features still work but there is a ton of open issues and PRs which are ignored. It's fine if it works but no one cares if you found a bug or have ideas on how to improve something, even worse if you want to contribute.
    - rollcat 1y ago
      Yep it's unfortunate, "it works for me" until it doesn't. OTOH it's not a moving target. Docker historically has been quite infamous for that, we were talking about half-lives for features, as if they were unstable isotopes. It took initiatives like OCI to get things to settle. K8s tries to solve the most complex problems, at the expense of leaving simple things stranded. If we had something like OCI for clustering, it would most likely take the same shape.
ClumsyPilot 1y ago
> Kubernetes is simply too resource-intensive to run on a $10/month VPS with just 1 shared vCPU and 2GB of RAM To put this in perspective, that’s less compute than a phone released in 2023, 12 years ago, Samsung Galaxy S4. To find this level of performance in a computer, we have to go to The main issue is that Kubernetes has created good API and primitives for managing cloud stuff, and managing a single server is still kinda crap despite decades of effort. I had K3S on my server, but replaced with docker + Traefik + Portainer - it’s not great, but less idle CPU use and fewer moving parts
jdsleppy 1y ago
I really like `DOCKER_HOST=ssh://... docker compose up -d`, what do you miss about Deployments?
JamesSwift 1y ago
Just go with a cloud provider that offers free control plane and shove a bunch of side projects into 1 node. I end up around $50 a month on GCP (was a bit cheaper at DO) once you include things like private docker registry etc. The marginal cost of an additional project on the cluster is essentially $0
figmert 1y ago
> What I really wish for is a lightweight alternative offering a Kubernetes-compatible API that runs well on inexpensive VPS instances. The gap between enterprise-grade container orchestration and affordable hobby hosting remains frustratingly wide. Depending on how much of the Kube API you need, Podman is that. It can generate containers and pods from Kubernetes manifests [0]. Kind of works like docker compose but with Kubernetes manifests. This even works with systemd units, similar to how it's outlined in the article. Podman also supports most (all?) of the Docker api, thus docker compose, works, but also, you can connect to remote sockets through ssh etc to do things. [0] https://docs.podman.io/en/latest/markdown/podman-kube-play.1.html https://docs.podman.io/en/latest/markdown/podman-kube-play.1... [1] https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html https://docs.podman.io/en/latest/markdown/podman-systemd.uni...
- sciencesama 1y ago
  You can use firecracker !
- jillyboel 1y ago
  The docs don't make it clear, can it do "zero downtime" deployments? Meaning it first creates the new pod, waits for it to be healthy using the defined health checks and then removes the old one? Somehow integrating this with service/ingress/whatever so network traffic only goes to the healthy one?
  - ffsm8 1y ago
    I can't speak on it's capabilities, but I feel like I have to ask: for what conceivable reason would you even want that extra error potential with migrations etc? It means you're forced to make everything always compatible between versions etc. For a deployment that isn't even making money and is running on a single node droplet with basically no performance... Why?
    - jillyboel 1y ago
      > I can't speak on it's capabilities, but I feel like I have to ask: for what conceivable reason would you even want that extra error potential with migrations etc? It's the default behavior of a kubernetes deployment which we're comparing things to. > It means you're forced to make everything always compatible between versions etc. For stateless services, not at all. The outside world just keeps talking to the previous version while the new version is starting up. For stateful services, it depends. Often there are software changes without changes to the schema. > For a deployment that isn't even making money I don't like looking at 504 gateway errors > and is running on a single node droplet with basically no performance I'm running this stuff on a server in my home, it has plenty of performance. Still don't want to waste it on kubernetes overhead, though. But even for a droplet, running the same application 2x isn't usually a big ask.
AdrianB1 1y ago
I am curious why your no revenue projects need the complexity, features and benefits of something like Kubernetes. Why you cannot just to it the archaic way of compiling your app, copy the files to a folder and run it there and never touch it for the next 5 years. If it is a dev environment with many changes, its on a local computer, not on VPS, I guess. Just curious by nature, I am.
mbreese 1y ago
> I'm constantly reinventing solutions to problems that Kubernetes already solves Another way to look at this is the Kubernetes created solutions to problems that were already solved at a lower scale level. Crontabs, http proxies, etc… were already solved at the individual server level. If you’re used to running large coordinated clusters, then yes — it can seem like you’re reinventing the wheel.
MyOutfitIsVague 1y ago
> Kubernetes is simply too resource-intensive to run on a $10/month VPS with just 1 shared vCPU and 2GB of RAM. That's more than what I'm paying for far fewer resources than Hetzner. I'm paying about $8 a month for 4 vCPUs and 8GB of RAM: https://www.hetzner.com/cloud https://www.hetzner.com/cloud Note that the really affordable ARM servers are German only, so if you're in the US you'll have to deal with higher latency to save that money, but I think it's worth it.
- andrewmcwatters 1y ago
  Thank you for sharing this. Do you have a referral link we can use to give you a little credit for informing us?
  - MyOutfitIsVague 1y ago
    Sure, if you still want it: https://hetzner.cloud/?ref=WwByfoEfJJdv https://hetzner.cloud/?ref=WwByfoEfJJdv I guess it gives you 20 euros in credit, too. That's nice.
- fhcbix 1y ago
  I recently set up an arm64 VPS at netcup: https://www.netcup.com/en/server/arm-server https://www.netcup.com/en/server/arm-server Got it with no location fee (and 2x storage) during the easter sale but normally US is the cheapest.
  - MyOutfitIsVague 1y ago
    That's pretty cheap. I have 4 vCPUs, 8GB RAM, 80GB disk, and 20TB traffic for €6. NetCup looks like it has 6VCPU, 8GB RAM, 256 GB, and what looks like maybe unlimited traffic for €5.26. That's really good. And it's in the US, where I am, so SSH would be less painful. I'll have to think about possibly switching. Thanks for the heads up.
artdigital 1y ago
I run my private stuff on a hosted vultr k8s cluster with 1 node for $10-$20 a month. All my hobby stuff is running on that "personal cluster" and it is that perfect sweetspot for me that you're talking about I don't use ingresses or loadbalancers because those cost extra, and either have the services exposed through tailscale (with tailscale operator) for stuff I only use myself, or through cloudflare argo tunnels for stuff I want internet accessible (Once a project graduates and becomes more serious, I migrate the container off this cluster and into a proper container runner)
lenerdenator 1y ago
The thing is, most of those enterprise-grade container orchestrations probably don't need k8s either. The more I look into it, the more I think of k8s as a way to "move to micro services" without actually moving to micro services. Loosely coupled micro services shouldn't need that level of coordination if they're truly loosely coupled.
SEJeff 1y ago
Have you tried k3s? I think it would run on a tiny vps like that and is a full stack. Instead of etcd it has sqlite embedded.
parliament32 1y ago
Why not minikube or one of the other resource-constrained k8s variants? https://minikube.sigs.k8s.io/ https://minikube.sigs.k8s.io/
jjwiseman 1y ago
I use Caprover to run about 26 services for personal projects on a Hetzner box. I like its simplicity. Worth it just for the one-click https cert management.
huksley 1y ago
I believe that Kubernetes is something you want to use if you have 1+ SRE full-time on your team. I actually got tired with complexity of kubernetes, AWS ECS and docker as well and just build a tool to deploy apps natively on the host. What's wrong with using Linux native primitives - systemd, crontab, postgresql or redis native package? Whose should work as intended, you don't need them in container.
byrnedo 1y ago
Please try https://github.com/skateco/skate https://github.com/skateco/skate, this is pretty much the exact same reason why I built it!

thenewwazoo 1y ago

> I'm constantly reinventing solutions to problems that Kubernetes already solves—just less efficiently. But you've already said yourself that the cost of using K8s is too high. In one sense, you're solving those solutions more efficiently, it just depends on the axis you use to measure things.

randallsquared 1y ago
The original statement is ambiguous. I read it as "problems that k8s already solves -- but k8s is less efficient, so can't be used".
AdrianB1 1y ago
That picture with the almost-empty truck seems to be the situation that he describes. He wants the 18 wheeler truck, but it is too expensive for just a suitcase.

sweettea 1y ago

Have you seen k0s or k3s? Lots of stories about folks using these to great success on a tiny scale, e.g. https://news.ycombinator.com/item?id=43593269 https://news.ycombinator.com/item?id=43593269

mikepurvis 1y ago
Or microk8s. I'm curious what it is about k8s that is sucking up all these resources. Surely the control plane is mostly idle when you aren't doing things with it?
- mdaniel 1y ago
  There are 3 components to "the control plane" and realistically only one of them is what you meant by idle. The Node-local kubelet (that reports in the state of affairs and asks if there is any work) is a constantly active thing, as one would expect from such a polling setup. The etcd, or it's replacement, is constantly(?) firing off watch notifications or reconciliation notifications based on the inputs from the aforementioned kubelet updates. Only the actual kube-apiserver is conceptually idle as I'm not aware of any compute that it, itself, does only in response to requests made of it Put another way, in my experience running clusters, in $(ps auwx) or its $(top) friend always show etcd or sqlite generating all of the "WHAT are you doing?!" and those also represent the actual risk to running kubernetes since the apiserver is mostly stateless[1] 1: but holy cow watch out for mTLS because cert expiry will ruin your day across all of the components
  - icedchai 1y ago
    I've noticed that etcd seems to do an awful lot of disk writes, even on an "idle" cluster. Nothing is changing. What is it actually doing with all those writes?
    - mdaniel 1y ago
      Almost certainly it's the propagation of the kubelet checkins rippling through etcd's accounting system[1]. Every time these discussions come up I'm always left wondering "I wonder if Valkey would behave the same?" or Consul (back when it was sanely licensed). But I am now convinced after 31 releases that the pluggable KV ship has sailed and they're just not interested. I, similarly, am not yet curious enough to pull a k0s and fork it just to find out 1: related, if you haven't ever tried to run a cluster bigger than about 450 Nodes that's actually the whole reason kube-apiserver --etcd-servers-overrides exists because the torrent of Node status updates will knock over the primary etcd so one has to offload /events into its own etcd

nvarsj 1y ago

Just do it like the olden days, use ansible or similar. I have a couple dedicated servers I fully manage with ansible. It's docker compose on steroids. Use traefik and labeling to handle reverse proxy and tls certs in a generic way, with authelia as simple auth provider. There's a lot of example projects on github. A weekend of setup and you have a pretty easy to manage system.

nicce 1y ago
What is the advantage of traefik over oldschool Nginx?
- gonzo41 1y ago
  There's very little advantage IMO. I've used both. I always end up back at Nginx. Traefik was just another configuration layer that got in the way of things.
  - nvarsj 1y ago
    Traefik is waaay simpler - 0 config, just use docker container labels. There is absolutely no reason to use nginx these days. I should know, as I spent years building and maintaining a production ingress controller for nginx at scale, and I'd choose Traefik every day over that.
- c0balt 1y ago
  Traefik has some nice labeling for docker that allows you to colocate your reverse proxy config with your container definition. It's slightly more convenient than NGINX for that usecase with compose. It effectively saves you a dedicated vietualhost conf by setting some labels. One can read more here: https://doc.traefik.io/traefik/routing/providers/docker/ https://doc.traefik.io/traefik/routing/providers/docker/ This obviously has some limits and becomes significantly less useful when one requires more complex proxy rules.
- nvarsj 1y ago
  Basically what c0balt said. It's zero config and super easy to set everything up. Just run the traefik image, and add docker labels to your other containers. Traefik inspects the labels and configures reverse proxy for each. It even handles generating TLS certs for you using letsencrypt or zerossl.

404mm 1y ago

I found k3s to be a happy medium. It feels very lean and works well even on a Pi, and scales ok to a few node cluster if needed. You can even host the database on a remote mysql server, if local sqlite is too much IO.

hkon 1y ago

I've used caprover a bunch

Alupis 1y ago

> Kubernetes is simply too resource-intensive to run on a $10/month VPS with just 1 shared vCPU and 2GB of RAM I hate sounding like an Oracle shill, but Oracle Cloud's Free Tier is hands-down the most generous. It can support running quite a bit, including a small k8s cluster[1]. Their k8s backplane service is also free. They'll give you 4 x ARM64 cores and 24GB of ram for free. You can split this into 1-4 nodes, depending on what you want. [1] https://www.oracle.com/cloud/free/ https://www.oracle.com/cloud/free/

lakomen 1y ago
[dead]
waveringana 1y ago
the catch is: no commercial usage and half the time you try to spin up an instance itll tell you theres no room left
- SOLAR_FIELDS 1y ago
  That limitation (spinning up an instance) only exists if you don't put a payment card in. If you put a payment card in, it goes away immediately. You don't have to actually pay anything, you can provision the always free resources, but obviously in this regard you have to ensure that you don't accidentally provision something with cost. I used terraform to make my little kube cluster on there and have not had a cost event at all in over 1.5 years. I think at one point I accidentally provisioned a volume or something and it cost me like one cent.
- Alupis 1y ago
  > no commercial usage I think that's if you are literally on their free tier, vs. having a billable account which doesn't accumulate enough charges to be billed. Similar to the sibling comment - you add a credit card and set yourself up to be billed (which removes you from the "free tier"), but you are still granted the resources monthly for free. If you exceed your allocation, they bill the difference.
  - 1y ago

investa 1y ago

SSH up/down can be scripted. Or maybe look into Kamal? Or use Digital Ocean app service. Got integration, cheap, just run a container. But get your postgres from a cheaper VC funded shop :)

kartikarti 1y ago

Virtual Kubelet is one step forward towards Kubernetes as an API https://github.com/virtual-kubelet/virtual-kubelet https://github.com/virtual-kubelet/virtual-kubelet

czhu12 1y ago

This is exactly why I built https://canine.sh https://canine.sh -- basically for indie hackers to have the full experience of Heroku with the power and portability of Kubernetes. For single server setups, it uses k3s, which takes up ~200MB of memory on your host machine. Its not ideal, but the pain of trying to wrangle docker deployments, and the cheapness of hetzner made it worth it.

satvikpendem 1y ago
How does it compare to Coolify and Dokploy?
- czhu12 1y ago
  Neither of those use kubernetes unfortunately, the tool has kind of a bad rap, but every company I’ve worked at has eventually migrated on to kubernetes
  - satvikpendem 1y ago
    Sure, I'm looking for more of a personal project use case where it doesn't much matter to me whether it uses Kubernetes or not, I'm more interested in concrete differences.
    - czhu12 1y ago
      Ah yeah then I’d say the biggest difference is the fact that it can use to helm to install basically anything in the world to your cluster

singron 1y ago

NixOS works really well for me. I used to write these kinds of idempotent scripts too but they are usually irrelevant in NixOS where that's the default behavior.

lewo 1y ago
And regarding this part of the article > Particularly with GitOps and Flux, making changes was a breeze. i'm writing comin [1] which is GitOps for NixOS machines: you Git push your changes and your machines fetch and deploy them automatically. [1] https://github.com/nlewo/comin https://github.com/nlewo/comin

vrosas 1y ago

Why not just use something like Cloud Run? If you're only running a microVM deploying it there will probably be at or near free.

pachevjoseph 1y ago

I’ve been using https://www.coolify.io/ https://www.coolify.io/ self hosted. It’s a good middle ground between full blown k8s and systemd services. I have a home lab where I host most of my hobby projects though. So take that into account. You can also use their cloud offering to connect to VPSs

nullpoint420 1y ago

Have you tried nixOS? I feel like it solves the functional aspect you're looking for.

osigurdson 1y ago

Podman is a fairly nice bridge. If you are familiar with Kubernetes yaml, it is relatively easy to do docker-compose like things except using more familiar (for me) K8s yaml. In terms of the cloud, I think Digital Ocean costs about $12 / month for their control plane + a small instance.

turtlebits 1y ago

I'm a cheapskate too, but at some point, the time you spend researching cheap hosting, signing up and getting deployed is not worth the hassle of paying a few more $ on bigger boxes.

CoolCold 1y ago

6$/m - will likely bring you peace of mind - Netcup hosting VPS 1000 ARM G11 6 vCore (ARM64) 8 GB RAM 256 GB NVMe

Jipazgqmnm 1y ago
They also have regular promotions that offer e.g. double the disk space. There you get 6 vCore (ARM64) 8 GB RAM 512 GB NVMe for 6 $ / m - traffic inclusive. You can choose between "6 vCore ARM64, 8 GB RAM" and "4 vCore x86, 8 GB ECC RAM" for the same price. And much more, of course. https://www.netcup.com/en/server/vps https://www.netcup.com/en/server/vps

daitangio 1y ago

I developed a tiny wrapper around docker compose which work on my use case: https://github.com/daitangio/misterio https://github.com/daitangio/misterio It can manage multiple machine with just ssh access and docker install.

alex5207 1y ago

> I'm stuck with manual docker compose up/down commands over SSH Out of curiosity, what is so bad about this for smaller projects?

eigengrau 1y ago

It’s been a couple of years since I’ve last used it, but if you want container orchestration with a relatively small footprint, maybe Hashicorp Nomad (perhaps in conjunction with Consul and Traefik) is still an option. These were all single binary tools. I did not personally run them on 2G mem VPSes, but it might still be worthwhile for you to take a look. It looks like Nomad has a driver to run software via isolated fork/exec, as well, in addition to Docker containers.

aequitas 1y ago

I've ran K3s on a couple of Raspberry Pi's as a homelab in the past. It's lightweight and ran nice for a few years, but even so, one Pi was always dedicated as controller, which seemed like a waste. Recently I switched my entire setup (few Pi's, NAS and VM's) to NixOS. With Colmena[0] I can manage/update all hosts from one directory with a single command. Kubernetes was a lot of fun, especially the declarative nature of it. But for small setups, where you are still managing the plumbing (OS, networking, firewall, hardening, etc) yourself, you still need some configuration management. Might as well put the rest of your stuff in there also. [0] https://colmena.cli.rs/unstable/ https://colmena.cli.rs/unstable/

rcarmo 1y ago

What about Portainer? I deploy my compose files via git using it.

BiteCode_dev 1y ago

The solution to this is to not solve all the problems a billion dollar tech does on a personnal project. Let it not be idempotent. Let it crash sometimes. We lived without kubs for years and the web was ok. Your users will survive.

bigstrat2003 1y ago
Yeah, unless you're doing k8s for the purpose of learning job skills, it's way overkill. Just run a container with docker, or a web server outside a container if it's a website. Way easier and it will work just fine.

melodyogonna 1y ago

For $10 you can buy VPS with a lot more resources than that on both Contabo and Ovh

rollcat 1y ago

I've been using Docker swarm for internal & lightweight production workloads for 5+ years with zero issues. FD: it's a single node cluster on a reasonably powerful machine, but if anything, it's over-specced for what it does. Which I guess makes it more than good enough for hobby stuff - I'm playing with a multi-node cluster in my homelab and it's also working fine.

Taikonerd 1y ago
I think Docker Swarm makes a lot of sense for situations where K8s is too heavyweight. "Heavyweight" either in resource consumption, or just being too complex for a simple use case.
- psviderski 1y ago
  The only problem is Docker Swarm is essentially abandonware after Docker was acquired by Mirantis in 2019. Core features still work but there is a ton of open issues and PRs which are ignored. It's fine if it works but no one cares if you found a bug or have ideas on how to improve something, even worse if you want to contribute.
  - rollcat 1y ago
    Yep it's unfortunate, "it works for me" until it doesn't. OTOH it's not a moving target. Docker historically has been quite infamous for that, we were talking about half-lives for features, as if they were unstable isotopes. It took initiatives like OCI to get things to settle. K8s tries to solve the most complex problems, at the expense of leaving simple things stranded. If we had something like OCI for clustering, it would most likely take the same shape.

ClumsyPilot 1y ago

> Kubernetes is simply too resource-intensive to run on a $10/month VPS with just 1 shared vCPU and 2GB of RAM To put this in perspective, that’s less compute than a phone released in 2023, 12 years ago, Samsung Galaxy S4. To find this level of performance in a computer, we have to go to The main issue is that Kubernetes has created good API and primitives for managing cloud stuff, and managing a single server is still kinda crap despite decades of effort. I had K3S on my server, but replaced with docker + Traefik + Portainer - it’s not great, but less idle CPU use and fewer moving parts

jdsleppy 1y ago

I really like `DOCKER_HOST=ssh://... docker compose up -d`, what do you miss about Deployments?

JamesSwift 1y ago

Just go with a cloud provider that offers free control plane and shove a bunch of side projects into 1 node. I end up around $50 a month on GCP (was a bit cheaper at DO) once you include things like private docker registry etc. The marginal cost of an additional project on the cluster is essentially $0

figmert 1y ago

> What I really wish for is a lightweight alternative offering a Kubernetes-compatible API that runs well on inexpensive VPS instances. The gap between enterprise-grade container orchestration and affordable hobby hosting remains frustratingly wide. Depending on how much of the Kube API you need, Podman is that. It can generate containers and pods from Kubernetes manifests [0]. Kind of works like docker compose but with Kubernetes manifests. This even works with systemd units, similar to how it's outlined in the article. Podman also supports most (all?) of the Docker api, thus docker compose, works, but also, you can connect to remote sockets through ssh etc to do things. [0] https://docs.podman.io/en/latest/markdown/podman-kube-play.1.html https://docs.podman.io/en/latest/markdown/podman-kube-play.1... [1] https://docs.podman.io/en/latest/markdown/podman-systemd.unit.5.html https://docs.podman.io/en/latest/markdown/podman-systemd.uni...

sciencesama 1y ago
You can use firecracker !
jillyboel 1y ago
The docs don't make it clear, can it do "zero downtime" deployments? Meaning it first creates the new pod, waits for it to be healthy using the defined health checks and then removes the old one? Somehow integrating this with service/ingress/whatever so network traffic only goes to the healthy one?
- ffsm8 1y ago
  I can't speak on it's capabilities, but I feel like I have to ask: for what conceivable reason would you even want that extra error potential with migrations etc? It means you're forced to make everything always compatible between versions etc. For a deployment that isn't even making money and is running on a single node droplet with basically no performance... Why?
  - jillyboel 1y ago
    > I can't speak on it's capabilities, but I feel like I have to ask: for what conceivable reason would you even want that extra error potential with migrations etc? It's the default behavior of a kubernetes deployment which we're comparing things to. > It means you're forced to make everything always compatible between versions etc. For stateless services, not at all. The outside world just keeps talking to the previous version while the new version is starting up. For stateful services, it depends. Often there are software changes without changes to the schema. > For a deployment that isn't even making money I don't like looking at 504 gateway errors > and is running on a single node droplet with basically no performance I'm running this stuff on a server in my home, it has plenty of performance. Still don't want to waste it on kubernetes overhead, though. But even for a droplet, running the same application 2x isn't usually a big ask.

AdrianB1 1y ago

I am curious why your no revenue projects need the complexity, features and benefits of something like Kubernetes. Why you cannot just to it the archaic way of compiling your app, copy the files to a folder and run it there and never touch it for the next 5 years. If it is a dev environment with many changes, its on a local computer, not on VPS, I guess. Just curious by nature, I am.

mbreese 1y ago

> I'm constantly reinventing solutions to problems that Kubernetes already solves Another way to look at this is the Kubernetes created solutions to problems that were already solved at a lower scale level. Crontabs, http proxies, etc… were already solved at the individual server level. If you’re used to running large coordinated clusters, then yes — it can seem like you’re reinventing the wheel.

MyOutfitIsVague 1y ago

> Kubernetes is simply too resource-intensive to run on a $10/month VPS with just 1 shared vCPU and 2GB of RAM. That's more than what I'm paying for far fewer resources than Hetzner. I'm paying about $8 a month for 4 vCPUs and 8GB of RAM: https://www.hetzner.com/cloud https://www.hetzner.com/cloud Note that the really affordable ARM servers are German only, so if you're in the US you'll have to deal with higher latency to save that money, but I think it's worth it.

andrewmcwatters 1y ago
Thank you for sharing this. Do you have a referral link we can use to give you a little credit for informing us?
- MyOutfitIsVague 1y ago
  Sure, if you still want it: https://hetzner.cloud/?ref=WwByfoEfJJdv https://hetzner.cloud/?ref=WwByfoEfJJdv I guess it gives you 20 euros in credit, too. That's nice.
fhcbix 1y ago
I recently set up an arm64 VPS at netcup: https://www.netcup.com/en/server/arm-server https://www.netcup.com/en/server/arm-server Got it with no location fee (and 2x storage) during the easter sale but normally US is the cheapest.
- MyOutfitIsVague 1y ago
  That's pretty cheap. I have 4 vCPUs, 8GB RAM, 80GB disk, and 20TB traffic for €6. NetCup looks like it has 6VCPU, 8GB RAM, 256 GB, and what looks like maybe unlimited traffic for €5.26. That's really good. And it's in the US, where I am, so SSH would be less painful. I'll have to think about possibly switching. Thanks for the heads up.

artdigital 1y ago

I run my private stuff on a hosted vultr k8s cluster with 1 node for $10-$20 a month. All my hobby stuff is running on that "personal cluster" and it is that perfect sweetspot for me that you're talking about I don't use ingresses or loadbalancers because those cost extra, and either have the services exposed through tailscale (with tailscale operator) for stuff I only use myself, or through cloudflare argo tunnels for stuff I want internet accessible (Once a project graduates and becomes more serious, I migrate the container off this cluster and into a proper container runner)

lenerdenator 1y ago

The thing is, most of those enterprise-grade container orchestrations probably don't need k8s either. The more I look into it, the more I think of k8s as a way to "move to micro services" without actually moving to micro services. Loosely coupled micro services shouldn't need that level of coordination if they're truly loosely coupled.

SEJeff 1y ago

Have you tried k3s? I think it would run on a tiny vps like that and is a full stack. Instead of etcd it has sqlite embedded.

parliament32 1y ago

Why not minikube or one of the other resource-constrained k8s variants? https://minikube.sigs.k8s.io/ https://minikube.sigs.k8s.io/

jjwiseman 1y ago

I use Caprover to run about 26 services for personal projects on a Hetzner box. I like its simplicity. Worth it just for the one-click https cert management.

huksley 1y ago

I believe that Kubernetes is something you want to use if you have 1+ SRE full-time on your team. I actually got tired with complexity of kubernetes, AWS ECS and docker as well and just build a tool to deploy apps natively on the host. What's wrong with using Linux native primitives - systemd, crontab, postgresql or redis native package? Whose should work as intended, you don't need them in container.

byrnedo 1y ago

Please try https://github.com/skateco/skate https://github.com/skateco/skate, this is pretty much the exact same reason why I built it!