9 ms·

Show HN: Edka – Kubernetes clusters on your own Hetzner account

437 points by camil 1y ago

Hi HN, I’ve been working with Kubernetes for over a decade, since the alpha days, and was involved in kube-aws project before AWS launched EKS. For the past four years, I’ve been helping friends and small businesses cut costs by running Kubernetes on Hetzner Cloud, which I’ve found to be rock solid and by far the best priced provider. Provisioning a cluster on Hetzner is now straightforward, thanks to tools like k3s and hetzner-k3s, but configuring it for your specific needs still takes time and expertise. I built Edka to make that part easy: spin up a production ready cluster in ~2 minutes, then choose how low level or automated you want to go. How it works: Layer 1 – Cluster provisioning - Creates a k3s-based Kubernetes cluster on Hetzner (lightweight, easy to manage, scales well). Layer 2 – Add-ons - One-click deploy for metrics-server, cert-manager, and various operators; preconfigured for Hetzner, no extra setup needed. Layer 3 – Applications - Minimal config UIs for apps built on top of add-ons. - Example: Need PostgreSQL? Fill a few fields → platform installs CloudNativePG → provisions HA PostgreSQL with PITR → gives ready to use endpoints. Backups can be restored to any point in time with a click. Quick demo: https://edka.io/apps/ https://edka.io/apps/ Layer 4 – Deployments - Connect your CI to push container images to a public/private registry. - Edka updates deployments automatically (with semantic versioning rules), supports instant rollbacks, autoscaling, persistent volumes, secrets/env imports, and quick public exposure. Quick demo: https://edka.io/deployments/ https://edka.io/deployments/ Tech stack: TypeScript, React + Tailwind CSS, PostgreSQL, Redis, BullMQ, Vault + AWS KMS to encrypted sensitive data. The platform is still in beta and I’m building it in my spare time, so there are some rough edges, but I’d love feedback from anyone running Kubernetes on Hetzner, exploring alternatives to EKS/GKE/AKS or looking to automate their infrastructure with Kubernetes. More details: https://edka.io/ https://edka.io/ Thank you!

latchkey 1y ago
Great work. Just tried to email support@ and it bounced.
- camil 1y ago
  Thanks for letting me know. Apparently there was a wrong permission set for the Google group.
  - latchkey 1y ago
    Oh yea, that setting bites me all the time.
Lucasoato 1y ago
An Hetzner employee once told me that they’ve been trying for years to develop their own Kubernetes-as-a-service solution, I wonder if they’re still working on that or not.
- mdaniel 1y ago
  Years?! what. the. actual. fuck Well, I guess from a platform that has no intrinsic IAM offering, I take that back, I guess keeping track of whose special console password is the current one is, in fact, hard work
deleted 1y ago
[deleted]
SCHiM 1y ago
Could you explain: 1) What are the limitations of the scaling you do? Can I do this programmatically? I.e. send some requests to get additional pods of a specific type online? 2) What have you done in terms of security hardening? you mention hardened pods/cluster, but specifically, did you do pentest? Just follow best practice? Periodic scans? Stress tests?
- camil 1y ago
  Thanks for your questions! 1) The platform provides a control plane to help you deploy the cluster on your own Hetzner account, so you are in control of resources and pay direct usage costs to Hetzner. 2) Because you have full access to kubernetes cluster and it runs on your own Hetzner account, the security of the cluster is a shared responsibility and you can fine tune the configuration according to your requirements. The platform security is totally our responsibility. We try to follow best practices and internal penetration tests were conducted, but we're still in beta and try to see if there's interest for such product before launching the stable version.
barbazoo 1y ago
Love how focussed this is. I would have never guessed that there's an overlap between the circle of people wanting to run a prod workload on a K8s cluster and folks that need a GUI to set up and manage a K8s cluster would be that big but looks like I might be wrong.
- 0x457 1y ago
  > I would have never guessed that there's an overlap between the circle of people wanting to run a prod workload on a K8s cluster and folks that need a GUI to set up and manage a K8s cluster would be that big but looks like I might be wrong. Count how many GKE ad EKS users are out there?
- tormeh 1y ago
  Surely the appeal is more that someone will fix things if your k8s installation breaks?
zft 1y ago
Congratulations on the launch! Is there are plans to support Gitlab and gitlab registry (or any registry) ?
- camil 1y ago
  Thank you! Yes, both are planned. The registry will be a very easy implementation.
slig 1y ago
Congrats on shipping! I see that you have WordPress as a pro app. As someone who pays for WP hosting, what I'd like to see there is the ability to "fork" a WP instance, media, DB, everything, with a new hostname, that I can try things, updates, etc.
- camil 1y ago
  Thanks! Wordpress will be available for free, it is not currently finished. Probably next week will be ready.
upa11 1y ago
Great job. Love the project
- camil 1y ago
  Thank you!
czhu12 1y ago
Is this deploying K3s or full kubernetes with a control vs worker plane on different instances?
- camil 1y ago
  It is ready to use Kubernetes setup with separate control plane and node pools
- andix 1y ago
  k3s does support running separate control plane and worker node pools. It's not just for toy-project clusters, or single node clusters. k3s can also power rather big clusters.
MaKey 1y ago
What's the difference to a well established tool like kops (https://github.com/kubernetes/kops https://github.com/kubernetes/kops), which also supports Hetzner?
- worldsayshi 1y ago
  Does anyone know how this compares to using https://github.com/vitobotta/hetzner-k3s https://github.com/vitobotta/hetzner-k3s ?
- andix 1y ago
  I guess UI and commercial support.
- abound 1y ago
  There's also Talos, which also also supports Hetzner [1] and is similarly streamlined. Not quite the same idea but very similar. [1] https://www.talos.dev/v1.10/talos-guides/install/cloud-platforms/hetzner https://www.talos.dev/v1.10/talos-guides/install/cloud-platf...
- camil 1y ago
  Probably the easiest out there is https://github.com/vitobotta/hetzner-k3s https://github.com/vitobotta/hetzner-k3s. There are many options, depending on how low level you want to go. Hetzner terraform project is probably the most complex and complete, but it takes time to configure all those. The main idea was to provide simplification, not just to Kubernetes provisioning in Hetzner, but also to the most common apps and tools that extend Kubernetes capabilities, like ingress controllers, prometheus, elasticsearch, databases and so on.
andix 1y ago
There is this project to deploy k3s to Hetzner via Terraform: https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne... It's not the smoothest thing I've ever used, but it's all self hosted and everything can be fixed with some Terraform or SSH. Great to see some managed Kubernetes on Hetzner!
- trolla 1y ago
  This module is way better https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/tree/main https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/tr... I'm using it right now
  - andix 1y ago
    Thanks for the feedback. I've stumbled upon it when it when the project was quite new, and it looked promising. kube-hetzner seems to be a bit stuck, they have a big backlog for the next major release, but it might never happen.
  - camil 1y ago
    I agree, this is probably the most complete solution out there. My intentions with this project are to provide various layers of abstraction, not only for Kubernetes provisioning, but also for the most common apps and tools that are usually extending the Kubernetes capabilities and also allow some low level configuration options.
21sys 1y ago
I can't find this Spanish (?) company in the company register and there are none of the legally required information on the website. Not very trustworthy for a SaaS that stores your data and access keys. I'm confident that this is only a startup "day one" issue, but in times of increased scam and extortion can I be sure? Nope.
- camil 1y ago
  Hello there! Fair enough. As I mentioned in the original post, I built this as a side project, by myself, and I run it as a freelancer registered in Spain. It is not hard to find my public profile. You can check my Spanish VAT number, ESY1848661G. This is still in beta and currently looking to collect feedback and see if there is any interest in the market, before scaling it to a company. Thank you!
  - camil 1y ago
    Updated the privacy policy with missing details. You can also check officially here https://www.einforma.com/rapp/ficha/empresas?id=dWSG1MwtU312F_Eb5-jWUA https://www.einforma.com/rapp/ficha/empresas?id=dWSG1MwtU312... Also here: https://ceo.oepm.es/detalleExpediente?numExp=N0486066 https://ceo.oepm.es/detalleExpediente?numExp=N0486066
kopadudl 1y ago
Typo: One Cluser always free
- camil 1y ago
  Fixed, thanks!
trolla 1y ago
The pricing is misleading. You say on the landing page that the free plan is €0 per month, but in reality you still have to pay for the control plane nodes. The cheapest control plane node is €3.29 per month; if you want high-availability - you need to double that. So in reality you would pay €20 per month for the standard plan and also extra for control plane nodes. Also, with this Terraform module I can setup a similar setup without paying any extra fee for your service: https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/tree/main https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/tr... The Terraform module takes care of ingress, certificates, metrics etc Anyway, congrats for shipping
- parchley 1y ago
  Actually you want 3 nodes for high availability, two will do you nothing good
- 0x457 1y ago
  > if you want high-availability - you need to double that. Triple. 1 and 2 nodes will give failure allowance of zero.
figassis 1y ago
I have yet to see a guide to automate k8s on Hetzner's beefy bare metal instances. True, you want cattle, but being able to include some bare metal instance with amazing CPUs and memory would be great, and I do just that. My clusters include both cloud and bare metal instances. In the past I had used Hetzner virtual switch to create a shared L2 network between cloud and bare metal nodes. Now I just use tailscale. But the TF and other tools are using the API to add and kill nodes, if you could pass a class of nodes to those tools that they know can't create but are able to wipe and rebuild, this would be ideal.
- SOLAR_FIELDS 1y ago
  Would CAPH solve your issue? https://github.com/syself/cluster-api-provider-hetzner https://github.com/syself/cluster-api-provider-hetzner you would wire up cluster autoscaler to point to it using clusterapi https://github.com/kubernetes/autoscaler/tree/a9cb59fdd2dd888529af74929313f3f55c73e7c9/cluster-autoscaler/cloudprovider/clusterapi https://github.com/kubernetes/autoscaler/tree/a9cb59fdd2dd88...
CuriouslyC 1y ago
Why would I use Edka vs using Linode's free Kubernetes offering?
- czhu12 1y ago
  Linode pricing is probably 3-4x more expensive than Hetzner, who does not offer managed kubernetes.
- camil 1y ago
  This was designed for Hetzner, which I still believe has the best offer on the market comparing price, performance and stability. On top of that, the platform offers some ready to deploy add-ons that simplify the configuration after the initial cluster provisioning.
  - chatmasta 1y ago
    What Hetzner-specific functionality did you need to design that you wouldn’t need in a “deploy to arbitrary set of VMs” scenario?
    - camil 1y ago
      Hetzner was an easy choice because you can attach persistent volumes, expose services using their load balancers, servers are fast and easy to provision and they probably have the best pricing. I run multiple clusters in Hetzner for over 4 years now and only had minor issues. Sometimes they do not have enough instances on a specific region, sometimes provisioning new instances can be delayed or they send emails to reboot the instances due to patches to their hypervisors. But most of the times runs stable. Few of my clusters have 100% uptime for more than 2 years.
boredhacker3 1y ago
exactly what i was looking for. I will give it a shot !
- camil 1y ago
  Thank you! Please feel free to ask any questions.
dorongrinstein 1y ago
[flagged]
- lagniappe 1y ago
  Are you blowing out the birthday candles on someone else's cake?
- turtlebits 1y ago
  $5/month per core for self hosted? Thats way too expensive.
deleted 1y ago
[deleted]
everfrustrated 1y ago
Has anybody found a good way to use encrypted disks with Hetzner yet?
- winrid 1y ago
  Their installer script supports LUKS. Setup dropbear, and have another encrypted instance that runs a cron that runs a script every minute to check for the dropbear port on all instances and sshes in and passes the key to boot. This is what I do for fastcomments anyway for ovh and hetzner
- M4t7e 1y ago
  If you need disk encryption on Hetzner, I built a Terraform module that sets up a Kubernetes cluster with encrypted disks enabled by default: https://github.com/hcloud-k8s/terraform-hcloud-kubernetes https://github.com/hcloud-k8s/terraform-hcloud-kubernetes
- ralala 1y ago
  What is the threat model you want to mitigate using encryption at rest? Is it that a physical disk is not properly wiped after usage? Then you could just use luks and store the key anywhere else, e.g. another machine or an external volume…
- bflesch 1y ago
  Encrypted disks are easily setup with archlinux + LUKS + tinySSH, you can remote unlock via SSH.
- adamcharnock 1y ago
  To answer from a Kubernetes perspective: Both OpenEBS Mayastor and LocalZFS now support disk encryption.
zgk7iqea 1y ago
typo on the website: one cluser always free
deknos 1y ago
Am i the only one who is confused about "Hetzner" in the title and "AWS KMS" in the body?
- camil 1y ago
  Thanks for the feedback! Didn't plan to bring any confusion with that. The AWS KMS is used by the platform to encrypt/decrypt sensitive data before/after storing it in Vault and is part of the tech stack used to develop the platform.
  - deknos 1y ago
    it's more the thing, that if you put secrets on AWS, you are STILL dependent on AWS even if you run things on hetzner. It would be better, if you find a solution for secrets maintenance which runs on hetzner..
betaby 1y ago
Site doesn't answer how storage is 'solved'. Is this solution uses local folder provisioning when using PostgreSQL for example.
- camil 1y ago
  Sorry for that, I wasn't expecting such interest. There are still undocumented parts, but happy to answer any question. It uses https://github.com/hetznercloud/csi-driver https://github.com/hetznercloud/csi-driver to attach persistent volumes to PostgreSQL pods.
  - __turbobrew__ 1y ago
    I wonder how trustworthy hetzner distributed storage is. I always saw hetzner as just a control plane to allocate bare compute and nothing more, I wouldn’t necessarily trust their managed storage solutions, but I also don’t have much experience with it. Honestly I’m kindof surprised that something like rook it not used instead, but I guess it is easier to trust hetzner storage and hope for the best.
- pwmtr 1y ago
  If you are looking for Postgres on Hetzner, you may want to check out Ubicloud. We host in various bare metal providers, including Hetzner. (I am the lead engineer building Ubicloud PostgreSQL, so if you have questions I can answer them)
mfrye0 1y ago
This is incredibly timely. I've been an AWS customer for 10+ years and have been having a tough time with them lately. Looking at potentially moving off and considering options. My theory is that with terraform and a container based infra, that it should be pretty easier with Claude Code to migrate wherever.
- adamcharnock 1y ago
  This is exactly what we [1] do! We migrate clients out of AWS and into Hetzner bare-metal k8s clusters, and then we also become the client's DevOps team (typically for a lot less than Amazon charges) I will say that there is a fair bit of lifting required to spin up a k8s cluster on bare metal, particularly for things such as monitoring and distributed block storage (we use OpenEBS). I would ballpark it as a small number of months. It is likely easier on their cloud offering, but we've found that to be a little less reliable than we would hope. Happy to chat more: adam@... [1] https://lithus.eu https://lithus.eu
- topspin 1y ago
  I'm using AWS for small k8s clusters. I stay away from most of the "managed" AWS products except S3 and ECR. My k8s stack is packer + tofu + k3s + zfs: It's easy, concise, self managed, and costs are easy to predict.
- derfabianpeter 1y ago
  You might also want to review https://loopback.cloud https://loopback.cloud for that purpose
rumblefrog 1y ago
This looks great! Haven't tried it yet, but should I presume this also does k8s and OS updates for me? Or how managed is it?
- camil 1y ago
  Thanks for the feedback! The platform is mostly self service, but it is very easy to upgrade the Kubernetes version, just change the version in the cluster configuration. For OS updates, you can replace the nodes and it will automatically pick the latest OS image from Hetzner. I also run it isolated for some small companies, as a fully managed service, so that option is available as well.
rumblefrog 1y ago
I wonder how long before Hetzner adds something like managed Kubernetes to their native product line. They already have S3 compatible object storages, load balancers and more.
- Hamuko 1y ago
  No idea about the timing but I imagine it's coming. Would make a lot of sense, especially if you can combine it with the hardware servers. You could get a lot of grunt in your cluster for a lot less than for example AWS.
- mdaniel 1y ago
  In case others were curious https://docs.hetzner.com/storage/object-storage/supported-actions https://docs.hetzner.com/storage/object-storage/supported-ac... https://docs.hetzner.com/cloud/load-balancers/overview#:~:text=up%20to%2020%20load%20balancers%20across%20all%20projects https://docs.hetzner.com/cloud/load-balancers/overview#:~:te...
- hobofan 1y ago
  Given how rarely they offer specific software solutions and at what pace, I would say 5+ years from now or never.
sciencesama 1y ago
is there a selfhosted version of this ?
- import 1y ago
  You can just install k3s maybe?
marcosscriven 1y ago
When I was looking into this, I instead setup Proxmox on Hetzner (which you can do natively from ISO). From there it was much easier just using it for whatever I wanted, including K3S
bflesch 1y ago
I cant seem to figure out where this company is located and if it is a scam or not. Website has no imprint, no contact address. There is one email address in the privacy statement but it is "redacted by cloudflare". Also in privacy statement it says "Edka Digital S.L." but no idea which country it is registered it. For me it does not pass the smell test. No physical address, no idea who is running it, no idea if company is indeed registered or not. The pricing FAQ at least talks about VAT and I assume it is EU VAT but could be anything.
- camil 1y ago
  Hello there, as I mentioned in the post, I build this as a side project by my self and I'm running it as a freelancer registered in Spain, you can check my VAT number ESY1848661G. I was planning to collect some feedback and honestly didn't expect such interest in the project. I will make the necessary adjustments to the privacy policy and terms of service. When I started this, I had in mind to convert it into a company, but I'm still running it as a freelancer. Thanks for your feedback! I will correct my mistake.
  - bflesch 1y ago
    Hey, thanks for your immediate reply. Congrats for starting your own business. If you're Spanish-based maybe something like "aviso legal" at [1] or "legal notice" (imprint) at hetzner [2] is needed so people can validate that you/your company actually exist. I'm not familiar with Spanish S.L. (Sociedad Limitada) but it seems to be a private, share-based legal entity with minimum 3000 EUR share capital and at least one director. It seems the share capital does not need to be paid in full [3] which is a risk for potential customers if things go wrong. If you're based in a EU country I'd suggest to also clearly communicate all these legal information, because it's easier for potential customers to build trust into your services. [1] https://www.hola.com/aviso-legal/ https://www.hola.com/aviso-legal/ [2] https://www.hetzner.com/legal/legal-notice/ https://www.hetzner.com/legal/legal-notice/ [3] https://www.lawants.com/en/sl-spain/#:~:text=minimum%20share%20capital%20of%203%2C000%20EUR.%20Interestingly%2C%20this%20amount%20does%20not%20necessarily%20need%20to%20be%20fully%20paid%20up%20before%20the%20company%E2%80%99s%20formation https://www.lawants.com/en/sl-spain/#:~:text=minimum%20share...
rumblefrog 1y ago
I tried to deploy a small cluster in the US VA region, but the cluster status kept flipping between Failed and Creating with no clear way of troubleshooting it: 7ad975fb-3c8e-47a9-b03d-9e6bec81f0db
- camil 1y ago
  Hello there, sorry for that I will look into it right now.
marcosscriven 1y ago
What are the connectivity options between heztner dedicated servers? I see they allow you to pay to have in a single rack, with a dedicated switch. But does that introduce a risk of single point of failure in the rack power or switch?
physix 1y ago
This is a great idea. I really like it! We considered reaching out in May, but held back because we want to run on bare metal. Any chance to get this provisioned on bare metal at Hetzner? We have K8S running on bare metal there. It's a slog to get it all working, but for our use case, having a dedicated 10G LAN between nodes (and a bare metal Cassandra cluster in the same rack) makes a big difference in performance. Also, from a cost perspective. We run AX41-NVMe dedicated servers that cost us about EUR 64 per server with a 10G LAN, all in the same rack. Getting the same horsepower using Cloud instances I guess would be a CCX43, which costs almost double.
- adamcharnock 1y ago
  We're setting up a data-heavy client at the moment who has a similar need. We're working with Hetzner's custom solutions team to provision a multi-AZ setup, with 25G networking and 100G AZ interconnects. Link in bio if you want to chat, email is adam@...
- VoidWhisperer 1y ago
  Are you asking if it can provision bare metal servers with hetzner in a similar way to what it is doing with cloud servers, or if it can manage clusters on your hetzner bare metal servers (in the case of the second, a tool like Rancher might be better)
  - physix 1y ago
    I was thinking more of the former, whereby I "bring my own servers". I haven't really thought it through yet, whether that even makes sense.
    - VoidWhisperer 1y ago
      That might be a bit challenging unless they sort out an integration directly with hetzner as I don't think their API supports anything related to bare metal provisioning, just cloud and 'storage boxes'
    - moondev 1y ago
throwmeaway222 1y ago
Great job!
adamcharnock 1y ago
This certainly looks like a pleasingly straight-forward way to spin up k8s. I do notice that this deploys onto their cloud offering, which we've (https://lithus.eu https://lithus.eu) found to be a little shaky in a few places. We deploy clients onto their bare metal line-up which we find to be pretty rock solid. The worst that typically happens is the scheduled restart of an upstream router, which we mitigate via multi-AZ deployments. That being said, there is a base cluster size under which a custom bare-metal deployment isn't really viable in terms of economics/effort. So I'll definitely keep an eye on this.
JanMa 1y ago
A bit off topic, but you might want to rethink the name. It is very close to EDEKA, the largest German supermarket chain. They have a very large IT division (https://it.edeka https://it.edeka) and judging from the name of your project I was expecting it to be one of their projects.
- camil 1y ago
  Well, I had this since 2011, and in 2018 a new disease was labeled EDKA ( that is the first result you get when you google for edka). I became aware about the german supermarket few years after also. I could consider it at some point, but is very hard to find something available these days...
- physix 1y ago
  me too
sneak 1y ago
off topic: k8s aside, what are people using to receive webhooks from github/gitea/gitlab and do builds/deploys? is the generally accepted way to put deploy credentials into CI secrets and do it that way?
- mdaniel 1y ago
  I'm sure for 10 people you'll get 15 answers, but for my money OIDC is the way, the truth, and the light. GitHub and GitLab offer it, one can have federated auth from within a k8s Pod to anything that trusts OIDC, and realistically one can do it from anything that has intrinsic identity. That's also how AWS Identity Anywhere works, just with more X509 I really loved this talk about using Let's Encrypt for IAM Anywhere https://www.youtube.com/watch?v=M1hXUcBMf1Q https://www.youtube.com/watch?v=M1hXUcBMf1Q I have personally also set up EKS Anywhere <https://github.com/aws/eks-anywhere#readme https://github.com/aws/eks-anywhere#readme> with OIDC, so one need not have a "smart cloud" to get that done, but it places the burden upon security the cluster's identity upon the operator https://gitlab.com/-/snippets/2302594 https://gitlab.com/-/snippets/2302594
camil 1y ago
What a great fortune to reach top of HN and get product feedback, then get rate limited by GitHub ( that was an easy fix though...) Then Hetzner gets a partial outage, failing to create new instances, causing most of the new cluster to get stuck in "creating" mode. Thanks everyone for the interest, I was not expecting this from a side project and I'll do my best to improve the platform in the future and address all feedback received.
- lagniappe 1y ago
  It happens to almost all of us :) No worries, you did great! Thanks for sharing your project with us
  - camil 1y ago
    Thank you for your kind words! I really appreciate it.
- js4ever 1y ago
  About Hetzner it's nothing new, since July, 1-2 of 10 deployments will stay stuck in creating state for hours or forever. It's really annoying, especially when you deploy a multi node cluster.
  - camil 1y ago
    Probably I don't deploy enough or the EU regions are more reliable, but I was not experiencing issues during cluster provisioning. It is true that since July there were various small issues, for me mostly around removing provisioned resources, especially firewall configurations. Also noticed the web sockets are failing most of the times now and have to hard refresh the page to see the updates.
Seattle3503 1y ago
Hetzner is working on their own managed offering too, but it doesn't seem like anyone has an idea when it will land. https://www.reddit.com/r/hetzner/comments/18yhy89/seems_like_we_are_finally_getting_managed/ https://www.reddit.com/r/hetzner/comments/18yhy89/seems_like...
- k8sToGo 1y ago
  Considering how bad their object storage offering is, I'd never use this I think
  - krystofbe 1y ago
    I am having no issues with their object storage
    - k8sToGo 1y ago
      Hetzner subreddit is filled with posts of issues
  - camil 1y ago
    I am aware of that. I'm also not using their object storage. Still prefer GCS mostly and sometimes S3 or R2. For the compute offering, I didn't experience major issues. I have some clusters with 100% uptime for more than 2 years.
julienmarie 1y ago
What is the difference with Syself.com ? I was looking into them recently ?
- camil 1y ago
  I didn't have the chance to test their platform yet, but I expect it to be a mature product. My intentions with this platform are to make it more accessible to developers and small companies that do not have Kubernetes knowledge yet or want to spin clusters fast for development, testing etc.
flowerthoughts 1y ago
My biggest issue with k8s on Hetzner is that there is no way of going from tiny (1 machine) to medium (10-100 machines) purely on bare metal. I was able to get Gateway API to provide ingress nodes after a bit of Envoy tinkering, but storage is an issue. You can't run Ceph Rook on a single machine (too heavy,) and I couldn't find a think local volume wrapper that lets me easily migrate from local volume management to distributed. Feels like there should be a PV daemon that can do local, and transparent migrations and is lightweight enough to run on a single machine. Once my PV has been migrated to Ceph, the proxy could configure itself away from that job.
Banou 1y ago
Any plans to expand further than Hetzner? They're pretty restrictive on certain usages(e.g VPNs), I'd be really interested in support for Datapacket for example.
- camil 1y ago
  Was innitially designed for Hetzner, but they way it has been bult, will allow expanding to any provider. Especially to providers that offer their own Kubernetes distribution.
- mdaniel 1y ago
  Based on their stunningly hilarious "API"(sic), unlikely; their "create a server" mutation doesn't even support any user data https://api.datapacket.com/#definition-ServerInput https://api.datapacket.com/#definition-ServerInput
tzahifadida 1y ago
For anyone interested I am in the last stages of building a course operating around Kube-Hetzner (https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne... 3k github stars). Basically a lot of scripts that shows how this works and how to perform backups, restores etc... and a lot of exercises for common use cases and failure troubleshooting. Intentionally NOT abstracting away anything so you can see how this works. Without understanding you are going to get stuck. Join the waiting list here: https://shipacademy.dev https://shipacademy.dev
librasteve 1y ago
It’s a little off topic, but to mention Raku now has support for the Hetzner API https://raku.land/zef:wayland/WWW::CloudHosting::Hetzner https://raku.land/zef:wayland/WWW::CloudHosting::Hetzner That way we can use Raku as a scripting language for deployment.
- mdaniel 1y ago
  It's not a little off topic, it's way off topic for this thread. Your "for deployment" is missing the "of what" part, given the staggeringly thin surface area of your API. "I can reboot a server" is nowhere near a fully functioning Kubernetes control plane. Talk about the rest of the owl Anyway, if you're proud of your toy, do its own Show HN
oriettaxx 1y ago
how do you create instances i hetzner? with cloud init? I created for me some scripts to add node (Hetzner instance with Ubuntu) in a Docker Swarm, and I would really love to find something as you did but for Docker Swarm
reverseblade2 1y ago
I just use microk8s in Ubuntu on hetzner takes around 1 minute to set it up
curtisszmania 1y ago
[dead]
blessedcavapoo1 1y ago
[dead]
EDM115 1y ago
post it on the Hetzner subreddit, they regularly watch it :)
footlong2 1y ago
[dead]
dexcs 1y ago
Does it support root servers and / or is it planned?
fduran 1y ago
Hola, I may try this out (once the HN hug of death slows down). At SadServers we ran a GKE cluster with 0.2 vCPUs workload for a PoC at it was costing us CAD $50/day until we figured out thing like GCP using a pod not doing anything just keeping a node up using 1 CPU, hidden Prometheus metrics etc.
Too 1y ago
Why isn’t Hetzner providing a managed control plane themselves? Seems like a small investment to make for the return of converting massive amounts of customers stuck on GKE/EKS/AKS. That, plus a good IAM story, and they will be seen as a full cloud solution, not just a basement of servers. They have the infrastructure, it just need to be packaged.
2start 1y ago
sounds and looks great! There are many good projects out there to deploy Kubernetes https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne..., https://github.com/vitobotta/hetzner-k3s https://github.com/vitobotta/hetzner-k3s. However, I like that yours is manageable via Dashboard and you install basic applications. Feels way smoother than Terraform templates or CLI tools. Setting up kubernetes yourself with k3s and deploying a HA database with cnpg is so simple nowadays. It works great until some day you face an issue that you can't fix without diving deep into the details e.g. debugging your CNI or the implementations of specific operators to understand reconciliation errors. I faced countless of these issues such as: - your Postgres CNPG failed during failover and can't recover on it's own - Postgres WAL archiving suddenly doesn't work anymore after an update. - Cluster networking on some nodes doesn't work anymore due to an incompatibility of kube-proxy and Cilium after an update Unless it's your core business I'd limit myself to deploying applications on Kubernetes instead of managing Kubernetes itself. At least for use cases where reliability matters. There are some reasonably priced managed Kubernetes offerings out there. For example syself (https://syself.com https://syself.com) built a managed Kubernetes on Hetzner. The prices are super reasonable. For 300 Euros per month you can enjoy Hetzner prices with an experience that beats AWS.
- kazanz 1y ago
  I'm going to second the Syself (https://syself.com https://syself.com) recommendation. We're running our entire PaaS platform (Sherpa.sh) on top of Hetzner with Syself. We are 5x more affordable than Vercel and a big piece of that is because Syself allows us to operate autoscaling k8s clusters without an army of ops engineers. We tried hetzner-k3s, and many other solutions. But ran into many of these same problems. Syself on the other hand, has solved all the sticky hardware and networking challenges at scale with their own customer operators. Plus they're support is super responsive and helpful. The one downside, is they don't have a dashboard yet, so you need to be comfortable in the CLI. But I'm sure they are working on it. I don't think I would manage kubernetes any other way at this point.
softwaredam 1y ago
To me this seems more than a side project. Really well done. Nowadays, Hetzner has many Kubernetes providers. One that I really like is Syself.com. They provide production-ready Kubernetes including self-healing, bare-metal support, OIDC integration and many more. As a cloud and migration expert, I have evaluated them and they turned out to perform very well.
- softwaredam 1y ago
  https://syself.com/ https://syself.com/ is what I am talking about.
lucasrattz 1y ago
How does it compare to established solutions like https://syself.com https://syself.com? Does it support bare metal servers too?
dev_ago 1y ago
[dead]
dev_ago 1y ago
[dead]
PaleHazy 1y ago
For those who like k3s and are familiar with it, this looks like a great natural initiative. I never really bought into k3s myself for whatever reason, even on RPis I don’t feel too constrained on resources. A similar approach I’ve been using is using Syself (running it for about half a year now with no issues). They contributed the Cluster API provider for Hetzner, which is obviously Chad. Of course, I wish CAPI were a bit less involved, but that’s exactly where external help makes sense. I am totally fully sold on CAPI, but then again always need your base cluster. I still want to set up CAPI on my own, but still preferred having “someone else” host the control plane. With CAPI, though, you have the flexibility to do either. Also Talos and Omni are cool and pretty chill on Hetzner. Fantastic tools. Still though I just like the "raw-ish" feel to kubeadm and CAPI feels the most "native" solution. In terms of addons, I have my very curated yaml manifestos. And ArgoCD Application sets. That part is hard for me to replace, but that should defenitely be married with a UI.

437 points by camil 1y ago

latchkey 1y ago

Great work. Just tried to email support@ and it bounced.

camil 1y ago
Thanks for letting me know. Apparently there was a wrong permission set for the Google group.
- latchkey 1y ago
  Oh yea, that setting bites me all the time.

Lucasoato 1y ago

An Hetzner employee once told me that they’ve been trying for years to develop their own Kubernetes-as-a-service solution, I wonder if they’re still working on that or not.

mdaniel 1y ago
Years?! what. the. actual. fuck Well, I guess from a platform that has no intrinsic IAM offering, I take that back, I guess keeping track of whose special console password is the current one is, in fact, hard work

deleted 1y ago

[deleted]

SCHiM 1y ago

Could you explain: 1) What are the limitations of the scaling you do? Can I do this programmatically? I.e. send some requests to get additional pods of a specific type online? 2) What have you done in terms of security hardening? you mention hardened pods/cluster, but specifically, did you do pentest? Just follow best practice? Periodic scans? Stress tests?

camil 1y ago
Thanks for your questions! 1) The platform provides a control plane to help you deploy the cluster on your own Hetzner account, so you are in control of resources and pay direct usage costs to Hetzner. 2) Because you have full access to kubernetes cluster and it runs on your own Hetzner account, the security of the cluster is a shared responsibility and you can fine tune the configuration according to your requirements. The platform security is totally our responsibility. We try to follow best practices and internal penetration tests were conducted, but we're still in beta and try to see if there's interest for such product before launching the stable version.

barbazoo 1y ago

Love how focussed this is. I would have never guessed that there's an overlap between the circle of people wanting to run a prod workload on a K8s cluster and folks that need a GUI to set up and manage a K8s cluster would be that big but looks like I might be wrong.

0x457 1y ago
> I would have never guessed that there's an overlap between the circle of people wanting to run a prod workload on a K8s cluster and folks that need a GUI to set up and manage a K8s cluster would be that big but looks like I might be wrong. Count how many GKE ad EKS users are out there?
tormeh 1y ago
Surely the appeal is more that someone will fix things if your k8s installation breaks?

zft 1y ago

Congratulations on the launch! Is there are plans to support Gitlab and gitlab registry (or any registry) ?

camil 1y ago
Thank you! Yes, both are planned. The registry will be a very easy implementation.

slig 1y ago

Congrats on shipping! I see that you have WordPress as a pro app. As someone who pays for WP hosting, what I'd like to see there is the ability to "fork" a WP instance, media, DB, everything, with a new hostname, that I can try things, updates, etc.

camil 1y ago
Thanks! Wordpress will be available for free, it is not currently finished. Probably next week will be ready.

upa11 1y ago

Great job. Love the project

camil 1y ago
Thank you!

czhu12 1y ago

Is this deploying K3s or full kubernetes with a control vs worker plane on different instances?

camil 1y ago
It is ready to use Kubernetes setup with separate control plane and node pools
andix 1y ago
k3s does support running separate control plane and worker node pools. It's not just for toy-project clusters, or single node clusters. k3s can also power rather big clusters.

MaKey 1y ago

What's the difference to a well established tool like kops (https://github.com/kubernetes/kops https://github.com/kubernetes/kops), which also supports Hetzner?

worldsayshi 1y ago
Does anyone know how this compares to using https://github.com/vitobotta/hetzner-k3s https://github.com/vitobotta/hetzner-k3s ?
andix 1y ago
I guess UI and commercial support.
abound 1y ago
There's also Talos, which also also supports Hetzner [1] and is similarly streamlined. Not quite the same idea but very similar. [1] https://www.talos.dev/v1.10/talos-guides/install/cloud-platforms/hetzner https://www.talos.dev/v1.10/talos-guides/install/cloud-platf...
camil 1y ago
Probably the easiest out there is https://github.com/vitobotta/hetzner-k3s https://github.com/vitobotta/hetzner-k3s. There are many options, depending on how low level you want to go. Hetzner terraform project is probably the most complex and complete, but it takes time to configure all those. The main idea was to provide simplification, not just to Kubernetes provisioning in Hetzner, but also to the most common apps and tools that extend Kubernetes capabilities, like ingress controllers, prometheus, elasticsearch, databases and so on.

andix 1y ago

There is this project to deploy k3s to Hetzner via Terraform: https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne... It's not the smoothest thing I've ever used, but it's all self hosted and everything can be fixed with some Terraform or SSH. Great to see some managed Kubernetes on Hetzner!

trolla 1y ago
This module is way better https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/tree/main https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/tr... I'm using it right now
- andix 1y ago
  Thanks for the feedback. I've stumbled upon it when it when the project was quite new, and it looked promising. kube-hetzner seems to be a bit stuck, they have a big backlog for the next major release, but it might never happen.
- camil 1y ago
  I agree, this is probably the most complete solution out there. My intentions with this project are to provide various layers of abstraction, not only for Kubernetes provisioning, but also for the most common apps and tools that are usually extending the Kubernetes capabilities and also allow some low level configuration options.

21sys 1y ago

I can't find this Spanish (?) company in the company register and there are none of the legally required information on the website. Not very trustworthy for a SaaS that stores your data and access keys. I'm confident that this is only a startup "day one" issue, but in times of increased scam and extortion can I be sure? Nope.

camil 1y ago
Hello there! Fair enough. As I mentioned in the original post, I built this as a side project, by myself, and I run it as a freelancer registered in Spain. It is not hard to find my public profile. You can check my Spanish VAT number, ESY1848661G. This is still in beta and currently looking to collect feedback and see if there is any interest in the market, before scaling it to a company. Thank you!
- camil 1y ago
  Updated the privacy policy with missing details. You can also check officially here https://www.einforma.com/rapp/ficha/empresas?id=dWSG1MwtU312F_Eb5-jWUA https://www.einforma.com/rapp/ficha/empresas?id=dWSG1MwtU312... Also here: https://ceo.oepm.es/detalleExpediente?numExp=N0486066 https://ceo.oepm.es/detalleExpediente?numExp=N0486066

kopadudl 1y ago

Typo: One Cluser always free

camil 1y ago
Fixed, thanks!

trolla 1y ago

The pricing is misleading. You say on the landing page that the free plan is €0 per month, but in reality you still have to pay for the control plane nodes. The cheapest control plane node is €3.29 per month; if you want high-availability - you need to double that. So in reality you would pay €20 per month for the standard plan and also extra for control plane nodes. Also, with this Terraform module I can setup a similar setup without paying any extra fee for your service: https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/tree/main https://github.com/hcloud-k8s/terraform-hcloud-kubernetes/tr... The Terraform module takes care of ingress, certificates, metrics etc Anyway, congrats for shipping

parchley 1y ago
Actually you want 3 nodes for high availability, two will do you nothing good
0x457 1y ago
> if you want high-availability - you need to double that. Triple. 1 and 2 nodes will give failure allowance of zero.

figassis 1y ago

I have yet to see a guide to automate k8s on Hetzner's beefy bare metal instances. True, you want cattle, but being able to include some bare metal instance with amazing CPUs and memory would be great, and I do just that. My clusters include both cloud and bare metal instances. In the past I had used Hetzner virtual switch to create a shared L2 network between cloud and bare metal nodes. Now I just use tailscale. But the TF and other tools are using the API to add and kill nodes, if you could pass a class of nodes to those tools that they know can't create but are able to wipe and rebuild, this would be ideal.

SOLAR_FIELDS 1y ago
Would CAPH solve your issue? https://github.com/syself/cluster-api-provider-hetzner https://github.com/syself/cluster-api-provider-hetzner you would wire up cluster autoscaler to point to it using clusterapi https://github.com/kubernetes/autoscaler/tree/a9cb59fdd2dd888529af74929313f3f55c73e7c9/cluster-autoscaler/cloudprovider/clusterapi https://github.com/kubernetes/autoscaler/tree/a9cb59fdd2dd88...

CuriouslyC 1y ago

Why would I use Edka vs using Linode's free Kubernetes offering?

czhu12 1y ago
Linode pricing is probably 3-4x more expensive than Hetzner, who does not offer managed kubernetes.
camil 1y ago
This was designed for Hetzner, which I still believe has the best offer on the market comparing price, performance and stability. On top of that, the platform offers some ready to deploy add-ons that simplify the configuration after the initial cluster provisioning.
- chatmasta 1y ago
  What Hetzner-specific functionality did you need to design that you wouldn’t need in a “deploy to arbitrary set of VMs” scenario?
  - camil 1y ago
    Hetzner was an easy choice because you can attach persistent volumes, expose services using their load balancers, servers are fast and easy to provision and they probably have the best pricing. I run multiple clusters in Hetzner for over 4 years now and only had minor issues. Sometimes they do not have enough instances on a specific region, sometimes provisioning new instances can be delayed or they send emails to reboot the instances due to patches to their hypervisors. But most of the times runs stable. Few of my clusters have 100% uptime for more than 2 years.

boredhacker3 1y ago

exactly what i was looking for. I will give it a shot !

camil 1y ago
Thank you! Please feel free to ask any questions.

dorongrinstein 1y ago

[flagged]

lagniappe 1y ago
Are you blowing out the birthday candles on someone else's cake?
turtlebits 1y ago
$5/month per core for self hosted? Thats way too expensive.

deleted 1y ago

[deleted]

everfrustrated 1y ago

Has anybody found a good way to use encrypted disks with Hetzner yet?

winrid 1y ago
Their installer script supports LUKS. Setup dropbear, and have another encrypted instance that runs a cron that runs a script every minute to check for the dropbear port on all instances and sshes in and passes the key to boot. This is what I do for fastcomments anyway for ovh and hetzner
M4t7e 1y ago
If you need disk encryption on Hetzner, I built a Terraform module that sets up a Kubernetes cluster with encrypted disks enabled by default: https://github.com/hcloud-k8s/terraform-hcloud-kubernetes https://github.com/hcloud-k8s/terraform-hcloud-kubernetes
ralala 1y ago
What is the threat model you want to mitigate using encryption at rest? Is it that a physical disk is not properly wiped after usage? Then you could just use luks and store the key anywhere else, e.g. another machine or an external volume…
bflesch 1y ago
Encrypted disks are easily setup with archlinux + LUKS + tinySSH, you can remote unlock via SSH.
adamcharnock 1y ago
To answer from a Kubernetes perspective: Both OpenEBS Mayastor and LocalZFS now support disk encryption.

zgk7iqea 1y ago

typo on the website: one cluser always free

deknos 1y ago

Am i the only one who is confused about "Hetzner" in the title and "AWS KMS" in the body?

camil 1y ago
Thanks for the feedback! Didn't plan to bring any confusion with that. The AWS KMS is used by the platform to encrypt/decrypt sensitive data before/after storing it in Vault and is part of the tech stack used to develop the platform.
- deknos 1y ago
  it's more the thing, that if you put secrets on AWS, you are STILL dependent on AWS even if you run things on hetzner. It would be better, if you find a solution for secrets maintenance which runs on hetzner..

betaby 1y ago

Site doesn't answer how storage is 'solved'. Is this solution uses local folder provisioning when using PostgreSQL for example.

camil 1y ago
Sorry for that, I wasn't expecting such interest. There are still undocumented parts, but happy to answer any question. It uses https://github.com/hetznercloud/csi-driver https://github.com/hetznercloud/csi-driver to attach persistent volumes to PostgreSQL pods.
- __turbobrew__ 1y ago
  I wonder how trustworthy hetzner distributed storage is. I always saw hetzner as just a control plane to allocate bare compute and nothing more, I wouldn’t necessarily trust their managed storage solutions, but I also don’t have much experience with it. Honestly I’m kindof surprised that something like rook it not used instead, but I guess it is easier to trust hetzner storage and hope for the best.
pwmtr 1y ago
If you are looking for Postgres on Hetzner, you may want to check out Ubicloud. We host in various bare metal providers, including Hetzner. (I am the lead engineer building Ubicloud PostgreSQL, so if you have questions I can answer them)

mfrye0 1y ago

This is incredibly timely. I've been an AWS customer for 10+ years and have been having a tough time with them lately. Looking at potentially moving off and considering options. My theory is that with terraform and a container based infra, that it should be pretty easier with Claude Code to migrate wherever.

adamcharnock 1y ago
This is exactly what we [1] do! We migrate clients out of AWS and into Hetzner bare-metal k8s clusters, and then we also become the client's DevOps team (typically for a lot less than Amazon charges) I will say that there is a fair bit of lifting required to spin up a k8s cluster on bare metal, particularly for things such as monitoring and distributed block storage (we use OpenEBS). I would ballpark it as a small number of months. It is likely easier on their cloud offering, but we've found that to be a little less reliable than we would hope. Happy to chat more: adam@... [1] https://lithus.eu https://lithus.eu
topspin 1y ago
I'm using AWS for small k8s clusters. I stay away from most of the "managed" AWS products except S3 and ECR. My k8s stack is packer + tofu + k3s + zfs: It's easy, concise, self managed, and costs are easy to predict.
derfabianpeter 1y ago
You might also want to review https://loopback.cloud https://loopback.cloud for that purpose

rumblefrog 1y ago

This looks great! Haven't tried it yet, but should I presume this also does k8s and OS updates for me? Or how managed is it?

camil 1y ago
Thanks for the feedback! The platform is mostly self service, but it is very easy to upgrade the Kubernetes version, just change the version in the cluster configuration. For OS updates, you can replace the nodes and it will automatically pick the latest OS image from Hetzner. I also run it isolated for some small companies, as a fully managed service, so that option is available as well.

rumblefrog 1y ago

I wonder how long before Hetzner adds something like managed Kubernetes to their native product line. They already have S3 compatible object storages, load balancers and more.

Hamuko 1y ago
No idea about the timing but I imagine it's coming. Would make a lot of sense, especially if you can combine it with the hardware servers. You could get a lot of grunt in your cluster for a lot less than for example AWS.
mdaniel 1y ago
In case others were curious https://docs.hetzner.com/storage/object-storage/supported-actions https://docs.hetzner.com/storage/object-storage/supported-ac... https://docs.hetzner.com/cloud/load-balancers/overview#:~:text=up%20to%2020%20load%20balancers%20across%20all%20projects https://docs.hetzner.com/cloud/load-balancers/overview#:~:te...
hobofan 1y ago
Given how rarely they offer specific software solutions and at what pace, I would say 5+ years from now or never.

sciencesama 1y ago

is there a selfhosted version of this ?

import 1y ago
You can just install k3s maybe?

marcosscriven 1y ago

When I was looking into this, I instead setup Proxmox on Hetzner (which you can do natively from ISO). From there it was much easier just using it for whatever I wanted, including K3S

bflesch 1y ago

I cant seem to figure out where this company is located and if it is a scam or not. Website has no imprint, no contact address. There is one email address in the privacy statement but it is "redacted by cloudflare". Also in privacy statement it says "Edka Digital S.L." but no idea which country it is registered it. For me it does not pass the smell test. No physical address, no idea who is running it, no idea if company is indeed registered or not. The pricing FAQ at least talks about VAT and I assume it is EU VAT but could be anything.

camil 1y ago
Hello there, as I mentioned in the post, I build this as a side project by my self and I'm running it as a freelancer registered in Spain, you can check my VAT number ESY1848661G. I was planning to collect some feedback and honestly didn't expect such interest in the project. I will make the necessary adjustments to the privacy policy and terms of service. When I started this, I had in mind to convert it into a company, but I'm still running it as a freelancer. Thanks for your feedback! I will correct my mistake.
- bflesch 1y ago
  Hey, thanks for your immediate reply. Congrats for starting your own business. If you're Spanish-based maybe something like "aviso legal" at [1] or "legal notice" (imprint) at hetzner [2] is needed so people can validate that you/your company actually exist. I'm not familiar with Spanish S.L. (Sociedad Limitada) but it seems to be a private, share-based legal entity with minimum 3000 EUR share capital and at least one director. It seems the share capital does not need to be paid in full [3] which is a risk for potential customers if things go wrong. If you're based in a EU country I'd suggest to also clearly communicate all these legal information, because it's easier for potential customers to build trust into your services. [1] https://www.hola.com/aviso-legal/ https://www.hola.com/aviso-legal/ [2] https://www.hetzner.com/legal/legal-notice/ https://www.hetzner.com/legal/legal-notice/ [3] https://www.lawants.com/en/sl-spain/#:~:text=minimum%20share%20capital%20of%203%2C000%20EUR.%20Interestingly%2C%20this%20amount%20does%20not%20necessarily%20need%20to%20be%20fully%20paid%20up%20before%20the%20company%E2%80%99s%20formation https://www.lawants.com/en/sl-spain/#:~:text=minimum%20share...

rumblefrog 1y ago

I tried to deploy a small cluster in the US VA region, but the cluster status kept flipping between Failed and Creating with no clear way of troubleshooting it: 7ad975fb-3c8e-47a9-b03d-9e6bec81f0db

camil 1y ago
Hello there, sorry for that I will look into it right now.

marcosscriven 1y ago

What are the connectivity options between heztner dedicated servers? I see they allow you to pay to have in a single rack, with a dedicated switch. But does that introduce a risk of single point of failure in the rack power or switch?

physix 1y ago

This is a great idea. I really like it! We considered reaching out in May, but held back because we want to run on bare metal. Any chance to get this provisioned on bare metal at Hetzner? We have K8S running on bare metal there. It's a slog to get it all working, but for our use case, having a dedicated 10G LAN between nodes (and a bare metal Cassandra cluster in the same rack) makes a big difference in performance. Also, from a cost perspective. We run AX41-NVMe dedicated servers that cost us about EUR 64 per server with a 10G LAN, all in the same rack. Getting the same horsepower using Cloud instances I guess would be a CCX43, which costs almost double.

adamcharnock 1y ago
We're setting up a data-heavy client at the moment who has a similar need. We're working with Hetzner's custom solutions team to provision a multi-AZ setup, with 25G networking and 100G AZ interconnects. Link in bio if you want to chat, email is adam@...
VoidWhisperer 1y ago
Are you asking if it can provision bare metal servers with hetzner in a similar way to what it is doing with cloud servers, or if it can manage clusters on your hetzner bare metal servers (in the case of the second, a tool like Rancher might be better)
- physix 1y ago
  I was thinking more of the former, whereby I "bring my own servers". I haven't really thought it through yet, whether that even makes sense.
  - VoidWhisperer 1y ago
    That might be a bit challenging unless they sort out an integration directly with hetzner as I don't think their API supports anything related to bare metal provisioning, just cloud and 'storage boxes'
  - moondev 1y ago

throwmeaway222 1y ago

Great job!

adamcharnock 1y ago

This certainly looks like a pleasingly straight-forward way to spin up k8s. I do notice that this deploys onto their cloud offering, which we've (https://lithus.eu https://lithus.eu) found to be a little shaky in a few places. We deploy clients onto their bare metal line-up which we find to be pretty rock solid. The worst that typically happens is the scheduled restart of an upstream router, which we mitigate via multi-AZ deployments. That being said, there is a base cluster size under which a custom bare-metal deployment isn't really viable in terms of economics/effort. So I'll definitely keep an eye on this.

JanMa 1y ago

A bit off topic, but you might want to rethink the name. It is very close to EDEKA, the largest German supermarket chain. They have a very large IT division (https://it.edeka https://it.edeka) and judging from the name of your project I was expecting it to be one of their projects.

camil 1y ago
Well, I had this since 2011, and in 2018 a new disease was labeled EDKA ( that is the first result you get when you google for edka). I became aware about the german supermarket few years after also. I could consider it at some point, but is very hard to find something available these days...
physix 1y ago
me too

sneak 1y ago

off topic: k8s aside, what are people using to receive webhooks from github/gitea/gitlab and do builds/deploys? is the generally accepted way to put deploy credentials into CI secrets and do it that way?

mdaniel 1y ago
I'm sure for 10 people you'll get 15 answers, but for my money OIDC is the way, the truth, and the light. GitHub and GitLab offer it, one can have federated auth from within a k8s Pod to anything that trusts OIDC, and realistically one can do it from anything that has intrinsic identity. That's also how AWS Identity Anywhere works, just with more X509 I really loved this talk about using Let's Encrypt for IAM Anywhere https://www.youtube.com/watch?v=M1hXUcBMf1Q https://www.youtube.com/watch?v=M1hXUcBMf1Q I have personally also set up EKS Anywhere <https://github.com/aws/eks-anywhere#readme https://github.com/aws/eks-anywhere#readme> with OIDC, so one need not have a "smart cloud" to get that done, but it places the burden upon security the cluster's identity upon the operator https://gitlab.com/-/snippets/2302594 https://gitlab.com/-/snippets/2302594

camil 1y ago

What a great fortune to reach top of HN and get product feedback, then get rate limited by GitHub ( that was an easy fix though...) Then Hetzner gets a partial outage, failing to create new instances, causing most of the new cluster to get stuck in "creating" mode. Thanks everyone for the interest, I was not expecting this from a side project and I'll do my best to improve the platform in the future and address all feedback received.

lagniappe 1y ago
It happens to almost all of us :) No worries, you did great! Thanks for sharing your project with us
- camil 1y ago
  Thank you for your kind words! I really appreciate it.
js4ever 1y ago
About Hetzner it's nothing new, since July, 1-2 of 10 deployments will stay stuck in creating state for hours or forever. It's really annoying, especially when you deploy a multi node cluster.
- camil 1y ago
  Probably I don't deploy enough or the EU regions are more reliable, but I was not experiencing issues during cluster provisioning. It is true that since July there were various small issues, for me mostly around removing provisioned resources, especially firewall configurations. Also noticed the web sockets are failing most of the times now and have to hard refresh the page to see the updates.

Seattle3503 1y ago

Hetzner is working on their own managed offering too, but it doesn't seem like anyone has an idea when it will land. https://www.reddit.com/r/hetzner/comments/18yhy89/seems_like_we_are_finally_getting_managed/ https://www.reddit.com/r/hetzner/comments/18yhy89/seems_like...

k8sToGo 1y ago
Considering how bad their object storage offering is, I'd never use this I think
- krystofbe 1y ago
  I am having no issues with their object storage
  - k8sToGo 1y ago
    Hetzner subreddit is filled with posts of issues
- camil 1y ago
  I am aware of that. I'm also not using their object storage. Still prefer GCS mostly and sometimes S3 or R2. For the compute offering, I didn't experience major issues. I have some clusters with 100% uptime for more than 2 years.

julienmarie 1y ago

What is the difference with Syself.com ? I was looking into them recently ?

camil 1y ago
I didn't have the chance to test their platform yet, but I expect it to be a mature product. My intentions with this platform are to make it more accessible to developers and small companies that do not have Kubernetes knowledge yet or want to spin clusters fast for development, testing etc.

flowerthoughts 1y ago

My biggest issue with k8s on Hetzner is that there is no way of going from tiny (1 machine) to medium (10-100 machines) purely on bare metal. I was able to get Gateway API to provide ingress nodes after a bit of Envoy tinkering, but storage is an issue. You can't run Ceph Rook on a single machine (too heavy,) and I couldn't find a think local volume wrapper that lets me easily migrate from local volume management to distributed. Feels like there should be a PV daemon that can do local, and transparent migrations and is lightweight enough to run on a single machine. Once my PV has been migrated to Ceph, the proxy could configure itself away from that job.

Banou 1y ago

Any plans to expand further than Hetzner? They're pretty restrictive on certain usages(e.g VPNs), I'd be really interested in support for Datapacket for example.

camil 1y ago
Was innitially designed for Hetzner, but they way it has been bult, will allow expanding to any provider. Especially to providers that offer their own Kubernetes distribution.
mdaniel 1y ago
Based on their stunningly hilarious "API"(sic), unlikely; their "create a server" mutation doesn't even support any user data https://api.datapacket.com/#definition-ServerInput https://api.datapacket.com/#definition-ServerInput

tzahifadida 1y ago

For anyone interested I am in the last stages of building a course operating around Kube-Hetzner (https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne... 3k github stars). Basically a lot of scripts that shows how this works and how to perform backups, restores etc... and a lot of exercises for common use cases and failure troubleshooting. Intentionally NOT abstracting away anything so you can see how this works. Without understanding you are going to get stuck. Join the waiting list here: https://shipacademy.dev https://shipacademy.dev

librasteve 1y ago

It’s a little off topic, but to mention Raku now has support for the Hetzner API https://raku.land/zef:wayland/WWW::CloudHosting::Hetzner https://raku.land/zef:wayland/WWW::CloudHosting::Hetzner That way we can use Raku as a scripting language for deployment.

mdaniel 1y ago
It's not a little off topic, it's way off topic for this thread. Your "for deployment" is missing the "of what" part, given the staggeringly thin surface area of your API. "I can reboot a server" is nowhere near a fully functioning Kubernetes control plane. Talk about the rest of the owl Anyway, if you're proud of your toy, do its own Show HN

oriettaxx 1y ago

how do you create instances i hetzner? with cloud init? I created for me some scripts to add node (Hetzner instance with Ubuntu) in a Docker Swarm, and I would really love to find something as you did but for Docker Swarm

reverseblade2 1y ago

I just use microk8s in Ubuntu on hetzner takes around 1 minute to set it up

curtisszmania 1y ago

[dead]

blessedcavapoo1 1y ago

[dead]

EDM115 1y ago

post it on the Hetzner subreddit, they regularly watch it :)

footlong2 1y ago

[dead]

dexcs 1y ago

Does it support root servers and / or is it planned?

fduran 1y ago

Hola, I may try this out (once the HN hug of death slows down). At SadServers we ran a GKE cluster with 0.2 vCPUs workload for a PoC at it was costing us CAD $50/day until we figured out thing like GCP using a pod not doing anything just keeping a node up using 1 CPU, hidden Prometheus metrics etc.

Too 1y ago

Why isn’t Hetzner providing a managed control plane themselves? Seems like a small investment to make for the return of converting massive amounts of customers stuck on GKE/EKS/AKS. That, plus a good IAM story, and they will be seen as a full cloud solution, not just a basement of servers. They have the infrastructure, it just need to be packaged.

2start 1y ago

sounds and looks great! There are many good projects out there to deploy Kubernetes https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner https://github.com/kube-hetzner/terraform-hcloud-kube-hetzne..., https://github.com/vitobotta/hetzner-k3s https://github.com/vitobotta/hetzner-k3s. However, I like that yours is manageable via Dashboard and you install basic applications. Feels way smoother than Terraform templates or CLI tools. Setting up kubernetes yourself with k3s and deploying a HA database with cnpg is so simple nowadays. It works great until some day you face an issue that you can't fix without diving deep into the details e.g. debugging your CNI or the implementations of specific operators to understand reconciliation errors. I faced countless of these issues such as: - your Postgres CNPG failed during failover and can't recover on it's own - Postgres WAL archiving suddenly doesn't work anymore after an update. - Cluster networking on some nodes doesn't work anymore due to an incompatibility of kube-proxy and Cilium after an update Unless it's your core business I'd limit myself to deploying applications on Kubernetes instead of managing Kubernetes itself. At least for use cases where reliability matters. There are some reasonably priced managed Kubernetes offerings out there. For example syself (https://syself.com https://syself.com) built a managed Kubernetes on Hetzner. The prices are super reasonable. For 300 Euros per month you can enjoy Hetzner prices with an experience that beats AWS.

kazanz 1y ago
I'm going to second the Syself (https://syself.com https://syself.com) recommendation. We're running our entire PaaS platform (Sherpa.sh) on top of Hetzner with Syself. We are 5x more affordable than Vercel and a big piece of that is because Syself allows us to operate autoscaling k8s clusters without an army of ops engineers. We tried hetzner-k3s, and many other solutions. But ran into many of these same problems. Syself on the other hand, has solved all the sticky hardware and networking challenges at scale with their own customer operators. Plus they're support is super responsive and helpful. The one downside, is they don't have a dashboard yet, so you need to be comfortable in the CLI. But I'm sure they are working on it. I don't think I would manage kubernetes any other way at this point.

softwaredam 1y ago

To me this seems more than a side project. Really well done. Nowadays, Hetzner has many Kubernetes providers. One that I really like is Syself.com. They provide production-ready Kubernetes including self-healing, bare-metal support, OIDC integration and many more. As a cloud and migration expert, I have evaluated them and they turned out to perform very well.

softwaredam 1y ago
https://syself.com/ https://syself.com/ is what I am talking about.

lucasrattz 1y ago

How does it compare to established solutions like https://syself.com https://syself.com? Does it support bare metal servers too?

dev_ago 1y ago

[dead]

dev_ago 1y ago

[dead]

PaleHazy 1y ago

For those who like k3s and are familiar with it, this looks like a great natural initiative. I never really bought into k3s myself for whatever reason, even on RPis I don’t feel too constrained on resources. A similar approach I’ve been using is using Syself (running it for about half a year now with no issues). They contributed the Cluster API provider for Hetzner, which is obviously Chad. Of course, I wish CAPI were a bit less involved, but that’s exactly where external help makes sense. I am totally fully sold on CAPI, but then again always need your base cluster. I still want to set up CAPI on my own, but still preferred having “someone else” host the control plane. With CAPI, though, you have the flexibility to do either. Also Talos and Omni are cool and pretty chill on Hetzner. Fantastic tools. Still though I just like the "raw-ish" feel to kubeadm and CAPI feels the most "native" solution. In terms of addons, I have my very curated yaml manifestos. And ArgoCD Application sets. That part is hard for me to replace, but that should defenitely be married with a UI.