6 ms·

We use Nomad from Hashicorp, it's super simple. Never liked the complexity K8s brings along.

by lkurusa 4y ago

dimitrios1 4y ago
I never liked the cost Hashicorp products bring along.
- OJFord 4y ago
  What, zero? (GP didn't say anything about using Hashicorp-managed products, they're open source and beer-free to use. Another comment says Hashicorp's platform doesn't even offer Nomad (yet?) anyway.)
  - pentium166 4y ago
    If you need features offered by the self-managed Enterprise version of a Hashicorp product, I've heard the price tag is something like low six figures per product.
  - dimitrios1 4y ago
    Using terraform without TFE is something I would never recommend to any large org. Been there, done that.
  - wlonkly 4y ago
    Hashicorp is very inflexible about support plans -- either you go all in on their Enterprise product, or you're self supported. By the time you've licensed Nomad, Consul and Vault -- because they interact and you will find Nomad support ends where Consul support begins, and so on -- it is a LOT of money.
- lkurusa 4y ago
  Hmm. Please could you explain further? I'm genuinely curious what costs you associate with Hashicorp products.
  - 3np 4y ago
    It depends on your size. For a fairly minimal close-to-best-practices you'll need for each DC, each on a separate physical host (I may be missing something): 3 x Consul server 3 x Nomad server 2/3 x Vault server It's long since I operated k8s but IIRC I think you can get similar capabilities and redundancy with 3-5 machines? That's before you start looking at actual runner nodes, load balancers, proxies, logging and monitoring infra, etc... Unless you cheat (which I think many do) or you're big enough, that overhead can be meaningful.
    - schmichael 4y ago
      (Disclosure: Nomad team lead) FWIW we recognized this was too much overhead for many users. Nomad 1.3 supports service discovery so you can start without Consul, and 1.4 will support secure variables to get folks farther along without requiring Vault. So 3 Nomad servers should give you a pretty featureful and highly available cluster these days.
      - orthecreedence 4y ago
        Yeah or, like, spin up three medium servers in different zones and have each server run all three services. We did that for a production setup for years and it worked fantastically. There's no need to have nomad/consul/vault all on different servers unless they are significantly underpowered or the workloads are crazy. If best practices say otherwise, then maybe they should be reconsidered.
      - 3np 4y ago
        Sure, but at this point there's so much else we get from Consul that, like, what's the point... I guess the path is set but I'd personally much prefer having a recognized deployment scenario be hosting Consul server and Nomad server on the same physical machines, and accommodating (be it through code or just docs) for making that play well with security, certs, and resource usage without becoming a confounding mess. Even Vault, if the operator accepts and/or mitigates the sidechannel aspects - from a security perspective that still shouldn't be a step down from anything Nomad-specific? Seeing as HC already provides solutions for all of these supposed to be serving for Nomad, doesn't it make more sense to make them play together smoother and nice on the same machine rather than reinventing a lesser wheel for each of them?
sph 4y ago
OT: I really do not like Hashicorp. Terraform has a terrible DSL, and terrible documentation. Also, I paid $70 for their VMWare Vagrant plugin ages ago, it was so buggy to be unusable, they were unresponsive on the Github repo, and they completely ignored my request for a refund under their own 30-day guarantee. Not very professional. I really don't get why people love that company so much.

We use Nomad from Hashicorp, it's super simple. Never liked the complexity K8s brings along.

dimitrios1 4y ago

I never liked the cost Hashicorp products bring along.

OJFord 4y ago
What, zero? (GP didn't say anything about using Hashicorp-managed products, they're open source and beer-free to use. Another comment says Hashicorp's platform doesn't even offer Nomad (yet?) anyway.)
- pentium166 4y ago
  If you need features offered by the self-managed Enterprise version of a Hashicorp product, I've heard the price tag is something like low six figures per product.
- dimitrios1 4y ago
  Using terraform without TFE is something I would never recommend to any large org. Been there, done that.
- wlonkly 4y ago
  Hashicorp is very inflexible about support plans -- either you go all in on their Enterprise product, or you're self supported. By the time you've licensed Nomad, Consul and Vault -- because they interact and you will find Nomad support ends where Consul support begins, and so on -- it is a LOT of money.
lkurusa 4y ago
Hmm. Please could you explain further? I'm genuinely curious what costs you associate with Hashicorp products.
- 3np 4y ago
  It depends on your size. For a fairly minimal close-to-best-practices you'll need for each DC, each on a separate physical host (I may be missing something): 3 x Consul server 3 x Nomad server 2/3 x Vault server It's long since I operated k8s but IIRC I think you can get similar capabilities and redundancy with 3-5 machines? That's before you start looking at actual runner nodes, load balancers, proxies, logging and monitoring infra, etc... Unless you cheat (which I think many do) or you're big enough, that overhead can be meaningful.
  - schmichael 4y ago
    (Disclosure: Nomad team lead) FWIW we recognized this was too much overhead for many users. Nomad 1.3 supports service discovery so you can start without Consul, and 1.4 will support secure variables to get folks farther along without requiring Vault. So 3 Nomad servers should give you a pretty featureful and highly available cluster these days.
    - orthecreedence 4y ago
      Yeah or, like, spin up three medium servers in different zones and have each server run all three services. We did that for a production setup for years and it worked fantastically. There's no need to have nomad/consul/vault all on different servers unless they are significantly underpowered or the workloads are crazy. If best practices say otherwise, then maybe they should be reconsidered.
    - 3np 4y ago
      Sure, but at this point there's so much else we get from Consul that, like, what's the point... I guess the path is set but I'd personally much prefer having a recognized deployment scenario be hosting Consul server and Nomad server on the same physical machines, and accommodating (be it through code or just docs) for making that play well with security, certs, and resource usage without becoming a confounding mess. Even Vault, if the operator accepts and/or mitigates the sidechannel aspects - from a security perspective that still shouldn't be a step down from anything Nomad-specific? Seeing as HC already provides solutions for all of these supposed to be serving for Nomad, doesn't it make more sense to make them play together smoother and nice on the same machine rather than reinventing a lesser wheel for each of them?

sph 4y ago

OT: I really do not like Hashicorp. Terraform has a terrible DSL, and terrible documentation. Also, I paid $70 for their VMWare Vagrant plugin ages ago, it was so buggy to be unusable, they were unresponsive on the Github repo, and they completely ignored my request for a refund under their own 30-day guarantee. Not very professional. I really don't get why people love that company so much.