7 ms·

After years of trial and error our team has come to the same conclusion. I know some people might consider this insanity, but we actually run all of our scripts

After years of trial and error our team has come to the same conclusion. I know some people might consider this insanity, but we actually run all of our scripts as a separate C# CLI application (The main application is a C# web server). Effectively no bash scripts, except as the entry point here and there. The build step and passing the executable around is a small price to pay for the gain in static type checking, being able to pull in libraries as needed, and knowing that our CI is not going to down because someone made a dumb typo somewhere. The other thing I would add is consider passing in all environment variables as args. This makes it easy to see what dependencies the script actually needs, and has the bonus of being even more portable.

baq 1y ago
> I know some people might consider this insanity Some people here still can’t believe YAML is used for not only configuration, but complex code like optimized CI pipelines. This is insane. You’re actually introducing much needed sanity into the process by admitting that a real programming language is the tool to use here. I can’t imagine the cognitive dissonance Lisp folks have when dealing with this madness, not being one myself.
- mschuster91 1y ago
  > Some people here still can’t believe YAML is used for not only configuration, but complex code like optimized CI pipelines. This is insane. It's miles better than Jenkins and the horrors people created there. GitLab CI can at least be easily migrated to any other GitLab instance and stuff should Just Work because it is in the end not much more than self contained bash scripts, but Jenkins... is a clown show, especially for Ops people of larger instances. On one side, you got 50 plugins with CVEs but you can't update them because you need to find a slot that works for all development teams to have a week or two to fix their pipelines again, and on the other side you got a Jenkins instance for each project which lessens the coordination effort but you gotta worry about dozens of Jenkins instances. Oh and that doesn't include the fact many old pipelines aren't written in Groovy or, in fact, in any code at all but only in Jenkins's UI... Github Actions however, I'd say for someone coming from GitLab, is even worse to work with than Jenkins.
  - deleted 1y ago
    [deleted]
- TeMPOraL 1y ago
  > I can’t imagine the cognitive dissonance Lisp folks have when dealing with this madness, not being one myself. After a decade trying to fight it, this one Lisper here just gave up. It was the only way to stay sane. I remain hopeful that some day, maybe within our lifetimes, the rapid inflation phase of software industry will end, and we'll have time to rethink and redo the fundamentals properly. Until then, one can at least enjoy some shiny stuff, and stay away from the bleeding edge, aka. where sewage flows out of pipe and meets the sea. (It's gotten a little easier now, as you can have LLMs deal with YAML-programming and other modern worse-is-better "wisdom" for you.)
- motorest 1y ago
  > Some people here still can’t believe YAML is used for not only configuration, but complex code like optimized CI pipelines. I've been using YAML for ages and I never had any issue with it. What do you think is wrong with YAML?
  - mst 1y ago
    Turing complete YAML ends up being an app specific terrible programming language. Many of us would rather use a less terrible programming language instead.
    - motorest 1y ago
      [flagged]
    - hadlock 1y ago
      Something went horribly wrong if your coworkers are putting switching logic inside your config
  - hadlock 1y ago
    I've been using YAML professionally for a decade and other than forgetting to wrap some values in quotes, has been an absolute non issue. Some people talk about YAML being a turing complete language, if people try to do that in your CI/CD system just fire them I'll allow helm style templating but that's about it.
- no_wizard 1y ago
  I'm shocked there isn't a 'language for config' that hasn't become the de facto standard and its YAML all the way down seemingly. I am with you 100%. It would really benefit from a language that intrinsically understood its being used to control a state machine. As it is, that is what nearly all folks want in practice is a way to run different things based on different states of CI. A lisp DSL would be perfect for this. Macros would make things alot easier in many respects. Unfortunately, there's no industry consensus and none of the big CI platforms have adopted support for anything like that, they all use variants of YAML (I always wondered who started it with YAML and why everyone copied that, if anyone knows I'd love to read about it). Honestly, I can say the same complaints hold up against the cloud providers too. Those 'infrastructure as code' SDKs really don't lean into the 'as code' part very well
  - mdaniel 1y ago
    > (I always wondered who started it with YAML and why everyone copied that, if anyone knows I'd love to read about it). I know this isn't a definite answer to your question, but it was still super interesting to me and hopefully it will inspire someone else to dig into finding the actual answer The best guess I have as far as CI/CD specifically appears to be <https://en.wikipedia.org/wiki/Travis_CI#:~:text=travis%20ci%20was%20the%20first%20ci%20service%20that%20provided%20services%20to%20open-source%20projects%20for%20free%20 https://en.wikipedia.org/wiki/Travis_CI#:~:text=travis%20ci%...> which launched in 2011 offering free CI and I found a reference to their .travis.yml in GitLab's repo in 2011, too - CruiseControl (2004) was "ant as a service," so it was XML https://web.archive.org/web/20040812214609/http://confluence.public.thoughtworks.org/display/CC/Getting+Started+With+CruiseControl https://web.archive.org/web/20040812214609/http://confluence... - Hudson (2007) https://web.archive.org/web/20140701020639/https://www.java.net//blog/kohsuke/archive/20070514/Hudson%20J1.pdf https://web.archive.org/web/20140701020639/https://www.java.... was also XML, and was by that point driving Maven 2 builds (also XML) - I was shocked that GitHub existed in 2008 https://web.archive.org/web/20081230235955/http://github.com/ https://web.archive.org/web/20081230235955/http://github.com... with an especial nod to no longer a pain in the ass and Not only is Git the new hotness, it's a fast, efficient, distributed version control system ideal for the collaborative development of software but this was just "for funsies" link since they were very, very late to the CI/CD game - I was surprised but k8s 1.0.0 still had references to .json PodSpec files in 2010 https://github.com/kubernetes/kubernetes/blob/v1.0.0/examples/storm/zookeeper.json https://github.com/kubernetes/kubernetes/blob/v1.0.0/example... - cloud-init had yaml in 2010 https://github.com/openstack-archive/cloud-init/blob/0.7.0/doc/examples/cloud-config-gluster.txt https://github.com/openstack-archive/cloud-init/blob/0.7.0/d... so that's a plausible "it started here" since they were yaml declarations of steps to perform upon machine boot (and still, unquestionably, my favorite user-init thing) - just for giggles, GitLab 1.0.2 (2011) didn't even have CI/CD https://gitlab.com/gitlab-org/gitlab/-/tree/v1.0.2 https://gitlab.com/gitlab-org/gitlab/-/tree/v1.0.2 -- however, while digging into that I found .travis.yml in v2.0.0 (also 2011) so that's a very plausible citation <https://gitlab.com/gitlab-org/gitlab/-/blob/v2.0.0/.travis.yml https://gitlab.com/gitlab-org/gitlab/-/blob/v2.0.0/.travis.y...> - Ansible 1.0 in 2012 was also "execution in yaml" https://github.com/ansible/ansible/blob/v1.0/examples/playbooks/get_url.yml https://github.com/ansible/ansible/blob/v1.0/examples/playbo...
ozim 1y ago
I don’t think it is insanity quite the opposite - insanity is trying to force everything in yaml or pipeline. I have seen people doing absolutely insane setups because they thought they have to do it in yaml and pipeline and there is absolutely no other option or it is somehow wrong to drop some stuff to code.
- motorest 1y ago
  > I don’t think it is insanity quite the opposite - insanity is trying to force everything in yaml or pipeline. I'm not sure I understood what you're saying because it sounds too absurd to be real. The whole point of a CICD pipeline is that it automates all aspects of your CICD needs. All mainstream CICD systems support this as their happy path. You specify build stages and build jobs, you manage your build artifacts, you setup how things are tested, deployed and/or delivered. That's their happy path. And you're calling the most basic usecases of a standard class if tools as "insanity"? Please help me explain what point you are trying to make.
  - ozim 1y ago
    In the article Strange Way to Enforce Status Checks with Merge Queue. All aspects of your CICD pipeline - rebasing PRs is not 'basic CICD' need. CICD pipeline should take a commit state and produce artifacts from that state, not lint and not autofix trivial issues. Everything that is not "take code state - run tests - build - deploy (eventualy fail)" is insanity. Autofixing/linting for example should be separate process waay before CICD starts. And people do stuff like that because they think it is part of integration and testing. Trying to shove it inside is insanity.
robinwassen 1y ago
Did a similar thing when we needed to do complex operations towards aws. Instead of wrapping the aws cli command I wrote small Go applications using the boto3 library. Removed the headaches when passing in complex params, parsing output and and also made the logic portable as we need to do the builds on different platforms (Windows, Linux and macOS).
7bit 1y ago
> The other thing I would add is consider passing in all environment variables as args. This makes it easy to see what dependencies the script actually needs, and has the bonus of being even more portable. This is the dumbest thing I see installers do a lot lately.
noworriesnate 1y ago
I've used nuke.build for this in the past. This makes it nice for injecting environment variables into properties and for auto-generating CI YAML to wrap the main commands, but it is a bit of a pain when it comes to scaling the build. E.g. we did infrastructure as code using Pulumi, and that caused the build code to dramatically increase to the point the Nuke script became unwieldy. I wish we had gone the plain C# CLI app from the beginning.
mst 1y ago
Honestly, "using the same language as the application" is often a solid choice no matter what the application is written in. (and I suspect that for any given language somebody might propose as an exception to that rule, there's more than one team out there doing it anyway and finding it works better for them than everything else they've tried)

baq 1y ago

> I know some people might consider this insanity Some people here still can’t believe YAML is used for not only configuration, but complex code like optimized CI pipelines. This is insane. You’re actually introducing much needed sanity into the process by admitting that a real programming language is the tool to use here. I can’t imagine the cognitive dissonance Lisp folks have when dealing with this madness, not being one myself.

mschuster91 1y ago
> Some people here still can’t believe YAML is used for not only configuration, but complex code like optimized CI pipelines. This is insane. It's miles better than Jenkins and the horrors people created there. GitLab CI can at least be easily migrated to any other GitLab instance and stuff should Just Work because it is in the end not much more than self contained bash scripts, but Jenkins... is a clown show, especially for Ops people of larger instances. On one side, you got 50 plugins with CVEs but you can't update them because you need to find a slot that works for all development teams to have a week or two to fix their pipelines again, and on the other side you got a Jenkins instance for each project which lessens the coordination effort but you gotta worry about dozens of Jenkins instances. Oh and that doesn't include the fact many old pipelines aren't written in Groovy or, in fact, in any code at all but only in Jenkins's UI... Github Actions however, I'd say for someone coming from GitLab, is even worse to work with than Jenkins.
- deleted 1y ago
  [deleted]
TeMPOraL 1y ago
> I can’t imagine the cognitive dissonance Lisp folks have when dealing with this madness, not being one myself. After a decade trying to fight it, this one Lisper here just gave up. It was the only way to stay sane. I remain hopeful that some day, maybe within our lifetimes, the rapid inflation phase of software industry will end, and we'll have time to rethink and redo the fundamentals properly. Until then, one can at least enjoy some shiny stuff, and stay away from the bleeding edge, aka. where sewage flows out of pipe and meets the sea. (It's gotten a little easier now, as you can have LLMs deal with YAML-programming and other modern worse-is-better "wisdom" for you.)
motorest 1y ago
> Some people here still can’t believe YAML is used for not only configuration, but complex code like optimized CI pipelines. I've been using YAML for ages and I never had any issue with it. What do you think is wrong with YAML?
- mst 1y ago
  Turing complete YAML ends up being an app specific terrible programming language. Many of us would rather use a less terrible programming language instead.
  - motorest 1y ago
    [flagged]
  - hadlock 1y ago
    Something went horribly wrong if your coworkers are putting switching logic inside your config
- hadlock 1y ago
  I've been using YAML professionally for a decade and other than forgetting to wrap some values in quotes, has been an absolute non issue. Some people talk about YAML being a turing complete language, if people try to do that in your CI/CD system just fire them I'll allow helm style templating but that's about it.
no_wizard 1y ago
I'm shocked there isn't a 'language for config' that hasn't become the de facto standard and its YAML all the way down seemingly. I am with you 100%. It would really benefit from a language that intrinsically understood its being used to control a state machine. As it is, that is what nearly all folks want in practice is a way to run different things based on different states of CI. A lisp DSL would be perfect for this. Macros would make things alot easier in many respects. Unfortunately, there's no industry consensus and none of the big CI platforms have adopted support for anything like that, they all use variants of YAML (I always wondered who started it with YAML and why everyone copied that, if anyone knows I'd love to read about it). Honestly, I can say the same complaints hold up against the cloud providers too. Those 'infrastructure as code' SDKs really don't lean into the 'as code' part very well
- mdaniel 1y ago
  > (I always wondered who started it with YAML and why everyone copied that, if anyone knows I'd love to read about it). I know this isn't a definite answer to your question, but it was still super interesting to me and hopefully it will inspire someone else to dig into finding the actual answer The best guess I have as far as CI/CD specifically appears to be <https://en.wikipedia.org/wiki/Travis_CI#:~:text=travis%20ci%20was%20the%20first%20ci%20service%20that%20provided%20services%20to%20open-source%20projects%20for%20free%20 https://en.wikipedia.org/wiki/Travis_CI#:~:text=travis%20ci%...> which launched in 2011 offering free CI and I found a reference to their .travis.yml in GitLab's repo in 2011, too - CruiseControl (2004) was "ant as a service," so it was XML https://web.archive.org/web/20040812214609/http://confluence.public.thoughtworks.org/display/CC/Getting+Started+With+CruiseControl https://web.archive.org/web/20040812214609/http://confluence... - Hudson (2007) https://web.archive.org/web/20140701020639/https://www.java.net//blog/kohsuke/archive/20070514/Hudson%20J1.pdf https://web.archive.org/web/20140701020639/https://www.java.... was also XML, and was by that point driving Maven 2 builds (also XML) - I was shocked that GitHub existed in 2008 https://web.archive.org/web/20081230235955/http://github.com/ https://web.archive.org/web/20081230235955/http://github.com... with an especial nod to no longer a pain in the ass and Not only is Git the new hotness, it's a fast, efficient, distributed version control system ideal for the collaborative development of software but this was just "for funsies" link since they were very, very late to the CI/CD game - I was surprised but k8s 1.0.0 still had references to .json PodSpec files in 2010 https://github.com/kubernetes/kubernetes/blob/v1.0.0/examples/storm/zookeeper.json https://github.com/kubernetes/kubernetes/blob/v1.0.0/example... - cloud-init had yaml in 2010 https://github.com/openstack-archive/cloud-init/blob/0.7.0/doc/examples/cloud-config-gluster.txt https://github.com/openstack-archive/cloud-init/blob/0.7.0/d... so that's a plausible "it started here" since they were yaml declarations of steps to perform upon machine boot (and still, unquestionably, my favorite user-init thing) - just for giggles, GitLab 1.0.2 (2011) didn't even have CI/CD https://gitlab.com/gitlab-org/gitlab/-/tree/v1.0.2 https://gitlab.com/gitlab-org/gitlab/-/tree/v1.0.2 -- however, while digging into that I found .travis.yml in v2.0.0 (also 2011) so that's a very plausible citation <https://gitlab.com/gitlab-org/gitlab/-/blob/v2.0.0/.travis.yml https://gitlab.com/gitlab-org/gitlab/-/blob/v2.0.0/.travis.y...> - Ansible 1.0 in 2012 was also "execution in yaml" https://github.com/ansible/ansible/blob/v1.0/examples/playbooks/get_url.yml https://github.com/ansible/ansible/blob/v1.0/examples/playbo...

ozim 1y ago

I don’t think it is insanity quite the opposite - insanity is trying to force everything in yaml or pipeline. I have seen people doing absolutely insane setups because they thought they have to do it in yaml and pipeline and there is absolutely no other option or it is somehow wrong to drop some stuff to code.

motorest 1y ago
> I don’t think it is insanity quite the opposite - insanity is trying to force everything in yaml or pipeline. I'm not sure I understood what you're saying because it sounds too absurd to be real. The whole point of a CICD pipeline is that it automates all aspects of your CICD needs. All mainstream CICD systems support this as their happy path. You specify build stages and build jobs, you manage your build artifacts, you setup how things are tested, deployed and/or delivered. That's their happy path. And you're calling the most basic usecases of a standard class if tools as "insanity"? Please help me explain what point you are trying to make.
- ozim 1y ago
  In the article Strange Way to Enforce Status Checks with Merge Queue. All aspects of your CICD pipeline - rebasing PRs is not 'basic CICD' need. CICD pipeline should take a commit state and produce artifacts from that state, not lint and not autofix trivial issues. Everything that is not "take code state - run tests - build - deploy (eventualy fail)" is insanity. Autofixing/linting for example should be separate process waay before CICD starts. And people do stuff like that because they think it is part of integration and testing. Trying to shove it inside is insanity.

robinwassen 1y ago

Did a similar thing when we needed to do complex operations towards aws. Instead of wrapping the aws cli command I wrote small Go applications using the boto3 library. Removed the headaches when passing in complex params, parsing output and and also made the logic portable as we need to do the builds on different platforms (Windows, Linux and macOS).

7bit 1y ago

> The other thing I would add is consider passing in all environment variables as args. This makes it easy to see what dependencies the script actually needs, and has the bonus of being even more portable. This is the dumbest thing I see installers do a lot lately.

noworriesnate 1y ago

I've used nuke.build for this in the past. This makes it nice for injecting environment variables into properties and for auto-generating CI YAML to wrap the main commands, but it is a bit of a pain when it comes to scaling the build. E.g. we did infrastructure as code using Pulumi, and that caused the build code to dramatically increase to the point the Nuke script became unwieldy. I wish we had gone the plain C# CLI app from the beginning.

mst 1y ago

Honestly, "using the same language as the application" is often a solid choice no matter what the application is written in. (and I suspect that for any given language somebody might propose as an exception to that rule, there's more than one team out there doing it anyway and finding it works better for them than everything else they've tried)