6 ms·

Thank god trends like Pulumi and the new AWS sdk is emerging. General purpose programing languages are getting more expressive by the day, why do we use data s

Thank god trends like Pulumi and the new AWS sdk is emerging. General purpose programing languages are getting more expressive by the day, why do we use data serialization languages instead for configs? it doesn't make any sense. Configuration is code not data.

reallydontask 7y ago
The main problem is that, in my experience, the majority of DevOps teams are Ops teams that have been renamed and refocused towards automation. These are people that by and large don't want to code, not saying that they can't or won't. To be fair this has been in Windows shops, where scripting has only recently (last 5-10 years) taken off, so you've got a lot of windows admins that the closest they've been to code is Batch scripting with a bit of Powershell. This is a big change for them As it happens i read about pulumi recently and I've put it on mt list of todo things, but I can't see that I'll be able to sell it to our team and our team is blessed (cursed?) with three former developers
- dragonwriter 7y ago
  > The main problem is that, in my experience, the majority of DevOps teams are Ops teams that have been renamed and refocused towards automation. I'm on a relatively new team kind of like that (it's not a former Ops team but most of the team was pulled from Ops/DBA/analysis teams, and fits the description, and it's not a really a DevOps but more of a Dev+Ops team) and the general reaction of the team to being introduceed to the AWS CDK has been “we need to move to that as soon as we can”, even from people who very vocally never wanted to be programmers. And that's with, in many cases, a couple months experience with both programming and YAML IAC in the form of CloudFormation.
- toyg 7y ago
  > These are people that by and large don't want to code I disagree. That's the stereotype that tool builders have of such people. Good ops people have always loved coding, or we wouldn't be living on the mountains of Bash scripts also known as "Linux distributions". (Besides, people who don't like to code won't like writing tons of declarative markup either. So there is little point in the current approach either way.)
  - reallydontask 7y ago
    It might be a stereotype but it's also my experience, which granted is limited and Windows based, which as I pointed out in the previous comment hasn't been really onboard with the scripting experience until relatively recently As to the declarative markup, for instance Azure Devops still doesn't have feature parity between YAML pipelines and classic pipelines, so while you could well be right about the same resistance to yaml, it's not, necessarily an issue yet.
    - toyg 7y ago
      Most devops tools are built on Linux for Linux, then ported to Windows later as an afterthought, so I don’t think platforms are much of a factor. It’s a defect in the production pipeline somewhere, something like feedback from potential users not reaching developers until it’s too late. It doesn’t help that, in some cases, vendors just impose what is going to happen, and the community is simply forced to put up with it.
q3k 7y ago
I still have my reservations against Pulumi in particular, though. I've only really skimmed their documentation, but the idea that calling 'new' against a VM class instantiates a VM in production seems to be to magical for my taste, and might be an indication that it's yet another product that targets the happy path more than what we tend to experience in production: failures.
solatic 7y ago
> General purpose programing languages are getting more expressive by the day You know, once upon a time, we understood that declarative approaches to software engineering were superior to imperative approaches, when declarative approaches are feasible. Declarative approaches are much safer and easier to test, at a cost of only being able to express what the tool accepting the declarative approach can understand. Imperative approaches are strictly worse for any problem set where a declarative approach solves the problem within performance requirements. The additional expressiveness of languages like Pulumi is the last thing I want. YAML is a horrible language for declarative system configuration because a) any sufficiently complex system will require you to generate your declarative codebase in the name of maintainability, b) generating code for any language where whitespace is significant will lead you to an early death, and c) stringly-typed languages are fundamentally unmaintainable at sufficient scale. But this is not an indictment of a declarative approach! It is an indictment of YAML. > Configuration is code not data. Data > code. Data does not need to be debugged. The best code you can have is deleted code - deleted code does not need to be maintained, updated, or patched. Code is a necessary evil we write in order to build operable systems, not a virtue in and of itself.
- UserIsUnused 7y ago
  There are declarative general purpose programing languages. That data you are talking about does need to be debugged, like Helm charts and pipeline definitions. Sure data is better, but config is code, not data.
  - solatic 7y ago
    Generators need to be debugged, not data. It's very easy to test a generator - a few unit tests checking whether, for a given input, the generator produced the expected output, and you're set. Data sometimes needs to be cleaned, but there's no such thing as a bug in data. Whether the generated declarative output produces the expected behavior on the part of the tool interpreting the declarative output is part of the tool's contract, not the generator or the declarative output. If you need to check the tool's behavior then either a) you wrote the tool or b) you're writing an acceptance test for the tool, which is an entirely different endeavor.
falcolas 7y ago
> Thank god trends like Pulumi and the new AWS sdk is emerging. The more things change, the more they stay the same. The pendulum is swinging back towards scripting languages. But give it a few years, and we'll be railing against scripting languages for not being idempotent enough, and moving back to the latest version of YAML and XML (perhaps we'll go with TOML this time) with custom interpreters. We have seen this half a dozen times already, between turing complete DSLs in ruby, to plain bash, perl, and python scripts. The other side are the DSLs written using YAML, JSON, and XML (and every other config language written in history).
sgslo 7y ago
> new AWS SDK Could you share some details on that? A quick Google search didn't reveal anything about major changes to the SDK.
- dragonwriter 7y ago
  Probably referring to the CDK (Cloud Development Kit), a newish SDK for developing Cloudformation infrastructure-as-code in general-purpose programming languages. https://aws.amazon.com/cdk/ https://aws.amazon.com/cdk/
- JonathanHedley 7y ago
  I believe the OP is referring to the AWS CDK (Cloud Development Kit) - https://docs.aws.amazon.com/cdk/latest/guide/home.html https://docs.aws.amazon.com/cdk/latest/guide/home.html
jimbokun 7y ago
> Configuration is code not data. As any Lisper knows, code is data is code.

reallydontask 7y ago

The main problem is that, in my experience, the majority of DevOps teams are Ops teams that have been renamed and refocused towards automation. These are people that by and large don't want to code, not saying that they can't or won't. To be fair this has been in Windows shops, where scripting has only recently (last 5-10 years) taken off, so you've got a lot of windows admins that the closest they've been to code is Batch scripting with a bit of Powershell. This is a big change for them As it happens i read about pulumi recently and I've put it on mt list of todo things, but I can't see that I'll be able to sell it to our team and our team is blessed (cursed?) with three former developers

dragonwriter 7y ago
> The main problem is that, in my experience, the majority of DevOps teams are Ops teams that have been renamed and refocused towards automation. I'm on a relatively new team kind of like that (it's not a former Ops team but most of the team was pulled from Ops/DBA/analysis teams, and fits the description, and it's not a really a DevOps but more of a Dev+Ops team) and the general reaction of the team to being introduceed to the AWS CDK has been “we need to move to that as soon as we can”, even from people who very vocally never wanted to be programmers. And that's with, in many cases, a couple months experience with both programming and YAML IAC in the form of CloudFormation.
toyg 7y ago
> These are people that by and large don't want to code I disagree. That's the stereotype that tool builders have of such people. Good ops people have always loved coding, or we wouldn't be living on the mountains of Bash scripts also known as "Linux distributions". (Besides, people who don't like to code won't like writing tons of declarative markup either. So there is little point in the current approach either way.)
- reallydontask 7y ago
  It might be a stereotype but it's also my experience, which granted is limited and Windows based, which as I pointed out in the previous comment hasn't been really onboard with the scripting experience until relatively recently As to the declarative markup, for instance Azure Devops still doesn't have feature parity between YAML pipelines and classic pipelines, so while you could well be right about the same resistance to yaml, it's not, necessarily an issue yet.
  - toyg 7y ago
    Most devops tools are built on Linux for Linux, then ported to Windows later as an afterthought, so I don’t think platforms are much of a factor. It’s a defect in the production pipeline somewhere, something like feedback from potential users not reaching developers until it’s too late. It doesn’t help that, in some cases, vendors just impose what is going to happen, and the community is simply forced to put up with it.

q3k 7y ago

I still have my reservations against Pulumi in particular, though. I've only really skimmed their documentation, but the idea that calling 'new' against a VM class instantiates a VM in production seems to be to magical for my taste, and might be an indication that it's yet another product that targets the happy path more than what we tend to experience in production: failures.

solatic 7y ago

> General purpose programing languages are getting more expressive by the day You know, once upon a time, we understood that declarative approaches to software engineering were superior to imperative approaches, when declarative approaches are feasible. Declarative approaches are much safer and easier to test, at a cost of only being able to express what the tool accepting the declarative approach can understand. Imperative approaches are strictly worse for any problem set where a declarative approach solves the problem within performance requirements. The additional expressiveness of languages like Pulumi is the last thing I want. YAML is a horrible language for declarative system configuration because a) any sufficiently complex system will require you to generate your declarative codebase in the name of maintainability, b) generating code for any language where whitespace is significant will lead you to an early death, and c) stringly-typed languages are fundamentally unmaintainable at sufficient scale. But this is not an indictment of a declarative approach! It is an indictment of YAML. > Configuration is code not data. Data > code. Data does not need to be debugged. The best code you can have is deleted code - deleted code does not need to be maintained, updated, or patched. Code is a necessary evil we write in order to build operable systems, not a virtue in and of itself.

UserIsUnused 7y ago
There are declarative general purpose programing languages. That data you are talking about does need to be debugged, like Helm charts and pipeline definitions. Sure data is better, but config is code, not data.
- solatic 7y ago
  Generators need to be debugged, not data. It's very easy to test a generator - a few unit tests checking whether, for a given input, the generator produced the expected output, and you're set. Data sometimes needs to be cleaned, but there's no such thing as a bug in data. Whether the generated declarative output produces the expected behavior on the part of the tool interpreting the declarative output is part of the tool's contract, not the generator or the declarative output. If you need to check the tool's behavior then either a) you wrote the tool or b) you're writing an acceptance test for the tool, which is an entirely different endeavor.

falcolas 7y ago

> Thank god trends like Pulumi and the new AWS sdk is emerging. The more things change, the more they stay the same. The pendulum is swinging back towards scripting languages. But give it a few years, and we'll be railing against scripting languages for not being idempotent enough, and moving back to the latest version of YAML and XML (perhaps we'll go with TOML this time) with custom interpreters. We have seen this half a dozen times already, between turing complete DSLs in ruby, to plain bash, perl, and python scripts. The other side are the DSLs written using YAML, JSON, and XML (and every other config language written in history).

sgslo 7y ago

> new AWS SDK Could you share some details on that? A quick Google search didn't reveal anything about major changes to the SDK.

dragonwriter 7y ago
Probably referring to the CDK (Cloud Development Kit), a newish SDK for developing Cloudformation infrastructure-as-code in general-purpose programming languages. https://aws.amazon.com/cdk/ https://aws.amazon.com/cdk/
JonathanHedley 7y ago
I believe the OP is referring to the AWS CDK (Cloud Development Kit) - https://docs.aws.amazon.com/cdk/latest/guide/home.html https://docs.aws.amazon.com/cdk/latest/guide/home.html

jimbokun 7y ago

> Configuration is code not data. As any Lisper knows, code is data is code.