11 ms·

There's so much more to CI/CD than the build definitions (e.g. dashboarding, log viewing/retention, access control, manual interventions, secret management, tes

by diggs 4y ago

There's so much more to CI/CD than the build definitions (e.g. dashboarding, log viewing/retention, access control, manual interventions, secret management, test reporting, etc.) and while some of your points resonate very strongly with me (e.g. local builds), I can't help but wonder what the endgame is here? You've raised $27m to what? Give away a free tool for engineers to define their builds in? While they continue to use and pay for another service to actually run the builds? I assume you intend to replace the CI service at some point and move up the stack to monetize? Without more transparency its easy to imagine something like... Step 1. Drive uptake of your tool by selling people on the pitfalls of "CI lock-in" Step 2. Introduce your own CI solution, which people can now easily switch to Step 3. Lock people in

shykes 4y ago
Good question. No, we don't intend to replace the CI service. We think of CI as infrastructure, and there are plenty of great infrastructure providers out there. And you are absolutely correct that for Dagger to succeed, it has to remain agnostic to infrastructure providers, which means we cannot build an infrastructure provider business ourselves. If the experience of hosting Dagger becomes so bad that it affects the developer experience, we might ship such a feature as a stopgap and even charge for it - but it is not a strategic priority. The sooner other infrastructure providers offer a fantastic hosting experience, the better. That is not where we see the biggest opportunity for Dagger. However, there is a great opportunity to help businesses manage their software supply chain. What is running where, and how did it get there? Who is authorized to run which pipelines on which machines, and which which inputs? What is the diff between this dev environment and staging? Staging and production? Etc. Keep in mind this is not limited to your production pipeline; there are staging environments, development environments, all running pipelines all day long. It's hard to simply know everything that is going on. Each time Dagger runs your pipeline, it can produce telemetry about every aspect of your supply chain. Git SHAs, docker image checksums and cryptographic signatures, specific versions of kubernetes and cloudformation configurations, before and after templating, test results, monitoring status... It also integrates with your secrets management backends, your developer's local source code. Basically every node in your supply chain can be a node in your DAG if you want it to. The logical next step is to give you a centralized place to send all this telemetry, and tools to extract insights from it. You could also perhaps manage the configuration of your various Dagger engines in one location. Another product that is often requested is a visual DAG debugger. When a pipeline break, you want to know why, and staring at your CI logs is definitely not the best experience for that. With a web UI, there's a lot we can do there. The business opportunity boils down to this: if CI/CD pipelines are software, there ought to be a platform for developing that software, and an ecosystem of developers creating value around that platform. Dagger aspires to create that missing developer ecosystem. If we succeed, there will be no shortage of business opportunities. If we fail, none of the other features will matter.
- stingraycharles 4y ago
  > Each time Dagger runs your pipeline, it can produce telemetry about every aspect of your supply chain. To me, that still doesn’t seem to go to the core of what value it brings to table. I understand that Dagger can do all this, and that businesses would like to know what runs where and how everything interacts, but… it doesn’t explain to me the CI / build pipeline angle? How does knowing the telemetry around my build pipelines translate to better software, or cost savings, or other improvements? If there’s a registry to exchange “components” of a build pipeline, what does that bring me what a regular python / java / etc package can’t? Don’t get me wrong, I think there are plenty of problems with CI (I have a ~200 job CI pipeline on fire in front of me), I just can’t seem to connect the dots here. :)
- laurentb 4y ago
  Hey Solomon, question coming from a sales guy among this crowd... Who does this impact the most and what outcomes does that help them achieve? At the end of the day, you need to pay the bills, for you and your team, who is going to sign the dotted line and tell you that yes "I need to invest $100k on this because it solves a major pain!"? Whilst I can see this as a nice to have, I'm having a hard time understanding who your target market is going to be... At the end of the day, I presume this is going to end up becoming a full fledged company, with its sales team, marketing and what not. What are you thinking about in terms of revenue channels so far ?
  - abrahms 4y ago
    As someone who does a bunch of CI/CD work, the answer is: supply chain and the security therein is a major focus of enterprises right now. Security folks as well as ops folks are keen on this. Big question is can those two groups convince the developers that this is a good idea (though vice versa should be fairly easy at a certain company size).
    - laurentb 4y ago
      That's fine, I don't deny that, companies like circleCI and co are doing really well, but Docker is also critical to a lot of enterprise as well and yet the business model was not as strong as initially thought. I'm curious to understand what the pricing is going to be and whether it's going to be well recieved or not (and right now, there is nowhere on dagger.io's website where pricing is mentioned)
taude 4y ago
I didn't spend a lot of time with the site, but I interpreted Dagger as the Terraform of CICD definitions....
- shykes 4y ago
  Yes, that's a reasonable approximation.
awinter-py 4y ago
IMO the best open source infra lock-in strategy is kube took a while for E/AKS to catch up with G, things like ingress still vary in DX across clouds 'the same everywhere but it only works here' (no comment on morality of this, quality of kube generally, or whether this team will do the same. docker IMO missed the chance to be a cloud host or a standard interface)
- shitloadofbooks 4y ago
  Can you elaborate on your concerns? I've not seen them occur in practice at all. Sure each provider does things differently, but they still work with primitive Kubernetes manifests at the end of the day. A migration from one to the other is nothing more than changing some annotations or potentially transforming the "shape" of some lists or maps.
  - awinter-py 4y ago
    concern: like every system that we think of as cloud software, the 'cloud platform integration' half of it never gets open sourced same as like, aws extending mysql and postgres to provide RDS features -- they don't open source those pieces, so 'stock oss' DBs don't have fancy RDS features like backup + restore with kube, platform-integrated components like network, ingress + storage 1) require custom work on each cloud, so new features won't be available on all clouds at the same time. But also 2) that means some components, ingress specifically, have different interfaces on different clouds when they do finally get implemented.
- koffiezet 4y ago
  Completely off-topic, but k8s actually removes a lot of the vendor lock-in. Imagine having to migrate a k8s based app from AWS to Azure vs a pure AWS-based setup, provisioned with cloudformation. Sure it'll involve some work, but I know what I'd pick. I've personally ran workloads originally intended to be deployed on GKE on on-prem openshift clusters with very few changes, other than indeed the mentioned ingress stuff. That was also the initial goals of k8s - make it universal so it's adopted as a standard by everyone else to make yourself competitive against the AWS juggernaut. And it worked.
maccard 4y ago
> You've raised $27m to what? Give away a free tool for engineers to define their builds in? While they continue to use and pay for another service to actually run the builds? I assume you intend to replace the CI service at some point and move up the stack to monetize? Assuming that dagger does take over the world, this has the docker story of "this is a widely used tool that a bunch of people are using and we can't monetise without resorting to lock-in", which is a huge bummer.

by diggs 4y ago

shykes 4y ago

Good question. No, we don't intend to replace the CI service. We think of CI as infrastructure, and there are plenty of great infrastructure providers out there. And you are absolutely correct that for Dagger to succeed, it has to remain agnostic to infrastructure providers, which means we cannot build an infrastructure provider business ourselves. If the experience of hosting Dagger becomes so bad that it affects the developer experience, we might ship such a feature as a stopgap and even charge for it - but it is not a strategic priority. The sooner other infrastructure providers offer a fantastic hosting experience, the better. That is not where we see the biggest opportunity for Dagger. However, there is a great opportunity to help businesses manage their software supply chain. What is running where, and how did it get there? Who is authorized to run which pipelines on which machines, and which which inputs? What is the diff between this dev environment and staging? Staging and production? Etc. Keep in mind this is not limited to your production pipeline; there are staging environments, development environments, all running pipelines all day long. It's hard to simply know everything that is going on. Each time Dagger runs your pipeline, it can produce telemetry about every aspect of your supply chain. Git SHAs, docker image checksums and cryptographic signatures, specific versions of kubernetes and cloudformation configurations, before and after templating, test results, monitoring status... It also integrates with your secrets management backends, your developer's local source code. Basically every node in your supply chain can be a node in your DAG if you want it to. The logical next step is to give you a centralized place to send all this telemetry, and tools to extract insights from it. You could also perhaps manage the configuration of your various Dagger engines in one location. Another product that is often requested is a visual DAG debugger. When a pipeline break, you want to know why, and staring at your CI logs is definitely not the best experience for that. With a web UI, there's a lot we can do there. The business opportunity boils down to this: if CI/CD pipelines are software, there ought to be a platform for developing that software, and an ecosystem of developers creating value around that platform. Dagger aspires to create that missing developer ecosystem. If we succeed, there will be no shortage of business opportunities. If we fail, none of the other features will matter.

stingraycharles 4y ago
> Each time Dagger runs your pipeline, it can produce telemetry about every aspect of your supply chain. To me, that still doesn’t seem to go to the core of what value it brings to table. I understand that Dagger can do all this, and that businesses would like to know what runs where and how everything interacts, but… it doesn’t explain to me the CI / build pipeline angle? How does knowing the telemetry around my build pipelines translate to better software, or cost savings, or other improvements? If there’s a registry to exchange “components” of a build pipeline, what does that bring me what a regular python / java / etc package can’t? Don’t get me wrong, I think there are plenty of problems with CI (I have a ~200 job CI pipeline on fire in front of me), I just can’t seem to connect the dots here. :)
laurentb 4y ago
Hey Solomon, question coming from a sales guy among this crowd... Who does this impact the most and what outcomes does that help them achieve? At the end of the day, you need to pay the bills, for you and your team, who is going to sign the dotted line and tell you that yes "I need to invest $100k on this because it solves a major pain!"? Whilst I can see this as a nice to have, I'm having a hard time understanding who your target market is going to be... At the end of the day, I presume this is going to end up becoming a full fledged company, with its sales team, marketing and what not. What are you thinking about in terms of revenue channels so far ?
- abrahms 4y ago
  As someone who does a bunch of CI/CD work, the answer is: supply chain and the security therein is a major focus of enterprises right now. Security folks as well as ops folks are keen on this. Big question is can those two groups convince the developers that this is a good idea (though vice versa should be fairly easy at a certain company size).
  - laurentb 4y ago
    That's fine, I don't deny that, companies like circleCI and co are doing really well, but Docker is also critical to a lot of enterprise as well and yet the business model was not as strong as initially thought. I'm curious to understand what the pricing is going to be and whether it's going to be well recieved or not (and right now, there is nowhere on dagger.io's website where pricing is mentioned)

taude 4y ago

I didn't spend a lot of time with the site, but I interpreted Dagger as the Terraform of CICD definitions....

shykes 4y ago
Yes, that's a reasonable approximation.

awinter-py 4y ago

IMO the best open source infra lock-in strategy is kube took a while for E/AKS to catch up with G, things like ingress still vary in DX across clouds 'the same everywhere but it only works here' (no comment on morality of this, quality of kube generally, or whether this team will do the same. docker IMO missed the chance to be a cloud host or a standard interface)

shitloadofbooks 4y ago
Can you elaborate on your concerns? I've not seen them occur in practice at all. Sure each provider does things differently, but they still work with primitive Kubernetes manifests at the end of the day. A migration from one to the other is nothing more than changing some annotations or potentially transforming the "shape" of some lists or maps.
- awinter-py 4y ago
  concern: like every system that we think of as cloud software, the 'cloud platform integration' half of it never gets open sourced same as like, aws extending mysql and postgres to provide RDS features -- they don't open source those pieces, so 'stock oss' DBs don't have fancy RDS features like backup + restore with kube, platform-integrated components like network, ingress + storage 1) require custom work on each cloud, so new features won't be available on all clouds at the same time. But also 2) that means some components, ingress specifically, have different interfaces on different clouds when they do finally get implemented.
koffiezet 4y ago
Completely off-topic, but k8s actually removes a lot of the vendor lock-in. Imagine having to migrate a k8s based app from AWS to Azure vs a pure AWS-based setup, provisioned with cloudformation. Sure it'll involve some work, but I know what I'd pick. I've personally ran workloads originally intended to be deployed on GKE on on-prem openshift clusters with very few changes, other than indeed the mentioned ingress stuff. That was also the initial goals of k8s - make it universal so it's adopted as a standard by everyone else to make yourself competitive against the AWS juggernaut. And it worked.

maccard 4y ago

> You've raised $27m to what? Give away a free tool for engineers to define their builds in? While they continue to use and pay for another service to actually run the builds? I assume you intend to replace the CI service at some point and move up the stack to monetize? Assuming that dagger does take over the world, this has the docker story of "this is a widely used tool that a bunch of people are using and we can't monetise without resorting to lock-in", which is a huge bummer.