9 ms·

Docker and StriderCD: Brilliant Continuous Integration In A Box

54 points by niall_ohiggins 13y ago

nickstinemates 13y ago
I just set this up accessible over the internet, building my blog. I've since removed the service/webhooks, but feel free to login at http://stinemat.es:3000 http://stinemat.es:3000 to play around. It's extremely cool.
- asabjorn 13y ago
  Do you have a demo account or an invite code we can use to register?
  - nickstinemates 13y ago
    The demo account is available on the blog post :)
- niall_ohiggins 13y ago
  Glad you like it! If you have any thoughts we'd love to hear them over at https://github.com/Strider-CD/strider https://github.com/Strider-CD/strider
StavrosK 13y ago
Docker sounds brilliant, but I haven't had much luck with deploying my (moderately complex) Django apps on it. For example, I want the postgres db data to live outside the container, but volumes can't currently do that with Dockerfiles: they're initialized at "docker build" and then the files are just overwritten when trying to mount an external volume. Another use case is trying to spin up a development container (where the code resides on the host), but, again, shared volumes overwrite the directory. You can ADD your code initially and then mount the volume when you run it, but it's added hassle.
- agumonkey 13y ago
  Same here, I'm still not clear on how to modularize things (even thought the aufs layers seems module friendly)
- shykes 13y ago
  Hey StavrosK, This is by design, "I want the postgres db data to live outside the container" is a deployment-specific decision, and should not be hardcoded into the container itself. You (or someone you distribute this container to) might not want the db data to live outside the container, or might want it to live at a specific path on his machine. That's why a Dockerfile lets you specify that /var/lib/postgres is persistent, but doesn't let you specify how to persist it. Now, if I remember our IRC discussion correctly, in your case you're hitting a limitation of docker's implementation of volumes, which is that the container cannot specify an initial state for its volume - it can only start empty. That is a limitation that we're going to fix. But we need to make sure we fix it without breaking portability of containers. I hope this helps.
  - StavrosK 13y ago
    Yep, it does help, thanks. Your reason is very clear and sane, it's just frustrating for me because my process is: I launch a VM, see all my memory being eaten up for nothing, then say "man, I'm so going to turn this into an awesome docker container", and, 30 minutes later, I hit a snag and have to quit. Your proposal of having the container specify an initial state would certainly work. Exactly how to do this will take some thinking (what if the external volume has some files in it? What if it's empty but the user needs it empty? Etc), but I think this is a necessary feature for a large variety of use cases. Thanks for the reply!
  - zgohr 13y ago
    Hey shykes, This is something I'm still not quite clear on. According to the issues in mention here: https://github.com/dotcloud/docker/issues/1185 https://github.com/dotcloud/docker/issues/1185 it is possible to do what StavrosK is asking. What am I missing? Why would a volume need an initial state when when the goal can be accomplished with a bind mount? I also +1'd the documentation issue because I can't seem to wrap my head around what the current state of volumes and mounting is. Appreciate any direction you could point me in.
francesca 13y ago
this is awesome. well done.
andridk 13y ago
Ah, this is deploying StriderCD with Docker, and not the other way the around. Was really hoping to see a build server that could spit out Docker images :)
- niall_ohiggins 13y ago
  That is also in the pipeline :)
routelastresort 13y ago
I had major problems with Strider's web interface (running from a Docker instance) just dying on me. I think it's got a long way to go before it's a viable replacement/equivalent for Travis-CI.
- niall_ohiggins 13y ago
  We haven't seen that bug. https://hosted.stridercd.com https://hosted.stridercd.com runs on Docker. We'd love to fix this for you. Can you file a bug at http://github.com/Strider-CD/strider http://github.com/Strider-CD/strider? Many thanks and sorry you are having issues.
- malandrew 13y ago
  Speaking of that, how is TravisCI's commercial service? I know that they are used for a lot of open-source projects, but I haven't seen them get any love or attention from the HN crowd since launching their paid plans for private repos.
timr 13y ago
For those of you looking for a hosted CI system, I highly, highly recommend CircleCI...we've been using it for continuous deployment, and it's been just about perfect. It's one of those tools that Just Works, and makes your life a little bit easier.
- malandrew 13y ago
  How does it compare to the other hosted CI solutions for private repos out there?
  - niall_ohiggins 13y ago
    Strider is designed to be extremely customizable. We have found that in the real world, CI tools are like code editors. Organizations need to heavily extend and modify the tool for their own individual workflow and process. For that reason we believe the best CI tools are Open Source (like Travis and Jenkins). We are working on a hosted Strider offering where each customer will be able to customize their version independently of one another. Strider's value proposition is extensibility and customization. It's the opposite of a "one size fits all" approach (Travis does an amazing job at that already).
  - arohner 13y ago
    Hi, founder of Circle here. Our focus is on developer productivity. That means we try to make things fast, reliable and low overhead. As far as differentiators, we're pretty fast. We can automatically parallelize your test suite across N VMs. We automatically inspect your source tree, and figure out how to run the tests, without configuration. It's not perfect, but a substantial percentage of customers get a working build on their first try. We have some pretty cool features in the pipeline that haven't been in generally available products, we hope to blow peoples' minds with them when they're released :-) Let me know if there's anything else I can answer.
    - timr 13y ago
      "As far as differentiators, we're pretty fast. We can automatically parallelize your test suite across N VMs. We automatically inspect your source tree, and figure out how to run the tests, without configuration. It's not perfect, but a substantial percentage of customers get a working build on their first try." Yes, yes and YES. I love that there's a comment in this thread that claims that CI systems have to be like Jenkins to be marketable. CircleCI was great precisely because it worked basically out of the box, and then the configuration to get it to do more complicated stuff (like, say, running migrations) was as easy as specifying a unix command. I set up our CI system on CircleCI in an afternoon, and we haven't had to touch it since. That's pretty remarkable.
tombh 13y ago
Did it take anyone else ages to download all the files from `docker pull strider/strider'? Was taking over 30 mins for me, so I gave up and installed everything by hand without docker in about 5 mins!
- shykes 13y ago
  Hi, docker maintainer here. The docker registry is getting hit really hard because of the explosive growth of the project - we're in the process of upgrading the infrastructure, should get much better very soon.
  - tombh 13y ago
    It's a good problem to have I guess :)
- shad42 13y ago
  Thanks for reporting this, we're investigating any latency issues.

I just set this up accessible over the internet, building my blog. I've since removed the service/webhooks, but feel free to login at http://stinemat.es:3000 http://stinemat.es:3000 to play around. It's extremely cool.

asabjorn 13y ago
Do you have a demo account or an invite code we can use to register?
- nickstinemates 13y ago
  The demo account is available on the blog post :)
niall_ohiggins 13y ago
Glad you like it! If you have any thoughts we'd love to hear them over at https://github.com/Strider-CD/strider https://github.com/Strider-CD/strider

StavrosK 13y ago

Docker sounds brilliant, but I haven't had much luck with deploying my (moderately complex) Django apps on it. For example, I want the postgres db data to live outside the container, but volumes can't currently do that with Dockerfiles: they're initialized at "docker build" and then the files are just overwritten when trying to mount an external volume. Another use case is trying to spin up a development container (where the code resides on the host), but, again, shared volumes overwrite the directory. You can ADD your code initially and then mount the volume when you run it, but it's added hassle.

agumonkey 13y ago
Same here, I'm still not clear on how to modularize things (even thought the aufs layers seems module friendly)
shykes 13y ago
Hey StavrosK, This is by design, "I want the postgres db data to live outside the container" is a deployment-specific decision, and should not be hardcoded into the container itself. You (or someone you distribute this container to) might not want the db data to live outside the container, or might want it to live at a specific path on his machine. That's why a Dockerfile lets you specify that /var/lib/postgres is persistent, but doesn't let you specify how to persist it. Now, if I remember our IRC discussion correctly, in your case you're hitting a limitation of docker's implementation of volumes, which is that the container cannot specify an initial state for its volume - it can only start empty. That is a limitation that we're going to fix. But we need to make sure we fix it without breaking portability of containers. I hope this helps.
- StavrosK 13y ago
  Yep, it does help, thanks. Your reason is very clear and sane, it's just frustrating for me because my process is: I launch a VM, see all my memory being eaten up for nothing, then say "man, I'm so going to turn this into an awesome docker container", and, 30 minutes later, I hit a snag and have to quit. Your proposal of having the container specify an initial state would certainly work. Exactly how to do this will take some thinking (what if the external volume has some files in it? What if it's empty but the user needs it empty? Etc), but I think this is a necessary feature for a large variety of use cases. Thanks for the reply!
- zgohr 13y ago
  Hey shykes, This is something I'm still not quite clear on. According to the issues in mention here: https://github.com/dotcloud/docker/issues/1185 https://github.com/dotcloud/docker/issues/1185 it is possible to do what StavrosK is asking. What am I missing? Why would a volume need an initial state when when the goal can be accomplished with a bind mount? I also +1'd the documentation issue because I can't seem to wrap my head around what the current state of volumes and mounting is. Appreciate any direction you could point me in.

francesca 13y ago

this is awesome. well done.

andridk 13y ago

Ah, this is deploying StriderCD with Docker, and not the other way the around. Was really hoping to see a build server that could spit out Docker images :)

niall_ohiggins 13y ago
That is also in the pipeline :)

routelastresort 13y ago

I had major problems with Strider's web interface (running from a Docker instance) just dying on me. I think it's got a long way to go before it's a viable replacement/equivalent for Travis-CI.

niall_ohiggins 13y ago
We haven't seen that bug. https://hosted.stridercd.com https://hosted.stridercd.com runs on Docker. We'd love to fix this for you. Can you file a bug at http://github.com/Strider-CD/strider http://github.com/Strider-CD/strider? Many thanks and sorry you are having issues.
malandrew 13y ago
Speaking of that, how is TravisCI's commercial service? I know that they are used for a lot of open-source projects, but I haven't seen them get any love or attention from the HN crowd since launching their paid plans for private repos.

timr 13y ago

For those of you looking for a hosted CI system, I highly, highly recommend CircleCI...we've been using it for continuous deployment, and it's been just about perfect. It's one of those tools that Just Works, and makes your life a little bit easier.

malandrew 13y ago
How does it compare to the other hosted CI solutions for private repos out there?
- niall_ohiggins 13y ago
  Strider is designed to be extremely customizable. We have found that in the real world, CI tools are like code editors. Organizations need to heavily extend and modify the tool for their own individual workflow and process. For that reason we believe the best CI tools are Open Source (like Travis and Jenkins). We are working on a hosted Strider offering where each customer will be able to customize their version independently of one another. Strider's value proposition is extensibility and customization. It's the opposite of a "one size fits all" approach (Travis does an amazing job at that already).
- arohner 13y ago
  Hi, founder of Circle here. Our focus is on developer productivity. That means we try to make things fast, reliable and low overhead. As far as differentiators, we're pretty fast. We can automatically parallelize your test suite across N VMs. We automatically inspect your source tree, and figure out how to run the tests, without configuration. It's not perfect, but a substantial percentage of customers get a working build on their first try. We have some pretty cool features in the pipeline that haven't been in generally available products, we hope to blow peoples' minds with them when they're released :-) Let me know if there's anything else I can answer.
  - timr 13y ago
    "As far as differentiators, we're pretty fast. We can automatically parallelize your test suite across N VMs. We automatically inspect your source tree, and figure out how to run the tests, without configuration. It's not perfect, but a substantial percentage of customers get a working build on their first try." Yes, yes and YES. I love that there's a comment in this thread that claims that CI systems have to be like Jenkins to be marketable. CircleCI was great precisely because it worked basically out of the box, and then the configuration to get it to do more complicated stuff (like, say, running migrations) was as easy as specifying a unix command. I set up our CI system on CircleCI in an afternoon, and we haven't had to touch it since. That's pretty remarkable.

tombh 13y ago

Did it take anyone else ages to download all the files from `docker pull strider/strider'? Was taking over 30 mins for me, so I gave up and installed everything by hand without docker in about 5 mins!

shykes 13y ago
Hi, docker maintainer here. The docker registry is getting hit really hard because of the explosive growth of the project - we're in the process of upgrading the infrastructure, should get much better very soon.
- tombh 13y ago
  It's a good problem to have I guess :)
shad42 13y ago
Thanks for reporting this, we're investigating any latency issues.