5 ms·

Docker interactive tutorial

192 points by dhrp 13y ago

dhrp 13y ago
The creator here. Please let me know if you have any questions or comments!
- simonebrunozzi 13y ago
  Hi dhrp, I assume you work for Dotcloud? Nice job so far. Where do you see this heading to? What's your view of what will happen in the virtualization world? Best,
  - dhrp 13y ago
    Hi. Thanks! My personal take on where Docker is heading to? There are so many things people can do with this, it's hard to summarize. What I personally care most about? As a designer ex-entrepreneur and front-end developer, the thing that gets me going most is the idea that I'm able to "just run" an application. No more difficult than from the Mac store. For example Trac (a wiki system), Wordpress, Django apps, Mailservers, torrent-servers. Basic stuff which just makes it easier for me to deploy my creations, and those of others.
    - buster 13y ago
      Absolutely love docker, hope to see it mature even more :) I am currently playing around with it and building a messaging platform playground. One "pain" so far is that docker's IPAddress assignment is not very flexible. Will it be possible to assign IP addresses to containers (e.g. from "docker run")? Or have a better control what IPAddresses are used (like giving a network range on docker -d)? If i am not mistaken docker saves changes in containers through aufs and keeps those changes as separate images on disk, right? I'm currently working with containers which keep their state on the host OS (by mount bindings) and thus, i don't want to keep old images of not-running containers. Will there be some switch to disable that or clean up old ones? Maybe i am misinterpreting something, but i'm new to docker ;) Anyway, keep up the great work, i am very impressed with docker, kudos!!
      - tmzt 13y ago
        Currently using the unionize.sh script works very well for me. http://blog.docker.io/2013/04/unionize-network-superpowers-for-your-docker-containers/ http://blog.docker.io/2013/04/unionize-network-superpowers-f... You have to run ./unionize <bridge> <container sha1> <ip address> after starting the container, but that brings up a new interface inside the container with that IP and connects it to the bridge. This is useful for having private IPs between containers of an application, for accessing databases or similar.
        buster 13y ago
        I was starting with unionize as well, but the fact is that docker looks up for available IP adresses on its own. So you can give docker -d the -b parameter and pass an existing bridge and it will go through that bridges IP space and assign IP adresses already. Also that way the IP address shows in "docker inspect" which it doesn't with unionize.sh (i think). The problem here is that the built-in IP Address allocator is rather stupid and doesn't even try to ping an address before assigning it. I got it to interfere with my network heavily when it assigned my gateways IP Address to a container ;) It'd be nice if the whole IP Address allocation was more pluggable or configurable. Right now it's some code deeply tied into the whole system (i think) and i fear i don't have the Go skills to change that myself :( (for example, i think i would have been able to write a little bit of Go to assign IP Addresses the way i want to, if the system would be more pluggable)
  - dhrp 13y ago
    Oh, and yes, I do work for dotCloud.
- anonymousDan 13y ago
  What kind of security issues/misconfigurations do people typically have/need to watch out for? How does this compare to alternatives?
zhemao 13y ago
Haha, I had a good laugh upon hearing that Docker cannot run inside of itself. Is this an LXC limitation? Clearly the solution was to run Docker inside of KVM inside of Docker. But seriously, nice job. I haven't used docker yet because I want to play around with the standard lxc utilities first. But this is pretty awesome.
- aegiso 13y ago
  It's not an LXC limitation, but seeing as this case hasn't gotten the love it involves jimmying things all over the place. https://github.com/dotcloud/docker/wiki/Docker-in-Docker https://github.com/dotcloud/docker/wiki/Docker-in-Docker
  - dhrp 13y ago
    Thank you for posting this answer. Indeed, I think the short answer is: It is not impossible, but there are limitations.
- jpetazzo 13y ago
  Actually, I've ran Docker within QEMU within Docker (using v9fs so that QEMU could use the container's FS as root FS). Works, but painfully slow and not very resource-efficient :-)
  - shykes 13y ago
    I will add that Docker is getting an architecture upgrade, and in the future will support nesting :)
  - zhemao 13y ago
    Yeah, I was kidding about Docker in KVM in Docker. I know it would be a lot of overhead.
- ams6110 13y ago
  Would there be a useful reason to actually do this (run Docker in Docker), or is it more just a novelty?
agumonkey 13y ago
Enjoyed it, more would be welcome, only if you have more time to give of course. ps: especially on committing and layering.
- dhrp 13y ago
  Yeah. The committing and layering stuff is not the easiest to grasp. There is actually a section on the docs that does a decent job explaining these concepts but it is a bit hidden: http://docs.docker.io/en/latest/terms/ http://docs.docker.io/en/latest/terms/
itry 13y ago
I never understood what docker is or who is supposed to use it. Can somebody enlighten me?
- dhrp 13y ago
  It depends. Do you mind me asking, what is your background/profession? And have you deployed a virtual machine before to run an application?
  - itry 13y ago
    Im a coder and owner of a small SaaS company. What I do to run applications is this: I fire up a vm or dedicated server on some provider and run my stuff on it. I use Amazon and a couple of other providers.
    - dhrp 13y ago
      Ok. That's great. Then let me try to give you some examples #: Imagine you have setup your SaaS to run from some containers (1 container with your web app, 1 container with your worker, 1 container with your queue and a database somewhere. Now let's walk through a scenario for a significant new release of your web app: 1) Package your new web app, Launch it for testing (on the same host, cheap) to point to a testing database. 2) Fails? Rebuild, test immediately. 3) Happy? Now relaunch your container to connect to the production database 4) Everything works completely? Now re-route your traffic to the already warmed up container. Chances of failure? < 0.1% Some other ideas: - Package your worker. Run it once (on the same host), more load? run it multiple times, run it on multiple servers. -- it is so much quicker and cheaper than spinning up virtual machines. - So your developer made some changes.. He packages it and you run it. It fails. You now just save the entire container including the last state, logs and everything exactly as you crashed it. And hand that back to him. Hope it helps. these examples are based on intended use, because right now the whole development is still moving so fast production deployments are no yet recommended.
      - itry 13y ago
kclay 13y ago
Well thanks I plan on using docker to deploy a jvm app soon, this should help.
- tmzt 13y ago
  This should help a bit more with your specific case: http://blogs.atlassian.com/2013/06/deploy-java-apps-with-docker-awesome/ http://blogs.atlassian.com/2013/06/deploy-java-apps-with-doc...
seiji 13y ago
Why is the docker hype machine cranked up to 11?
- shykes 13y ago
  Hi seiji, I'm the author of Docker. There is hype around docker, for sure. The optimist in me likes to think that it's because people find the project useful and are excited about the possibilities of containers in general - which I believe are huge. Me and the dotCloud team have been working on container technology since 2008. For a long time it felt like preaching in the desert - mostly because it required exotic patches to the kernel which made widespread adoption difficult. So it's very rewarding to see more people adopt containers, and of course it's great to be on the right side of the hype for a change. But if we hadn't been at the right place at the right time, someone else would have done it instead. Containers are just too important and useful to not blow up.
  - e12e 13y ago
    Could you share some thoughts (or links) on the differences between LXC and linux-vserver? I played a little bit with that, and it seemed to promise "true jails" for Linux -- and LXC seems to be a natural successor -- any comment of what we've gained/lost from the "transition"? I'd also love to hear what people think about the relationship between freebsd+jails, solaris+zones and Linux+LXC/docker and/or if it would make sense to modularize the back-end so that "docker" (as in the daemon/management tools) would work for maintaining jails and/or zones as well? It'd be fun to be able to run docker+LXC under GNU Debian/Linux, and docker+jails under GNU Debian/kFreebsd (and ditto for the Debian-like/Ubunut-based solaris distros)... Maybe not useful, but interesting...
    - anonymousDan 13y ago
      Ditto, I'd like to hear a bit more about the security related strengths and weaknesses of docker and its alternatives.
    - shykes 13y ago
      > Could you share some thoughts (or links) on the differences between LXC and linux-vserver? vserver, lxc and openvz were 3 competing projects to add process-level isolation to the linux kernel. We used all 3 of them extensively (the ancestor of docker was based on vserver, then ported to openvz, then finally to lxc). They all had pros and cons, but in the end the only meaningful difference is that lxc got merged upstream, and the others didn't. > would make sense to modularize the back-end so that "docker" (as in the daemon/management tools) would work for maintaining jails and/or zones as well? Absolutely. That is the goal, and starting with 0.8 Docker's architecture will be modular enough to support it. See this blog post for details: http://blog.docker.io/2013/08/getting-to-docker-1-0/ http://blog.docker.io/2013/08/getting-to-docker-1-0/
kinleyd 13y ago
I've been playing a bit with Docker and found I couldn't get the kernel option to work (the linux-aufs_friendly kernel just wouldn't work on my Arch setup). However, that forced me to the option of using vagrant to set up a vm with docker configured and I found that great - I recommend this option over playing with your kernel. I'm a long time virtualbox user but had never played with headless vms and never realized how easy that is with vagrant. Additionally, using vagrant to get a coreos vm running, with docker all set up, was pretty cool. So far I'm finding the payload of the vms rather heavy (with the os overhead), but I haven't really got down to setting up individual docker app containers. I'm looking forward to that, and even more to what could lie ahead for this space: vagrant, coreos, docker, chef/puppet all look to making a very promising convergence.
- mateuszf 13y ago
  Probably it won't help you, but Docker is working on my Arch with linux-aufs_friendly kernel.
  - kinleyd 13y ago
    I suspect it must have something to do with my set up. My first suspicion was my nvidia drivers (304xx) but I recall not even being able to boot in console mode. I'll give it another shot later but for the moment vagrant's doing a fine job putting it all together.
limist 13y ago
8 steps, 15-20 minutes, and you'll get an initial understanding of a much better way to do devops/deployment, including scalability. Thanks Docker!

192 points by dhrp 13y ago

dhrp 13y ago

The creator here. Please let me know if you have any questions or comments!

simonebrunozzi 13y ago
Hi dhrp, I assume you work for Dotcloud? Nice job so far. Where do you see this heading to? What's your view of what will happen in the virtualization world? Best,
- dhrp 13y ago
  Hi. Thanks! My personal take on where Docker is heading to? There are so many things people can do with this, it's hard to summarize. What I personally care most about? As a designer ex-entrepreneur and front-end developer, the thing that gets me going most is the idea that I'm able to "just run" an application. No more difficult than from the Mac store. For example Trac (a wiki system), Wordpress, Django apps, Mailservers, torrent-servers. Basic stuff which just makes it easier for me to deploy my creations, and those of others.
  - buster 13y ago
    Absolutely love docker, hope to see it mature even more :) I am currently playing around with it and building a messaging platform playground. One "pain" so far is that docker's IPAddress assignment is not very flexible. Will it be possible to assign IP addresses to containers (e.g. from "docker run")? Or have a better control what IPAddresses are used (like giving a network range on docker -d)? If i am not mistaken docker saves changes in containers through aufs and keeps those changes as separate images on disk, right? I'm currently working with containers which keep their state on the host OS (by mount bindings) and thus, i don't want to keep old images of not-running containers. Will there be some switch to disable that or clean up old ones? Maybe i am misinterpreting something, but i'm new to docker ;) Anyway, keep up the great work, i am very impressed with docker, kudos!!
    - tmzt 13y ago
      Currently using the unionize.sh script works very well for me. http://blog.docker.io/2013/04/unionize-network-superpowers-for-your-docker-containers/ http://blog.docker.io/2013/04/unionize-network-superpowers-f... You have to run ./unionize <bridge> <container sha1> <ip address> after starting the container, but that brings up a new interface inside the container with that IP and connects it to the bridge. This is useful for having private IPs between containers of an application, for accessing databases or similar.
      - buster 13y ago
        I was starting with unionize as well, but the fact is that docker looks up for available IP adresses on its own. So you can give docker -d the -b parameter and pass an existing bridge and it will go through that bridges IP space and assign IP adresses already. Also that way the IP address shows in "docker inspect" which it doesn't with unionize.sh (i think). The problem here is that the built-in IP Address allocator is rather stupid and doesn't even try to ping an address before assigning it. I got it to interfere with my network heavily when it assigned my gateways IP Address to a container ;) It'd be nice if the whole IP Address allocation was more pluggable or configurable. Right now it's some code deeply tied into the whole system (i think) and i fear i don't have the Go skills to change that myself :( (for example, i think i would have been able to write a little bit of Go to assign IP Addresses the way i want to, if the system would be more pluggable)
- dhrp 13y ago
  Oh, and yes, I do work for dotCloud.
anonymousDan 13y ago
What kind of security issues/misconfigurations do people typically have/need to watch out for? How does this compare to alternatives?

zhemao 13y ago

Haha, I had a good laugh upon hearing that Docker cannot run inside of itself. Is this an LXC limitation? Clearly the solution was to run Docker inside of KVM inside of Docker. But seriously, nice job. I haven't used docker yet because I want to play around with the standard lxc utilities first. But this is pretty awesome.

aegiso 13y ago
It's not an LXC limitation, but seeing as this case hasn't gotten the love it involves jimmying things all over the place. https://github.com/dotcloud/docker/wiki/Docker-in-Docker https://github.com/dotcloud/docker/wiki/Docker-in-Docker
- dhrp 13y ago
  Thank you for posting this answer. Indeed, I think the short answer is: It is not impossible, but there are limitations.
jpetazzo 13y ago
Actually, I've ran Docker within QEMU within Docker (using v9fs so that QEMU could use the container's FS as root FS). Works, but painfully slow and not very resource-efficient :-)
- shykes 13y ago
  I will add that Docker is getting an architecture upgrade, and in the future will support nesting :)
- zhemao 13y ago
  Yeah, I was kidding about Docker in KVM in Docker. I know it would be a lot of overhead.
ams6110 13y ago
Would there be a useful reason to actually do this (run Docker in Docker), or is it more just a novelty?

agumonkey 13y ago

Enjoyed it, more would be welcome, only if you have more time to give of course. ps: especially on committing and layering.

dhrp 13y ago
Yeah. The committing and layering stuff is not the easiest to grasp. There is actually a section on the docs that does a decent job explaining these concepts but it is a bit hidden: http://docs.docker.io/en/latest/terms/ http://docs.docker.io/en/latest/terms/

itry 13y ago

I never understood what docker is or who is supposed to use it. Can somebody enlighten me?

dhrp 13y ago
It depends. Do you mind me asking, what is your background/profession? And have you deployed a virtual machine before to run an application?
- itry 13y ago
  Im a coder and owner of a small SaaS company. What I do to run applications is this: I fire up a vm or dedicated server on some provider and run my stuff on it. I use Amazon and a couple of other providers.
  - dhrp 13y ago
    Ok. That's great. Then let me try to give you some examples #: Imagine you have setup your SaaS to run from some containers (1 container with your web app, 1 container with your worker, 1 container with your queue and a database somewhere. Now let's walk through a scenario for a significant new release of your web app: 1) Package your new web app, Launch it for testing (on the same host, cheap) to point to a testing database. 2) Fails? Rebuild, test immediately. 3) Happy? Now relaunch your container to connect to the production database 4) Everything works completely? Now re-route your traffic to the already warmed up container. Chances of failure? < 0.1% Some other ideas: - Package your worker. Run it once (on the same host), more load? run it multiple times, run it on multiple servers. -- it is so much quicker and cheaper than spinning up virtual machines. - So your developer made some changes.. He packages it and you run it. It fails. You now just save the entire container including the last state, logs and everything exactly as you crashed it. And hand that back to him. Hope it helps. these examples are based on intended use, because right now the whole development is still moving so fast production deployments are no yet recommended.
    - itry 13y ago

kclay 13y ago

Well thanks I plan on using docker to deploy a jvm app soon, this should help.

tmzt 13y ago
This should help a bit more with your specific case: http://blogs.atlassian.com/2013/06/deploy-java-apps-with-docker-awesome/ http://blogs.atlassian.com/2013/06/deploy-java-apps-with-doc...

seiji 13y ago

Why is the docker hype machine cranked up to 11?

shykes 13y ago
Hi seiji, I'm the author of Docker. There is hype around docker, for sure. The optimist in me likes to think that it's because people find the project useful and are excited about the possibilities of containers in general - which I believe are huge. Me and the dotCloud team have been working on container technology since 2008. For a long time it felt like preaching in the desert - mostly because it required exotic patches to the kernel which made widespread adoption difficult. So it's very rewarding to see more people adopt containers, and of course it's great to be on the right side of the hype for a change. But if we hadn't been at the right place at the right time, someone else would have done it instead. Containers are just too important and useful to not blow up.
- e12e 13y ago
  Could you share some thoughts (or links) on the differences between LXC and linux-vserver? I played a little bit with that, and it seemed to promise "true jails" for Linux -- and LXC seems to be a natural successor -- any comment of what we've gained/lost from the "transition"? I'd also love to hear what people think about the relationship between freebsd+jails, solaris+zones and Linux+LXC/docker and/or if it would make sense to modularize the back-end so that "docker" (as in the daemon/management tools) would work for maintaining jails and/or zones as well? It'd be fun to be able to run docker+LXC under GNU Debian/Linux, and docker+jails under GNU Debian/kFreebsd (and ditto for the Debian-like/Ubunut-based solaris distros)... Maybe not useful, but interesting...
  - anonymousDan 13y ago
    Ditto, I'd like to hear a bit more about the security related strengths and weaknesses of docker and its alternatives.
  - shykes 13y ago
    > Could you share some thoughts (or links) on the differences between LXC and linux-vserver? vserver, lxc and openvz were 3 competing projects to add process-level isolation to the linux kernel. We used all 3 of them extensively (the ancestor of docker was based on vserver, then ported to openvz, then finally to lxc). They all had pros and cons, but in the end the only meaningful difference is that lxc got merged upstream, and the others didn't. > would make sense to modularize the back-end so that "docker" (as in the daemon/management tools) would work for maintaining jails and/or zones as well? Absolutely. That is the goal, and starting with 0.8 Docker's architecture will be modular enough to support it. See this blog post for details: http://blog.docker.io/2013/08/getting-to-docker-1-0/ http://blog.docker.io/2013/08/getting-to-docker-1-0/

kinleyd 13y ago

I've been playing a bit with Docker and found I couldn't get the kernel option to work (the linux-aufs_friendly kernel just wouldn't work on my Arch setup). However, that forced me to the option of using vagrant to set up a vm with docker configured and I found that great - I recommend this option over playing with your kernel. I'm a long time virtualbox user but had never played with headless vms and never realized how easy that is with vagrant. Additionally, using vagrant to get a coreos vm running, with docker all set up, was pretty cool. So far I'm finding the payload of the vms rather heavy (with the os overhead), but I haven't really got down to setting up individual docker app containers. I'm looking forward to that, and even more to what could lie ahead for this space: vagrant, coreos, docker, chef/puppet all look to making a very promising convergence.

mateuszf 13y ago
Probably it won't help you, but Docker is working on my Arch with linux-aufs_friendly kernel.
- kinleyd 13y ago
  I suspect it must have something to do with my set up. My first suspicion was my nvidia drivers (304xx) but I recall not even being able to boot in console mode. I'll give it another shot later but for the moment vagrant's doing a fine job putting it all together.

limist 13y ago

8 steps, 15-20 minutes, and you'll get an initial understanding of a much better way to do devops/deployment, including scalability. Thanks Docker!