6 ms·
Show HN: Bocker – Docker implemented in 100 lines of bash
- WD-42 11y agoNice work. Great to see the advanced features of BTRFS put to use.
- vezzy-fnord 11y agoUsing btrfs subvolumes as the image format, that's a nice touch. On the same road as the hypothetical systemd packaging system (not that I'm very enthusiastic about that). The network, PID and mount namespaces are the ones unshared, plus a private /proc. I like tools like this because they're reality checks on how the basics of Linux containers are just a few essential system calls, and particularly that they're limited.
- zobzu 11y agoalso that they dont use a root daemon and random hard to debug, hard to verify things. it feels like most of them build large products around things to justify making money, while i like the simple, elegant, easy to verify little pieces of software (some call it "the unix way") unshare and iproute are pretty decent for ex.
- p8952 11y agoThank you. One of the things I found really interesting here is how much could be done with just basic userland tools, and how old some of those tools are. Docker was released in 2013, but support for kernel namespacing has been around since ~2007. That's quite a long time for such a great feature to go mainstream.
- hosh 11y agoYes. The basic technology has been there for a while. And the Docker source code has some eyebrow-raising parts to it. However, I've stated this in other threads: Docker isn't about containment. It's really about the packaging system. I don't think this technology demo gets that.
- p8952 11y agoI agree 100%. While the tech in Docker is great, it alone can't be easily monetized. The real value comes from things like DockerHub, and getting people to buy into the whole ecosystem.
- vezzy-fnord 11y agoWhich is probably why people were concerned about Docker's expansion into the clustering and orchestration markets, even if from a business perspective those are their only real holdouts to avoid commodification. The base Docker is easy to replace if the project goes out of hand, the services around it are trickier.
- hosh 11y agoThere are several tech streams converging there. A bigger chunk is the space that Mesos and Kubernetes occupy. And to be fair, I suspect the Docker folks were thinking less about clustering and orchestration so much as: (1) clustering and orchestration still sucks; (2) people want as good of an experience using docker as they do when spanning across multiple nodes; (3) let's make clustering and orchestration less sucky and use the 'Docker Way'[1] [1] 'Docker Way' is a pointer to the fuzzy, difficult-to-verbalize thing that Docker enables, namely in packaging.
- vezzy-fnord 11y agoclustering and orchestration still sucks I don't see that changing until we get proper single-system imaging, location transparency, process and IPC migration, process checkpointing and RPC-based servers for representing network and local resources as objects (be they file-based or other) in our mainstream systems. These things only really caught on in the HPC and scientific computing spaces, where you've had distributed task managers and workload balancers like HTCondor and MOSIX for decades. They've also been research interests in systems like Amoeba and Sprite, but sans that, not much. The likes of Mesos, the Mesosphere ecosystem with Marathon and Chronos, and Docker Swarm bring only the primitive parts of the whole picture. Some other stuff they can half-ass by (ab)using file system features like subvolumes, but overall I don't see them improving on all the suck.
- chjj 11y agoAgreed. That was pretty good. A dash of bash with 3 teaspoons of butter: https://en.wikipedia.org/wiki/Btrfs#Subvolumes_and_snapshots https://en.wikipedia.org/wiki/Btrfs#Subvolumes_and_snapshots
- sandGorgon 11y agonot sure if you ever saw this - https://www.phoronix.com/scan.php?page=news_item&px=CoreOS-Btrfs-To-EXT4-OverlayFS https://www.phoronix.com/scan.php?page=news_item&px=CoreOS-B... it might be interesting to see a version of your script using overlayfs
- p8952 11y agoInteresting, it was my understanding that Docker was going in the other direction. Moving off AuFS towards Btrfs due to issues getting AuFS patches into the mainline kernel. I'll have to look info CoreOS's reasons for going with Ext4.
- sandGorgon 11y agoDocker 1.6+ works with overlayfs on my debian box (in production). I thought AUFS was already deprecated
- kangsterizer 11y agoI've a more or less similar script that uses overlayfs. More less than more... since it uses machined really. But it's easy to to grab commands from there and put in bocker for ex. You just need to replace the btrfs commands by the overlayfs mount, it's almost nothing :) https://github.com/gdestuynder/mctl/ https://github.com/gdestuynder/mctl/ See also this terrible draft https://www.insecure.ws/linux/systemd_nspawn.html https://www.insecure.ws/linux/systemd_nspawn.html
- tlrobinson 11y agoHere's a proof-of-concept implementation of "docker pull" in bash (YMMV, I think it has broken since I wrote it): https://gist.github.com/tlrobinson/c85dca269f4405ad4201 https://gist.github.com/tlrobinson/c85dca269f4405ad4201
- p8952 11y agoLooks good! I'd marked 'docker pull' as out of scope because I thought it would be fairly hard to interface with their API from bash, looks like I was wrong.
- tlrobinson 11y agoThe only hard bit is parsing the JSON, which I use jq for (http://stedolan.github.io/jq/ http://stedolan.github.io/jq/) I think the v2 API requires hitting an auth endpoint too.
- CSDude 11y agoThere are some images still in registry v1, I recon a problem like that where it gives you another URL
- Bocker 11y agoApproved.
- asdfaoeu 11y agoThere's also https://github.com/docker/dockerlite https://github.com/docker/dockerlite . But not sure how current it is.
- eatitraw 11y agoNow we need bokku: https://github.com/progrium/dokku https://github.com/progrium/dokku
- josegonzalez 11y agoDokku written in bash? (I am one the maintainers of dokku, which is written in bash).
- yodasan 11y agoI think he's suggesting creating a similar project to dokku, but instead of using Docker, use Bocker.
- josegonzalez 11y agoIf installing via apt-get, that means creating a debian package that provides lxc-docker-1.6.2 (docker 1.7 broke backwards compatibility in output of docker ps). Otherwise, just patching the [Makefile](https://github.com/progrium/dokku/blob/master/Makefile#L85 https://github.com/progrium/dokku/blob/master/Makefile#L85) to check for bocker (linked to `/usr/local/bin/docker` of course) would suffice. Though I don't think that is what he is asking for.
- logicrime 11y agoHoly crap! I've been keeping up with the hype, yet having never used Docker and never needed it yet I can't help but become more skeptical now that I know that it's features aren't more complex than a little bit of bash. People give bash a hard time, but things like this really give me that warm, fuzzy feeling. esac4lyfe
- welly 11y agoWhy would it need to be more complex than "a little bit of bash" if it fulfils the task intended? Surely that is better than a convoluted, 1,000,000,000 LoC application that no one outside a handful of core developers understands? Right?
- antsar 11y agoI think you and logicrime are saying the same thing.
- zobzu 11y agoto be honest most of the work is done by iproute and unshare in this case. unshare basically calls clone() (syscall) with the correct namespace related paramters, iproute deals with veth.
- collyw 11y agobash is quite a neat language (for what it does) but has horrendous syntax.
- tobbyb 11y agoJust playing with this in a VM with an attached btrfs volume, a complete revelation. 96 lines! And it's actually pretty functional. This takes keeping it simple to a whole new level. The Wheezy image I use with LXC worked well enough, the minimal alpine image not so well, apk complaining about its database. User name spaces support would be nice, then we can play with unprivileged containers. And Overlayfs would be a nifty alternative to btrfs, it's in kernel 3.18, and 4.04 adds support for multiple lower layers. But this btrfs implementation is cool too. Cgroups support will be somewhere on that list too. Cgroups and namespaces is in the kernel. General Linux ecosystem for networking, storage and distributed systems is already extensive. The possibilities are endless. So now LXC, Docker, Rkt and Nspawn have Bocker for company.
- deleted 11y ago[deleted]
- cyansmoker 11y agoYes they do. Nice to learn about even more projects than I thought existed.
- vacri 11y agoInteresting work. And 10% of those lines are simply closing braces which can be collapsed to the previous line, and half a dozen lines can be reclaimed from the help function...
- tzm 11y ago$10M per LoC
- SFjulie1 11y agoI thought puppet/chef were the pit of the devops ridicule. Then I not only saw this, but also positive reactions to a readable code in which you have : echo 'nameserver 8.8.8.8' > "$btrfs_path/$uuid"/etc/resolv.conf This is wrong on so many level that I don't know where to begin with.
- zobzu 11y agoi dont think this part is needed at all, it just means the base image sux basically, so this is some patching up, which, well, is fine for demo purposes
- druiid 11y agoSadly that is pretty much emulating how docker does it. To the best of my knowledge the '8.8.8.8' and '8.8.4.4' name-servers are kind of hard-coded in a lot of containers (all?)
- Gigablah 11y agoCan you share why you think puppet/chef are the "pit of the devops ridicule"?
- dsmithatx 11y agoHaving implemented it across 10k+ servers in 12 datacenters I'd say he means Puppet is overly complex for what provides. Kind of like Docker. I think people equate Chef to the same over-complexity. Having now used Salt and starting to play with Ansible I'm growing an extreme dislike for Puppet and the weeks of my life I can never recover dealing with things that Salt has made so much easier.
- spydum 11y agoI'm missing something, what actually "executes" the command here echo "$2" > "$btrfs_path/$uuid/$uuid.cmd" Is something watching for .cmd? Is this some behavior of util-linux (for which, my few seconds couldn't find solid documentation on)?
- inopinatus 11y agoIf my dry read of the code is correct, that line is for the "bocker ps" routine to be able to print the command later. The run command itself is executed next, inside of ip netns exec.
- spydum 11y agoAhhh, no.. I was looking at line 57 and somehow missed the actual call to $2 on 59. Clear as a bell now
- pekk 11y agoWow, Docker in 100 lines! It runs as root? Oh. It is written in bash? Oh. It needs a ton of manual setup? Oh. It doesn't actually implement the package format which is most of the point of Docker? Oh. So is it that easy to reimplement Docker? Despite the obvious snarky intent, it appears not.
- alwaysagopher 11y ago@Shykes, is that your alter-ego account?
- pekk 11y agoNo, that's a dumb joke. Actually I don't really even use Docker personally. HN's standard middlebrow dismissal of Docker is to claim that it's nothing except LXC, which really misses the point entirely. But at least it lets people pretend they are smarter, which is what really counts.
- Spivak 11y agoThey are rightfully dismissive of Docker because it's just the current cycle of trendy abstractions. It's a barely passable solution to a bigger problem. "We're running many services on a single machine. But this is complicated and difficult to update and maintain." "We took our machine, ran a virtualization platform on it, and split each service into its own VM. But this comes at the cost of increased resource usage." "Instead of separate VMs we created a container format to decrease the overhead while retaining many of the benefits of virtualization. But this is still resource heavy, and insecure as the containers will rarely see updates." "So we created 'lightweight' containers which are very thin wrapper around the base OS so that containers can take advantage of updated shared libraries to mitigate the security problems, and further decrease the overhead." "We're running many services on a single machine..." The cycle will eventually come around and we'll be in a better place having learned that was was really needed was improvements to the base OS, package management, more robust MAC policy, and name-spacing rather than containers.
- 11y ago
- npx 11y agoIf anyone wants to earn some huge brownie points with me (and who wouldn't), you could implement a PaaS on top of Joyent's Triton system[1]. Purely in terms of the cost structure you could offer with such a PaaS, this could be a Heroku killer. Huge bonus points if it's totally open source! 1. https://github.com/joyent/sdc-docker https://github.com/joyent/sdc-docker
- anh79 11y agoWhen I first saw the HN title, I was so stunning. Weird, it's not my tool https://github.com/icy/bocker https://github.com/icy/bocker ;) The author of "bocker" (not my bocker) has a great idea. I would learn from the script. Docker is not magic anymore.!!
- e12e 11y agoNice. As I just started playing with zfsonlinux, I'm tempted to "port" this from btrfs to zfs... Should allow for migrating "images"/snapshots with zfs send...
- joeevans1000 11y agoThe emperor is wearing fewer clothes.
- BonoboBoner 11y agoYou published it? You should have gotten a "micro-docker"-like hashtag trend going and then pitch your idea to VCs. The main "Lighter than Docker" startup would be valued at around 5-7 billion right now.
- simi_ 11y agohttps://sandstorm.io/ https://sandstorm.io/ ?
- tritium 11y agoJust a quick interjection: Something that one smart developer can do in 100 lines on any interpretter is never worth billions of dollars. Someone who can do it, on the other hand, is certainly worth hundreds of thousands of dollars, annually. Meanwhile, so many people continue to marvel at what can be done with an interpretter and a turing-complete language. Yet, the last thing we need is yet another turing complete language. Unfortunately, the problem with turing machines, virtual or otherwise, is that they're so good at faihfully reduplicating themselves...
- mpdehaan2 11y ago"Something that one smart developer can do in 100 lines on any interpretter is never worth billions of dollars." 100 lines maybe not, but docker is pretty lightweight glue riding on existing technology that did most of the heavy lifting. The valuation IS lopsided because it sort of did a "name grab" around the underpinning technology (sort of like "AJAX" was "XMLHttpRequest") and packaged it in a way that made it a more useful (some) and more importantly, talked about in a way people could understand, mostly giving it a name and describing some common practices was what happened. The original idea was definitely clever though, and it is getting people who didn't adopt immutable systems before to start to understand immutable systems, even if the future is not actually going to be Docker. Yet, Docker is getting the press versus the higher level software that needs to exist to drive Docker. While this makes it very hard for other projects to get VC attention, I think that's maybe a good thing for them if they don't know it - you want to bootstrap if you can anyway, and I hope many of them do. While this isn't a super robust implementation or anything, I think it's important to show that Docker is more or less glue around existing concepts, and that there's still room to make better things. Don't get me wrong, immutable systems are GREAT. Docker deserves points for getting people to understand them, and the ideas of private registries and Dockerfiles (though also not original) are good parts. Microservices? Not really neccessary for most people, that's more of a human problem. It sort of conflates immutable with microservices and makes you hop out of the norm to do the basics, but ok, sure, that's somewhat like a celebrity actor advocating for environmental causes. Still a good thing. But is it a giant leap forward? Not as much as people think, compared to AMI builds, and you see folks running Docker on top of EC2 in cases (yes, they boot faster - but AMIs gave you immutable and things like Packer allowed redistributable images; stacking images is kind of sketchy if you don't control them all). But it's enough to make people use them, and that's a win, and someday the management software for it may be smart enough to make it feel as easy and powerful as that (fingers crossed for ECS?). The 100 liner at least has the advantage of reminding people when people say "Docker is great" they mostly mean "I like this immutable systems thing and describing systems in text", and the other properties of Docker, and reminds people that if they can do better and try to make a better thing, they should also still try.
- arianvanp 11y agoAdd support for GPG-signed `btrfs pull` and `btrfs push` and I'm totally sold! I've been working on something similar to this but on top of systemd-nspawn, which already does some stuff for you. systemd-nspawn is nice because I run systemd in all my containers and thus allows me to easily do logging etc. I don't really dig the docker-microservices mantra that much. I just use them as glorified VMs I guess. (And yes, you should run an init system in your containers [0]) [0] - https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/ https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zomb...
- amrit_b 11y agoThis is incredible.
- agumonkey 11y agoPersistent data structure FTW.
- sklogic 11y agoI honestly thought docker was just a little script when I saw it first, judging on the functionality.
- jenscow 11y agoThat script is the best description of Docker I've read.
- kzisme 11y agoAs an entry level developer - how does someone go about re-writing "x" in 100 lines of "x"? Is there a certain process that goes into developing something like this, and why is this a popular thing to do? (writing an existing software in lesser lines)
- MoOmer 11y ago1) Knowledge of the space is good, but can be learned while implementing. You'll need to know the language pretty well, in order to keep the line count down. 2) It's fun
- cyansmoker 11y agoHey, I just submitted a pull request for the "exec" command :)