7 ms·

Google Open-Sourced Kubernetes to Boost Its Cloud

84 points by kalgen 11y ago

mark_l_watson 11y ago
I am a little surprised that Google is join this route. Making Google Cloud into a huge business must be a high priority. I used Borg for a while in 2013, and it was amazingly, specially the logging for tracking down runtime problems.
- fweespeech 11y ago
  Why? Google hands out all sorts of papers and such a few years after the fact. This is just another example, an open source version of Borg that will always be a few years behind.
  - thockingoog 11y ago
    While it is true that Borg is 10 years ahead of Kubernetes, we have the benefit of knowing where Borg arrived rather than following the same winding course.
    - fweespeech 11y ago
      Of course, but unless there is a massive maintainability flaw in Borg, its unlikely you can duplicate 10 years of effort AND maintain pace with Borg's/Borg's successor(s) progress. Well, without a huge influx of developers to the open source side.
      - thockingoog 11y ago
        First, there are a LOT of people working on Kubernetes right now. Second, of course it will be asymptotic for a long time. But keep in mind that we don't want or need everything that Borg has, and Borg does not have everything that Kubernetes does. Kubernetes is not a clone of Borg - it is inspired by the lessons and experiences we got by doing Borg. Often that means we learned how NOT to do something. If we can get 75% as functional as Borg in 1/3 the time, we'll be doing pretty darn well. I think we're on track for that.
        fweespeech 11y ago
        > If we can get 75% as functional as Borg in 1/3 the time, we'll be doing pretty darn well. I think we're on track for that. I'm sure you can do that. That doesn't invalidate my statement that it isn't surprising Google would do this precisely because it'll only be 75% as functional as Borg at any given point in time.
amelius 11y ago
Any references to the publications containing the real blueprint?
- kalgen 11y ago
  Borg: http://research.google.com/pubs/pub43438.html http://research.google.com/pubs/pub43438.html Omega: http://research.google.com/pubs/pub41684.html http://research.google.com/pubs/pub41684.html
dkarapetyan 11y ago
Google open source projects in general are just shitty. The reason is that they don't open source the whole thing and the code ends up being full of assumptions that don't hold because you are either not running things at google scale or are missing certain key bits of the "secret sauce". In any event google compute is a terrible user experience compared to the likes of AWS and other cloud providers. Heck, even the shittiest VPS providers tend to be better than google compute. So open sourcing their "secret sauce" as the article puts is still missing key bits so I don't know how many people actually fall for the good will part.
- thockingoog 11y ago
  What key bits do you think are missing from Kubernetes? I'm sorry you seem to have had a bad experience with GCE, but please know that Kubernetes runs on several other clouds, too, with no crippleware or anything. It is 100% open.
  - viklas 11y ago
    And 100% on-track for awesome. Yes, sometimes development/testing for new kubernetes features 'feels' like it's focussed-first on GCE functionality (before other platforms) and earlier on, it had some hooks that weren't great (like GCE-only external load balancers and storage). But hey, it's not even v1.0 yet - and all those things are either fixed or being worked on already. And as a non-GCE user, you aren't a second-class citizen. It works everywhere. We've deployed successfully in AWS, vagrant and bare-metal (in the garage), so far. All with 'one-command' automated deployment and re-use of our pod & service specs throughout. Roadmap/Architecture-wise, it would be good to see a more 'pluggable' approach for 3rd party integration (more like an Open Stack model), but again, we're still pre-v1.0... Also, I think the google-folk here are being very 'reasonable' in their replies. Your comment was mis-directed & ill-informed. Go do some reading or watch Kelsey Hightower's presentation from a couple of months ago: http://chariotsolutions.com/screencast/philly-ete-2015-16-kelsey-hightower-managing-containers-at-scale-with-coreos-and-kubernetes/ http://chariotsolutions.com/screencast/philly-ete-2015-16-ke...
- cmcluck 11y ago
degenerate 11y ago
This has nothing to do with the article, but I absolutely hate the way wired underlines their article links. It might fit with their magazine style design but it's so incredibly distracting. I always want to bump it 1px up or down!
bsaul 11y ago
Is it me or does anyone else interprets all the recent wave of open sourcing as a trend to not create open standards anymore, but instead open source the technology altogether (without even creating the standard). Have standards failed ?
- jsprogrammer 11y ago
  Why would you want your code private? Instead, let anyone work on it and benefit.
  - mbreese 11y ago
    The point of open protocols and standards isn't to keep the code private, there are plenty of open source SMTP or IMAP servers.
- VikingCoder 11y ago
  Standards are great when multiple companies want to provide the same service, which must communicate with software written by other companies. Compiler toolchain, devops, sysadmin, languages, runtimes, etc don't really fit that picture.
- mateuszf 11y ago
  This is a very good debate regarding this very topic: https://www.youtube.com/watch?v=kRVWjC6osuw https://www.youtube.com/watch?v=kRVWjC6osuw TLDR: we are in a stage where we don't know yet what functionality needs to be supported, it is not a good time to form standards, it is a time for checking possibilities and finding best technical solutions.
- obstinate 11y ago
  My 2c, not speaking as a Google rep but as myself, is that if you have a problem only a single actor needs to solve, you're going to end up with software, not a standard. To get a standard, you have to have a situation where the value of cooperation is higher than the cost. There are plenty of people now who need to solve the container problem, but Googlers been working on this shit for years, before it was really on anyone else's horizon. Google employees incepted the cgroup feature way back in 2006, to solve problems that were already being felt acutely at that time within Google. Folks have been working on this stuff a long time before it mattered to anyone else, and that's why what's coming out is software rather than standards. There is no way a big company is going to delay solutions to an urgent strategic problem in order to be part of a democratic process for the sake of a few people's ideals. Maybe if they'd seen it coming five or ten years in advance, to give enough time for the standardization process to occur, but Google was far too small and the future far too uncertain in 2001 to predict what might be needed in 2006.
justinsb 11y ago
I've been working with Kubernetes a fair bit (I'm contributing to porting it to AWS). It's an exciting time: I believe that we'll be running everything in containers within the next few years, and Kuberentes solves some of the big missing pieces with actually running Docker in production across a cluster. There are other competing systems trying to do the same thing (including Docker Inc itself), but I think Kubernetes has a huge advantage by having years of experience of what did and didn't work well in Borg. Most importantly of all though, I have found it a great project to use, and a really great project to contribute to. I think that contributions are the lifeblood of open-source projects, and I give Kubernetes 10/10 for their community and processes, which I think bodes very well for the future.
digitalzombie 11y ago
Kubernetes is alpha software right now. I was at a meet up last month and there are a lot of missing features from what I recall. They're planning to fix one of these feature with some project call Ubernetes. While people are raving about containers, there are still security issues with containers no? I think VM will be here to stay for a long while and while we might have to pay a performance and memory hit for them, they offer better isolation.
- 4ad 11y ago
  > there are still security issues with containers no? In Linux, perhaps. However FreeBSD jails and illumos zones are rock-solid. There's this crazy hype around containers these days and people just ignore the stable, secure, and tried technology, I don't understand it at all! FreeBSD and illumos are not Linux, but their still Unix-like, it's not like you'd have to use OpenVMS. Plus you'd get other benefits too, like DTrace and ZFS. And on illumos now you can even run Linux binaries in a zone. So why do people simply pretend these secure technologies don't exist? Can someone explain?
  - mateuszf 11y ago
    Because there are lot more Linux users / developers than BSD.
    - 4ad 11y ago
      I don't think that "X has more users than Y" is a valid complete argument in itself, otherwise nothing would ever change. Of course it matters in the grand scheme of things, but the causal relationship is more complex, involves many degrees of freedom and goes both ways. Before docker came, Linux had LXC, which wasn't as popular as docker is now, but it was certainly known and used by people. So when docker came, LXC had more users than docker, and yet docker surpassed LXC in popularity in weeks, so the "X has more users than Y" state can be changed by various factors and it's not enough to keep the system in equilibrium. So yes, the fact that Linux is more widely used than FreeBSD and illumos in the developer community is certainly an important factor, but I don't see anyone ever saying "FreeBSD is great but we want to use something supported by a larger community", or "illumos is great, but we don't have expertise with it", which are certainly important arguments to consider when making a decision. But I hardly see anyone making these arguments, or any other arguments really. It's like these systems don't even exist. At first I attributed this to the "X has more users than Y" factor, but then I see people having particular problems with Linux container technology, in areas such as security, virtual networking, etc. And these are problems already solved by FreeBSD and illumos. Surely when you have a problem you look for alternative solutions that don't have these problems? But I don't see people looking over the alternatives at all. As I said, there are many valid reasons not to use these other technologies, but I am very perplexed that people refuse to even acknowledge the existence of them. And now that illumos can run Linux inside a zone (and FreeBSD did this too 15 years ago, and still does for 32-bit binaries, I believe work is well underway to extend this to 64-bit as well), I think the "I only know Linux" argument loses some potency, you can run Linux after all...

84 points by kalgen 11y ago

I am a little surprised that Google is join this route. Making Google Cloud into a huge business must be a high priority. I used Borg for a while in 2013, and it was amazingly, specially the logging for tracking down runtime problems.

fweespeech 11y ago
Why? Google hands out all sorts of papers and such a few years after the fact. This is just another example, an open source version of Borg that will always be a few years behind.
- thockingoog 11y ago
  While it is true that Borg is 10 years ahead of Kubernetes, we have the benefit of knowing where Borg arrived rather than following the same winding course.
  - fweespeech 11y ago
    Of course, but unless there is a massive maintainability flaw in Borg, its unlikely you can duplicate 10 years of effort AND maintain pace with Borg's/Borg's successor(s) progress. Well, without a huge influx of developers to the open source side.
    - thockingoog 11y ago
      First, there are a LOT of people working on Kubernetes right now. Second, of course it will be asymptotic for a long time. But keep in mind that we don't want or need everything that Borg has, and Borg does not have everything that Kubernetes does. Kubernetes is not a clone of Borg - it is inspired by the lessons and experiences we got by doing Borg. Often that means we learned how NOT to do something. If we can get 75% as functional as Borg in 1/3 the time, we'll be doing pretty darn well. I think we're on track for that.
      - fweespeech 11y ago
        > If we can get 75% as functional as Borg in 1/3 the time, we'll be doing pretty darn well. I think we're on track for that. I'm sure you can do that. That doesn't invalidate my statement that it isn't surprising Google would do this precisely because it'll only be 75% as functional as Borg at any given point in time.

amelius 11y ago

Any references to the publications containing the real blueprint?

kalgen 11y ago
Borg: http://research.google.com/pubs/pub43438.html http://research.google.com/pubs/pub43438.html Omega: http://research.google.com/pubs/pub41684.html http://research.google.com/pubs/pub41684.html

dkarapetyan 11y ago

Google open source projects in general are just shitty. The reason is that they don't open source the whole thing and the code ends up being full of assumptions that don't hold because you are either not running things at google scale or are missing certain key bits of the "secret sauce". In any event google compute is a terrible user experience compared to the likes of AWS and other cloud providers. Heck, even the shittiest VPS providers tend to be better than google compute. So open sourcing their "secret sauce" as the article puts is still missing key bits so I don't know how many people actually fall for the good will part.

thockingoog 11y ago
What key bits do you think are missing from Kubernetes? I'm sorry you seem to have had a bad experience with GCE, but please know that Kubernetes runs on several other clouds, too, with no crippleware or anything. It is 100% open.
- viklas 11y ago
  And 100% on-track for awesome. Yes, sometimes development/testing for new kubernetes features 'feels' like it's focussed-first on GCE functionality (before other platforms) and earlier on, it had some hooks that weren't great (like GCE-only external load balancers and storage). But hey, it's not even v1.0 yet - and all those things are either fixed or being worked on already. And as a non-GCE user, you aren't a second-class citizen. It works everywhere. We've deployed successfully in AWS, vagrant and bare-metal (in the garage), so far. All with 'one-command' automated deployment and re-use of our pod & service specs throughout. Roadmap/Architecture-wise, it would be good to see a more 'pluggable' approach for 3rd party integration (more like an Open Stack model), but again, we're still pre-v1.0... Also, I think the google-folk here are being very 'reasonable' in their replies. Your comment was mis-directed & ill-informed. Go do some reading or watch Kelsey Hightower's presentation from a couple of months ago: http://chariotsolutions.com/screencast/philly-ete-2015-16-kelsey-hightower-managing-containers-at-scale-with-coreos-and-kubernetes/ http://chariotsolutions.com/screencast/philly-ete-2015-16-ke...
cmcluck 11y ago

degenerate 11y ago

This has nothing to do with the article, but I absolutely hate the way wired underlines their article links. It might fit with their magazine style design but it's so incredibly distracting. I always want to bump it 1px up or down!

bsaul 11y ago

Is it me or does anyone else interprets all the recent wave of open sourcing as a trend to not create open standards anymore, but instead open source the technology altogether (without even creating the standard). Have standards failed ?

jsprogrammer 11y ago
Why would you want your code private? Instead, let anyone work on it and benefit.
- mbreese 11y ago
  The point of open protocols and standards isn't to keep the code private, there are plenty of open source SMTP or IMAP servers.
VikingCoder 11y ago
Standards are great when multiple companies want to provide the same service, which must communicate with software written by other companies. Compiler toolchain, devops, sysadmin, languages, runtimes, etc don't really fit that picture.
mateuszf 11y ago
This is a very good debate regarding this very topic: https://www.youtube.com/watch?v=kRVWjC6osuw https://www.youtube.com/watch?v=kRVWjC6osuw TLDR: we are in a stage where we don't know yet what functionality needs to be supported, it is not a good time to form standards, it is a time for checking possibilities and finding best technical solutions.
obstinate 11y ago
My 2c, not speaking as a Google rep but as myself, is that if you have a problem only a single actor needs to solve, you're going to end up with software, not a standard. To get a standard, you have to have a situation where the value of cooperation is higher than the cost. There are plenty of people now who need to solve the container problem, but Googlers been working on this shit for years, before it was really on anyone else's horizon. Google employees incepted the cgroup feature way back in 2006, to solve problems that were already being felt acutely at that time within Google. Folks have been working on this stuff a long time before it mattered to anyone else, and that's why what's coming out is software rather than standards. There is no way a big company is going to delay solutions to an urgent strategic problem in order to be part of a democratic process for the sake of a few people's ideals. Maybe if they'd seen it coming five or ten years in advance, to give enough time for the standardization process to occur, but Google was far too small and the future far too uncertain in 2001 to predict what might be needed in 2006.

justinsb 11y ago

I've been working with Kubernetes a fair bit (I'm contributing to porting it to AWS). It's an exciting time: I believe that we'll be running everything in containers within the next few years, and Kuberentes solves some of the big missing pieces with actually running Docker in production across a cluster. There are other competing systems trying to do the same thing (including Docker Inc itself), but I think Kubernetes has a huge advantage by having years of experience of what did and didn't work well in Borg. Most importantly of all though, I have found it a great project to use, and a really great project to contribute to. I think that contributions are the lifeblood of open-source projects, and I give Kubernetes 10/10 for their community and processes, which I think bodes very well for the future.

digitalzombie 11y ago

Kubernetes is alpha software right now. I was at a meet up last month and there are a lot of missing features from what I recall. They're planning to fix one of these feature with some project call Ubernetes. While people are raving about containers, there are still security issues with containers no? I think VM will be here to stay for a long while and while we might have to pay a performance and memory hit for them, they offer better isolation.

4ad 11y ago
> there are still security issues with containers no? In Linux, perhaps. However FreeBSD jails and illumos zones are rock-solid. There's this crazy hype around containers these days and people just ignore the stable, secure, and tried technology, I don't understand it at all! FreeBSD and illumos are not Linux, but their still Unix-like, it's not like you'd have to use OpenVMS. Plus you'd get other benefits too, like DTrace and ZFS. And on illumos now you can even run Linux binaries in a zone. So why do people simply pretend these secure technologies don't exist? Can someone explain?
- mateuszf 11y ago
  Because there are lot more Linux users / developers than BSD.
  - 4ad 11y ago
    I don't think that "X has more users than Y" is a valid complete argument in itself, otherwise nothing would ever change. Of course it matters in the grand scheme of things, but the causal relationship is more complex, involves many degrees of freedom and goes both ways. Before docker came, Linux had LXC, which wasn't as popular as docker is now, but it was certainly known and used by people. So when docker came, LXC had more users than docker, and yet docker surpassed LXC in popularity in weeks, so the "X has more users than Y" state can be changed by various factors and it's not enough to keep the system in equilibrium. So yes, the fact that Linux is more widely used than FreeBSD and illumos in the developer community is certainly an important factor, but I don't see anyone ever saying "FreeBSD is great but we want to use something supported by a larger community", or "illumos is great, but we don't have expertise with it", which are certainly important arguments to consider when making a decision. But I hardly see anyone making these arguments, or any other arguments really. It's like these systems don't even exist. At first I attributed this to the "X has more users than Y" factor, but then I see people having particular problems with Linux container technology, in areas such as security, virtual networking, etc. And these are problems already solved by FreeBSD and illumos. Surely when you have a problem you look for alternative solutions that don't have these problems? But I don't see people looking over the alternatives at all. As I said, there are many valid reasons not to use these other technologies, but I am very perplexed that people refuse to even acknowledge the existence of them. And now that illumos can run Linux inside a zone (and FreeBSD did this too 15 years ago, and still does for 32-bit binaries, I believe work is well underway to extend this to 64-bit as well), I think the "I only know Linux" argument loses some potency, you can run Linux after all...