February 10, 2016

New York

Join the Container Revolution: The Cloud Native Computing Foundation

Okay, so I may be the only human on the planet that connects those two things, so guys, the Cloud Native Computing Foundation—it kind of kills me that I'm excited about meetings, and about these kind of pan-industrial meetings. I don't know what's wrong with me. Am I the only one? Is the CNCF exciting? Am I legitimately excited about this? You wanna kick this off ? >> Yeah, sure sure.

I think the answer is you're the only one >> [LAUGH] >> Okay that's fair >> But no, no I think that with the amount of stuff that's happening in the industry right now in terms of how people are trying to figure out how they're going to solve their problems, I think that there's a lot of open questions, there's a lot of parts to this right? The way we think about it at Mesosphere is the datacenter operating system which means its more than just scheduling, it's more than just monitoring, more than just resource allocation—there's a lot of different components, and I think to figure all those pieces out we really need to come together as an industry, as a community, and figure out what are the APIs between all these components, how do they interface, how do they work together, and that's why I think the CNCF is gonna be a ton of fun.

>> Clayton do you share the excitement? >> I do, I think one of the challenges that's going on right now has been that there's a huge amount of technology, and people are making these transitions from having very tall stacks in the datacenter in the enterprise, systems of record is a great example from the first talk—big complex systems that are suddenly, we're gonna break these all apart, move them all across the datacenter.

We're gonna move them under the covers, you're gonna need persistent storage, but don't worry it's gonna be hooked up over the network it's gonna be magic. >> Like what Matt said you're gonna rip them down recreate them again it's like don't worry whatever you build you're gonna destroy.

You're not gonna loose anything it's gonna be fine. And I think that concern like how people start building distributed systems this is what we're talking about right, that's what cloud native means is you're a distributed system and you have to deal with what it means to be a distributed system.

Building a community and sponsoring the sorts of things in the community that enable people and organizations and companies, enterprises and open source contributors to build those correctly is hugely important like this falls over unless we all know how to make the next steps.

>> So how does it fall over? This may be my fixation on failure modes but so how does it fall over and how do we think that CNCF might be able to address those failure modes? >> Well I think everybody out there building the software is trying to build pieces that help to deal with that.

So people build redundant data stores so you know CoreOS with SED, the idea of redundant data stores, the idea of building something that survives a failure is really important and and then how do you use that predictably from client applications, so Zookeeper some of the other consensus space configuration stores that lets you do this real time stuff.

There's a whole bunch of patterns around that, that people have already built up, and I think the job of the CNCF is to help try to formalize, guide, and put names and labels on some of these things, so that people know what to look for, to know what to find, to hook them together.

>> So inventing kinda of way of talking about this. Jake, is that your take as well? Does that make sense? >> Yeah I think really a lot of people have a really strong vision about what this stuff should look like. And it's really about just, like you said, giving them names to say like we are defining the software-defined networking piece, or this is the scheduler piece so these are all the pieces, these are how they talk together because without that, you will end up being able to cobble a stack together out of these components, it's just wasting the smallest amount of effort and getting there the fastest with the most cogent picture of what the final vision actually looks like.

>> Mathew? As the British guy, by the way, I'm the baffled and scared guy. I had no idea who you were talking about before, and I didn't know who was in the Super Bowl until a few days beforehand >> [LAUGH] >> So I feel a little bit—I'm the least technical person on this panel, I think the Cloud Native Computing Foundation has an opportunity to make things simpler.

A lot of things we talked about earlier on about wanting to adopt software because you wanna go faster. But it being very confusing, and it's confusing because there are lots of vendors like us out there right, what do these guys do, and all of that kind of stuff. I think the CNCF can be good if it's a place for code, but it can be the best if it's also a way to make this all go faster, make it simpler, make it easier for everybody to figure out.

Okay so here is how I do this, here's some paterns that I can use, this is where this fits and I think that's a great opportunity for the CNCF if we can get it right. >> Yeah I think so and I think of course and as you say like vendors, we're making it worse, right we're like don't worry, trust us, we're gonna make it better.

>> Yeah. >> But it is true though I think because we do have, I think you mentioned that everyone's kinda got their way of doing it, and some of the opinions really overlap and some of them really diverge in ways we don't even articulate, or we don't wanna talk about. And I think that one of the challenges is that we are creating a lot of confusion.

And of course everyone is like "oh I know how to make everything go faster. Everyone uses my thing." >> [LAUGH] >> It's like okay yes and by the way like everyone using my pick would actually make it go faster but the>> [LAUGH] >> Okay so I mean, you could definitely do that I don't wanna take that option off the table but I think the reality is tha— and if you listened to what Matt was saying about, what I heard was >> Yeah>> You're gonna be picking different tools for different jobs.

You're gonna be re-visiting those decisions you gotta be ready to be like, we did this thing and turns out that thing might be a good fit for this kind of a problem but we may need to switch it, and I think we need to be able to kinda constantly re-address that and plug and play different things.

>> Can we do a quick audience poll? Out of all of you folks who were adopting Docker (I saw you put your hands up earlier on) who's gonna just adopt the one thing from one vendor? Show your hands. Right no one. So that's a real problem the CNCF should be able to address if we do our job well.

>> If we do our job well, yeah I agree. And for me and I've been saying this for a while, but for me the model is the IETF >> Cuz you and I are old enough to remember that, right? >> [LAUGH] >> Although apparently according to Dave, I can talk about history and it's not creepy I guess, that's great.

This is the advantage of looking like I'm 12 still. >> [LAUGH] >> I keep waiting for puberty to roll in, I must be the only prepubescent man who has actually procreated >> [LAUGH] >> Did I say that or think it?. OkayI'm sorry. >> LAUGH] >> So let's get off of pubescence. So how do I actually do it? This is great.

So we all agree like hey we should allow, we should define these interfaces, define some of these places, let's actually get dirty and how do we actually do it? >> So first going through the process, >> First you have to have an election >> Have that election, yup, still more governance to be done, so we gotta to get that stuff all taken care off.

And then I think we go into this with two perspectives. One is sort of standards and interfaces and I think that's kind of IETF perspective, and then I think that's going to be really valuable. I would love to see us figure out the VFS for the datacenter, whatever that means, and I'd love for us to figure out what some of these interfaces, POSIX-like interfaces look like, but I think to actually >> [CROSSTALK] >> Exactly, exactly.

Even if you only use one implementation of it in your organization, that's fine. The fact that interface still exists gives the flexibility for organizations, for the industry to actually evolve and to innovate. And I think the other thing that I would love to see in CNCF which to me would really make it be successful, to answer your question, is to also have some component code, to also have some component of whether it's a reference stack or yeah, I am not sure exactly what it is, but to have something that can also stand on its own that people can use.

So serving like an Apache-like function in addition to an IETF-like function. >> Yeah. Yeah. >> So is this the love child between Apache and IETF? >> [LAUGH] It could be. >> They get together for a night of passion and they—I mean…>> [LAUGH] I mean Linux and Apache are very different—Linux Foundation and Apache are extremely different.

>> Oh it would never work. It's a one night stand, they would obviously never work. >> [LAUGH] So they'd have to. >> [CROSSTALK] >> One's for individuals, one's for corporations. I actually think if you could make that child. >> [LAUGH] >> It's the original odd couple >> [LAUGH] >> Help us.

>> And I think the interesting thing that I think of too is that one of the points that came up at the December meeting of the CNCF was, there's a lot of concern when we say standards, right? Someone who was there brought up the, oh, great now we're gonna go through SOA again and this will be just like WSDL and SOAP the first time around for all the real graybeards in the audience.

>> God, no. >> Yeah exactly and I think there's a very legitimate concern when we talk about, what's the role of a foundation coming in and imposing standards probably doesn't help anybody. And all these vendors fighting or disagreeing about what the best option is, but there is the opportunity to find the places of similarity and to work towards common similarities. Where the community benefits is, people coming together around the idea of the standards and interface, and whether they are hard interfaces or soft interfaces or formal standards or informal standards.

>> I'm wondering is the fact that it's all open, changes things a bit, because I mean you look at a decade ago with web services, you're dealing with I think of a bunch of proprietary things, you think the fact that all the stuff is open changes the game? >> I think that's a great point is that, you can't actually stop someone from going and creating something better than what you have today, and I think that's actually what we're seeing now, maybe compared to 10 or 20 years ago, you have this enormous booming open source, right.

The number of projects on github daily is exploding, it's an exponential curve up into the right. And that idea that someone can go out and take the best technology ideas, take papers coming out of Google or take ideas that have worked in the private enterprise and turning those into an open source of project and then building something better to help distributed systems is happening in a hundred ways on every open source project today. There's no way to stop that.

It's all about trying to incubate and grow those so that they can work together, and you can help people find how those can mix and match. >> So a decade ago standards bodies were kind of serving almost a gating function, like you can't do this until you you've done that, whereas now as you say everyone is just gonna do it anyway and it's more of a guidance function. Jake, does that makes sense to you? >> Yeah.

I was gonna say, as someone who has been using open source software for my entire professional career, I was a little bit confused how we were gonna impose anything and anybody, right? So the idea is we really have to deliver value for people to pick up any of these stuff, and somebody who's actually writing software everyday—I think the really interesting part is when software does get contributed and when the foundation does end up owning pieces of software, how does that affect the choice for those components of the infrastructure, and how does that affect the API's that get picked? Do the API choices of those contributed pieces of software end up becoming de facto standards.

And is that something we wanna embrace or avoid? >> Yeah. Just to add that I think that's why the CNCF also has to have a code component. Cuz if there's not a code component— >> If it's just preaching from the pulpit, it's not enough. >> Yeah. And the new standards are the ones that people naturally adopt because they make the most sense.

People actually implement them. I think a good example is CNI right now, I think Clay just did CNI work. We'll probably do some CNI work in Mesos, and that did not come from a foundation but I think what the CNCF can do is tell the community, "hey this where people are going, this is why people are going this way, maybe it's not standard but it is something that you should be looking at very, very closely and its adopted by these people and there's actually code behind this >> And these four things work with these five other things and all of these nine projects have made a commitment to work with one another even where they may be alternatives.

>> Yeah I like I like the IETF in the early days the IETF motto was "rough consensus and running code." >> Right the Dave Clark Seminar. >> Right exactly, Dave Clark and he looked at the way Internet mail evolved, sendmail was the reference implementation for RFC822 for Internet email.

So everybody had a reference implementation to work off, but over time other things came along and now unless you rely on that kind of thing, nobody uses sendmail and we use something that is much easier to configure. But it was a great way to really drive the industry forward and it worked in a way at the time the alternative to the IETF was all these standards bodies you know, and that's what got you things like GSM—a lot of the telephony standards and it was incredibly painful, and they would do crazy things like come up with an encryption standard that was broken and everybody had that in their phone you couldn't possibly change it right? And the IETF model was able to get all these vendors—and a lot of whom really didn't like each other—into a room and have them agree on a common set of interfaces and APIs and I think if we can achieve that in CNCF, which I hope that's what you were talking about, that kind of thing.

You're gonna have code that's in the CNCF that's part of it, it's officially a CNCF project but it doesn't exclude alternatives. I think that is gonna make it successful. >> The early history of RFCs is very interesting. The RFCs were actually designed—now we think of RFCs of being basically a spec, but that really was not the intent.

The intent was that there is a bunch of thinking in these that at the time totally proprietary organizations. That we want to actually get out there while it's not fully formed that the whole point is to actually have this discussion so we are not kind of not presenting fait accompli—we're not kind of presenting these finished artifacts but we are presenting the way we're think about the problem. Which I think actually, to a certain degree—this is where the openness almost counts against us.

Cuz like, oh we know, we discussed that on GitHub issue 8 billion. >> [LAUGH] >> I was like didn't you get on that GitHub issue? He was like I tried to but then my browser crashed because there were so many comments on it—and I think sometimes there's so much mayhem that it can be very hard to kind of—I mean what do you guys think? I mean CNCF can be still something.

>> I think that's great a great segue as were into the idea of incubation. So that we talk about what is the role of Apache today? To incubate communities. And I think one of the things we're talking about here is we're talking about the distributed systems community. We want to incubate the ideas, the technologies, the parts and at least some level of interop between the different parts of what you need in a system.

I don't think anybody in this room would probably disagree that you can look at some of these technologies out there, and be like yeah these are roughly alike. And trying to find those points of commonality, to find incubation. To incubate the technology projects that are struggling for reason, open source communities that can benefit from a little bit of assistance in terms of community building, and ensuring that there's some idea of trying to build these things in a better way will help those communities become stronger and the benefit of having some of the experience on the CNCF already is many of these people have technologies that fit at different levels on these stacks and trying to find well we don't really have a technology in this particular area, what are the open source projects that really represent the thing that is needed to support customers and employers in their actual day to day.

>> So one thing I wonder is that do you think that we will want to have a middle ground between a CNCF Apache-like Incubator Project and just a non-CNCF related project? Are we gonna wanna have a CNCF-aspiring project are we going to are we going to have CNCF gold, silver and bronze God I hope not.

>> I think that is actually defined in the governance isn't it? There are officially donated pieces of software and then like various levels of association in the CNCF >> Okay so then we get to actually you bring up a very - >>Oops >> [CROSSTALK] >> well also I think that one of the criticisms of the kind of neo-foundations—open source foundations—a lot of them are pay to play.

And CNCF is really no exception about it. What do you guys think about pay to play? Well let me give you that hot potato. >> Well speaking as a small company that didn't pay to play, I was elected to the governing board by the silver members so you know we all paid to be in the silver tier but if you look at platinum and governing board.

At the same time you've also got the pragmatic thing, where you've got to get a foundation funded right, so you're not just going to do work, you need to pay for things, you need to pay for staff to run the foundation so I think it goes with the territory, pay to play is a good way to get the thing funded but you've got to come up with a governance model that works for everyone, not just the people who pay to play and I think that is the trick >> Yeah, does anyone want to take this one, I mean.

>> [LAUGH] So I mean, I come from Apache right? >> [LAUGH] >> Very very very different model. Apache is for the individual, driven by the individuals, obviously there's a lot of organizations that contribute to Apache, in sort of an organization perspective but when you really think about it at the end of the day, who Apache really is empowering is an individual who's writing code that compiles, that runs and then they can contribute to the project.

And I think one interesting take on a direction that we can think about taking something like CNCF or foundations of this nature is to a realm where we do kind of marry in some of those concepts of individuals contributing code to a body of technology, a body of software whatever it is that represents the pieces of the foundation and I think that can be really interesting.

I think I've chatted with you about this. I think it would be very interesting if some of the money that went to pay to play, actually paid for engineers >> Yes. >> that worked on that project, that were working the technology. They have no third party vendors in the way of manipulating their perspectives.

They get to be the Linus of Linux, working for the Linux Foundation, just contributing to the technology. And like everything, we're constantly trying to evolve, but we do and I think that includes foundations. >> Yeah I kinda share your feeling but I'm uncomfortable with some of the pay to playness.

But Matthew I hear what your saying but we need to be able to kinda get lift off. I just think it's incumbent upon us to make sure that we are pumping that money back into the open-source communities. I wanna see that like, if you spend it on meetups and you spend it on the people that are working the stuff on weekends and boring their spouses with it because they're so enthusiastic about it.

I mean that that leads me where, I don't know Clay, what's your take? >> I think I agree one of the concerns that I think a lot of people have with open source in general is who supports open source? I mean I've been working with Red Hat, that's something that we talk about a lot, but you can't support everything in the world, so there certainly comes a point where you've to say, okay well if we wanna make this the foundation of a distributed system that were building, we are gonna redeploy our entire infrastructure on technology A or technology B, how do we know that we can trust that it will still be around and I think there's a very valuable role that Apache and other organisations in the past have played around that, like the Linux model in a sense benefited not just from big corporate backers who were willing to spend time and engineering dollars, but also building a mutual ground for individuals and other contributors and the money, in some respects on these pay to play foundations, I agree with Ben absolutely should be used to benefit the communities because ultimately we want this piece of technology to get better and stay around.

It doesn't know any good when big chunks of their infrastructure drop out of site, and there's no guarantee in today's open source economy that those projects will stay around, especially one's that might be starter funded or come from contributors who are doing this as a day job, there's a lot of more uncertainty in some of the newer technologies, so I think there's value in creating a a stable brand and a stable incubation process.

>> On the one hand it is all open you do kind of wonder especially with the prolonged GitHub outage, if they're like, sorry guys we had this firmware bug in this RAID controller that we shouldn't have bought and it's all gone, all GitHub is gone, we're sorry, it's like you've got other copies of this right, you just wonder ho many people would be like, well I'm dead I'm dead.

That was my only copy, but we should't take it for granted the kind of longevity of open source. And I think the role of the Cloud Native Computing Foundation should be to try to set up that kind of long term vision for five years, ten years out, what are the open source technologies that support and donation is one path, incubation is another path, engineering support kinda like ad hoc engineering support is another great way some of these projects can thrive and succeed until they find enough community purchase to be self sustaining.

>> Yeah I try to stay away from the money stuff as much as possible, but you can think of it as if some of these less sexy pieces that do get donated or do find a home in CNCF projects, if the money could be used to build momentum or keep momentum going on some of those projects I think that's a really benevolent application of pay to play.

I think in that case you can deliver more value than you're taking from the contributers. >> Right. If you're viewing it as opportunity then how can we now take these resources? And actually there's a very important resource that we haven't talked about that I'm personally am incredibly excited about and that is this cluster that Intel has donated. Intel has donated the cluster, SuperNap has donated the datacenter space and the power which is very significant. So this is 500 machines, and I think in the world of containers you're like 500, like that's not a big number, it's like no, no, no, no jackass.

That's actually a big number, like 500 actual physical machines, 500 to use, like that's a lot of compute power. >> And that's to get to the individual, how many individuals today have access to 500 machines, and not just 500 machines, 500 very nice machines. >> These aren't iPhone caliber PCs.

>> How many people actually know the amount of DRAM in a 2U. People are like, I know the Amazon instance sizes, it's like okay, news flash, there's half a terabyte of DRAM in a 2U and we have 500 of those. So to me, one of the things I'm excited about it's just for you to say about like how many open source developers have access to this stuff it's like, yeah I will see your 64 gig instance size and raise you a little bit. Why don't you go take even 10 of these boxes—I mean that's a tremendous amount of DRAM, CPU, network—and let's go play with these things. Lets play with them at scale, let's go beat on them.

So I'm very excited about this. >> Yeah, one of the criticisms actually about open source and talking to the Facebook folks—Facebook infrastructure folks said they use a lot of open source because they run into scale issues, because a billion people show up every day to Facebook. And that is a big challenge for low open source, is that everybody can contribute code but actually doing that kind of testing is super important. That and we should be very grateful to both what SuperNAP and Intel have done.

>> Well and I think with people and kinda open source projects are like, okay I get it, on my laptop I'll create like 10,000 of these, that's not what we meant, that's not what we meant by scale so I think it's gonna be—I'm very excited about that cluster. >> So there is one really fun irony of all of that though which is on the call a couple of weeks ago when we were chatting about how people would use this cluster, there were some humans that were doing manual scheduling which machines go to which…>> [LAUGH] >> I hate to take you inside the sausage factor. >> [LAUGH] >> but it's like, what is this IPMI? It's like yes I'm sorry Welcome to- >> [CROSSTALK] >> And actually in all honesty you're actually dealing with a software substrate that's still proprietary, you're dealing with IPMI, you're dealing with firmware—firmware is the enemy by the way, firmware is trying to hinder forward progress. >> [INAUDIBLE] >> That is the body of software that is not open so you've got no idea, you actually have no idea. >> Oh come on.

>> [LAUGH] >> It is so awful. It is so awful >> One step at a time and I think that we need to take these steps like we have these distributed technologies that maybe a few enterprises have tested to destruct. And it's always one of those cases which is you never really know how a distributed system works until you put it in the real world and then you start to abuse it.

And I think that sort of, we will get—that kind of contribution enables and we wanna look for more opportunities, to enable those sorts of, to give individuals open source projects, the opportunities to really blossom in spaces they can't today through funding or access to these sorts of high performance environments.

>> Even if the scale was not from a performance perspective but from a management perspective, it's like how many things are gonna deal with the IPMI, be able to auto config on a bunch of machines. >> That experience alone for everybody working in the CNCF, working with these machines is gonna be really important.

>> And remind people, you need to be reminded about what actual hardware looks like. Jake are you as excited about the hardware as I am? >> I'm actually kinda terrified. >> [LAUGH] >> Are we even gonna be able to come to consensus about how these 500 machines should be networked together? That's a pretty contentious issue even among different organizations that operate at scale.

>> Actually that's a very good point, I think we haven't figured this stuff out, but I think the intent is that this will be allocated in blocks of 10. So you don't have to make one decision about how these things are networked together because, yeah. And then of course you have all of the kind of legacy vendors, they've got their own opinion about how this stuff should be done, and yeah of course we should be using your thing. We get it, but yeah, there's lots of potential.

>> This is a purely selfish angle on this when you think about like what it is that we're actually trying to do. Like in the container space and distribute systems and orchestration, there's more open source databases today than ever before. There's a new open source database every week, they specialize, they do things differently, we've got systems that can survive failures across datacenters, we're building software-defined networking—There's a ton of extremely extremely exciting stuff. And so from a purely selfish perspective I think one of the things that the CNCF should try and do is convey that excitement to everyone else.

Which is, here's what you should be excited about if you don't know already, here is how you can find these things, see how they work for you. And I don't know that it's called like a taste maker or a fashion picker, but the idea of trying to help people find the things that will transform, everybody is doing this in different ways, the vendors are doing this, bloggers are doing this, but the CNCF can really form a role and say like, this is something you guys should know about because it can actually enable you to go to that next level.

>> Well, and even just letting people know like, here are 5 different things, I mean because even the different things might know about one another in some cases. Just helping people understand what's out there. I do think we're kind of in an era where there's a lot of confusion in terms of open source, it's hard to navigate.

And it's been hard, you guys mentioned this too about how, I've been doing open source my entire career more or less, but the last five years have changed for open source. It has gone non-linear, I mean, Matthew is that uh— >> No I was just thinking about for folks in the audience, can we do do a quick straw poll? So if we're gonna take on this role of communicating these things out, making it easier to find and figure out.

So let's come up with different ways of doing that, and so this is one of the questions questions that came up in the last CNCF call I was on. So for you folks out there, we're gonna give you a couple of different options. So if the CNCF did a whole bunch of essentially technical content about how to build cloud native applications, what does that look like architecture things like that, would that be interesting to guys? Let's take a look.

Okay, it's good. All right, second example, if the CNCF ran an event, would you go to another event on containers? Okay, all right. That's pretty good, and then the third part part is, if there was the CNCF built reference applications for containers and cloud native applications, would you use those? Okay, interesting. >> So sounds like what I saw is, looks like case studies, best practices, and then followed by the conference.

>> Interesting. >> Yeah, so this is actually really helpful for us to know because a conference is really expensive to pull off. And a bunch of us wonder, it's like for the price of that conference, remind me how many technical writers we can engage? >> [LAUGH] >> So I would say that our bias, maybe because we all suffer from conference fatigue, but if that's wrong, definitely let us know because this ultimately it really needs to be all of our CNCF.

I mean to me, the problem we need to solve is the problem that Matt's got of actually trying to navigate the chaos and build stuff and rebuild stuff and deploy quickly and that's what we should be judged on and held accountable by. >> Any other closing thoughts? Will we look back on this as an era of just ridiculous optimism after an acrimonious divorce when we've all declared unconditional war on each other, or will we view this as the dawn of a new era? >> [LAUGH] >> It has to be one extreme or the other. >> [LAUGH] >> Anyway guys thank you very much I'm very excited, I think we are lucky to have so many technologists that despite their own technology, despite their own angle, have really an uphand industry view of this so I feel lucky to have gotten to know you guys and I'm looking forward to working with you.

Thank you guys. >> Thank you.

Speakers:

Ben Hindman: Founder, Mesosphere

Jake Moshenko: Founder of Quay, CoreOS

Clayton Coleman: Lead OpenShift Engineer, Red Hat

Mathew Lodge: COO, Weaveworks

Moderator:

Bryan Cantrill: CTO, Joyent