A New Era: Containers in the Enterprise

The Future of Containers in the Enterprise

February 10, 2016

New York

A New Era: Containers in the Enterprise

Hi everybody, I assume I'm on. I was just telling Bryan about Avenue Q, if you haven't seen it, my favorite two songs are Everyone is a Little Bit Racist and the Internet is For Porn, which—it really is like adult Sesame Street, and I'm an ex New Yorker so it's nice to be home.

And in fact I lived, when I left New York, I lived across the street, at 305 West 50th. So to get my mojo back now that I live in Northern California, and have completely wimped out and can't do anything anymore, tough. I stood on the street corner in front of my old building, and just yelled at tourists, and told them this street belonged to me, and then I pushed them down an took their cab.

I took their cab and said welcome back to New York. I don't know if we're going anywhere yet. So the best part about giving the kickoff today is that every presentation that comes after me gets smarter and smarter so, you won't even remember if I'm not smart enough to tell you everything about the Container Evolution.

But I'm gonna try and set it in enterprise perspective. For those of you I don't know, or who I haven't talked to, I've been with Forester about seven years, before that I built and bought software in every technology era for the last 100 years. Starting out actually on Wall Street and with Tibco, for a decade in the 90s.

So, we were all about building real time messaging middleware systems, the containers of the 90s, then focussed all on virtualization management, so I was trying to build tools to help manage hundreds or thousands of VMs back when people only had about 30 of them. So we were a little early to the game, but it's always been about moving faster.

I work on our infrastructure and operations team and write for what's called a Cloud Playbook. So we spend most of our time talking to enterprises who are making the transition from highly virtualized datacenters to some sort of cloudy future, and that goes down multiple paths.

What I did is I just grouped this into three quick sections. What do we mean by digital transformation in 2016? Who's doing it, and why are they? How does the three main evolutions that we're tracking at the same time, cloud platforms, devops processes and containers tie together in the enterprise, how real is it today and how soon do I think it's gonna become even more real in the enterprise? Analysts love to name things and especially eras, define eras.

Forester has labeled the era that we are in now as the age of the customer, and really it's distinct from the information age. And in the the information age, technologists like all of us in this room who own the information own the technology, and the democratization of mobile and cloud technology means that we don't own it any more, the customers own it. And the companies that are transforming to succeed in this age of the customer are the ones that are shifting investment from datacenter efficiency spending to customer experience spending.

That means focusing more and more of their investment dollars on applications and experiences that directly win, serve, and retain customers. And so what it means is because of the ubiquity of cloud technologies, we are changing three major things at once, the infrastructure under the applications, development styles (how we build them), and application architectures, how we design them.

For the past seven or eight years especially to traditional infrastructure and operations folks, cloud has quite often seemed like existential competition, and I think we started exiting that phase in 2013, 2014. More and more companies we talk to don't see cloud now as an existential threat, they see it as a set of innovation tools that let them decide whether they are gonna be digital predators or digital prey.

And I think in this room based on some of the companies that I saw signed up here, we have a mix. People who are disruptors in this particular industry and start building things with containers and devops practices and build on cloud platforms by default and have never thought of anything else.

But also existing enterprises that are creating transformation, digital disruption teams within their own organisations, using the same tools. So rather than cloud being something competitive to IT, cloud is a set of tools you can use to do some preying yourself on other people in your industry.

What we are really see is it's a lot of things going on at once, but there's three evolutions in the data center are all tied together. We've obviously spent the last ten years virtualizing the hell out of everything in the data center, and it was really the fact that the enterprise got comfortable with virtualization that allowed public cloud to happen.

If you can virtualize something, it's not that great a leap to let someone else run your virtual machine for you. And now pushing even deeper, the fundamental difference with containers is it's not being driven by datacenter efficiency. IT teams are not bringing containers into the enterprise.

Developers are bringing containers in, and in fact in the really early early days of virtualization, developers brought that in too. Back in the early 2000s, really early, 16 years ago now, developers were bringing virtual machines into enterprises to run multiple things under their desk because they only had one box.

So it's the same thing again. Developers, and if you're a developer, thank you, you drive everything. Development styles are changing obviously from traditional waterfall, more agile, through whatever you want to define DevOps in your organization, but it's tearing down that artificial wall between dev and operations.

Actually it's been a real wall for a long time, so tearing down a real wall. And then application architectures, and this is probably where we are the least mature in the enterprise. This shifting from large monolithic apps that have evolved into multi-tiered and now we're going towards microservices.

Containers can accelerate all three of these. You don't need containers to do any of this, but using containers, it's gonna go a lot faster. And the reason that I think we can do three different things at once like this, is because of the ubiquity of now cloud infrastructure platforms, both on premises and in public clouds.

So I thought I'd start by saying what does it look like for cloud infrastructure, what does it look like for DevOps in the enterprise, and what does it look like for containers. So the current state, if you wanna solve a crime on Law and Order, just follow the money or if you wanna figure out who's gonna be president, just follow the money.

And if you wanna figure out what's happening in cloud, follow the apps. Enterprises are shedding applications faster than we even predicted they were a couple of years ago. So the data on the left, we run huge surveys, about seven or eight of them every year, to different types of I&O buyers.

Forester is very role-based, so we field surveys to infrastructure and operations teams, to developers, to software buyers, and to budget holders, executives. Because we do that, we often get very different views of what real adoption is of a technology. If you ask software buyers how much SaaS a company is using, you get a radically different view of it than if you ask I&O professionals how much SaaS their company is using. This was to our infrastructure survey, to I&O teams, and about 65% of all the apps they were running at the end of 2014, were on traditional hosting, or on-premise deployments.

The rest, a mixture of SaaS on some sort of private cloud infrastructure or public cloud. Two years, fast-forward, the end of 2016, the prediction is 52% left in the datacenter. 48% of apps moving, so by the beginning of 2017, and this trend has been accelerating for the past 2 and a half years, we're gonna have more than 50% of apps run outside of sort of a traditional virtualized datacenter environment or hosting platform.

That's sort of a tipping point. Once we got to 50% virtualization, we never looked back. We're not gonna un-virtualize a huge swath of applications anymore, and we're not gonna move a swath of applications back into a traditional datacenter environment. It's not gonna happen. Then we ask about cloud deployment styles and multi-cloud is finally growing, vendors have been talking about it for a long time, but multi- cloud in the enterprise is finally becoming real.

What I did, is that I grabbed data just for North America here, and yes, we know that Mexico is in North America, I apologize for that. For the diagram that just shows Canada and US. It's like we call it the Trump Chart, it says [LAUGH] Mexico is not in North America,but what we've been tracking every year, is the 3 most primary deployment types.

On premises private, off premises hosted private. So someone else probably runs it for you but it's still dedicated and then pure, shared, pay for use public cloud. 26% for public cloud in North America doesn't seem very high, it was 10% three years ago. The big myth of public cloud computing, is enterprises are all using Amazon, enterprises were all using Azure.

The growth is incredible, but actual adoption by enterprise production infrastructure and operations teams has been much lower over the years than any of the cloud providers would have you say. That's changing. What's significant is what's been changing, is private clouds. Private clouds are really growing up if you asked and we asked about private cloud three or four years ago, we generally found out that it was a slightly faster VM vending machine.

Not a lot of developer self service, not a lot of real automation, not a lot of integration with dev processes so people were calling whatever they were doing with their VMWare environment a private cloud, because vendors were telling them to do that, and marketing to them that anything you did with the virtual machine in your premises you could call a private cloud.

Private cloud is really growing up, and so this adoption is real. We asked specifically about self-service, developer access, and fully automated provisioning. So we've got 40% saying "I've got my own private in place" with over 50 in development by the end of the year. Hosted private 30, public 26, that's big it'll be 33 by the end of this year, so public is growing faster in terms of rate than the other two.

What it means is that this hybrid cloud is not some future state. You don't need to go buy a hybrid cloud. You've got one, if you're using any cloud. Virtually nobody calls Forester and says I need a hybrid cloud solution, who can sell me a hybrid cloud solution? They call and say I wanna use the public cloud and that means I've got a hybrid problem, a hybrid data problem, a hybrid app problem, a hybrid management problem, hybrid monitoring and hybrid integration.

So hybrid is actually a set of problems, it's not a solution and I had a client last week say that hybrid is the problem why would I want to do this to myself? Why would anyone want to just split across three different platforms? First, because developers insist we do this cuz they want to try something new every six months that exists on a different platform which is great, but also, we need hybrid infrastructure because modern apps are hybrid and we are not going back.

They are increasingly hybrid. You build them in one place, you build the social media and search components in one cloud, you test them in another cloud, you run your mobile backend in a different cloud platform or on premise, you connect to store point of sales systems, you grab an ecosystem of marketing partners that already exist in the cloud, yet you're gonna keep a big chunk of core transactional data and database data on premises.

So modern applications are hybrid, they're composite they're multi channel. So it's just the state of infrastructure and its the way we have to go, and most enterprises are starting to get their head around the fact that this is just the future, but it's not fun, because every time we've pulled things apart from each other in the past from client/server onto virtualization, we've trashed all of our management styles, we've trashed monitoring, we've trashed performance management, and availability management, so we have to rethink all of that again and especially with the container evolution.

So microservices, I didn't want get down the microservices hole, because I can't tell you how to refactor your app to be microservices, but I can say the benefits of doing it are very clear, and this is probably where we're lagging right now the most in the enterprise, which is transforming applications from monoliths to really being based on loose collections of highly distributed microservices.

The benefits seem obvious you can incrementally change different components, you have a lot more implementation choices, for each individual component, you can build run time robustness into your architecture rather than relying on the infrastructure and you've got multicloud deployment options.

I was telling Bryan last night, he's young enough that he can talk about history and it's cute. When I talk about history, it just is pathetic. So starting in the 60s or the 70s, I'm not gonna say that—we have been doing this for 40 years and these containers represent the next wave of us having the tools we need to start carving applications into smaller bits. So microservices is happening. I don't have a good sense of how many enterprise apps have currently been converted to microservices maybe someday we'll find out how to ask that question.

What we did do is try to test DevOps adoption this year. And it's pretty hard to test adoption of a concept or an idea, how much have you adopted this idea of DevOps in you organization? So what we did is we backed into it with some data. We looked at different survey data to check average release cadence in different industries use of elastic public cloud platforms in different industries and the percentage of developers in those companies that we're working on customer facing applications, that were driven by customer experience demands.

We found one obvious and expected hot area of DevOps and one that wasn't clear, services, marketing, advertising and consulting. They are in the front ends of building new applications that touch customers. Of course they're further along in the DevOps path than others. But other manufacturing and pharma we didn't expect. That's pharmaceutical manufacturing, medical manufacturing, some agriculture, aerospace.

Companies you don't really associate with DevOps or rapid releases and when we talked to some of the people from those industries what we found is they're just now coming to the realization that they're also software companies, you know that jet engine and manufacturers, like GE are software companies.

So they're entering this they've never based their business on software before so they're sort of coming to DevOps late but they can start scratch by doing these new systems of engagement they're building with DevOps practices which basically means inviting the infrastructure team to be part of the design.

The other three areas that were the hottest are what we would expect. High tech manufacturing, retail, [FEEDBACK] sorry retail >> [LAUGH] Did I do something bad? Retail and financial services and we have some great representatives here from financial services, love New York I was in financial services for a long time.

The reason DevOps heat is there is because they have the best customers. They are picky and demanding so any one working with financial services knows that they'll cut your throat out as soon as they'll send you a check. So what we're finding in industries for DevOps heat is the people that are on the front lines that need to retain customers who someone else is trying to steal, and it's the companies who are just now deciding that they need to be software companies.

And they wanna do this the right way and they are learning from their more advanced or DevOps-hot peers. So then there's a real Container updates. So here's a great—don't what you wanna see a number like that 8% of people are using Containers in production. give us a hand, 8% that's actually not such a bad number.

Remember I said that public cloud adoption in the enterprise real used for production workloads and was only 10% in 2012. So give us three or four years and we expect that to be on the same trajectory. That's actually a little bit higher than I thought it would be. So we asked this in our 2000 Global Developer Survey and we tried that question we did ask the question in two ways.

We asked are you using Containers for production deployments? Then are you using Containers/Docker anywhere in your software delivery chain? Which could be in a couple of developer's laptops. Those numbers are in the 30s the average globally is 31% which, is much higher than we thought it would be.

Of people using Containers somewhere ion the delivery pipeline. And remember we took a broad description of Containers if we're using platform as a service from companies like our hosts or many other platforms as a service platforms, you're using Containers most of them were based on Containers we just might not have known.

Then when we looked at it by role, we found a real tight correlation between the different roles that are using Containers. We expected it to be that work for software companies in the valley bubble within half a block of market street that work in technology services firms of course.

We didn't quite expect the number to be so high for working in an internal IT department. So that's great, I think conferences like this and generally the entire cloud ecosystem we're seeing a lot more infection, back and forth between the disruptors and the innovation centers of the disrupted large enterprises but that was higher than I thought and by industry we are matching exactly the same industries that are hot in DevOps.

So Containers can really accelerate your path to all of this transformation. Of course we always ask about challenges too. This was a smaller survey done by our consulting group and we asked about top challenges of people who are actually deploying containers in production today.

No surprise, security is number one. I almost wish we would take security off of our surveys for challenges because it's the default answer, anyone checks about any new technology, what are you scared of with your new car? Security, so what we try to do to discount the security, but I don't wanna discount it too much, security is important, understanding how best practices for securing Containers who owns them trusting who built them and creating secure certified repositories, it's not easy and the fact that people are asking about it means that we as an ecosystem need to focus on that, and the news for the last 12 months in the Container ecosystem is that everyone is focusing on security, and variable performance, integration with existing tools and management really all come together around trust.

The enterprise yet doesn't think, doesn't trust that it has the operational skills to run Containers alongside VMs, or probably in VMs they are a little more confident of but Containers alone not so confident. But looking at none of these are show stoppers we are not like 80% of people are concerned that Containers aren't secure or 75 are worried that they have a lack of digital certification, these are all relatively low concern numbers and that's good, they are not roadblocks. There's something else I was gonna say about that and I forgot oh, I was gonna say that they are not roadblocks and we can prove that by looking at the data, when we cut data about Container concerns against adoption plans we see absolutely no difference between the planned adoption rate for people that are highly concerned about security and those that aren't and that's exactly what we see in the public cloud, everyone will say what do you concern about the public cloud is security.

That seems to have no impact on how fast they are adopting public cloud, so part of that might be survey artifacts. Inquiry volume is also exploding at Forester. We were joking last year that we were just going to start picking up phone hello Forester, the answer is Doctor or Containers, what's your question, because that's all our development team was taking queries about last year for quite a while.

We went from virtually nothing in the first half of 2014 to we'll do probably 120 in this quarter if January is any indication it's going to be more than that and the inquiry topics are basically bucketed into "who's really doing this and I'm I the first in my industry?" Help me understand the ecosystem, we tracked 45 different vendors operating system vendors, virtualization vendors, PaaS platform vendors even some SaaS ISPs, and given infrastructure players who have all announced some support for Containers.

So it's hard for enterprise customers to understand what that means when VMWare announces support for Containers and Microsoft in Azure and in Windows all announce support for containers some one has got to figure out what that means so that's been a lot of the questions and then where they are shifting now is much more to "help me navigate the ecosystem" to solve a particular persistent storage problem or security problem or just to answer a question about monitoring.

The biggest ramp up we've seen in the last couple of weeks is, if I put these things in production, how I'm I going to monitor a set of 200 microservices living across six physical machines when I've designed my whole environment to monitor virtual machines for infrastructure resiliency so the big question is what do I monitor, how do I monitor it? And I think this is something as industry we need to answer, I can't answer that question yet, I'm hoping to find out more about that today.

But what I think is happening is that the next wave of public cloud growth and spending is gonna be be driven by core systems of record. The first wave of cloud was driven by systems of engagement, all the client-facing applications that we really need rapid release cadence, but the money is gonna come when we start migrating core business apps or components of them to the public cloud and that's happening it's just not happening in earnest yet, but that's really where Containers can help power this.

The challenge now with breaking down applications and migrating components to different clouds is that there are a lot of different ways to do that. We outline five in some research that we've been working on lately that have varying degrees of risk, require varying degrees of investment and give varying degrees of payback.

You know you can lift and shift it now. It's virtualized, just move it. Re-host it on another cloud platform. That really doesn't add any value over time. And cloud as a place to just get cheap servers is really so 2010. Cloud is a place to build exciting new applications and for developers to turn ideas into great software faster.

So lift and shift, no. Lift and extend, re-host it and at least use some cloud native technology: messaging, database, object storage, something to extend how the application performs and to make it take more advantage of the new platform it's going on. Very popular. You can rebuild it completely, very few people have the budget or the stomach for that especially for core applications, you can completely replace it with SaaS, that's happening to some degree.

Again, that's huge projects it's generally taking a year and a half or what's becoming the most popular which is hybrid extend. Break it into at least a couple of smaller services, leave stuff on premise that should be there and the stuff you move make sure you build it with new cloud native tools and technologies. So the hybrid extend, the challenges all of these require someone in your organization to not only understand the benefits of different cloud platforms, but to understand where Containers make sense.

And I think that's what, I think today I'm looking forward to working with the other presenters on stage to understand how different companies are, what on ramps they are taking to Containers. And I think you really have to match your company's skill level and tolerance for risk to how you enter the Container ecosystem.

You can do it yourself, you can assign someone to go out and wade into the Container format battles and runtime battles and pick a time when you think you've established the format that will probably be the most portable eventually. Requires a lot of platform engineering skills especially if you're gonna try to build your own private platform internally, Container based platform.

Pretty high risk tolerance. And it kind of requires at this point that you spend a lot of time getting involved in the ecosystem contributing upstream code. And there are plenty of large enterprises that are doing that but that takes quite a bit of investment and you're competing with a shrinking talent pool.

The shrinking talent pool only because everybody is competing for the same talents. You can enter with a public cloud platform Container as a Service, made with Azure, Google, AWS. They're gonna make a couple of decisions for you probably about an orchestration platform to start with, or orchestration tool.

But you're still gonna have to assemble it yourself, and that's okay. That might be the way you wanna enter the ecosystem, but also consider entering it to a traditional Platform as a Service. We hear a lot of inquiry topics about when is the Container ecosystem gonna solve persistant storage problems or networking problems or some sort of security problems and a lot of people that have been building Platform as a Service platforms for years and selling them say we've solved these, we've been based on Containers for a long time.

We have examples and best practices for doing these things around Containers already. So I think that Containers are gonna give Platform as a Service in a big shot in the arm. I had one awful presenter say Platform as a Service is a huge zero billion dollar market and I think that we need to—Containers could really be what powers the next generation of Platform as a Service because building that yourself is gonna get harder and harder over time, I think.

So, like I said everyone coming after me is gonna be smarter and especially know a lot more about Container deployment, so I'm really looking forward to the retail and financial services case studies we've got coming up. I'm gonna come back later and a panel with some experts from Wall Street.

So we can see how Containers are really working there, we have the foundation representatives coming in to give us an update on what's going on with the foundation, and advanced track technical sessions as Bryan mentioned. So eight years ago I was helping people manage hundreds or even maybe thousands of VMs in the data center. Five years ago it was—four or five years ago it was managing tens, hundreds of thousands of VMs in a public cloud, and I think three or four years from now I'm gonna be spending most of my time talking to people who are managing hundreds or thousands of Containers, so let's make it happen thank you for being here and I appreciate it.



Dave Bartoletti: Principal Analyst, Forrester