In the summer of 2014, Box was feeling the pain of a decade’s worth of hardware and software infrastructure that wasn’t keeping up with the company’s needs.
A platform that allows its more than 50 million users (including governments and big businesses like General Electric) to manage and share content in the cloud, Box was originally a PHP monolith of millions of lines of code built exclusively with bare metal inside of its own data centers. It had already begun to slowly chip away at the monolith, decomposing it into microservices. And “as we’ve been expanding into regions around the globe, and as the public cloud wars have been heating up, we’ve been focusing a lot more on figuring out how we run our workload across many different environments and many different cloud infrastructure providers,” says Box Cofounder and Services Architect Sam Ghods. “It’s been a huge challenge thus far because all these different providers, especially bare metal, have very different interfaces and ways in which you work with them.”
At DockerCon that June, Google announced the release of its Kubernetes container management system, and Ghods was won over. “We looked at a lot of different options, but Kubernetes really stood out, especially because of the incredibly strong team of Borg veterans,” he says, referencing Google’s internal container orchestrator Borg. Plus, “The fact that on day one it was designed to run on bare metal just as well as Google Cloud meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as well.”
Box deployed Kubernetes in a cluster in a production data center just six months later. Kubernetes was then still pre-beta, on version 0.11. They started small: The very first thing Ghods’s team ran on Kubernetes was a Box API checker that confirms Box is up. Next came some daemons that process jobs. The first live service, which the team could route to and ask for information, was launched a few months later. At that point, Ghods says, “We were comfortable with the stability of the Kubernetes cluster. We started to port some services over, then we would increase the cluster size and port a few more, and that’s ended up to about 100 servers in each data center that are dedicated purely to Kubernetes. And that’s going to be expanding a lot over the next 12 months, probably to many hundreds if not thousands.”
While observing teams who began to use Kubernetes for their microservices, “we immediately saw an uptick in the number of microservices being released,” Ghods notes. “There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices.”
By Ghods’s estimate, Box is still several years away from his goal of being a 90-plus percent Kubernetes shop. And he predicts the same happening across the industry, as Kubernetes has the opportunity to be the new cloud platform. Kubernetes provides an API consistent across different cloud platforms including bare metal, and “I don’t think people have seen the full potential of what’s possible when you can program against one single interface,” he says. “The same way AWS changed infrastructure so that you don’t have to think about servers or cabinets or networking equipment anymore, Kubernetes enables you to focus exclusively on the containers that you’re running, which is pretty exciting. That’s the vision.”
Box, with its early decision to use bare metal, embarked on its Kubernetes journey out of necessity. But Ghods says that even if companies don’t have to be agnostic about cloud providers today, Kubernetes may soon become the industry standard, as more and more tooling and extensions are built around the API. “The same way it doesn’t make sense to deviate from Linux because it’s such a standard,” Ghods says, “I think Kubernetes is going down the same path. It is still early days. When you’re on the cutting edge you can expect to bleed a little. But the bottom line is, this is where the industry is going. Three to five years from now it’s really going to be shocking if you run your infrastructure any other way.”
To learn more about what’s ahead for Box with Kubernetes, check out this in-depth case study.
Interested in more Kubernetes content? Get on the CNCF newsletter list for more Kubernetes information and updates.