Guest post by Bob DeRosa, Senior DevOps Architect at Broadridge
When you’re a global fintech leader and the foremost provider of investor communications in an ever-accelerating world, it’s essential that your applications are both reliable and scalable. The cloud excels in these areas and can greatly enhance both the internal and external customer experience when working with your products.
At Broadridge, we are in the process of transitioning our CloudBees CI installation from on-premises to cloud-based Kubernetes application development. We also wanted to ensure our DevOps team would have reliable backup and recovery tools throughout the transition process and beyond.
The team was initially using a monolithic design for our CloudBees CI installation, which posed a few problems:
- Runaway jobs: One team’s runaway job could affect those of other teams.
- Adding resources: The process of adding resources like disk space or processing power could take weeks.
- VM maintenance: A great deal of time was being spent on the maintenance and patching of VMs.
Moving to the cloud would offer more flexibility when scaling, increased reliability, and the ability for us to use infrastructure as a service. Kubernetes was a clear choice for hosting our applications, since CloudBees is already optimized for Kubernetes. What’s more, it would enable us to create more customized images for developers, simplify patching and rolling back, and reduce infrastructure costs across the board.
Ramping Up for Cloud Migration Preparation
For any cloud transition, you must make adequate preparations ahead of time, especially when your applications are processing trillions of dollars each day and even a minute of downtime means money lost. To that end, we hired people that had prior experience with Kubernetes and set proper expectations how much time it would require to complete a move of this magnitude.
Before diving headfirst into the transition, our DevOps team created a proof-of-concept (PoC) to test their assumptions about the software. It was essential to test the PoC upfront, as implementing changes can become increasingly difficult as the transition progresses. We discovered that the file system we originally planned to use would not work for our purposes.
Choosing the Right Tools Was a Critical Step
We knew that security, monitoring, and application backup were essential requirements for the migration to be a success. We chose Aqua, Datadog, and Kasten K10 by Veeam to accomplish these objectives.
When creating new infrastructure to house the applications, the DevOps team devised an Infrastructure-as-Code (IaC) system and stored the code in version control as part of the CI/CD pipeline. This enabled everything in the pipeline to be both auditable and repeatable.
IaC provides us with much needed consistency across multiple different development environments, ensuring that code will behave similarly all the way from development to production. Using IaC, we were able to sort out the majority of the project’s issues far before production.
Installing the Applications
Once the infrastructure was in place, the next step was to install the applications. Doing so in Kubernetes can be challenging, but much of the work can be alleviated by using a Helm Chart. Helm, put simply, is a package manager for Kubernetes that enables third parties to create their own installers. Helm Charts greatly reduce the complexity associated with installing vendor software and eliminate the need to write and maintain YAML manifests. We used Helm Charts to quickly install both Kasten K10 and CloudBees CI, as well as the other third-party software.
It should be noted that a transition to the cloud is a great opportunity to capture your best practices as code, keeping reusability for future devs top-of-mind. Our DevOps team ironed out the problems they were running into with EKS and implemented the solutions into Terraform modules that other developers could use to jumpstart their own EKS projects.
Early Results Look Promising
While we’re still in the process of fully migrating our extensive Cloudbees CI installation, we’ve already begun to see positive results.
- The reusable EKS Terraform modules and Helm Charts have increased the productivity of many other teams within Broadridge. Our Developers can essentially drag and drop the components they need to start their projects and focus more on what matters.
- We adopted the practice of wrapping third-party Helm Charts in our own charts, so we can customize third-party software installations without needing to maintain changes to the code.
- We developed a system to automatically synchronize third-party updates, so that as soon as an update drops, it becomes available within the internal Helm repository for review and implementation.
The improved flexibility of our cloud-based development environment became increasingly evident when we were asked to implement CloudBees CD. Since the environment was already deployed, and Helm can be used for CloudBees CD, installing it took only a couple of days.
Our next step will be migrating existing internal customers to the new cloud-based platform, while continuing to scale to accommodate future customers and projects.
Watch this on-demand webinar, in which I provide more details about our migration.