Guest post originally published on the Weaveworks blog by Jeff Krupinski, Weaveworks Senior Director of Sales
Have you ever thought about how retailers or restaurant chains have adopted cloud-native technology with edge presence at their brick and mortar stores? You may have seen some customer testimonials online from Chick-fil-A and Nordstrom that tackle these challenges and Weaveworks is at the core of helping others as well. At first glance, it can be a challenge to understand how to manage 100s or even 1000s of stores with the same applications running at each location. In this post, we will talk about 4 common challenges that the retail sector faces when modernizing its infrastructure at the edge.
This blog post is based on Weaveworks’ experience of partnering with organizations who adopted GitOps to better enhance Kubernetes cluster management and application delivery processes at the edge. Many of these customers have tried DIY Kubernetes and pipelines but were stuck in a manual deployment process that took hours and even days to update simple changes. With Weave GitOps, these customers can now schedule automatic updates and changes, and roll back to previous configurations without having to worry about manual interventions, especially at late hours or even weekends. And to completely avoid downtime, something that is absolutely not acceptable in the retail sector.
Challenge #1: Avoiding Cluster Configuration Drift with Multiple Target Environments
As we know in the cloud native ecosystem, containerization has brought simplicity: if it works on my laptop, it will work running anywhere. The abstraction of system configurations tied to an application has gone away with containerization. Same holds true for Kubernetes clusters. As long as your cluster has the same configurations as your dev, integration, staging, and production environments, the process of deploying applications or services through the pipeline should be seamless.
This is usually not the case because clusters can run on different hardware and are built differently from location to location. Typically, we find that these locations don’t use the same configuration especially if there are multiple teams/operators setting up clusters at stores.
Another example is when it’s time to update Kubernetes or the components itself which can vary from location if the update lands or not. Using templates and profiles within Weave GitOps assures that clusters have identical components such as observational and monitoring solutions, networking technologies, ingress, etc. Operators are removing direct access through kubectl when using GitOps workflows and only allowing changes to a cluster with approved pull requests via Git. Updates via Git automatically preserve change requests and give an accurate audit log.
Picture this, you have 100 stores that have a simple 5 node cluster running locally at each location. Operators can assure that clusters are always looking at the same git repository to sync with the latest changes, or roll back should a new change damage an application or set of applications.
In addition, you can even time your deployments for a certain time of day, as a new change is made in sync, or manually if needed (not recommended but you still have the ability to.). We typically don’t recommend manual updates unless it’s absolutely required. From past experience, you want to align to specific guidelines rather than making this a manual change. Policy from Weave GitOps can help automate the process and make manual changes a last resort.
Challenge #2: Updating a Running Application or Workload Service
This process is very similar to the 1st challenge where we discussed the cluster deployments, but we are now discussing the application deployment process. If you need to increase the amount of application instances during busy times or you need to update an application to allow for a new credit card, or some other change that has been causing users to leave the application because they can’t customize their order, resulting in revenue losses.
You can immediately resolve this through Kubernetes and autoscale processes via configuration deployment methodologies within Weave GitOps. Cloud native technologies with Weave GitOps allow for real time changes,so you can at once reduce the amount of revenue loss when you see certain trends.
The biggest problem we help delivery teams overcome is that they can’t keep each store or restaurant synched with the latest version of their application. Common causes are a failing pipeline resulting in only 25% of the locations getting an updated app deployed or certain stores having restrictions when apps can be deployed and it’s hard to understand which ones were done correctly. With Weave GitOps, you can implement a pull methodology where all locations are updated at your time of choosing.
Whether it be immediately, after hours, or manually due to any CVEs detected, Weave GitOps has the power to create this pipeline with reconciliation statistics. This is a crucial feature, especially to operation and delivery teams so they can determine the status after the workloads were deployed. Weave GitOps allows our retail customers to deliver updates and changes to all locations at once when initiating a change through a Git pull.
Challenge #3: Visualizing All Clusters in a Single Pane of Glass
Many teams we supported in the past are struggling with managing many clusters in various environments. One client stated that of the several hundreds of stores, they have to manage clusters that run in many different configurations including some using EKS, some using on premise Raspberry Pis, and some using reused hardware with vSphere. Not only was it difficult to stand up each cluster, operations teams are just adding bandaids to clusters to keep them running because they don’t have the time to redeploy a proper cluster to each environment.
Weave GitOps can visualize different environments in a single pane of glass view using CAPI (Cluster API) technology. The CAPI templates allowed the team to build and update clusters effortlessly through the UI. Not only were operations and engineering teams able to consume the interface today, but even the developers can now consume these clusters without having to contact their IT department via Service Now tickets. This self-service option has already components the DevSecOps team required built into each cluster so they can be confident consumers wouldn’t forget to add any required services to manage clusters accurately.
Challenge #4: Shifting Responsibility Left
A small operations team can be a true challenge for expanding and growing retailers as we have seen with another one of our clients. The operation team wears many hats as they architect environments and operate as well as maintain uptime as well. Due to bandwidth, the team wasn’t able to address requests from the growing development team in a timely manner anymore so a safe and reliable solution needed to be found. With the adoption of our Trusted Delivery engine in Weave GitOps, the team was able to include 150+ policies out of the box and bring their own specifications as well.
With the ownership now shifted left to the developers on managing their own applications in a safe and reliable fashion, the operation team was able to scale again. Ticket resolution time was cut by 50%immediately and they were able to shift their focus on architecture designs to accommodate a growing business. Since the writing of this post, they have now implemented GitOps pipelines for all of their stores and are starting the same for their HQ internal application process.
Final Thoughts
In summary, retail customers with edge locations are taking advantage of Weave GitOps and are sleeping a little more peacefully at night. Not only are these companies attracting great talent since they are using new cloud native approaches, they are decreasing technical debt and making decisions based on their technology growth. Their customers have noted the great experience which is driving upward revenue streams that were stale or trending downwards before making the move to Weave GitOps.
Weave GitOps provide developers and operator teams a consistent way to increase development velocity and deployment reliability for cloud-native applications. It enables declarative infrastructure, built-in security guardrails, and end-to-end pipeline automation. Request a Demo or reach out to sales@weave.works to learn more.