By Asaf Liveanu May 11, 2022

Guest post by Asaf Liveanu, Co-Founder & CPO at Finout

Two computer screens, 1 mobile phone and headphone on a desk

Cost optimization is a growing concern for organizations rapidly moving towards open-source and cloud-native projects based on Kubernetes. While flexibility remains one of the key strengths of Kubernetes, it often leads to excessive spending, due to overprovisioning of workload resources. The primary challenge with cost projection and control in a Kubernetes cost management ecosystem is the complexity and lack of visibility on how Kubernetes cluster resources are stacked with the underlying infrastructure. In kubernetes, to manage overall costs, it is important to identify pod-level resource consumption and assess how that impacts the workload. 

In this article, we discuss the challenges of controlling pod-level consumption and the FinOps strategies you can implement for efficient cost monitoring and management at the pod level. 

Challenges of Controlling Kubernetes Pod Resource Costs Consumption

When dealing with Kubernetes, controlling the cost of resource consumption is difficult for both startups and established organizations. It’s particularly confusing and complex to analyze the cost incurred by each pod, as developers tend to overestimate resource allocation, resulting in spending much more than they use. While challenges differ according to use case, here are some of the most common problems that restrict optimum cost control. 

Resource Asymmetry 

A typical Kubernetes ecosystem may consist of multiple workloads with different resource requirements. Applications rely on different types of resources which vary from common resources like CPU, Memory to more specialized resources like GPU accelerators or high speed storage.. Such resource asymmetry is handled by the kube-scheduler, which factors a scoring algorithm to assign the right node for each workload. The scoring algorithm is often a combination of multiple factors including resource availability and user defined conditions. Therefore comprehending how and when a node is to be allocated to a workload can be complex. 

Fluctuating Resource Consumption

Another crucial factor related to controlling consumption cost is the varied usage of resources by Kubernetes workloads. As resource usage is constantly changing for dynamic workloads, these changes in resource consumption can lead to fresh provisioning, realignment, or destruction of pods within nodes.. 

Missing Requests and Limit 

Request and Limit parameters help optimally allocate resources to pod workloads within a node. In a node with multiple pods, if one among them does not have a defined request and limit, the pod tends to consume all resources available in the node. In such instances, other pods never get the resources they require, impacting the overall performance and cost of the cluster. 

Implementing FinOps Methodologies for Pod-Level Cost Allocation

The FinOps methodology boosts business value by helping cross-functional business verticals collaborate to achieve cloud spending goals. The model also helps consolidate the expenses of all business units by bringing financial accountability and data-driven decision-making to the forefront.

Using the FinOps methodology on a Kubernetes cost management ecosystem will help you manage costs based on your usage of different parameters, including resources, objects, infrastructure components, features, and more. In this section, we explore some common cost-management FinOps strategies at the pod level. 

Set Pod Requests and Limits 

Rightsizing pods is one of the most recommended approaches to limiting budget overrun. Request and Limit are two parameters that help define and control the resource usage for containers within pods. As a pod can contain multiple containers, you should set the parameters for all containers to get the aggregate request and limit that the pod requires. 

Based on the value of these parameters, Kubernetes pod scheduler can choose the node for the pod. This allows the pods to function normally, ensuring better stability and less resource wastage. Defining the right values for the limit also ensures that the pod is only getting the amount of resources requested by the limit and prevents starving other pods due to resource unavailability. 

Enforce Tenant-Level Logical Separation for Kubernetes Components

Kubernetes clusters can be logically partitioned into small virtual partitions, known as namespaces. You can assign these partitions to different services or teams, where each namespace can function independently in isolation. Using the ResourceQuota object, each namespace can be allocated with the quantity of objects and a fair share of the Kubernetes cluster’s resources, including:

Breaking down a cluster into multiple namespaces and monitoring each namespace helps identify which team or service has more cost overhead, as well as ways to optimize it.  

Use Labels to Allocate Cloud Costs for Each Tenant Based on Usage

Kubernetes labels offer a popular approach when it comes to identifying Kubernetes objects and aligning them into groups. Labels are key-value pairs that can be attached to Kubernetes objects such as pods. These enable FinOps teams to identify the pod-level resource usage for different environments or applications. 

Tagging with labels also helps identify over-provisioned resources. Based on the usage, cost can be allocated or adjusted for individual teams, applications, or services. It is recommended to map labels for all business units incurring cloud cost management and align them with consumption logs for shared resources. This makes cost analysis, reporting, and optimization easier for individual teams, services, or business lines.

Use Cloud Cost Monitoring Tools to Avoid Overspending 

Since cloud-native tools offer limited features for tracking expenses, companies often adopt third-party cost management tools to forecast budgets and monitor cluster expenses. These tools provide real-time visibility of pod-level resource utilization and offer insightful reports about the expenses incurred. Automation is one of the most crucial features that such tools offer, helping to detect, analyze, and report any resource usage abnormalities before they adversely affect cost. Some popular cost-monitoring tools include Kubecost, CloudZero, Harness, and Loft.

Configure Quality of Service (QoS) for Each Pod

Kubernetes configurations provide three Quality of Service (QoS) classes for pods: Guaranteed, Burstable, and Best Effort. These classes help achieve higher utilization of node resources, resulting in less wastage and more cost benefits. Based on the defined resource limit and request parameters, you can assign pods to one of the three classes. QoS classes help the Kubernetes scheduler determine the scheduling of the pods on nodes, ensuring the nodes’ resources are maximally utilized. They also decide the order of the eviction priority of the pods when a node gets low on resources.  

Utilize Vertical and Horizontal Autoscaling to Optimize Allocation for Workloads

Kubernetes’ autoscaling function ensures that cost is only incurred for the resources consumed. Kubernetes provides two pod autoscaling mechanisms: 

With autoscaling, Kubernetes can quickly adapt to the change in demand by ensuring that the right size and number of pods are being used. This results in improved performance while reducing resource wastage and cost. 

Use Affinity with Taints and Tolerations to Control Pod Scheduling 

Kubernetes Affinity and Taints and Tolerations allow users to control where pods will get scheduled in the cluster. This is especially important if there are nodes in the cluster that are aimed at running specific workloads such as CPU accelerated workloads. If a Pod that does not consume these resources gets allocated to this node it will result in resource wastage. By defining affinity with taints and tolerations one can restrict which Pods are allowed to be scheduled. This ensures that Pods with workloads that would utilize these specific resources would be scheduled in the node.

Conclusion 

A recent survey projected that over 88% of organizations already use kubernetes to support their production workloads. Though this confirms the rising popularity of Kubernetes for supporting container orchestration, the complex ecosystem has inherent challenges. For organizations that operate intricate Kubernetes clusters to support dynamic workloads, overprovisioning compute resources is a common construct. While leveraging resources on-demand supports high availability, these organizations often have to deal with cost overruns. 

Traditional approaches to calculating resource consumption and related costs mostly don’t work or are too perplexing to achieve manually. As your organization matures, you may want to optimize spending by strategically using your resources. The FinOps methodology helps with cloud cost management by blending cost, agility, and quality at the pod level. 

Why Finout: While AWS charges you by the instance, what you really care about is your pods. With no-agent integration, utilize your existing Datadog or Prometheus, and get pod-level granularity of your spend. In other words – add AWS kubernetes to your CUR. Contact us to learn more