Member post originally published on the Fairwinds blog by Joe Pelletier
The widespread adoption of containerized applications has fundamentally changed how organizations develop, deploy, and manage their software infrastructure. Kubernetes is fundamental to this change, because it makes it possible to manage containerized workloads and services at scale. One of the cornerstones of its effective operation is Kubernetes governance, the policies and procedures that govern how Kubernetes is configured, managed, and secured.
Without Kubernetes governance, platform teams are challenged with onboarding new applications and teams. Governance enables consistency, predictability, and repeatability — enabling platform teams to build “paved roads” to production so engineering teams use cloud-native infrastructure to deploy applications more frequently — a key metric for the DevOps Research and Assessment team (DORA).
What is Kubernetes Governance?
Kubernetes governance dictates the rules of operation in your Kubernetes environment, ensuring an aligned and controlled management of your Kubernetes clusters. It includes management of your Kubernetes resources, role-based access control, scheduling, and upgrades. Governance also includes the process for making decisions related to Kubernetes, including how to manage feature requests, security issues, and bug fixes. It’s important to have governance in place because Kubernetes introduces new technologies and different complexities for development, operations, and security teams. Without governance, challenges emerge, such as:
- Insufficient visibility into Kubernetes cluster activity and growth
- Difficulty managing multiple software versions across the organization
- Issues tracking user roles, responsibilities, and privileges across multiple teams and environments
- Considerable time required to identify role violations, assess governance risks, and perform compliance checks
- Problems enforcing policies and procedures across teams
Kubernetes governance initiatives help ensure Kubernetes meets your organization’s policy requirements, adheres to best practices, and meets relevant regulatory requirements.
Best Practices for Kubernetes Deployment
To maximize the benefits of a Kubernetes implementation, follow these five best practices:
1. Security configurations
Must be established and enforced by your security team, ideally through automation and robust policies.
Set resource requests and limits on workloads to maximize infrastructure utilization while still ensuring optimal application performance.
3. Reliability
Ensure workloads are configured with liveness probes and readiness probes, and follow Infrastructure as Code (IaC) best practices. IaC ensures infrastructure is auditable and consistent.
Once Kubernetes deployment increases beyond a single application, enforcing policy is critical. Tools and automation can help you prevent common misconfigurations and enable IT compliance. It can also promote a service ownership model because users are comfortable deploying, knowing that guardrails are in place to enforce the policies. One open source tool for cloud native environments is theOpen Policy Agent (OPA), which offers policy-based controls.
5. Monitoring and alerting
In an ephemeral environment like Kubernetes, it’s important to make sure that your infrastructure and applications are running. There are a number tools available to optimize monitoring.
Important Policies for Kubernetes Governance
Cluster-wide policies
Organizations can deploy both cluster-wide and namespace-specific (or application-specific) policies. Usually, cluster-wide policies apply to all workloads and may relate to security, efficiency, and reliability categories. A few important policies include:
- Memory and CPU requests should be set
- Liveness and readiness probes should be set
- Image pull policy should be set to “Always”
- Container should not have dangerous capabilities
Namespace policies
Namespace-specific policies enforce standards for specific app teams or services when you need an increased level of security, efficiency, or reliability. You can use namespaces to create different ‘tenants’ within a shared Kubernetes cluster for teams; these teams must adhere to a common set of best practices that avoid disruption to other cluster tenants, such as resource exhaustion or security violations.
Enforcement points
Enforcement of policies can happen in multiple stages. Kubernetes governance enforces policies by delivering feedback to engineers in the tools they use, at the time they need it.
Cost allocation
Kubernetes spend increases proportionally based on the number of clusters, where apps and services are deployed, and how they are configured. Platform engineering teams need to allocate and showback costs in a business-relevant context to manage spend.
Resource labeling
Using namespaces or labels to map costs to a Kubernetes component helps you allocate costs to individual business units. The Kubernetes Vertical Pod Autoscaler (VPA) uses historical memory and CPU usage of workloads in conjunction with current pod usage to generate recommendations for resource requests and limits.
Cost avoidance and optimization
Cost avoidance means reducing usage and optimizing costs to get a better cloud rate. Platform engineering teams can achieve these goals by shipping applications more rapidly, optimizing cloud usage, and reducing risks.
Infrastructure as Code scanning
IaC enables you to use a configuration language to provision and manage infrastructure, applying the repeatability, transparency, and testing of modern software development to infrastructure management. IaC reduces error and configuration drift, allowing engineers to focus on work that contributes to larger business objectives.
Container image scanning
A vulnerability in your base image exists in every container that contains that base image. Staying up to date with patches to your base image, controlling permissions, and eliminating images not required for deployment to production helps increase the security of your container images.
Deprecated APIs
Kubernetes deprecates versions of APIs to minimize the need to maintain older APIs and push organizations to use more secure, up-to-date versions. When an application or service includesdeprecated or removed API versions, find and update them to the latest stable version.
Upgrade add-ons
Add-ons provide additional Kubernetes functionality. Sometimes add-ons require upgrades, but it’s important to check whether the latest version is compatible with your cluster. This process can be slow and difficult without tools that detect add-on changes automatically.
Policy enforcement and guardrails
In Kubernetes environments consisting of multiple Kubernetes clusters and teams of developers, having defined policies and an automated method of enforcing those policies is essential. These guardrails prevent deployment of changes that break something in production, allow a data breach, or enable configurations that do not scale properly.
Compliance analysis
Most organizations must comply with multiple security and privacy regulations, such as SOC 2, HIPAA, ISO27001, GDPR, PIPEDA, and many more. Adopting defined Kubernetes compliance requirements policies and enforcing them across all clusters automatically is critical to achieving compliance goals, as well as automating compliance analysis to assess whether you are meeting changing requirements.
Implementing Kubernetes Governance
Kubernetes governance aligns with cloud native computing strategy, enabling platform teams to apply guardrails automatically that implement and enforce policies. These policies can help you implement and run Kubernetes reliably, securely, and cost-efficiently. This enables you to maximize your investment in the K8s platform without worrying about whether you are meeting your organization’s policy requirements.