Guest post originally published on the Nirmata blog by Jim Bugwadia
Introduction
In Kubernetes, policies are a special type of configuration resource that control other configuration or runtime behaviors. For example, a simple policy declaration may be, “HTTP (non-encrypted) endpoints are not allowed”.
Kubernetes allows several types of policies to be configured. In this post, we introduce the four types of policies available in Kubernetes and provide guidance on how they should be used.
API Objects
Kubernetes has several API objects, like NetworkPolicies, Roles, ResourceQuotas, role-based access control configurations, etc. that act as policies and control other configuration and runtime behaviors. For example, a Network Policy defines what traffic is allowed to, and from, an application.
Admission Controllers
Kubernetes has built-in admission controllers that operate directly in the API server and intercept API requests for mutation and validation. For example, the DefaultIngressClass admission controller can be used to mutate an API request to create an Ingress and configure a default Ingress class if none is specified. The built-in admission controllers can be enabled or disabled using the API server –enable-admission-plugins and –disable-admission-plugins flags. Some built-in admission controllers may allow additional configuration via a file accessible to the API server.
ValidatingAdmissionPolicy
Kubernetes version 1.26 introduced an alpha feature to allow more flexible admission control checks using a language called Common Expression Language (CEL). The CEL validation is defined in a new resource called ValidatingAdmissionPolicy. Since this is an alpha feature, both the validatingadmissionpolicy admission controller and admissionregistration.k8s.io/v1alpha1 group/version have to be enabled for it to be used.
Dynamic Admission Controls
In addition to built-in admission controllers, Kubernetes allows execution of admission control decisions in a separate process. These processes are referred to as admission webhooks. This allows extensibility of the API server with CNCF policy engines like Kyverno and OPA/Gatekeeper that use Custom Resources to manage policy artifacts.
Which Kubernetes policy types should you use?
It is important to first learn and properly use the built-in Kubernetes API objects like NetworkPolicies and RBAC resources.
The built-in admission controllers are also important to learn about. However, it may not be clear how and when to use the different types of admission controllers. For managed Kubernetes services, it may not be possible to change API server configurations or files required for additional configurations. This limits their usage to clusters where the API server can be configured. Even in those cases, re-configuring the API server is an operation that may cause downtime and is not the most flexible way to enable and enforce policies.
The ValidatingAdmissionPolicy is more flexible, as it allows custom checks to be written using the Common Expressions Language (CEL). Since these validation checks run directly in the API server, the ValidatingAdmissionPolicy avoids the complexity of having to maintain a separate highly-available (HA) workload for policy enforcement. However, this feature is currently an alpha stage feature and only supports basic validation checks for data within an API request, along with lookups for a few other parameters like namespace labels. Additional API server, or external service calls, are not supported. Also, currently other policy types such as mutation rules, generate rules, and cleanup policies are not supported. Additionally, reporting and background scans are also not supported.
Admission webhooks, and policy engines like Kyverno and OPA/Gatekeeper, require managing a highly-available workload to intercept API requests and enforce policies which can be an operational challenge and requires careful management as with any other critical workload. As a benefit, these engines allow complex validation checks, with flexible API server and external service lookups, for data-driven policy decisions. Beyond validation checks, policy engines like Kyverno also offer mutation, generation, image verification, and cleanup policies. And, in addition to execution of policies at admission controls, policy engines can be executed in the CI/CD pipeline via command-line interfaces. For reporting, engines like Kyverno also produce policy reports that are easily consumable as an Kubernetes API server resource.
Conclusion
Due to extensibility and versatility, Kubernetes is being widely used as a platform for building platforms. Policies are critical for proper Kubernetes operations, and allow platform engineering teams to provide the necessary guardrails.
As with other critical components, Kubernetes provides a lot of flexibility and choices for policy management. For production clusters with mission critical workloads, all four Kubernetes policy types will likely be used.
If you are interested in learning more about Kubernetes policy and governance, including a in-depth comparison of the four Kubernetes policy types, grab a free copy of our new eBook: The Ultimate Guide to Policy-based Governance, Security & Compliance for Kubernetes.