By Sandeep Rajan, Software Engineer, Infoblox
Addons extend the functionality of Kubernetes. Some of the addons like CoreDNS and kube-proxy are considered essential to the functionality of a Kubernetes cluster and are shipped along with cluster management tools like Kubeadm, Kops, etc. There are other addons which can be installed as an option and are not as critical, but help users manage their clusters (e.g., Calico, Kubernetes Dashboard, NodelocalDNS, etc.)
The purpose of the cluster-addons project is to be able to manage addons in a better way with the help of Operators.
An Operator is a method of packaging, deploying and managing a Kubernetes application.
Each Operator will have its own CustomResourceDefinition (CRD), where users will be able to manage the addons in terms of installing, upgrading, choosing the version, etc. of the addon by modifying the CRD without being dependent on the cluster management tools.
The need for cluster-addon Operators
Today, cluster management tools like Kubeadm, Kops, Cluster API, etc., are bundled with essential addons such as CoreDNS and kube-proxy.
Some of the challenges faced are:
- Users want to have more flexibilty when managing addons installed in their clusters. Currently, addons get automatically managed alongside the lifecycle of the cluster. For Example, if a user upgrades/downgrades the version of Kubernetes, the version of the addon(s) gets upgraded/downgraded along with it.
- It is becoming difficult to manage cluster management tools due to the addons becoming increasingly complex.
In the past, there has been an attempt to solve these problems via the bash version of addon-manager. However, the addon-manager was unsuccessful and hasn’t been widely adopted.
The cluster-addons project focuses on solving these challenges by introducing Operators. Operators allow users to decouple the lifecycle of the addons from the lifecycle of the cluster.
Each Operator is defined by a CRD. Users will be able to tailor the addon in a limited way (E.g., installing, upgrading) through the CRD. Each Operator has its own Controller. A Controller is a control loop that watches the state of your addon resources and makes or requests changes where needed. The controller tries to move the current state of the addon resources closer to the desired state.
The CoreDNS Operator
CoreDNS has been the default DNS addon since Kubernetes v1.11 and is one of the most essential addons for having a functional Kubernetes cluster.
The CoreDNS Operator has been specifically highlighted here as CoreDNS is one of the most complicated addons. It is the first Operator to be implemented to showcase the power of the Operator and its capability of making CoreDNS easier to manage on a cluster.
As part of its basic functionality, the CoreDNS Operator is capable of installing CoreDNS, upgrading the CoreDNS version, and migrating the Corefile automatically during upgrade to ensure it is up to date and compatible with any version of CoreDNS, providing the user with a seamless experience.
The Operator continuously monitors the CoreDNS resources (Deployment, ConfigMap, Service, etc.) with the help of a controller.
The CoreDNS specifications can be modified through the CustomResource (CR). An example CoreDNS CR is as follows:
apiVersion: addons.x-k8s.io/v1alpha1
kind: CoreDNS
metadata:
name: coredns-operator
namespace: kube-system
spec:
version: 1.7.0
dnsDomain: cluster.local
dnsIP: 10.96.0.10
corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
forward . /etc/resolv.conf {
max_concurrent 1000
}
cache 30
loop
reload
loadbalance
}As seen above, the CoreDNS Version, DNS Domain, DNS IP and Corefile can be modified by modifying the CR spec. The Controller will register the changes applied to the CR and will reconcile with the CoreDNS resources to reflect the changes.
To learn about how to install the CoreDNS Operator on your cluster, you can follow the instructions here.
Creating your own Operator
The cluster-addon operators are created with the help of Kubebuilder, a framework for creating your own Kubernetes APIs using CRDs.
If you are interested in creating your own Operator, please check out these step-by-step instructions on how to create and deploy your Operator.
Status of the project
Today, The cluster-addon repository contains many Operators such as CoreDNS, NodelocalDNS, kube-proxy, etc., which can be installed and enable users to manage these addons on their Kubernetes clusters.
The cluster-addon project also contains the addon-installer library, which functions as a small implementation for applying addon operators to the Kubernetes cluster using kubectl and Kustomize.
Future work
We are working towards integrating the Operators into cluster management tools such as Kubeadm, Kops and Cluster API.
Resources
If you are interested in the cluster-addons project, want to explore Operators, want to discuss addons or new features, or get stuck somewhere, you can contact us at:
Infoblox delivers the next-level network experience with its Secure Cloud-Managed Network Services. Interested in joining us? See our openings here.