By David Eads + Kevin Wang September 26, 2022

Guest post from Huawei by David Eads (@deads2k), Kevin Wang (@kevin-wangzefeng)

Current users will know that KubeFed was an early answer to the issue of centralized management of multiple Kubernetes clusters– but as community engagement has slowed with the project since 2020, it’s now reached its EOL, leaving many searching for alternatives. Today we have two CNCF projects providing modern answers to the issue KubeFed accounted for. With active and vibrant communities to support them, Karmada and Open Cluster Management (OCM) are here to take up the challenge of managing fleets of clusters across the hybrid and multi-cloud landscape.

Limitations with KubeFed

The Kubernetes SIG KubeFed project allowed users to coordinate the configuration of multiple Kubernetes clusters from a single set of APIs in a hosting cluster. Federation v1 could distribute services to multiple Kubernetes clusters but could not handle other kinds of objects. The project has been archived by Kubernetes-SIG and never became a core feature. KubeFed v2 quickly replaced v1 and is used by operations staff around the world. It allows a single Kubernetes cluster to deploy multiple kinds of objects to multiple other Kubernetes clusters. KubeFed v2 also allows the “control plane” main cluster to manage the other clusters, including their resources and policy.

Chart showing KubeFed diagram flow

Impediments to wider KubeFed adoption:

  1. Incompatibility of Kubernetes APIs – the federated-resources API results in extra learning and adoption efforts when users migrate from single cluster to multi-cluster architecture.
  2. Lack of extensibility – users were unable to satisfy varying use cases with the rigid nature of KubeFed. This led to an explosion of highly diverged forks and made it hard to build a community around a standardized implementation.

Given the difficulty of proliferating CRDs and extensibility limitations, there is a lack of roadmap or meaningful development in the KubeFed project to address these issues. Let’s take a look at two alternatives that are quickly adopted by users around the world.

Karmada: Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration

Karmada (Kubernetes Armada) is a Kubernetes management system that enables you to run your cloud-native applications across multiple Kubernetes clusters and clouds, with no changes to your applications. By using Kubernetes-native APIs and providing advanced scheduling capabilities, Karmada enables truly open, multi-cloud Kubernetes.

Karmada aims to provide turnkey automation for multi-cluster application management in multi-cloud and hybrid cloud scenarios, with key features such as centralized multi-cloud management, high availability, failure recovery, and traffic scheduling.

As the natural successor of Federation (v1) and KubeFed (v2), Karmada inherited a lot of concepts from these two versions. For example:

For more details please refer to Karmada Concepts.

Karmada concepts diagram flow

Key features of Karmada

Most of the features in KubeFed have been reformed in Karmada. In addition, Karmada also provides extra features like, more powerful scheduling policies, Aggregated Kubernetes API, Customizing Resource Interpreter, Multi-Cluster Service Discovery, Multi-Cluster Ingress, etc.

Open Cluster Management: Simplify fleet management across the open hybrid cloud at scale

Open Cluster Management (OCM) provides a framework to enable any capability within the Kubernetes ecosystem to orchestrate itself across multiple clusters and cloud providers. The built-in primitives allow an understanding of cluster inventory, cluster workload placement, and cluster workload distribution. The hub-agent model of OCM allows for new kinds of agent behavior to be injected into managed clusters easily. New behaviors can further configure clusters or collect information about clusters (such as observability data or search indexing). The project also provides two multicluster capabilities out of the box: application delivery, orchestration, and policy-based governance.

Open Cluster Management diagram flow

Key features of OCM

Differences between Karmada and OCM

Both projects are ready to take up the challenge of managing fleets of clusters across the hybrid and multi-cloud landscape, but they have different philosophies when it comes to solving it. Karmada provides a more complete full stack end to end solution. OCM provides a robust modular framework and APIs that enable other Kubernetes ecosystem projects to integrate with it, to unlock multicluster capabilities.

In the future, there will be many use cases where both Karmada and OCM can be complementary to each other. There is already an ongoing collaboration between both project maintainers in the Kubernetes SIG-Multicluster community to standardize the Work API, which is a project that distributes Kubernetes objects between clusters.

Final thoughts

The Kubernetes community is desperately looking for alternatives to KubeFed. Today we explored two exciting projects Karmada and Open Cluster Management, both of which enhance the scale and the capabilities of multicluster management well beyond what KubeFed provided. We call upon the Kubernetes community to give both projects a try and let’s continue to head towards the multicluster future.

Authors

References