Refactoring Nutanix Kubernetes Platform for a Consistent Experience with Cilium
Challenge
Nutanix needed a more modern, efficient, and easier-to-manage CNI for its managed Kubernetes platform (NKP), especially as existing solutions posed complexity in configuration and lacked performance optimizations. Customers also required seamless networking between VMs and Kubernetes pods.
Solution
Nutanix adopted Cilium as the default CNI in NKP. Cilium was chosen for its simplicity, robust documentation, and advanced capabilities like DSR, Hubble observability, and identity-based security—all powered by eBPF.
Impact
Adopting Cilium significantly simplified Nutanix’s Kubernetes operations. The team found deployment and configuration much easier thanks to thorough documentation and a smooth Helm experience. Cilium’s features aligned closely with Nutanix’s roadmap and performance needs by enabling more precise traffic control, better policy enforcement, and smoother interoperability between Kubernetes workloads and VMs. Internally, the development experience has improved, and the team has had positive interactions with the Cilium community, reinforcing their long-term investment in the project.
Projects used
Highlights
- Cilium is now the default CNI deployed in NKP clusters.
- Seamless networking across Kubernetes and non-Kubernetes environments.
- Simplified deployment and Day 2 operations.
Finding a Seamless Solution in Cilium
Nutanix is a global leader in cloud software, offering a unified platform to run applications and manage data across on-premises data centers, public clouds, and edge. Its managed Kubernetes platform, Nutanix Kubernetes Platform (NKP), enables organizations to easily overcome Kubernetes Day 2 operational barriers, such as security, observability, reliability, policy management, and governance, when running Kubernetes on Nutanix infrastructure.
The Nutanix team became interested in Cilium after finding other open source solutions difficult to deploy, configure, and manage. They also wanted to implement Cilium as a replacement for kube-proxy to streamline configuration and simplify operations for customers.
When an opportunity came up to refactor the platform from scratch to provide a uniform experience across Nutanix infrastructure, the team incorporated Cilium as the default CNI plugin for NKP
“Cilium has very thorough and well-written documentation, and that makes a big difference not only for getting it up and running but also for the Day 2 experience,” said Daniel Lipovetsky, Senior Software Engineer at Nutanix. “We like that it’s forward-thinking and takes advantage of newer technologies. It also uses eBPF to deliver higher performance and remove some of the existing bottlenecks in traditional kernel and user-space networking.”
Forward-Thinking, Feature-Rich Networking
The team implemented Cilium because of features like Direct Server Return (DSR) for an efficient alternative to traditional overlay networking, its identity-based security model, and tunable eBPF map sizes making it ideal for high-performance environments with strict security requirements.
Cilium is now the CNI that is deployed whenever customers use NKP to create Kubernetes clusters on Nutanix managed infrastructure.
“Operations are much simpler since implementing Cilium,” said Lipovetsky. “Using a Helm chart made it incredibly easy to deploy and configure. Features like observability through Hubble are designed sensibly. They’re easy to deploy, easy to use, and essential to our roadmap.”
Consistent Network Policy from Pods to VMs
Another key goal was enabling seamless networking between Kubernetes clusters and traditional VM-based environments.
“There are still workloads that run best on VMs,” said Lipovetsky. “We want customers to have control between deploying on VMs or Kubernetes and be able to seamlessly cross that boundary. Cilium makes it easy for users to see traffic flows and define network policies between VMs and Kubernetes clusters running in the same environment.”
Cilium enables source IP preservation for more precise traffic control and cross-environment policy enforcement. Hubble also provides visibility into traffic flows both within the cluster, and between VMs and Kubernetes pods and services making unified network control simple and intuitive.
The Road Ahead: Expanding Cilium Across the Stack
Nutanix is actively extending its use of Cilium across more environments throughout the NKP stack. This shift enables an eBPF-native networking path, which improves performance, simplifies operations, and unlocks advanced features across hybrid workloads.
“We’ve already been using these features internally in development, and the experience has been great,” said Lipovetsky. “We’ve had good interactions with the development community and are excited to get more involved.”
Nutanix plans to further leverage Cilium’s capabilities to improve cross-environment consistency and deliver a more powerful, unified networking experience for customers regardless of where their workloads are running.