Delivering High Performance Workflows with Cybozu and Cilium
Challenge
Cybozu, a Japanese software company known for its customizable workplace platform Kintone, decided to modernize its infrastructure from traditional on-premises virtual machines to Kubernetes on bare metal. The team built their own CNI plugin but needed additional features that could support their performance and operational requirements while integrating with existing tooling.
Solution
Cybozu chose Cilium as a core part of its Kubernetes networking layer, primarily motivated by Cilium’s eBPF-based kube-proxy replacement offering Direct Server Return (DSR), Maglev hashing, and increased scalability. Cilium also provided network policy enforcement for improved security and Hubble’s observability for better visibility into traffic flows.
Impact
Cybozu has been running Cilium in production for more than four years, with the platform facilitating a seamless migration to Kubernetes and significantly improving network performance, scalability, and security. Cilium provided an efficient service networking layer, incrementally applicable network policies, and Hubble for detailed network observability, becoming an essential part of Cybozu’s transition from VMs to Kubernetes.
Projects used
By the numbers
1000+
bare metal servers
4+ years
running Cilium in production
5 engineers
working on Kubernetes networking
Cybozu Boosts Performance with Cilium on Bare Metal
Cybozu is a Japanese software company known for its customizable low code/no code workflow platform Kintone, which lets teams manage data, tasks, and communication in one central place.
The company had been running its infrastructure using both public cloud and traditional on-premises virtual machines, but it became clear to the department managing the private cloud deployment that it was time to modernize its infrastructure to take advantage of performance improvements offered by cloud native. The team migrated its systems to Kubernetes deployed on bare-metal servers.
As part of the transition, the team built its own CNI plugin, Coil, which worked well but lacked load balancing capabilities to provide high availability and network policies support to secure multi-tenant environments. To fill these gaps, they needed additional solutions that could support their performance and operational requirements, integrate with their existing tooling, and allow for the development of their own custom components where necessary.
“Load balancing was the biggest challenge. We needed to decide the best way to implement load balancing for critical services to enable high availability and the preservation of client information, such as IP addresses, for backend servers.” – Tomoya Terashima, software engineer at Cybozu
Scalable, Efficient Load Balancing with Cilium
Before implementing Cilium, the team researched various OSS projects, read through academic papers, and even considered developing its own load balancer.
“We were looking for high availability and resiliency,” said Terashima. “We realized our initial implementation with Calico and Coil didn’t have enough availability to handle Layer 4 connectivity like TCP connection or UDP streams. It also couldn’t take care of the system if some nodes were taken down or failed.”
Cybozu adopted Cilium as a core part of its Kubernetes networking layer as it was the only “all-in-one” solution for its load balancing and network policy needs.
One of the primary motivations was Cilium’s Layer 4 load balancer powered by its eBPF-based kube-proxy replacement, which offered a more efficient implementation of Kubernetes service load balancing. This feature provided several advantages:
- Support for Direct Server Return (DSR), which enabled better throughput for high-traffic services.
- Maglev hashing allowed for stable and efficient load distribution across backend pods ensuring uptime and availability.
- kube-proxy replacement removed the need for iptables rules, which increased scalability.
In addition to load balancing, Cilium also offered network policy enforcement which helped maintain the security of clusters.
Hubble also provided observability, which gave the team better visibility into traffic flows within the cluster.
“Cilium’s largest value was higher service availability and increased security through network policy. Node failures and service disruptions happen, but we have been able to keep our service online through Maglev hashing. Network policy is also very easy to use, not just for our infrastructure team, but also for engineers across other teams looking to secure their own applications”. – Tomoya Terashima, software engineer at Cybozu
Cybozu is leveraging Cilium in CNI chaining mode, which allows the team to use Cilium alongside its existing CNI plugin, Coil. This flexibility lets them retain control over parts of their networking stack while relying on Cilium for service handling, observability, and policy enforcement.
Cilium Aids Transition from VMs to Kubernetes
Cybozu has now been running Cilium in production for over four years. During that time, they successfully migrated many of their workloads from their legacy VM-based infrastructure to Kubernetes clusters built on top of bare metal servers.
“When we first deployed Cilium we were running only a few types of services on Kubernetes clusters but now that Cilium has helped us ensure our applications can communicate stably on Kubernetes, we can move the rest of our services off of our old virtual machine based cloud,” said Takuya Yoshikawa, Deputy General Manager of the Cloud Infrastructure Division at Cybozu. “Cilium helped make this move possible.”
Cilium has facilitated a seamless migration to the new environment, and significantly improved network performance, scalability and security in several ways:
- It provided a consistent and efficient service networking layer, which reduced the performance tuning and troubleshooting effort often associated with iptables-based kube-proxy.
- Network policies could be applied incrementally, which helped improve the security posture of migrated workloads without requiring major rewrites.
- Hubble offered detailed observability during migrations, making it easier to trace issues like dropped connections or unexpected traffic paths.
One Tool Fits All: The Future with Cilium
Overall, Cilium has become an essential component of Cybozu’s evolving infrastructure. It has the right features for production-level infrastructure that the team was looking for while also having an open and active community, and delivering a high performance, portable networking solution.
“We needed to first check that our applications could run scalably on our Kubernetes clusters, and Cilium helped us a lot,” Yoshikawa continued that as the company’s cloud is expected to grow, the challenge of designing Kubernetes networking will be to not only make it work, but make it scale. “Without scalable networking, our customers may suffer severe performance outages. Cilium will help us achieve this stability and scale in performance.”
In the near future, the team plans to replace its old Cilium BGP integration with a new BGP Control Plane which will provide a seamless way to configure BGP policies and add flexibility to its network configurations.
As part of its long term vision, Cybozu wants Cilium to become its primary CNI, rather than using it in tandem with the internally developed Coil. “We believe using Cilium as our primary CNI will allow us to unlock its full potential and give us more optimized Kubernetes networking,” said Terashima.
Key Impacts:
- Scalability for future growth: Provides the foundation for scaling the platform as the company’s cloud is expected to grow
- Consistent and efficient service networking layer: Reduced the performance tuning and troubleshooting effort often associated with iptables-based kube-proxy
- Incremental network policy application: Improved the security posture of migrated workloads without requiring major rewrites
- Enhanced observability: Hubble offered detailed observability during migrations, making it easier to trace issues like dropped connections or unexpected traffic paths
- Higher service availability: Maglev hashing enabled the team to keep services online even during node failures and service disruptions
- Increased security: Network policy enforcement improved security of clusters
- Successful infrastructure modernization: Enabled migration of workloads from legacy VM-based infrastructure to Kubernetes on 1,000+ bare metal servers