Member post originally published on the Palark blog by Dmitry Shurupov, editor-in-chief, Palark
In the recent werf v2.0 release, the CI/CD solution has unveiled a new deployment engine called Nelm. It has replaced Helm while retaining backward compatibility and introducing new prominent features. You can try them in werf right now with other tools to come around the corner. Let’s explore the history of the project and examine the path it’s traveled to this transformation.
The quickest introduction to werf
If you’re not aware of werf, it’s a CNCF Sandbox project and CLI tool for implementing CI/CD with Kubernetes. Dubbed “the missing part of your CI/CD system,” it is an opinionated, all-in-one solution covering the entire CI/CD pipeline, from the building of container images all the way to their deployment.
werf does not replace a CI system but rather integrates with the one you prefer. E.g., it comes with ready-to-use integrations for GitLab CI/CD and GitHub Actions. When you integrate your CI with werf, the latter is executed to build and deploy images as well as help with debugging, testing, and releasing your software.
If you were to look at your CI/CD as an assembly line — just like a real manufacturing process — werf is your universal industrial robot, capable of doing everything needed of it at every single workstation (i.e. CI/CD step) as the product (your software) makes its way little gradually until it’s finalized (run in Kubernetes).
Note: This article provides a bit more detailed introduction to what werf is.
How werf has evolved
werf was started in a DevOps service company that wished to automate its common CI/CD practices that it utilized for various customers. Back in 2016, it was a simple Ruby program that helped you build Docker images by executing docker build and adding some magic around it.
In the years that followed, it added more and more features that would be available out-of-the-box and easy to use. Initially, they were focused solely on building container images, but later, they would go on to cover other CI/CD steps as well.
If you’re not using werf, a common approach to implementing container-based CI/CD is to pick and integrate well-known tools that fulfill all your needs. It’s somewhat similar to bringing all those wonderful Unix-based CLI commands into a massive one-liner full of pipes.
That’s exactly what werf has done — become a “glue” for various tools which can be used with the CI system of your choice. However, it’s much more than merely a “glue”, since it comes with a number of extra features that are available on top of the solid foundation the cloud native ecosystem provides. Helm in werf serves as a great illustration of that.
What about Helm?
Deploying the apps to Kubernetes via werf was a major milestone in the project’s history. This is when Helm came into play, establishing itself as one of the tool’s pillars. That happened as early as 2017, much before Helm became a graduated CNCF project. Yet the choice was obvious even back then. Helm was quite mature and, what is more, shined as the de facto standard in the cloud native ecosystem whenever you needed to deploy something to Kubernetes.
However, as great a tool as it was, the pace of Helm’s development significantly stalled in the following years. Why? It’s essential to note that Helm is responsible for two things: packaging your software (think templates and charts) and deploying it to the clusters (helm install and everything it has to do with). While some Kubernetes users are not happy with the convenience and feature-richness of templates in Helm*, its charts have become the most common way to pack your apps for Kubernetes, a community-adopted standard. It’s still relevant for 2024: the vast majority of the software you want to run in K8s is described in Helm charts.
* By the way, our “Making the most out of Helm templates” article might prove handy for those interested in making their Helm templates more flexible and dynamic.
But that’s not Helm’s sole use case. As outlined above, Helm not only helps you describe the Kubernetes resources for your software, but it is also used to deploy to K8s. While many users rely solely on Helm to deploy their apps, nowadays, those are mostly basic scenarios lacking numerous additional features that other tooling provides.
Many of us would prefer the GitOps approach and the appropriate technologies to implement it — Argo and Flux. No wonder both of these CNCF projects graduated almost simultaneously in 2022 (Flux vs. Argo).
At the same time, Helm’s deployment features haven’t seen much improvement lately.
Since Helm was used in werf for both describing and deploying software, this slowdown affected werf and its potential in numerous use cases. A patched version of Helm has been used in werf for a long time, but that could not last forever.
After years of seeing that Helm just wasn’t going over the hump, the team behind werf decided to move forward with its fork and start Nelm as an alternative project.
Here comes Nelm
Nelm is an Open Source project born in the bowls of werf. It is a partial reimplementation of Helm, with its deployment subsystem rewritten from scratch and an improved chart management subsystem. It continues to develop Helm by adding new features while maintaining backward compatibility.
Features
Nelm already offers helpful features that Helm is missing. They cover various areas of the user experience. For example, Nelm relies on the Server-Side Apply (SSA) for updating Kubernetes resources, which adds significantly to consistency and robustness. While Helm relies on the 3-Way Merge (3WM), the SSA adopted in Nelm is a popular choice in the industry, which is also applied in Kubernetes (stable since v1.22), Flux, Argo CD, and kubectl kustomize.
Another example is advanced resource status tracking, which has made Nelm a much more transparent, user-friendly tool during the deployment process. It displays real-time status, logs, and resource events and even performs an automatic rollback in the event that issues arise. Moreover, with Nelm, you can preview your changes similarly to what kubectl apply --dry-run does, with no drawbacks foisted by helm-diff.
To learn about the intricacies of Nelm features, watch this recent CNCF-hosted webinar: “From improving Helm to developing Nelm: The evolution of deployments in werf”.
The planned features for Nelm are no less exciting. Remember the aforementioned complaints lodged about the Helm templates? Adopting an alternative templating mechanism, be it CUE or a full-featured language (such as TypeScript), is on the Nelm to-do list. Enjoying the Kustomize approach to defining your app’s resources? An advanced resource patching implementation is on the way too. GitOps enthusiasts might be happy to see a soon-anticipated Flux + Nelm integration.
Library and CLI tool
Developed as a cornerstone of werf, Nelm is primarily a library. That’s where the project’s efforts are currently focused. The good news is that this library can be used in any other software. If a Kubernetes-based tool or solution might benefit from Nelm’s features, it should be easy to embed them, similar to the way it’s done in werf already.
However, Nelm’s potential as a CLI tool — some kind of drop-in replacement for Helm — also seems to be huge. Consequently, a standalone nelm command expanding the library’s capabilities to terminal users will be available in the coming months as well.
Now, let’s back to the werf story.
werf 2.0
Nelm substituted a Helm fork under werf’s hoof and progressed through various werf release channels to become mature enough: from alpha to beta, then early access, and stable… However, at this point, the differences between Nelm and Helm have grown stark. Due to the stricter validation that the SSA has imposed, some werf users have had to fix long-standing issues in their Helm charts. While they were never detected in Helm, they could not pass the validation rendered in Nelm.
Although it did affect a few users and charts, it did not seem reasonable to force the users of the werf’s stable version to modify their charts. It also did not seem reasonable to weaken the Nelm validation mechanism, which would literally mean “artificially introducing several flaws into it” [to fully reproduce all Helm 3WM-specific issues].
Therefore, werf’s next major release — v2.0 — was executed at the end of April and formally announced in May. This version enabled Nelm by default for all werf users, including those who stick with the stable release channel. It’s opened up a new chapter in werf’s 8-year lifetime.
With Nelm as a default deployment engine, werf can embrace and offer new features at its own pace. This process has already started with the recently introduced werf.io/deploy-dependency-<name> annotation, providing a means to specify the order in which Kubernetes resources are deployed with more flexibility than ever.
Summary
werf has taken a long journey from a simple script to a comprehensive CI/CD solution gluing well-known tools to your favorite CI system. Helm has played a vital role in it, being one of the most essential components. However, with werf v2.0 released, it has now been replaced by Nelm.
Nelm was crafted by the werf team based on their experience using and improving Helm over the years, following the needs of numerous production users. It is backward-compatible with Helm, production-ready, ushers in a variety of improvements and new features, and has ambitious plans for the future.
As for using Nelm today, it is available as a library, and you can run werf to see it in action. You’re also most welcome to adopt it in other tools. A standalone CLI tool will follow shortly, allowing you to try out Nelm’s capabilities without werf. You might also be excited about a Flux integration soon to benefit from Nelm in your GitOps-based workflow. Stay tuned!