Flux project post by Scott Rigby

Hey everyone! 👋If you couldn’t join Flux maintainers Stefan Prodan & Philip Laine’s session at GitOps Days 2022, you can now watch the full talk online! This blog post outlines the main points of this jam-packed 90 min talk with links to skip to specific sections 🎬👀 Let’s check it out! 🕵️

Flux Project Overview

For new users, Stefan starts 0:48 with a brief Flux project overview. This includes some context in which the legacy version was written during the early days of Kubernetes – a history that might interest everyone. 2:35 He then summarizes what you can do with the current version Flux, and 3:55 the sub-project Flagger.

Screenshot showing flux code example

Flux OCI support for Helm

One of the most requested features in the last year was getting Helm OCI support into Flux. 7:50 Exciting news is Helm OCI support was shipped in flux2 v0.31 release! This means you can use Flux to pull charts from OCI-compatible container registries, auth using Kubernetes Docker secrets, username/password credentials, or tokens for registries like GitHub that support those.

Stefan describes plans within the next month 9:10 to support OIDC authentication, self-signed certs for self-hosted container registries, amd dependencies from charts in Git to charts in OCI. He also shows 12:00 how the manifests differ when migrating from HTTPS Helm repositories to OCI repositories. Spoiler – the changes are clear and minimal 🙂

OCI Support for Kubernetes Configs

Screenshot showing flux code example

What if you want to use OCI for more than just Helm? 17:50 Flux already supports storing your desired state in various formats – Helm charts, plain YAML manifests, Kustomize overlays, and others like Terraform configuration when installing optional components such as Weave TF-controller. Good news, 18:12 there is an RFC for doing exactly this – allowing users to choose OCI as the source of truth for everything inside your cluster. This would allow you to use cosign for verification instead of PGP, all other features should work the same. The RFC is now marked implementable, and work is already underway!

Flux GitHub Repository Dispatch

20:40 In the latest version of Flux, a community member added integration with GitHub Repository Dispatch API, allowing you to build promotion pipelines – for example, to instruct Flux on a Staging cluster to always install the latest available Helm chart, run Helm tests there, and once the application is running as expected, Flux will send an event to the Dispatch API and a GitHub Action can promote that version from Staging to Production via a Git commit. See this step-by-step guide in our docs.

Managing Kubernetes Jobs with Flux

Philip presents the challenges and some solutions for managing Jobs with Flux. 23:47 Historically, triggering Kubernetes Jobs has been problematic with every GitOps tool, because a Kubernetes Job resource represents a pod that runs to completion once. To trigger the job again, you must create a new Job resource or delete the Job and recreate it. While most traditional CI/CD pipelines offer this feature, using Flux to reconcile a Job in your cluster today will run once, at which point its part is over. This has been a challenge for getting some developers on board with GitOps.

26:42 Philip explains Current solutions that work well out of the box with Flux today: CronJob – if your use case allows the pods to run on a schedule, and Helm hooks – the go-to solution for developers who want pre or post-deploy actions. Unfortunately for non-Helm users, there isn’t an equivalent option. Philip addresses this with 27:54 potential use cases and solutions for managing Jobs with GitOps independent of Helm.

Screenshot showing DB migration code example

29:09 Phillip explains the database migration use case, the current issues, and demos an alternative technique using two Kustomizations that solves the majority of complaints around Job triggering and Flux.

37:09 Philip explains how pre and post-deploy jobs are very common in traditional CI/CD systems, however Kubernetes Jobs limit use cases due to running as a single image, and this is where Tekton can help. He explains the current issues with Tekton and several solutions around them. Including 39:29 an early solution by Kevin McDermott integrating Flux notification controller with Tekton’s event listener. Another solution is 41:50 the use of webhooks within Flagger’s canary resource to implement for example Helm tests and the various things you can do with Flagger’s built-in load tester logic.

Screenshot showing build your own loadtester code example

43:17 This brings us to the part of the presentation Philip calls “I am sorry Stefan” due to his “misuse” of tools Stefan has built such as Flagger 😂 One such use case we don’t usually hear about is the ability to build your own Loadtester to extend Flagger’s built-in tester with more functionality and come up with new ideas. Philip demos a custom Loadtester that triggers Tekton for complex pipelines as part of a pre-rollout 🤯as well as additional use cases you might want to look at when running Jobs. 53:40 He clarifies the benefit of using Tekton for these types of solutions, largely because the run result is stored in the Kube API as opposed to in-memory, allowing for savings over multiple runs.

Observability for GitOps Pipelines

Stefan 55:00 talks about observability features and tools built into Flux. One of the main features built into Flux v2 tool relies on the magic of kstatus, which supports not only Kubernetes built-in resource types like Deployments, but also any Custom Resource Definitions that integrate the kstatus standard. This allows Flux to wait and report back the status of a particular Custom Resource.

1:01:34 Stefan then gives an overview of how to set up monitoring with Prometheus stack, Loki & Grafana dashboards, using Flux. He also reviews and explains the configuration details of custom Flux dashboards, and demos on how to use the dashboards for monitoring in different scenarios for various developer roles.

Screenshot showing Weave GitOps Core - OSS UI for Flux

But what about dashboards for non-CLI users? 1:17:20 For this Stefan demos Weave GitOps Core, an Open Source Web UI for Flux! 1:18:57 Stefan starts at the login interface and explains feature options such as Dex integration for OIDC providers. 1:19:39 We get to see how the web UI displays clean lists of applications and sources. 1:20:29 He deploys a microservices demo app to show how the UI allows you to expand an application to see all the other Flux objects reconciled on the cluster, and drill down another level to see the Kubernetes objects defined by a Flux object such as a Kustomization – as well as their events, a visual graph of the resources and their sources, and real time status. There is also a view to see the YAML definition of each Flux object as it exists in the cluster, which helps with debugging for those who don’t want to use the CLI. 

The Video!

Here’s the full video if you’d like to watch it from start to finish:

Next Steps

To learn more about Flux, check out the Flux website and docs and give us a star on GitHub, and feel free to reach out on the CNCF Slack #flux channel (get an invite here).

Did you miss the GitOps Days conference? No worries, you can watch all sessions on-demand at the GitOps Days 2022 Playlist.