Guest post by Saylor Berman, Senior Software Engineer, F5 NGINX and Ciara Stacke, Senior Software Engineer, F5 NGINX

Using minimal Docker containers is a popular strategy in the world of containerization due to benefits of security and resource efficiency. These containers are stripped-down versions of traditional containers, designed to contain just the essential components needed to run an application. In some cases, a base container may contain nothing at all (this would be the `scratch` container). 

Many people approach the creation of Docker images by making it as simple as possible to get their app working. This involves choosing a base image such as `ubuntu` or `python` that contains all the necessary libraries and tooling to get up and running. While easy, these images have an increased attack surface and memory footprint due to all the extra “stuff” inside. 

When we developed NGINX Gateway Fabric, our implementation of the new Kubernetes Gateway API, we leveraged several tools and strategies to ensure security and optimization. In this article we cover our design decisions:

Reduced Surface

We chose to start as small as possible. In fact, we can’t get any smaller. Based on `scratch`, our `nginx- gateway-fabric` image simply contains the Golang binary and nothing else. There are a couple steps taken before we produce the final build, but we ensure that the final build only contains what is necessary, which is just the binary. The Dockerfile looks something like this: 

FROM alpine:3.18 
RUN apk add --no-cache libcap 
COPY ./build/out/gateway /usr/bin/ 
RUN setcap 'cap_kill=+ep' /usr/bin/gateway 
FROM scratch 
USER 102:1001 
ENTRYPOINT [ "/usr/bin/gateway" ] 

This multi-step Dockerfile allows us to use `alpine` with the latest version of libcap to copy our pre-built binary into the proper location and set the necessary permissions for us to manage nginx. We then use `scratch` as the base for our production image and set the user and group ID so that we can control and limit permissions that our container has access to. 

With this approach, the `nginx-gateway-fabric` image is roughly the size of the binary itself, without any extra bloat. The binary does not need any extra dependencies to run and we have kept the size and attack surface as small as possible. 

Image Security Scanning 

One of the most effective ways to keep your product secure is by doing regular security scanning. With the help of Trivy, we run regular image security scans as part of our Github CI/CD pipeline. Trivy scans the container image for any known vulnerabilities (CVEs) that exist in libraries or binary files. Using the `scratch` image protects us from any vulnerabilities in the base image, but Trivy can still catch any vulnerabilities in the libraries that are compiled in our Golang binary. Results of the scan are uploaded to the 

Github Security tab for our repository to make it easy for our team to see any issues that are found. 

Code Quality and Security 

In addition to minimizing our container images, we also prioritize code quality and security within our application, employing a security-first mindset. We prioritize security from the outset, thoroughly evaluating each design and feature with a focus on security. We proactively identify and safeguard assets at the early stages of our processes, ensuring their protection throughout the development lifecycle, and we adhere to best practices for secure design, including proper authentication, authorization, and encryption mechanisms. An important part in upholding these standards is our use of a robust code review process. Every contributor to the project must open a pull request with any changes, and each pull request must be approved by at least two project maintainers. In addition to this process, we also incorporate the use of linters and tools to maintain our high standards for code security and quality. 

Static Code Analysis 

One way to boost code quality is to use Static Code Analysis tools (commonly abbreviated as SAST). One of the key advantages of using SAST is the ability to detect vulnerabilities early in the development process. We integrate CodeQL, a static code analysis tool developed by GitHub, into our CI/CD pipeline, which scans and identifies any potential issues in the pull request, as well as in the main branch. This allows us to remediate any potential code quality or security issues before we merge a change to the main branch. Similar to the Trivy scanning, any potential issues are uploaded to the GitHub Security tab for our repository to make it easy for our team to see any issues that are found. 

Keeping Dependencies Up-to-Date 

Keeping your project’s dependencies up-to-date is crucial for security and performance. In the NGINX Gateway Fabric project, we utilize Dependabot to automate this process by continuously monitoring our project’s dependencies and automatically opening pull requests when updates are available. This ensures that we’re always using the latest, most secure versions of our dependencies. 

Dependabot also provides security alerts by monitoring the Common Vulnerabilities and Exposures (CVE) database. When a security vulnerability is identified in one of our project’s dependencies, Dependabot promptly notifies us and automatically opens a pull request with the necessary updates. This, along with scanning our Docker images, enables us to patch vulnerabilities quickly. 

Deployment Security Best Practices 

As a critical component in a Kubernetes environment, we strive to follow Kubernetes security best practices wherever possible. This includes: 

What’s next?

Hopefully these secure development practices have given you some ideas on how to minimize risk of exposure while also developing resilient apps and choosing platform tools. We’d love to hear if you’re interested in using NGINX Gateway Fabric to improve security posture of your Kubernetes platform. We encourage you to: