Project post by Eduardo Silva, creator of Fluent Bit
The Cloud Native Computing Foundation and the Fluent Bit community are thrilled to announce that Fluent Bit has been downloaded and deployed over one billion times, rapidly reaching a milestone that few software projects ever achieve. As Fluent Bit is helping users solve complex observability challenges across cloud-native, IoT, and bare metal environments, and is embedded into primary Kubernetes distributions, it has quickly become industry standard technology – a core element of any enterprise observability platform.
Designed with high performance in mind, Fluent Bit is a highly scalable logs and metrics processor. It has rapidly grown in popularity as it enables any organization to 1) Effectively manage observability data at scale and 2) Receive immediate insight into their systems’ performance. For context, in order for any IT organization to meet SLAs around availability and security, it must maintain a near real-time understanding of the state of its infrastructure. The challenge has always been how to quickly and efficiently get intelligence out of the mountain of log and event data created across distributed environments in the enterprise. And this problem continues to get more difficult, as organizations quickly adopt distributed, dynamic, cloud-native services – which provide many, substantial benefits, but also mean a more complex, ephemeral environment, and even more data. Fluent Bit addresses these issues by enabling log processing at petabyte scale, adding additional context to event data in Kubernetes based environments, reducing overall observability costs by routing to multiple backends, and serving as a vendor-neutral collector to avoid vendor lock-in.
Fluent Bit is a CNCF-Graduated project under the umbrella of Fluentd, alongside other foundational technologies such as Kubernetes and Prometheus. This status means it is proven in production and ready for deployment by almost any enterprise. Fluent Bit is downloaded between 2-5 million times daily. It is used in production by some of the largest and most complex organizations including: AWS Cloud, Google Cloud, Digital Ocean, VMware, Cisco, Lyft, LinkedIn, and Walmart.
“It has been great to see the Fluent Bit project grow to where it is now. Capturing and forwarding metrics and logs is a critical part of any observability system. Doing so in a reliable and lightweight way is a hard problem. The success of Fluent Bit speaks for itself. VMware Tanzu is happy to participate and lean on Fluent Bit as part of our cloud native solutions. Huge congrats to the Fluent Bit team!” – Joe Beda, Principal Engineer and Kubernetes co-founder, VMware Tanzu
“DigitalOcean successfully deploys Fluent Bit to thousands of nodes across multiple products and systems to provide both internal and customer log processing capability. Hitting a billion downloads is a really big milestone for Fluent Bit. Congratulations to the whole community for making a reliable solution for a critical problem.” – Bojan Djurkovic, Senior Software Engineer, DigitalOcean
“We’ve been using Fluent Bit for routing logs from some large Kubernetes clusters to the Kafka infrastructure. It has been working very well at large scale and has a very small footprint – both cpu and memory usage are amazingly low. It keeps chugging day-after-day without incident. It is rock solid and very high performance. It is easy to set up and easy to debug too. We have a new and broader application of Fluent Bit in development now, so we will continue to use Fluent Bit both within Kubernetes and outside Kubernetes.” – Brian McQueen, Senior SRE, LinkedIn
We invite those who are unfamiliar with the project to join us at FluentCon Europe, a co-located event with KubeCon Europe – May 16, 2022 onsite in Valencia, Spain to learn more about how folks are using Fluent Bit at scale. Additionally, we welcome users to join the 7,000+ members in the Fluent Slack channel where they can learn and share their use cases.