Guest post by Michelle Nguyen, Principal Software Engineer at New Relic

As a way to reemphasize our commitment to open source and open standards, we are sharing an exciting development in open source observability. Pixie, a real-time debugging platform for Kubernetes, has just announced their plugin system.

The plugin system allows users to export data from Pixie’s rich dataset to any tool which supports OpenTelemetry. This includes data such as pod details, full-body application requests, flame graphs, and more. Using Pixie’s auto-telemetry capabilities, this means you can immediately start exploring Pixie data within your favorite platform, without having to do any manual instrumentation.

We’ll discuss:

Open Specifications and APIs for Observability

In the days of proprietary agents, deploying a collector typically required that data be sent to its corresponding dashboard platform for analysis. Deploying another unsupported data source would require yet another product to adopt and learn. Most teams report that they are using up to 15 projects just for observability [1]! Ultimately, this leads to data and process silos across teams, making it impossible to build a complete picture of their cluster and applications.

Open source data collectors and standards, such as OpenTelemetry, solve this problem by improving interoperability in observability tooling. By leveraging data collectors that follow open standards, developers can send data to their platform of choice. Switching out that platform is as easy as updating a set of configs. This enables developers to flexibly select products that best fit their needs. Many projects have now adopted open standards, from Jaeger to Fluentd.

Pixie’s Plugin System

Screenshot shows Pixie Plugin System dashboard

New Relic is a proud contributor to Pixie, a CNCF sandbox project which has recently joined this movement by introducing the Pixie Plugin System

By leveraging technology like eBPF, Pixie is able to automatically collect telemetry data from your cluster without requiring any manual instrumentation. Some of the data that Pixie is able to collect are: 

All of this data is stored in-cluster for performance and security benefits, with only minor CPU impact (up to 5% overhead). 

Diagram showing data collected in-memory without redeploys or instrumentation to data converted to OpenTelemetry format and sent to OTel collector and then data sent to any OpenTelemetry supported destination

Pixie’s new plugin system utilizes open standards to leverage the strengths of other tools in the ecosystem. The initial version of the plugin system addresses Pixie’s data storage limitations. As Pixie stores its collected data in-cluster, this restricts data storage within Pixie to the last 24 hours. The plugin system now allows developers to export their Pixie data in the OpenTelemetry format to any tool for long-term data retention. This enables users to:

Getting Started

It’s easy to give the plugin system a try! Here are some materials to help you get started: