Guest post originally published on the New Relic blog by John Watson, lead engineer, and Lavanya Chockalingam, senior product marketing manager at New Relic

As a developer, you care about the performance of your applications, and you know just how challenging it can be to find the right instrumentation to capture telemetry data. One option is to use a proprietary agent, but what if you don’t want to be locked into that agent’s vendor?

You could use one of the open standards like OpenCensus or OpenTracing, but it can be tricky to decide which option is right for your application and workflows. Any option you choose must be able to scale and adapt to your needs as your applications change over time. But it’s also become increasingly important that the instrumentation you choose adheres to standards that can be shared across vendors and projects.

Vendor-neutral open standards matter. And OpenTelemetry, a Cloud Native Computing Foundation (CNCF) project, has created an open specification you can use to standardize how you collect and send telemetry data to backend platforms.

In this post, we’ll explain more about the OpenTelemetry project—how it works, the benefits it offers, and how to get started.

What is OpenTelemetry?

At its inception, OpenTelemetry formed by merging the OpenTracing and OpenCensus projects. OpenTelemetry offers a single set of APIs and libraries that standardize how you collect and transfer telemetry data. OpenTelemetry provides a secure, vendor-neutral specification for instrumentation so that you can send data to distinct backends of your choice, such as New Relic.

The following components make up the OpenTelemetry project:

(We’ll dive more into these components in the architecture section below.)

Key terms

If you’re new to OpenTelemetry, these are the key terms you need to know:

Benefits of OpenTelemetry

By combining OpenTracing and OpenCensus into a single open standard, OpenTelemetry provides the following benefits:

In short, instead of being bogged down in the nitty-gritty of decision making, you can focus your efforts on building amazing, reliable, solutions-based software. And that’s what it’s all about.

https://fast.wistia.net/embed/iframe/154rrpyjwv”

Using OpenTelemetry

The OpenTelemetry APIs and SDKs come with plenty of quick-start guides and documentation, so you can learn how to ingest data immediately. The quick-start guide for Java, for example, outlines how to acquire a tracer, create spans, add attributes, and propagate context across the various spans.

You will also find examples of common use cases to help get you started. These provide practical working code that demonstrates how to instrument HTTP/gRPC servers and clients, database connectors, and more.

After you have instrumented your application with OpenTelemetry trace APIs, you can send the trace data to an observability platform like New Relic or any other backend using one of the pre-built exporters in the OpenTelemetry registry.

Specifications for metrics and logs are still in the development phase, but once launched, they’ll play an important role in achieving OpenTelemetry’s main goal: to ensure libraries and frameworks have all built-in telemetry data types, which allows developers to ingest telemetry data without having to instrument it.

OpenTelemetry architecture components

Since OpenTelemetry is intended to be a cross-language framework for vendors and observability backends, it’s extremely flexible and extensible, yet it’s also quite complex. The architecture of a default implementation of OpenTelemetry breaks down into three components:

  1. The OpenTelemetry API
  2. The OpenTelemetry SDK, consisting of:
    • The Tracer pipeline
    • The Meter pipeline
    • A shared Context layer
  3. The Collector
Default implementation of OpenTelemetry architecture

Let’s take a look at each architecture component.

The OpenTelemetry API

Application developers use the Open Telemetry API to instrument their code, and library authors use it to write instrumentation directly into their libraries. The API does not address operational concerns or how data is sent to vendor backends.

We can break the  API into four parts:

  1. A Tracer API
  2. A Metrics API
  3. A Context API
  4. A set of semantic conventions
OpenTelemetry API diagram

The Tracer API

The Tracer API supports the generation of spans, which are named, timed operations representing a contiguous segment of work in a trace. Spans can be assigned a traceId and, optionally, be annotated with time-stamped events. A Tracer tags spans with a name and a version. When looking at your data, the name and version associated with a Tracer allow you to track the instrumentation library that generated the spans.

The Metric API

The Metric API provides access to various types of Metric instruments, such as Counters and  Observers. Counters allow you to count things. Observers allow you to measure values at discrete points in time; for example, you’d use Observers to “observe” values that aren’t taking place in the context of a Span, such as current CPU load or the total number of bytes free on a disk. See the specification for a list of instruments that cover the full range of possible metric use cases.

The Context API

The Context API adds context information—like W3C Trace ContextZipkin B3 headers, or New Relic distributed tracing headers—to spans and traces that use the same “context.” In addition, this API allows you to track how spans propagate within a system. The context is updated as a trace propagates from one process to the next. Metric instruments always have access to the current context.

Semantic conventions

Finally, the OpenTelemetry API contains a set of semantic conventions that includes guidelines and rules for naming spans, attributes, and for associating errors with spans (among other things). By encoding this in the API specification, the OpenTelemetry project ensures that all instrumentation, regardless of author or language, contains the same semantic information. This capability is enormously valuable to vendors looking to provide a consistent APM experience for all users.

https://fast.wistia.net/embed/iframe/sylxrsbow4″

The OpenTelemetry SDK

Next, the OpenTelemetry SDK is an implementation of the OpenTelemetry API. The SDK consists of roughly three parts, analogous to the APIs we just covered—a Tracer, a Meter, and a shared Context layer that ties it all together.

OpenTelemetry SDK architecture

Ideally, the SDK should fulfill 99% of standard use cases, but you can customize the SDK if needed. In the Tracer pipeline implementation, for example, you can customize anything aside from how the core implementation interacts with the shared Context layer. For example, within the implementation of the Tracer pipeline, you can customize the sampling algorithm that it uses.

The Tracer pipeline

The Tracer pipeline diagram

When configuring the SDK, you associate one or more SpanProcessors with the implementation of the Tracer pipeline. A SpanProcessor watches the lifecycle of spans, and delivers spans to a SpanExporter when appropriate. Built into the SDK is a simple SpanProcessor that forwards finished spans directly to the exporter one at a time.

The SDK also contains a batching implementation that forwards finished spans in batches on a configurable interval. However, since the SpanProcessor implementation accepts plug-ins, you can implement your own and assign it custom behavior. For example, if your telemetry backend supports viewing “in progress” spans, you could create a SpanProcessor implementation that sends spans for export at every change to the span state.

At the end of the Tracer pipeline is the SpanExporter. An exporter’s job is simple: convert the spans from their OpenTelemetry representations to whatever the telemetry backend requires and send them to that backend. Providing a custom SpanExporter is the easiest way for a telemetry vendor to participate in the OpenTelemetry ecosystem.

The Meter pipeline

The Meter diagram

The Meter pipeline is considerably more complex than the Tracer pipeline, as metrics are more complex than spans.  The following description is based on the Java implementation of the SDK and may vary across languages.

The Meter pipeline creates and maintains metrics instrumentation of various types, including Counters and Observers. Each instance of an instrument will need to be aggregated in some way. By default, Counters are aggregated by summing the values, and Observers are aggregated by taking the last value recorded. All of the instrument types have a default aggregation defined for them.

(As of this writing, the ability to configure custom aggregations for metrics instrumentation is still in the proposal stage.)

Implementations of the Meter Pipeline currently vary across languages, but in all cases, metric aggregations are delivered to the MetricExporter. Again, as in the case of spans, vendors can provide their own exporter to convert the aggregated data generated by the metric aggregators to the types that are needed for the telemetry backend.

OpenTelemetry supports two styles of exporter: “push”-based exporters, where the exporter sends data to the backend on a timed interval, and “pull”-based exporters, where the backend will query for the data when it wants it. New Relic is an example of a push-based backend, and Prometheus is a pull-based backend.

The shared Context layer

The shared Context layer sits between the Tracer and Meter pipelines and enables all non-observer metric recordings to take place in the context of an executing span. You can customize the Context with propagators, which propagate the span context both in and out of the system. As stated earlier, all OpenTelemetry SDK implementations provide an implementation of the W3C Trace Context specification, but you can optionally include Zipkin B3 propagation, or others.

The Collector

The Collector pipeline

The OpenTelemetry Collector is a stand-alone service that can ingest metrics and spans from various sources, including ZipkinJaeger, and OpenCensus. The collector can do tail-based sampling of spans, and enables exporting spans and metrics to a large number of vendor and open-source telemetry systems, including New Relic.

For more information about OpenTelemetry’s architecture, check out the full specification.

Start exploring OpenTelemetry today

Currently, OpenTelemetry is in beta phase. Right now is the ideal time to begin exploring what this project can do for you.

Check out this basic Java example that outlines how you can instrument your code to send metrics and traces to a backend like New Relic.

To find out more about other open source projects New Relic is involved in, visit New Relic Open Source.