Merging OpenTracing and OpenCensus: the path to convergence

Authors: Ted Young, Pritam Shah and the Technical Specifications Committee (Carlos Alberto, Bogdan Drutu, Sergei Kanzhelev and Yuri Shkuro).

The joint project has acquired the name: http://opentelemetry.io

Very, very briefly:

• We are creating a new unified set of libraries and specifications to enable telemetry monitoring. It will merge the OpenTracing and OpenCensus projects and provide a supported way to migrate.
• The reference implementation in Java will be available on April 24, and work on implementations in other languages ​​will begin in full on May 8, 2019. Get to know the schedule here.
• By September 2019, parity is planned with existing projects for C#, Golang, Java, NodeJS and Python. There is a lot of work ahead of us, but we can do it if we work in parallel. If you are interested in participating in this project, please register and let us know how you would like to contribute.
• When the implementation in each language becomes mature, the respective OpenTracing and OpenCensus projects will be closed. This means freezing old projects and the new project will continue to support the existing toolkit for two years using backwards compatibility.

project Overview

We are merging! The highest goal is to bring the OpenTracing and OpenCensus projects together into one common project.
The core of the new project will be a set of clean and well-thought-out interfaces, including the traditional assembly of libraries that implement these interfaces in the form of a so-called. SDK. The icing on the cake will be recommended standards for data and wire protocols, including common parts of the infrastructure.
The result will be a complete telemetry system suitable for monitoring microservices and other types of modern distributed systems, compatible with most major OSS and commercial backends.

Main events

24.04 — Reference candidate submitted for review.
8.05 - A team is formed, starting to work in all languages.
20.05 — Official launch of the project at Kubecon Barcelona.
6.09 - Implementations in C#, Golang, Java, NodeJS and Python achieve parity with peers.
6.11 - Official completion of the OpenTracing and OpenCensus projects.
20.11 — Farewell party in honor of the completion of projects at the Observability Summit, Kubecon San Diego.

Timeline of convergence

Migration for each language includes a production-ready SDK build, toolkit for popular libraries, documentation, CI, backwards compatibility tools, and closure of the related OpenCensus and OpenTracing projects (“sunset”). We have set an ambitious goal for September 2019 to reach parity for C#, Golang, Java, NodeJS and Python. We will be pushing the sunset date until all languages ​​are ready. But it's better to avoid it.
When reviewing goals, please consider your personal involvement, let us know by filling out registration form, or by saying hello in chats of Gitter projects opentracing и OpenCensus. You can view the chart as an infographic here.

Goal: First Draft Cross-Language Specification (due May 8th)

It is important to work as a team, even when working in parallel in different languages. The cross-language specification provides guidance for the project. Sounds prosaic, but it guarantees a consistent system that feels familiar regardless of the programming language.

Mandatory requirements for the first draft of the specification for language X:

• Definitions of general terminology.
• Model for describing distributed transactions, statistics and metrics.
• Clarifications on important issues that arose during the implementation.

This goal blocks the rest of the work, the first draft must be completed by May 8th.

Goal: First draft for data specification (completed by July 6th)

The data specification defines a common data format for traces and metrics so that data exported by all processes can be processed by the same telemetry infrastructure regardless of the data generation process. This includes the data schema for the tracing model described in the cross-language specification. Also included are metadata definitions for common operations that tracing uses to capture, such as HTTP requests, errors, and database queries. These semantic conventions are an example.

The first draft is based on the current OpenCensus data format and will contain the following:

• A data schema that implements a cross-language specification.
• Metadata definitions for common operations.
• JSON and Protobuf definitions.
• Implementation of reference clients.

Please note that there is also a wire protocol that distributes traces in-band, and which we would like to standardize as well. Distribution Format Trace-Context developed through the W3C.

Goal: parity across all major supported languages ​​(completion by September 6th)

We must achieve parity for the current language ecosystem in order to replace old projects with new ones.

• Interface definitions for tracing, metrics, and context propagation based on the cross-language specification.
• A ready-to-use SDK that implements these interfaces and exports Trace-Data. Where possible, the SDK will be built by porting an existing implementation from OpenCensus.
• Toolkit for popular libraries currently covered by OpenTracing and OpenCensus.

We also value backward compatibility and want to ensure a smooth transition from existing projects.

• The new SDK will have backwards compatibility with the current OpenTracing interfaces. They will allow legacy OpenTracing tools to work alongside newer tools in the same process, allowing users to gradually migrate their work.
• When a new SDK is ready, an upgrade plan will be created for current OpenCensus users. As with OpenTracing, legacy tools will be able to continue working alongside newer ones.
• By November, both OpenTracing and OpenCensus will be closed to receive changes. Backward compatibility with legacy tools will be maintained for two years.

Building a best-in-class SDK for each language takes a lot of work, and that's what we need the most.

Goal: Core Documentation (complete by September 6th)

Documentation is a critical success factor for any open source project. We want top-notch documentation and learning tools, and our tech writers are the most active developers on the project. Teaching developers how to properly monitor software is one of the biggest impacts we want to have on the world.

The following pieces of documentation are the minimum required to get started:

• Project Orientation.
• Observability 101.
• Beginning of work.
• Language guides (separately for each).

We welcome writers of all levels! Our new site is powered by Hugo using regular markup, so it's very easy to contribute.

Target: Registry v1.0 (completion by July 6th)

Registry - another critical component, improved version Open Tracing Registry.

• It is easy to find libraries, plugins, installers and other components.
• Easy management of Registry components.
• You can find out which SDK features are available in each language.

If you are interested in design, front-end and UX, we have a great project for personal participation.

Goal: Infrastructure for testing and release of software (completion by September 6)

To ensure we continue to deliver secure code you can rely on, we have a project commitment to building quality test and release pipelines. Please let us know if you can take care of testing, performance measurement and software release pipelines. We clearly define the level of production readiness, and the maturity of the testing infrastructure will be a major deciding factor for us.

Goal: close the OpenTracing and OpenCensus projects (completion by November 6th)

We plan to start closing old projects from September 6, if the new project reaches parity with them. 2 months later, with the parity of all languages, we plan to close the OpenTracing and OpenCensus projects. It should be understood like this:

• the repositories will be frozen, no more edits will be made.
• a two-year support period is planned for the current toolkit.
• users will be able to upgrade to the new SDK using the old toolkit.
• a gradual update will be possible.

Join

Any help would be welcome as this is a huge project. If you are interested in learning about observability, now is the time!

• Fill in registration form for direct participation and joining the team.
• Contact us on channels opentracing и OpenCensus Gitter.
• Check out Java reference implementationwhich is under development.

Source: habr.com