🥇Merging OpenTracing and OpenCensus: The Path to Convergence

Authors: Ted Young, Pritam Shah, and the Technical Specification Committee (Carlos Alberto, Bogdan Drutu, Sergey Kanzhelev, and Yuri Shkuro).

The collaborative project has been given the name: http://opentelemetry.io

In very brief:

We are creating a new unified set of libraries and specifications for telemetry observation. It will combine the OpenTracing and OpenCensus projects, and provide a supported way for migration.
The reference implementation in Java will be available on April 24, and work on implementations in other languages will fully begin on May 8, 2019. You can view the schedule here.
By September 2019, parity with existing projects for C#, Golang, Java, NodeJS, and Python is planned. We have a lot of work ahead, but we can manage if we work in parallel. If you are interested in participating in this project, please sign up and let us know how you would like to contribute.
When the implementation in each language reaches maturity, the relevant OpenTracing and OpenCensus projects will be closed. This means the old projects will be frozen, while the new project will continue to support the existing tooling for two years, using backward compatibility.

Project Overview

We are merging! The highest goal is to bring together the OpenTracing and OpenCensus projects into a single unified project.
The core of the new project will consist of clean and well-thought-out interfaces, including a traditional collection of libraries implementing these interfaces in the form of so-called SDKs. The cherry on top will be recommended standards for data and wire protocols, including common infrastructural components.
The result will be a comprehensive telemetry system suitable for monitoring microservices and other types of modern distributed systems, compatible with most major OSS and commercial base software.

Key dates

24.04 — Reference candidate provided for review.
08.05 — The team is being formed to start work in all languages.
20.05 — Official project launch at Kubecon Barcelona.
06.09 — Implementations in C#, Golang, Java, NodeJS, and Python achieve parity with their counterparts.
06.11 — Official completion of OpenTracing and OpenCensus projects.
20.11 — Farewell party to celebrate the completion of the projects at the Observability Summit, Kubecon San Diego.

Timeline of convergence

Migration for each language involves assembling an SDK ready for production deployment, tools for popular libraries, documentation, CI, backward compatibility tools, and phasing out related OpenCensus and OpenTracing projects ("sunset"). We set an ambitious goal for September 2019 — achieving parity for C#, Golang, Java, NodeJS, and Python. We will push back the sunset date until all languages are ready. But it is preferable to avoid this.
When reviewing the goals, please consider your personal participation and let us know by filling out the registration form, or by greeting in the project chats on Gitter OpenTracing and OpenCensus. You can view the schedule in infographic form here.

Goal: first draft of the cross-language specification (to be completed by May 8)

It’s important to work cohesively, even when working in parallel on different languages. The cross-language specification provides guidance for the project. It sounds prosaic, but it guarantees support for a cohesive system that feels familiar regardless of the programming language.

Mandatory requirements for the first draft of the specification for Language X:

Definitions of common terminology.
A model for describing distributed transactions, statistics, and metrics.
Clarifications on important issues that arise during implementation.

This goal blocks other work; the first draft must be completed by May 8.

Goal: first draft for the data specification (to be completed by July 6)

The data specification defines a common data format for traces and metrics so that data exported by all processes can be handled by the same telemetry infrastructure, regardless of the data generation process. This includes a data schema for the tracing model described in the cross-language specification. Also included are definitions of metadata for common operations that tracing uses for capture: for example, HTTP requests, errors, and database queries. These semantic agreements are examples.

The first draft is based on the current OpenCensus data format and will contain the following:

A data schema implementing the cross-language specification.
Definitions of metadata for common operations.
Definitions for JSON and Protobuf.
Implementation of reference clients.

Please note that there is also a wired protocol that distributes traces interband, and we would also like to standardize it. The distribution format Trace-Context is being developed through W3C.

Goal: parity across all major supported languages (completion by September 6th)

We need to achieve parity for the current language ecosystem by replacing old projects with new ones.

Interface definitions for tracing, metrics, and context distribution based on the cross-language specification.
A ready-to-use SDK that implements these interfaces and exports Trace-Data. If possible, the SDK will be created by porting the existing implementation from OpenCensus.
Toolkits for popular libraries currently covered in OpenTracing and OpenCensus.

We also value backward compatibility and want to ensure a smooth transition from existing projects.

The new SDK will have backward compatibility tools with the current OpenTracing interfaces. They will allow deprecated OpenTracing tools to work alongside new tools in the same process, enabling users to gradually migrate their developments.
When the new SDK is ready, an upgrade plan will be created for current OpenCensus users. As with OpenTracing, deprecated tools will continue to work alongside the new ones.
By November, both OpenTracing and OpenCensus will be closed to incoming changes. Backward compatibility with deprecated tools will be maintained for two years.

Creating a best-in-class SDK for each language requires immense effort, and this is what we need most.

Goal: core documentation (completion by September 6th)

A critical factor for the success of any open-source project is documentation. We want to have top-notch documentation and training resources, and our technical writers are the most active developers on the project. Teaching developers the right way to observe software is one of the most important impacts we want to have on the world.

The following parts of the documentation are minimally required to get started:

Project orientation.
Observability 101.
Getting started.
Language Guides (one for each).

We invite writers of all levels! Our new site is based on Hugo, using regular markup, making it quite easy to contribute.

Goal: Registry v1.0 (completion by July 6)

Registry — another critically important component, an improved version OpenTracing Registry.

Easily find libraries, plugins, installers, and other components.
Easy management of Registry components.
You can learn which SDK features are available in each language.

If you are interested in design, UI, and UX — we have a great project for personal involvement.

Goal: infrastructure for testing and releasing software (completion by September 6)

To continue delivering safe code that can be relied upon, we have a project commitment to create quality pipelines for testing and releasing software. Please let us know if you can take care of the pipelines for testing, performance measurement, and software release. We clearly indicate the production readiness level, and the maturity of the testing infrastructure will be a key deciding factor for us.

Goal: closure of OpenTracing and OpenCensus projects (completion by November 6)

We plan to start closing old projects on September 6, if the new project reaches parity with them. Two months later, when all languages are at parity, we plan to close the OpenTracing and OpenCensus projects. This should be understood as follows: