{"id":31014,"date":"2019-10-31T21:38:54","date_gmt":"2019-10-31T18:38:54","guid":{"rendered":"https:\/\/prohoster.info\/blog\/trassirovka-servisov-opentracing-i-jaeger\/"},"modified":"2019-10-31T21:38:54","modified_gmt":"2019-10-31T18:38:54","slug":"trassirovka-servisov-opentracing-i-jaeger","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/trassirovka-servisov-opentracing-i-jaeger","title":{"rendered":"Service Tracing, OpenTracing and Jaeger","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Service Tracing, OpenTracing and Jaeger\" src=\"\/wp-content\/uploads\/2019\/04\/045800f00c8da9702f62cf3de53df7b6.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nIn our projects, we utilize a microservices architecture. When performance bottlenecks arise, a significant amount of time is spent monitoring and analyzing logs. When logging the timings of individual operations to a log file, it is typically challenging to understand what triggered those operations, track the sequence of actions, or determine the time offsets of one operation relative to another across different services. <\/p>\n<p>To minimize manual labor, we decided to use one of the tracing tools. This article will discuss how and why tracing can be used, as well as our experiences with it. <br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h3>What problems can be solved with tracing<\/h3>\n<p><\/p>\n<ol>\n<li>Identifying performance bottlenecks both within a single service and across the entire execution tree among all involved services. For example:\n<ul>\n<li>A series of many short consecutive calls between services, for instance, for geocoding or querying the database.<\/li>\n<li>Long waits for input\/output, such as data transfer over the network or reading from disk.<\/li>\n<li>Long data parsing operations.<\/li>\n<li>Long operations that require CPU.<\/li>\n<li>Code segments that are unnecessary for obtaining the final result and can be removed or executed lazily.<\/li>\n<\/ul>\n<\/li>\n<li>Visually understand the order in which calls occur and what happens when an operation is executed.<br \/>\n<img decoding=\"async\" alt=\"Service Tracing, OpenTracing and Jaeger\" src=\"\/wp-content\/uploads\/2019\/04\/f22f14599568bff5573fe59eeeaea753.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nIt is clear that, for example, a request arrived at service WS -&gt; service WS supplemented data via service R -&gt; then sent a request to service V -&gt; service V loaded a lot of data from service R -&gt; went to service P -&gt; service P went back to service R -&gt; service V ignored the result and went to service J -&gt; and only then returned the response to service WS, while continuing to compute something else in the background.<br \/>\nWithout such a trace or detailed documentation for the entire process, it is very difficult to understand what is happening when looking at the code for the first time, especially since the code is scattered across various services and hidden behind a multitude of beans and interfaces.\n<\/li>\n<li>Gathering information about the execution tree for subsequent deferred analysis. At each step of execution, additional information can be added to the trace that is available at that stage, allowing further investigation into what input data led to the particular scenario. For example:\n<ul>\n<li>User ID<\/li>\n<li>Permissions<\/li>\n<li>Selected method type<\/li>\n<li>Log or execution error<\/li>\n<\/ul>\n<\/li>\n<li>Transforming traces into a subset of metrics and further analysis in the form of metrics.<\/li>\n<\/ol>\n<h3>What tracing can log. Span<\/h3>\n<p>In tracing, there is the concept of a span, which is analogous to a single log in the console. A span has:<\/p>\n<ul>\n<li>A name, usually this is the name of the method that was executed<\/li>\n<li>The name of the service in which the span was generated<\/li>\n<li>A unique ID<\/li>\n<li>Some meta-information in the form of key\/value that has been logged into it. For example, method parameters or whether the method finished with an error or not<\/li>\n<li>Start and end time of the execution of this span<\/li>\n<li>Parent span ID<\/li>\n<\/ul>\n<p>\nEach span is sent to the span collector for storage in the database for later viewing as soon as it completes its execution. Subsequently, a tree of all spans can be constructed by connecting them by parent ID. During analysis, it is possible to find, for example, all spans in a service that took longer than a specified time. Then, by clicking on a specific span, you can see the entire tree above and below this span.<\/p>\n<p><img decoding=\"async\" alt=\"Service Tracing, OpenTracing and Jaeger\" src=\"\/wp-content\/uploads\/2019\/04\/9cc97f1278af60f0efd5e393cd183c5c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Opentrace, Jaeger, and how we implemented this for our projects<\/h3>\n<p>\nThere is a general standard <noindex><a rel=\"nofollow\" href=\"https:\/\/opentracing.io\/docs\/overview\/\">Opentrace<\/a><\/noindex>, which describes how and what should be collected, without tying tracing to a specific implementation in any language. For example, in Java, all work with traces is done through the common Opentrace API, while underneath it there may be Jaeger or a blank default implementation that does nothing.<br \/>\nWe use <noindex><a rel=\"nofollow\" href=\"https:\/\/www.jaegertracing.io\/\">Jaeger<\/a><\/noindex> as the implementation of Opentrace. It consists of several components:<\/p>\n<p><img decoding=\"async\" alt=\"Service Tracing, OpenTracing and Jaeger\" src=\"\/wp-content\/uploads\/2019\/04\/1fd4ae8563569271b067861c81101d71.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<ul>\n<li>Jaeger-agent \u2014 a local agent that is usually on each machine and to which services log to a local default port. If there is no agent, then traces of all services on that machine are usually turned off<\/li>\n<li>Jaeger-collector \u2014 all agents send the collected traces to it, and it saves them in the chosen database<\/li>\n<li>Database \u2014 their preferred is Cassandra, but we use Elasticsearch, there are implementations for a couple of other databases, and an in-memory implementation that does not save anything to disk<\/li>\n<li>Jaeger-query \u2014 this service queries the database and returns the collected traces for analysis<\/li>\n<li>Jaeger-ui \u2014 this is the web interface for searching and viewing traces, it queries Jaeger-query<\/li>\n<\/ul>\n<p><img decoding=\"async\" alt=\"Service Tracing, OpenTracing and Jaeger\" src=\"\/wp-content\/uploads\/2019\/04\/afc415cb5d4f19e7ba18394c2c1a9dd5.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nA separate component can be called the Opentrace Jaeger implementation for specific languages, through which spans are sent to Jaeger-agent.<br \/>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/jaegertracing\/jaeger-client-java\">Connecting Jaeger in Java<\/a><\/noindex> The process involves implementing the interface io.opentracing.Tracer, after which all traces will be sent to the actual agent through it.<\/p>\n<p><img decoding=\"async\" alt=\"Service Tracing, OpenTracing and Jaeger\" src=\"\/wp-content\/uploads\/2019\/04\/5ce4bdfc06def7e4645bd58163e99dab.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nYou can also connect to Spring components <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/opentracing-contrib\/java-spring-cloud\">opentracing-spring-cloud-starter<\/a><\/noindex> and the implementation from Jaeger <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/opentracing-contrib\/java-spring-jaeger\">opentracing-spring-jaeger-cloud-starter<\/a><\/noindex> which will automatically configure tracing for everything that passes through these components, such as HTTP requests to controllers, database queries through JDBC, etc.<\/p>\n<h3>Logging traces in Java<\/h3>\n<p>\nAt the very top level, the first Span should be created, which can be done automatically by a Spring controller when receiving a request, or manually if there isn't one. It is then passed through to lower Scope levels. If any method further down wants to add a Span, it takes the current activeSpan from Scope, creates a new Span, indicates that its parent is the obtained activeSpan, and makes the new Span active. When calling external services, the current active span is passed to them, and those services create new spans tied to this span.<br \/>\nAll operations are done through an instance of Tracer, which can be obtained via the DI mechanism or GlobalTracer.get() as a global variable if the DI mechanism does not work. By default, if the tracer has not been initialized, a NoopTracer will be returned, which does nothing.<br \/>\nNext, the current scope is retrieved from the tracer through ScopeManager, a new scope is created from the current one with the new span attached, and subsequently, the created Scope is closed, which closes the created span and returns the previous Scope to an active state. The Scope is tied to the thread, so in multithreaded programming, it is essential to remember to pass the active span to another thread for the activation of that thread's Scope linked to this span.<\/p>\n<pre><code class=\"java\">io.opentracing.Tracer tracer = ...; \/\/ GlobalTracer.get()\n\nvoid DoSmth () {\n   try (Scope scope = tracer.buildSpan(\"DoSmth\").startActive(true)) {\n      ...\n   }\n}\nvoid DoOther () {\n    Span span = tracer.buildSpan(\"someWork\").start();\n    try (Scope scope = tracer.scopeManager().activate(span, false)) {\n        \/\/ Do things.\n    } catch(Exception ex) {\n        Tags.ERROR.set(span, true);\n        span.log(Map.of(Fields.EVENT, \"error\", Fields.ERROR_OBJECT, ex, Fields.MESSAGE, ex.getMessage()));\n    } finally {\n        span.finish();\n    }\n}\n\nvoid DoAsync () {\n    try (Scope scope = tracer.buildSpan(\"ServiceHandlerSpan\").startActive(false)) {\n        ...\n        final Span span = scope.span();\n        doAsyncWork(() -&gt; {\n            \/\/ STEP 2 ABOVE: reactivate the Span in the callback, passing true to\n            \/\/ startActive() if\/when the Span must be finished.\n            try (Scope scope = tracer.scopeManager().activate(span, false)) {\n                ...\n            }\n        });\n    }\n}<\/code><\/pre>\n<p>For multithreaded programming, there is also TracedExecutorService and similar wrappers that automatically propagate the current span into the thread when starting asynchronous tasks:<\/p>\n<pre><code class=\"java\">private ExecutorService executor = new TracedExecutorService(\n    Executors.newFixedThreadPool(10), GlobalTracer.get()\n);<\/code><\/pre>\n<p>For external HTTP requests, there is <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/opentracing-contrib\/java-apache-httpclient\">TracingHttpClient<\/a><\/noindex><\/p>\n<pre><code class=\"java\">HttpClient httpClient = new TracingHttpClientBuilder().build();<\/code><\/pre>\n<h3>The issues we encountered<\/h3>\n<p><\/p>\n<ul>\n<li>Beans and DI do not always work if the tracer is not used in a service or component, then <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/users\/autowired\/\" class=\"user_link\">Autowired<\/a><\/noindex> Tracer may not work and you will have to use GlobalTracer.get().<\/li>\n<li>Annotations do not work if it is not a component or service, or if the method call occurs from another method of the same class. You must be careful, check what works, and use manual trace creation if @Traced does not work. You can also attach an additional compiler for Java annotations, then they should work everywhere.<\/li>\n<li>In older Spring and Spring Boot, the auto-configuration of opentracing Spring Cloud does not work due to bugs in DI, so if you want traces to work automatically in Spring components, you can do it similarly to <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/opentracing-contrib\/java-spring-jaeger\/blob\/master\/opentracing-spring-jaeger-starter\/src\/main\/java\/io\/opentracing\/contrib\/java\/spring\/jaeger\/starter\/JaegerAutoConfiguration.java\">github.com\/opentracing-contrib\/java-spring-jaeger\/blob\/master\/opentracing-spring-jaeger-starter\/src\/main\/java\/io\/opentracing\/contrib\/java\/spring\/jaeger\/starter\/JaegerAutoConfiguration.java<\/a><\/noindex><\/li>\n<li>In Groovy, try-with-resources does not work, you must use try finally.<\/li>\n<li>Each service must specify its own spring.application.name under which the traces will be logged. Moreover, a separate name for production and testing is needed to avoid interference.<\/li>\n<li>If you use GlobalTracer and Tomcat, all services launched in this Tomcat have one GlobalTracer, so they will all have the same service name.<\/li>\n<li>When adding traces to a method, you must ensure that it is not called in a loop many times. You should add one common trace for all calls that logs the total execution time. Otherwise, excessive load will be created.<\/li>\n<li>Once in jaeger-ui, we made too large requests for a large number of traces and since we didn't wait for a response, we made it again. As a result, jaeger-query started consuming a lot of memory and slowing down Elastic. A restart of jaeger-query helped.<\/li>\n<\/ul>\n<h3>Sampling, storage, and viewing of traces<\/h3>\n<p>\nThere are three types <noindex><a rel=\"nofollow\" href=\"https:\/\/www.jaegertracing.io\/docs\/1.9\/sampling\/\">of trace sampling<\/a><\/noindex>:<\/p>\n<ol>\n<li>Const which sends and saves all traces.<\/li>\n<li>Probabilistic which filters traces with a certain given probability.<\/li>\n<li>Rate limiting that restricts the number of traces per second. These parameters can be configured on the client, either on jaeger-agent or in the collector. Currently, we use const 1 in our stack of validators since there are not too many requests, but they take a considerable amount of time. In the future, if this causes excessive load on the system, we can limit it.<\/li>\n<\/ol>\n<p>\nIf using Cassandra, by default it retains traces for only two days. We use <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/jaegertracing\/jaeger\/tree\/master\/plugin\/storage\/es\">Elasticsearch<\/a><\/noindex> and traces are stored indefinitely and are not deleted. A separate index is created for each day, for example, jaeger-service-2019-03-04. In the future, automatic cleanup of old traces needs to be configured.<\/p>\n<p>To view traces, you need to:<\/p>\n<ul>\n<li>Select the service you wish to filter traces by, for example, tomcat7-default for a service that is running in Tomcat and cannot have its own name.<\/li>\n<li>Then select the operation, time frame, and minimum operation duration, for instance, from 10 seconds, to capture only the long-running executions.<br \/>\n<img decoding=\"async\" alt=\"Service Tracing, OpenTracing and Jaeger\" src=\"\/wp-content\/uploads\/2019\/04\/bc712d4aabfe9c2ae961d3b8f9396a42.png\" style=\"display:block;margin: 0 auto;\" \/><\/li>\n<li>Go into one of the traces and see what caused the slowdown.<br \/>\n<img decoding=\"async\" alt=\"Service Tracing, OpenTracing and Jaeger\" src=\"\/wp-content\/uploads\/2019\/04\/16e9dd11903358f5498fe11034bbf848.png\" style=\"display:block;margin: 0 auto;\" \/><\/li>\n<\/ul>\n<p>\nAdditionally, if a request ID is known, it can be found by searching with tags if this ID is logged in the trace span.<\/p>\n<h3>Documentation<\/h3>\n<p><\/p>\n<ul>\n<li>OpenTracing documentation <noindex><a rel=\"nofollow\" href=\"https:\/\/opentracing.io\/docs\/overview\/what-is-tracing\/\">opentracing.io\/docs\/overview\/what-is-tracing<\/a><\/noindex><\/li>\n<li>Jaeger documentation <noindex><a rel=\"nofollow\" href=\"https:\/\/www.jaegertracing.io\/docs\/1.10\/\">www.jaegertracing.io\/docs\/1.10<\/a><\/noindex><\/li>\n<li>Jaeger Java integration <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/jaegertracing\/jaeger-client-java\">github.com\/jaegertracing\/jaeger-client-java<\/a><\/noindex><\/li>\n<li>Spring OpenTracing integration<br \/>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/jaegertracing\/jaeger-client-java\">github.com\/jaegertracing\/jaeger-client-java<\/a><\/noindex><br \/>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/opentracing-contrib\/java-spring-cloud\">github.com\/opentracing-contrib\/java-spring-cloud<\/a><\/noindex><\/li>\n<\/ul>\n<h3>Articles<\/h3>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/carprice\/blog\/340946\/\">habr.com\/ru\/company\/carprice\/blog\/340946<\/a><\/noindex> Jaeger OpenTracing and Microservices in a real project using PHP and Golang<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/eng.uber.com\/distributed-tracing\/\">eng.uber.com\/distributed-tracing<\/a><\/noindex> Evolving Distributed Tracing at Uber Engineering<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/opentracing.io\/guides\/java\/\">opentracing.io\/guides\/java<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/medium.com\/jaegertracing\/running-jaeger-agent-on-bare-metal-d1fc47d31fab\">medium.com\/jaegertracing\/running-jaeger-agent-on-bare-metal-d1fc47d31fab<\/a><\/noindex> Running Jaeger Agent on bare metal<\/li>\n<\/ul>\n<h3>Video<\/h3>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=qg0ENOdP1Lo\">www.youtube.com\/watch?v=qg0ENOdP1Lo<\/a><\/noindex> How We Used Jaeger and Prometheus to Deliver Lightning-Fast User Queries \u2014 Bryan Boreham<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=WRntQsUajow\">www.youtube.com\/watch?v=WRntQsUajow<\/a><\/noindex> Intro: Jaeger \u2014 Yuri Shkuro, Uber &amp; Pavol Loffay, Red Hat<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=fsHb0qK37bc\">www.youtube.com\/watch?v=fsHb0qK37bc<\/a><\/noindex> Serghei Iakovlev, \u201cA Little Story of a Great Victory: OpenTracing, AWS, and Jaeger\u201d<\/li>\n<\/ul>\n<p>Source: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/srg\/blog\/446752\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412 \u043d\u0430\u0448\u0438\u0445 \u043f\u0440\u043e\u0435\u043a\u0442\u0430\u0445 \u043c\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u043c\u0438\u043a\u0440\u043e\u0441\u0435\u0440\u0432\u0438\u0441\u043d\u0443\u044e \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u0443. \u041f\u0440\u0438 \u0432\u043e\u0437\u043d\u0438\u043a\u043d\u043e\u0432\u0435\u043d\u0438\u0438 \u0443\u0437\u043a\u0438\u0445 \u043c\u0435\u0441\u0442 \u0432 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e \u043c\u043d\u043e\u0433\u043e \u0432\u0440\u0435\u043c\u0435\u043d\u0438 \u0442\u0440\u0430\u0442\u0438\u0442\u0441\u044f \u043d\u0430 \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433 \u0438 \u0440\u0430\u0437\u0431\u043e\u0440 \u043b\u043e\u0433\u043e\u0432. \u041f\u0440\u0438 \u043b\u043e\u0433\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0438 \u0442\u0430\u0439\u043c\u0438\u043d\u0433\u043e\u0432 \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u044b\u0445 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0439 \u0432 \u043b\u043e\u0433-\u0444\u0430\u0439\u043b, \u043a\u0430\u043a \u043f\u0440\u0430\u0432\u0438\u043b\u043e, \u0441\u043b\u043e\u0436\u043d\u043e \u043f\u043e\u043d\u044f\u0442\u044c \u0447\u0442\u043e \u043f\u0440\u0438\u0432\u0435\u043b\u043e \u043a \u0432\u044b\u0437\u043e\u0432\u0443 \u044d\u0442\u0438\u0445 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0439, \u043e\u0442\u0441\u043b\u0435\u0434\u0438\u0442\u044c \u043f\u043e\u0441\u043b\u0435\u0434\u043e\u0432\u0430\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u0439 \u0438\u043b\u0438 \u0441\u043c\u0435\u0449\u0435\u043d\u0438\u0435 \u0432\u043e \u0432\u0440\u0435\u043c\u0435\u043d\u0438 \u043e\u0434\u043d\u043e\u0439 \u043e\u043f\u0435\u0440\u0430\u0446\u0438\u0438 \u043e\u0442\u043d\u043e\u0441\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0434\u0440\u0443\u0433\u043e\u0439 \u0432 \u0440\u0430\u0437\u043d\u044b\u0445 \u0441\u0435\u0440\u0432\u0438\u0441\u0430\u0445. \u0414\u043b\u044f \u043c\u0438\u043d\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u0438 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":22990,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-31014","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.1.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412 \u043d\u0430\u0448\u0438\u0445 \u043f\u0440\u043e\u0435\u043a\u0442\u0430\u0445 \u043c\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u043c\u0438\u043a\u0440\u043e\u0441\u0435\u0440\u0432\u0438\u0441\u043d\u0443\u044e.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/trassirovka-servisov-opentracing-i-jaeger\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.1.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0422\u0440\u0430\u0441\u0441\u0438\u0440\u043e\u0432\u043a\u0430 \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432, OpenTracing \u0438 Jaeger | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412 \u043d\u0430\u0448\u0438\u0445 \u043f\u0440\u043e\u0435\u043a\u0442\u0430\u0445 \u043c\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u043c\u0438\u043a\u0440\u043e\u0441\u0435\u0440\u0432\u0438\u0441\u043d\u0443\u044e.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/trassirovka-servisov-opentracing-i-jaeger\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T18:38:54+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T18:38:54+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Service tracing, OpenTracing and Jaeger | ProHoster","description":"In our projects, we use microservices.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/trassirovka-servisov-opentracing-i-jaeger","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0422\u0440\u0430\u0441\u0441\u0438\u0440\u043e\u0432\u043a\u0430 \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432, OpenTracing \u0438 Jaeger | ProHoster","og:description":"\u0412 \u043d\u0430\u0448\u0438\u0445 \u043f\u0440\u043e\u0435\u043a\u0442\u0430\u0445 \u043c\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u043c\u0438\u043a\u0440\u043e\u0441\u0435\u0440\u0432\u0438\u0441\u043d\u0443\u044e.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/trassirovka-servisov-opentracing-i-jaeger","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:38:54+00:00","article:modified_time":"2019-10-31T18:38:54+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"31014","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 04:06:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 03:26:04","updated":"2026-01-21 04:06:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/31014","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=31014"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/31014\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/22990"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=31014"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=31014"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=31014"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}