{"id":93358,"date":"2020-09-06T07:42:57","date_gmt":"2020-09-06T05:42:57","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/monitorim-sportmaster-kak-i-chem"},"modified":"2020-09-06T07:42:57","modified_gmt":"2020-09-06T05:42:57","slug":"monitorim-sportmaster-kak-i-chem","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/monitorim-sportmaster-kak-i-chem","title":{"rendered":"Monitoring Sportmaster \u2014 what and how","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>We started thinking about creating a monitoring system during the formation of our product teams. It became clear that our domain\u2014operations\u2014doesn't fit into these teams at all. Why is that?<\/p>\n<p>The thing is, all our teams are built around specific information systems, microservices, and front-end applications, so they don't see the overall health of the entire system. For example, they might not know how a small part in the deep backend affects the frontend. Their interest is limited to the systems that integrate with their own. If the team and its Service A are almost entirely unrelated to Service B, then that service is practically invisible to them.<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/sportmaster_lab\/blog\/517678\/\"><img decoding=\"async\" alt=\"Monitoring Sportmaster \u2014 what and how\" src=\"\/wp-content\/uploads\/2020\/09\/393621991a8997d449cb3adec71b6af5.png\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p>Our team, on the other hand, works with systems that are highly integrated with each other: there are many connections between them, forming a significant infrastructure. The operation of our online store depends on all these systems (which, by the way, are numerous). <\/p>\n<p>So, it turns out that our department doesn't belong to any specific team but is somewhat on the sidelines. In this whole scenario, our task is to understand comprehensively how information systems work, their functionality, integrations, software, network, hardware, and how all of this is interconnected.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><br \/>\nThe platform that powers our online stores looks like this:<\/p>\n<ul>\n<li>front<\/li>\n<li>middle-office<\/li>\n<li>back-office<\/li>\n<\/ul>\n<p>\nAs much as we'd like it to be, there\u2019s no such thing as all systems working smoothly and flawlessly. This is again due to the number of systems and integrations\u2014given our situation, some incidents are inevitable, regardless of the quality of testing. This is true both within a single system and in terms of their integration. We need to monitor the state of the entire platform as a whole, not just some isolated part of it. <\/p>\n<p>Ideally, monitoring the health status of the entire platform needs to be automated. And we came to monitoring as an inevitable part of this process. Initially, it was built only for the front end, while separate monitoring systems across layers have existed for network engineers, software administrators, and hardware specialists. All these people only monitored their specific levels, and no one had a comprehensive understanding.<\/p>\n<p>For instance, if a virtual machine crashes, in most cases only the administrator responsible for the hardware and the virtual machine is aware of it. The front team might see the fact that the application has crashed, but they have no information on the virtual machine failure. The administrator may know who the client is and have a rough idea of what is currently running on that virtual machine, assuming it's a large project. For smaller ones, they probably don't know. Regardless, the administrator needs to approach the owner to ask what was on that machine, what needs to be restored, and what needs to be changed. If something major breaks, it leads to a chaotic search \u2014 because no one sees the entire system.<\/p>\n<p>Ultimately, such fragmented stories impact the entire frontend, the users, and our core business function \u2014 online sales. Since we are not part of the teams and are engaged in the operation of all ecommerce applications within the online store, we took on the responsibility of creating a comprehensive monitoring system for the ecommerce platform. <\/p>\n<h2>System Structure and Stack<\/h2>\n<p>\nWe started by identifying several layers of monitoring for our systems, within which we would need to collect metrics. All of this needed to be integrated, which we accomplished in the first phase. Now, in this phase, we are refining the quality of metric collection across all our layers to build correlations and understand how the systems influence one another.<\/p>\n<p>The lack of comprehensive monitoring during the initial stages of application deployment (since we began building it when most systems were already in operation) resulted in a significant technical debt in setting up monitoring for the entire platform. We could not afford to focus solely on configuring monitoring for a single information system in detail, as the other systems would have remained unmonitored for some time. To address this issue, we defined a list of the most critical metrics for assessing the state of the information system by layers and began implementing it.<\/p>\n<p>Thus, we decided to tackle the problem in parts.<\/p>\n<p>Our system consists of:<\/p>\n<ul>\n<li>hardware; <\/li>\n<li>operating system; <\/li>\n<li>software; <\/li>\n<li>UI components in the monitoring application; <\/li>\n<li>business metrics; <\/li>\n<li>integration applications; <\/li>\n<li>information security; <\/li>\n<li>networks; <\/li>\n<li>traffic balancer. <\/li>\n<\/ul>\n<p>\n<img decoding=\"async\" alt=\"Monitoring Sportmaster \u2014 what and how\" src=\"\/wp-content\/uploads\/2020\/09\/687a5b8b39b3fd540a9b591f49a019d3.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAt the core of this system is monitoring itself. To understand the overall state of the entire system, it is essential to know what is happening with the applications at all these layers and across all the many applications. <\/p>\n<p>So, about the stack. <\/p>\n<p><img decoding=\"async\" alt=\"Monitoring Sportmaster \u2014 what and how\" src=\"\/wp-content\/uploads\/2020\/09\/b9a9b86d788990899e7f04e34fb879d9.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nWe use open-source software. At the center, we have Zabbix, which we primarily use as an alerting system. It is well-known for being ideal for monitoring infrastructure. What does this mean? It refers to the low-level metrics that every company with its own data center (and Sportmaster has its own data centers) monitors \u2014 server temperature, memory status, RAID status, and metrics from networking devices.<\/p>\n<p>We integrated Zabbix with the Telegram messenger and Microsoft Teams, which are actively used by our teams. Zabbix covers the layer of actual networks, hardware, and partially software, but it is not a cure-all. We enrich this data from some other services. For instance, regarding hardware levels, we directly connect via API to our virtualization system to retrieve data. <\/p>\n<p>Additionally, besides Zabbix, we use Prometheus, which allows monitoring metrics in a dynamic environment application. This means we can obtain application metrics via HTTP endpoints without worrying about which metrics to upload and which not. Based on this data, we can develop analytical queries.<\/p>\n<p>Data sources for the other layers, such as business metrics, fall into three categories.<\/p>\n<p>First, these are external business systems, like Google Analytics, from which we collect metrics from logs. We obtain data on active users, conversions, and everything related to the business. Second, there is the UI monitoring system, which should be described in more detail.<\/p>\n<p>We started with manual testing, which evolved into automated functional and integration tests. From that, we created monitoring, retaining only the core functionality and focusing on the markers that are as stable as possible and do not often change over time.<\/p>\n<p>The new team structure implies that all application activities are centralized around product teams, which is why we have stopped pure testing. Instead, we have turned our tests into UI monitoring, developed using Java, Selenium, and Jenkins (which serves as the launch and reporting system). <\/p>\n<p>We had many tests, but ultimately we decided to focus on the main road, a high-level metric. If we had a lot of specific tests, it would be difficult to maintain data relevance. Each subsequent release would significantly break the entire system, and we would only be engaged in fixing it. Therefore, we focused on fundamental aspects that rarely change and monitor only those.<\/p>\n<p>Finally, thirdly, the data source is a centralized logging system. We use Elastic Stack for logs, and then we can pull this data into our business metrics monitoring system. In addition, our own Monitoring API service, developed in Python, polls any services via API and retrieves data from them into Zabbix.<\/p>\n<p>Another indispensable attribute of monitoring is visualization. Ours is built on Grafana. Among various visualization systems, it stands out because you can visualize metrics from different data sources on the dashboard. We can gather high-level metrics of the online store, such as the number of orders placed in the last hour from the database, OS performance metrics on which this online store runs from Zabbix, and metrics of this application's instances from Prometheus. All of this will be on one dashboard. Clear and accessible.<\/p>\n<p>Let me mention security \u2014 we are currently enhancing the system, which we will later integrate with the global monitoring system. In my view, the main problems faced by e-commerce in the field of information security are related to bots, parsers, and brute force attacks. This needs to be monitored, as they can critically affect both the operation of our applications and our reputation from a business perspective. With the chosen stack, we successfully cover these tasks.<\/p>\n<p>Another important point is that application metrics are collected by Prometheus. It is also integrated with Zabbix. Additionally, we have sitespeed, a service that allows us to monitor parameters such as our page load speed, bottlenecks, page rendering, script loading, and more; it is also integrated via API. Thus, metrics are collected in Zabbix, and we also set alerts from there. Currently, all alerts are sent through the main channels (currently this is email and Telegram, and we recently added MS Teams). We plan to enhance the alerting system to enable smart bots to function as a service and provide monitoring information to all interested product teams. <\/p>\n<p>For us, metrics are important not only for individual information systems but also for overall metrics across the entire infrastructure used by applications: clusters <a class=\"wpil_keyword_link\" href=\"https:\/\/prohoster.info\/en\/server\/\"   title=\"of physical servers\" data-wpil-keyword-link=\"linked\"  data-wpil-monitor-id=\"1270\">of physical servers<\/a>, which host virtual machines, traffic load balancers, network load balancers, the network itself, and channel utilization. Additionally, we gather metrics for our own data centers (we have several, and the infrastructure is of considerable size). <\/p>\n<p><img decoding=\"async\" alt=\"Monitoring Sportmaster \u2014 what and how\" src=\"\/wp-content\/uploads\/2020\/09\/aaee82a501dce0eee31c5a81662cd8be.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nThe advantages of our monitoring system are that it allows us to see the operational state of all systems, evaluate their impact on each other and on shared resources. Ultimately, it enables resource planning, which is also part of our responsibilities. We manage server resources \u2014 a pool within e-commerce, adding and decommissioning new equipment, purchasing new hardware, conducting audits of resource utilization, and more. Each year, teams plan new projects and develop their systems, and it\u2019s important for us to provide them with resources. <\/p>\n<p>With the help of metrics, we observe trends in resource consumption by our information systems. Based on this information, we can plan accordingly. At the virtualization level, we collect data and see information on the available resources across our data centers. Within the data center, we can observe both utilization and actual distribution and consumption of resources, including both standalone servers and virtual machines, as well as clusters of physical servers where all these virtual machines are running smoothly. <\/p>\n<h2>Prospects<\/h2>\n<p>\nWe currently have the core of the system ready as a whole, but there are still quite a few aspects to work on. At a minimum, this includes the information security layer, but it is also important to reach the network, develop alerting, and address correlation issues. We have many layers and systems, and each layer has a multitude of metrics. It results in a Matryoshka doll, in the degree of a Matryoshka doll.<\/p>\n<p>Our task is ultimately to create the right alerts. For example, if there was a hardware issue, again, with a virtual machine where an important application was running, and that service was not reserved in any way. We will find out that the virtual machine has died. Then, business metrics will be alerted: users have disappeared, there are no conversions, the UI in the interface is unavailable, and software and services have also failed.<\/p>\n<p>With this arrangement, we will receive spam from alerts, which doesn't fit into the format of a proper monitoring system. The issue of correlation arises. Ideally, our monitoring system should say: 'Guys, your physical machine has died, and along with it, this application and such metrics', using a single alert instead of bombarding us with a hundred alerts. It should communicate the key issue \u2014 the cause, which facilitates the prompt resolution of the problem by localizing it.<\/p>\n<p>Our notification system and alert processing is built around a 24\/7 hotline service. All alerts deemed essential and included in the checklist are sent there. Every alert must have a description: what happened, what it means, and what it affects. Additionally, it should include a link to the dashboard and instructions on what to do in this case. <\/p>\n<p>This covers the requirements for setting up alerting. The situation can then develop in two directions \u2014 either there is a problem that needs to be addressed, or there has been a failure in the monitoring system. However, in any case, we need to investigate and understand the issue.<\/p>\n<p>On average, we receive about a hundred alerts per day, taking into account that the correlation of alerts has not yet been properly configured. If technical work needs to be done and we forcibly disable something, their number increases significantly. <\/p>\n<p>In addition to monitoring the systems we operate and collecting metrics deemed important on our side, the monitoring system allows for the collection of data for product teams. They can influence the composition of metrics within the information systems that we monitor. <\/p>\n<p>Our colleague may come in and request to add a metric that could be useful for both us and the team. For example, the team may find the basic metrics we have insufficient and need to track something specific. In Grafana, we create a space for each team and grant admin rights. Furthermore, if the team needs dashboards and they can't create them themselves or don't know how, we assist them. <\/p>\n<p>Since we are outside the team's value creation flow, their releases, and planning, we gradually arrive at a point where the releases of all systems are seamless and can be deployed daily without prior coordination with us. It's crucial for us to track these releases since they could potentially affect the application's performance and cause issues, which is critical. We use Bamboo for release management, from which we obtain data via API to see which releases have gone out in which information systems and their status. Most importantly, we know when.<\/p>\n<p>This way, we can see the correlation between new releases and emerging problems. The main idea is to understand how the system operates on all layers, quickly localize any issues, and fix them just as promptly. Often, the most time-consuming aspect is not solving the problem but finding the root cause. <\/p>\n<p>In this area, we want to focus on proactivity moving forward. Ideally, we would like to learn about an impending issue beforehand rather than afterward, so we can prevent it rather than just solve it. Sometimes there are false alarms from the monitoring system, caused both by human error and changes in the application. We are working on this by fine-tuning our approach and aiming to inform users about any manipulations with the monitoring system we undertake alongside them, or by conducting these activities during maintenance windows. <\/p>\n<p>So, the system has been up and running successfully since early spring... and it is showing quite a real profit. Of course, this is not its final version; we will be implementing many more useful features. But right now, with so many integrations and applications, we really can't do without monitoring automation.<\/p>\n<p>If you are also monitoring large projects with a significant number of integrations, please comment on what silver bullet you have found for this.<br \/>\n<br \/>Source: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/sportmaster_lab\/blog\/517678\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041e \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u0438 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433\u0430 \u043c\u044b \u0437\u0430\u0434\u0443\u043c\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u044d\u0442\u0430\u043f\u0435 \u0444\u043e\u0440\u043c\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u043e\u0432\u044b\u0445 \u043a\u043e\u043c\u0430\u043d\u0434. \u0421\u0442\u0430\u043b\u043e \u043f\u043e\u043d\u044f\u0442\u043d\u043e, \u0447\u0442\u043e \u043d\u0430\u0448\u0435 \u0434\u0435\u043b\u043e \u2014 \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u044f \u2014 \u0432 \u044d\u0442\u0438 \u043a\u043e\u043c\u0430\u043d\u0434\u044b \u043d\u0438\u043a\u0430\u043a \u043d\u0435 \u043f\u043e\u043f\u0430\u0434\u0430\u0435\u0442. \u041f\u043e\u0447\u0435\u043c\u0443 \u0442\u0430\u043a? \u0414\u0435\u043b\u043e \u0432 \u0442\u043e\u043c, \u0447\u0442\u043e \u0432\u0441\u0435 \u043d\u0430\u0448\u0438 \u043a\u043e\u043c\u0430\u043d\u0434\u044b \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u044b \u0432\u043e\u043a\u0440\u0443\u0433 \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u044b\u0445 \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c, \u043c\u0438\u043a\u0440\u043e\u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432 \u0438 \u0444\u0440\u043e\u043d\u0442\u043e\u0432, \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u043e\u0431\u0449\u0435\u0435 \u0441\u043e\u0441\u0442\u043e\u044f\u043d\u0438\u0435 \u0437\u0434\u043e\u0440\u043e\u0432\u044c\u044f \u0432\u0441\u0435\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u0432 \u0446\u0435\u043b\u043e\u043c \u043a\u043e\u043c\u0430\u043d\u0434\u044b \u043d\u0435 \u0432\u0438\u0434\u044f\u0442. \u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u043e\u043d\u0438 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":93359,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-93358","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.1.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041e \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u0438 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433\u0430 \u043c\u044b \u0437\u0430\u0434\u0443\u043c\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u044d\u0442\u0430\u043f\u0435 \u0444\u043e\u0440\u043c\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u043e\u0432\u044b\u0445 \u043a\u043e\u043c\u0430\u043d\u0434. \u0421\u0442\u0430\u043b\u043e \u043f\u043e\u043d\u044f\u0442\u043d\u043e, \u0447\u0442\u043e \u043d\u0430\u0448\u0435 \u0434\u0435\u043b\u043e \u2014 \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u044f \u2014 \u0432 \u044d\u0442\u0438 \u043a\u043e\u043c\u0430\u043d\u0434\u044b \u043d\u0438\u043a\u0430\u043a \u043d\u0435 \u043f\u043e\u043f\u0430\u0434\u0430\u0435\u0442.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/monitorim-sportmaster-kak-i-chem\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.1.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043c \u0421\u043f\u043e\u0440\u0442\u043c\u0430\u0441\u0442\u0435\u0440 \u2014 \u043a\u0430\u043a \u0438 \u0447\u0435\u043c | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041e \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u0438 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433\u0430 \u043c\u044b \u0437\u0430\u0434\u0443\u043c\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u044d\u0442\u0430\u043f\u0435 \u0444\u043e\u0440\u043c\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u043e\u0432\u044b\u0445 \u043a\u043e\u043c\u0430\u043d\u0434. \u0421\u0442\u0430\u043b\u043e \u043f\u043e\u043d\u044f\u0442\u043d\u043e, \u0447\u0442\u043e \u043d\u0430\u0448\u0435 \u0434\u0435\u043b\u043e \u2014 \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u044f \u2014 \u0432 \u044d\u0442\u0438 \u043a\u043e\u043c\u0430\u043d\u0434\u044b \u043d\u0438\u043a\u0430\u043a \u043d\u0435 \u043f\u043e\u043f\u0430\u0434\u0430\u0435\u0442.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/monitorim-sportmaster-kak-i-chem\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-09-06T05:42:57+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-09-06T05:42:57+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Monitoring Sportmaster \u2014 how and with what | ProHoster","description":"We started thinking about creating a monitoring system during the formation of product teams. It became clear that our operations work was not fitting into these teams at all.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/monitorim-sportmaster-kak-i-chem","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043c \u0421\u043f\u043e\u0440\u0442\u043c\u0430\u0441\u0442\u0435\u0440 \u2014 \u043a\u0430\u043a \u0438 \u0447\u0435\u043c | ProHoster","og:description":"\u041e \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u0438 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433\u0430 \u043c\u044b \u0437\u0430\u0434\u0443\u043c\u0430\u043b\u0438\u0441\u044c \u043d\u0430 \u044d\u0442\u0430\u043f\u0435 \u0444\u043e\u0440\u043c\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u043e\u0432\u044b\u0445 \u043a\u043e\u043c\u0430\u043d\u0434. \u0421\u0442\u0430\u043b\u043e \u043f\u043e\u043d\u044f\u0442\u043d\u043e, \u0447\u0442\u043e \u043d\u0430\u0448\u0435 \u0434\u0435\u043b\u043e \u2014 \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u044f \u2014 \u0432 \u044d\u0442\u0438 \u043a\u043e\u043c\u0430\u043d\u0434\u044b \u043d\u0438\u043a\u0430\u043a \u043d\u0435 \u043f\u043e\u043f\u0430\u0434\u0430\u0435\u0442.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/monitorim-sportmaster-kak-i-chem","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-09-06T05:42:57+00:00","article:modified_time":"2020-09-06T05:42:57+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"93358","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 11:49:43","updated":"2026-02-09 16:45:36","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/93358","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=93358"}],"version-history":[{"count":1,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/93358\/revisions"}],"predecessor-version":[{"id":158514,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/93358\/revisions\/158514"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/93359"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=93358"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=93358"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=93358"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}