{"id":80372,"date":"2020-05-05T13:42:27","date_gmt":"2020-05-05T11:42:27","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/devid-obrajen-xirus-metriki-metriki-metriki-chast-1"},"modified":"2020-05-05T13:42:27","modified_gmt":"2020-05-05T11:42:27","slug":"devid-obrajen-xirus-metriki-metriki-metriki-chast-1","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/devid-obrajen-xirus-metriki-metriki-metriki-chast-1","title":{"rendered":"David O'Brien (Xirus): Metrics! Metrics! Metrics! Part 1","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Recently, David O'Brien launched his own company, Xirus (https:\/\/xirus.com.au), focusing on Microsoft Azure Stack cloud products. These are designed for consistent development and deployment of hybrid applications in data centers, edge locations, remote offices, and the cloud.<\/p>\n

David trains individuals and companies on everything related to Microsoft Azure and Azure DevOps (formerly VSTS) and is still engaged in hands-on consulting and infrastructure automation. He has been a Microsoft MVP (Most Valuable Professional) awardee for 5 years and recently received the MVP Azure award. As a co-organizer of the Melbourne Microsoft Cloud and Datacenter Meetup, O'Brien regularly speaks at international conferences, merging his interest in traveling the world with his passion for sharing IT stories with the community. David's blog is located at david-obrien.net<\/a><\/noindex>, where he also publishes his online training courses on Pluralsight.<\/p>\n

The presentation discusses the importance of metrics for understanding what's happening in your environment and how your application performs. Microsoft Azure has a powerful and easy way to visualize metrics across all types of workloads, and the lecture explains how to leverage them all.<\/p>\n

At 3 AM on a Sunday, while you're sleeping, you are suddenly awoken by a text message alert: \u201ccritical application is not responding again.\u201d What is happening? Where is the bottleneck? In this talk, you'll learn about the services that Microsoft Azure offers clients for collecting logs and, in particular, the metrics of your cloud workloads. David will explain which metrics should concern you when operating on the cloud platform and how to access them. You'll learn about open-source tools and dashboard building, thus gaining enough knowledge to create your own dashboards.<\/a><\/noindex><\/p>\n

And if you're woken up again at 3 AM by a critical application crash notification, you'll be able to quickly diagnose the issue.<\/p>\n

Good afternoon, today we will talk about metrics. My name is David O\u2019Brien, I am a co-founder and owner of a small consulting Australian company called Xirus. Thank you again for coming here to spend your time with me. So, why are we here? To talk about metrics, or rather, I will tell you about them, and before doing anything, let\u2019s start with the theory.<\/p>\n

\"David
\n
\nI will explain what metrics are, what can be done with them, what to pay attention to, how to collect and incorporate metric gathering into Azure, and what metric visualization is. I will show you what these things look like in the Microsoft cloud and how to work with this cloud.<\/p>\n

Before we begin, I would like to ask those who use Microsoft Azure to raise their hands. And who works with AWS? I see a few hands. And with Google? ALI Cloud? One person! Great. So, what are metrics? The official definition from the National Institute of Standards and Technology of the United States is as follows: \u201cA metric is a measurement standard that describes the conditions and rules for measuring a property and serves to understand the results of the measurement.\u201d What does this mean? <\/p>\n

Let's take, for example, a metric for monitoring available disk space on a virtual machine. For instance, we are given the number 90, and this number represents a percentage, meaning that the amount of free disk space is 90%. I should note that it\u2019s not very interesting to read a 40-page PDF describing the definition of metrics. <\/p>\n

However, a metric does not say how the measurement result was obtained; it only shows the result. So, what do we do with metrics?<\/p>\n

First, we measure the value of something in order to use the measurement result later. <\/p>\n

\"David
\n
\nFor example, we learned the available disk space and can now use it, manage this memory, etc. After obtaining the metric results, we need to interpret them. For instance, if the metric shows a result of 90, we must understand what this number means: is it the volume of free space or the volume of used disk space in percentage or gigabytes, network latency of 90 ms, and so forth? In other words, we need to understand the significance of the metric value. For metrics to have any meaning, after interpreting one metric value, we must ensure the collection of multiple values. This is very important because many people do not realize the necessity of collecting metrics. Microsoft has made the process of obtaining metrics very easy, but you must ensure their collection yourself. These metrics are stored for only 41 days and disappear on the 42nd day. Therefore, depending on the characteristics of your external or internal equipment, you need to consider how to retain metrics for more than 41 days\u2014in the form of logs, records, etc. Thus, after collection, you should place them somewhere that allows you to retrieve all statistical changes to the metric results when needed. By placing them there, you can begin to work effectively with them. <\/p>\n

Only after you obtain the metric values, interpret them, and collect them, can you create an SLA\u2014a Service Level Agreement. This SLA may not hold much significance for your clients; it is more important for your colleagues, managers, those ensuring system operation, and concerned about its functionality. The metric can measure the number of tickets\u2014for example, if you receive 5 tickets a day, it indicates the response speed to user requests and how quickly issues are resolved. A metric should not simply state that your site loads in 20 ms or that the response speed is 20 ms; a metric encompasses much more than just one technical indicator. <\/p>\n

Therefore, the task of our discussion is to present you with a comprehensive picture of the essence of metrics. A metric serves to provide a full view of the process at a glance. <\/p>\n

\"David
\n
\nOnce we have the metrics, we can guarantee the system's operational status at 99%, because it's not just a glance at a log file stating that the system is running. A 99% uptime guarantee means that, for instance, in 99% of cases, the API responds at a normal speed of 30 ms. This is precisely what your users, colleagues, and managers care about. Many of our clients monitor web server logs, yet they fail to notice any errors in them and think everything is fine. For example, they see a network speed of 200 Mbps and think, 'Okay, all is great!'. But to achieve that 200, users require a response speed of 30 milliseconds, which is a metric that is not measured or collected in log files. Consequently, users are surprised that the website loads very slowly because, without the necessary metrics, they are unaware of the reasons for such behavior. <\/p>\n

However, since we have an SLA guaranteeing 100% uptime, clients begin to express their frustration, as it is indeed very difficult to use the site. Therefore, to establish an objective SLA, it is necessary to see the complete picture of the process created by the collected metrics. This is a subject of my ongoing debate with some providers, who, when creating an SLA, do not grasp what the term 'uptime' means and, in most cases, do not explain to their clients how their API works. <\/p>\n

If you have created a service, such as an API for a third party, you need to understand what the metric of 39.5 represents \u2013 a successful response, a response at a speed of 20 ms, or at a speed of 5 ms. It is you who must adapt their SLA to your own SLA, to your own metrics. <\/p>\n

Having figured all this out, you can start creating a stunning monitoring dashboard. Tell me, has anyone used the interactive visualization app Grafana? Great! I am a big fan of this open source tool because it is free and easy to use. <\/p>\n

\"David
\n
\nIf you haven't used Grafana yet, I'll show you how to work with it. Those born in the '80s and '90s might remember the caring Care Bears. I'm not sure how popular these bears were in Russia, but when it comes to metrics, we should be just as 'caring.' As I mentioned, you need a comprehensive view of how the entire system operates, and it shouldn't just concern your API, your website, or the service running on a virtual machine. <\/p>\n

\"David
\n
\nYou need to organize the collection of those metrics that most accurately reflect the performance of the entire system. Most of you are software developers, so your life is constantly changing, adapting to new product requirements. Just as you're concerned with coding processes, you need to pay attention to metrics. You should understand how each metric relates to every line of code you write. For instance, next week you are launching a new marketing campaign and expect a large influx of visitors to your site. To analyze this event, you'll need metrics, and perhaps an entire dashboard to track these people's activity. Metrics are essential for understanding how successful your marketing campaign is and how it actually performs. They will help you, for example, to develop an effective CRM \u2014 a customer relationship management system. <\/p>\n

So, let's dive into our cloud service, Azure. It's very easy to find and organize metric collection here because it has Azure Monitor. This monitor centralizes the management of your system's configuration. Each element of Azure that you want to use in your system comes with numerous default metrics enabled. It's a free application that works right 'out of the box' and doesn't require any prior setup \u2014 you don't need to write anything or 'attach' it to your system. We'll confirm this by watching the following demo. <\/p>\n

\"David
\n
\nAdditionally, there is an option to send these metrics to third-party applications such as the Splunk log storage and analysis system, the cloud-based log management application SumoLogic, the ELK log processing tool, and IBM Radar. However, there are some minor differences that depend on the resources you are using \u2013 virtual machines, network services, Azure SQL databases, meaning the use of metrics varies based on the functionalities of your working environment. I wouldn't say these differences are significant, but unfortunately, they do exist, and this should be taken into account. Enabling and forwarding metrics can be done in several ways: through the Portal, CLI\/PowerShell, or using ARM templates. <\/p>\n

\"David
\n
\nBefore we begin the first demonstration, I will answer any questions you may have. If there are no questions, let's proceed. The screen shows what the Azure Monitor page looks like. Can anyone of you say that this monitor is not working? <\/p>\n

\"David
\n
\nSo, now everything is fine, you see how the monitor services look. I can say that it is a great and very simple tool for daily tasks. With it, you can monitor applications, networks, and infrastructure. Recently, the monitoring interface was improved, and while previously the services were located in different places, now all information about the services is consolidated on the monitor's homepage. <\/p>\n

The metrics table is a tab accessible via HomeMonitorMetrics, where you can see all available metrics and select the ones you need. However, if you need to enable metric collection, you should use the directory path HomeMonitorDiagnostic settings and check the metrics' Enabled\/Disabled checkboxes. By default, almost all metrics are in an enabled state, but if you need to enable something additional, you will need to change the diagnostic status from Disabled to Enabled. <\/p>\n

\"David
\n
\nTo do this, click on the row of the selected metric and on the opened tab enable the diagnostic mode. If you plan to analyze the selected metric, after clicking on the Turn on diagnostic link, you should check the Send to Log Analytics checkbox in the window that appears. <\/p>\n

\"David
\n
\nLog Analytics is somewhat similar to Splunk but is cheaper. This service allows you to aggregate all your metrics, logs, and everything else you may need, and store them in the Log Analytics workspace. The service uses a special query processing language called KQL \u2013 Kusto Query Language, which we will review in the next demo. For now, note that it enables you to create queries regarding metrics, logs, terms, trends, patterns, etc., and build dashboards.<\/p>\n

So, we check the Send to Log Analytics checkbox and the LOG panel checkboxes: DataPlaneRequests, MongoRequests, and QueryRuntimeStatistics, and below in the METRIC panel \u2013 the Requests checkbox. Then we assign a name and save the settings. In the command line, this represents two lines of code. By the way, the Azure Cloud shell in this respect resembles Google, which also allows command line usage in your web browser. AWS has nothing like it, making Azure much more user-friendly in this context. <\/p>\n

For instance, I can run the demo through the web interface without using any code on my laptop. To do this, I must authenticate using my Azure account. Next, I can use, for example, Terraform, if you are already familiar with it, wait for the connection to the service, and get a Linux environment that Microsoft uses by default. <\/p>\n

\"David
\n
\nNext, I use Bash, which is integrated into Azure Cloud Shell. A very useful feature is the built-in IDE in the browser, a lightweight version of VS Code. After that, I can access my error metrics template, modify it, and tailor it to my needs. <\/p>\n

\"David
\n
\nBy configuring metric collection in this template, you can apply it to create metrics for your entire infrastructure. Once we have applied the metrics, gathered, and saved them, we will need to visualize them. <\/p>\n

\"David
\n
\nAzure Monitor focuses solely on metrics and does not provide a comprehensive overview of your system's status. You may have a number of other applications running outside the Azure environment. So if you need to monitor all processes and visualize all the collected metrics in one place, Azure Monitor is not suitable for that. <\/p>\n

To address this task, Microsoft offers the Power BI tool \u2013 a comprehensive software for business analysis that includes visualization of a wide range of data. It is quite an expensive product, and its cost depends on the necessary set of features. By default, it provides 48 types of processed data and is connected to SQL Azure data warehouses, Azure Data Lake Storage, Azure machine learning services, and Azure Databricks. Utilizing scalability, you can receive new data every 30 minutes. This may be sufficient for your needs or insufficient if you require real-time monitoring visualization. In this case, applications like the mentioned Grafana are recommended. Furthermore, Microsoft documentation describes the capability to send metrics, logs, and event tables using SIEM tools to visualization systems such as Splunk, SumoLogic, ELK, and IBM radar. <\/p>\n

23:40 min<\/p>\n

Continuation will be available very soon\u2026<\/p>\n

\r\n \"Play\r\n
<\/div>\r\n <\/div><\/center><\/p>\n

A little advertisement \ud83d\ude42<\/h3>\n

\nThank you for staying with us. Do you enjoy our articles? Want to see more interesting content? Support us by placing an order or recommending us to your friends, cloud VPS for developers starting at $4.99<\/a><\/noindex>, a unique entry-level server alternative that we have created for you:<\/b> The whole truth about VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps from $19 or how to properly share a server?<\/a><\/noindex> (options available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).<\/p>\n

Dell R730xd at half the price in the Equinix Tier IV data center in Amsterdam?<\/b> Only with us 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100TB starting at $199<\/a><\/noindex> in the Netherlands! Dell R420 \u2014 2x E5-2430 2.2GHz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB \u2014 from $99!<\/b><\/b> Read about how To build a corporate-class infrastructure using Dell R730xd E5-2650 v4 servers costing 9000 euros for peanuts?<\/a><\/noindex>
\n
Source: habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u041d\u0435\u0434\u0430\u0432\u043d\u043e \u0414\u044d\u0432\u0438\u0434 \u041e\u2019\u0411\u0440\u0430\u0439\u0435\u043d \u043e\u0442\u043a\u0440\u044b\u043b \u0441\u0432\u043e\u044e \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u0443\u044e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044e Xirus (https:\/\/xirus.com.au), \u0441\u043e\u0441\u0440\u0435\u0434\u043e\u0442\u043e\u0447\u0438\u0432\u0448\u0438\u0441\u044c \u043d\u0430 \u043e\u0431\u043b\u0430\u0447\u043d\u044b\u0445 \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u0430\u0445 Microsoft Azure Stack. \u041e\u043d\u0438 \u043f\u0440\u0435\u0434\u043d\u0430\u0437\u043d\u0430\u0447\u0435\u043d\u044b \u0434\u043b\u044f \u0441\u043e\u0433\u043b\u0430\u0441\u043e\u0432\u0430\u043d\u043d\u043e\u0433\u043e \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0430 \u0433\u0438\u0431\u0440\u0438\u0434\u043d\u044b\u0445 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u0439 \u0432 \u0446\u0435\u043d\u0442\u0440\u0430\u0445 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u0434\u0430\u043d\u043d\u044b\u0445, \u0432 \u043f\u043e\u0433\u0440\u0430\u043d\u0438\u0447\u043d\u044b\u0445 \u0440\u0430\u0441\u043f\u043e\u043b\u043e\u0436\u0435\u043d\u0438\u044f\u0445, \u0443\u0434\u0430\u043b\u0435\u043d\u043d\u044b\u0445 \u043e\u0444\u0438\u0441\u0430\u0445 \u0438 \u043e\u0431\u043b\u0430\u043a\u0435. \u0414\u044d\u0432\u0438\u0434 \u043e\u0431\u0443\u0447\u0430\u0435\u0442 \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u044b\u0445 \u043b\u0438\u0446 \u0438 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 \u0432\u0441\u0435\u043c\u0443, \u0447\u0442\u043e \u0441\u0432\u044f\u0437\u0430\u043d\u043e \u0441 Microsoft Azure \u0438 Azure DevOps (\u0431\u044b\u0432\u0448\u0435\u043c\u0443 VSTS) \u0438 \u0434\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":80373,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-80372","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47David O\u2019Brien (Xirus): Metrics! Metrics! Metrics! Part 1 | ProHoster","description":"Recently, David O\u2019Brien launched his own company, Xirus (https:\/\/xirus.com.au), focusing on Microsoft Azure Stack cloud products.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/devid-obrajen-xirus-metriki-metriki-metriki-chast-1","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0414\u044d\u0432\u0438\u0434 \u041e\u2019\u0411\u0440\u0430\u0439\u0435\u043d (Xirus): \u041c\u0435\u0442\u0440\u0438\u043a\u0438! \u041c\u0435\u0442\u0440\u0438\u043a\u0438! \u041c\u0435\u0442\u0440\u0438\u043a\u0438! \u0427\u0430\u0441\u0442\u044c 1 | ProHoster","og:description":"\u041d\u0435\u0434\u0430\u0432\u043d\u043e \u0414\u044d\u0432\u0438\u0434 \u041e\u2019\u0411\u0440\u0430\u0439\u0435\u043d \u043e\u0442\u043a\u0440\u044b\u043b \u0441\u0432\u043e\u044e \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u0443\u044e \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u044e Xirus (https:\/\/xirus.com.au), \u0441\u043e\u0441\u0440\u0435\u0434\u043e\u0442\u043e\u0447\u0438\u0432\u0448\u0438\u0441\u044c \u043d\u0430 \u043e\u0431\u043b\u0430\u0447\u043d\u044b\u0445 \u043f\u0440\u043e\u0434\u0443\u043a\u0442\u0430\u0445 Microsoft Azure Stack.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/devid-obrajen-xirus-metriki-metriki-metriki-chast-1","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-05-05T11:42:27+00:00","article:modified_time":"2020-05-05T11:42:27+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"80372","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 16:19:42","updated":"2022-09-28 07:38:12","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/80372","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=80372"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/80372\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/80373"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=80372"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=80372"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=80372"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}