{"id":30785,"date":"2019-10-31T21:37:25","date_gmt":"2019-10-31T18:37:25","guid":{"rendered":"https:\/\/prohoster.info\/blog\/stroitelnye-bloki-raspredelennyh-prilozhenij-nulevoe-priblizhenie\/"},"modified":"2019-10-31T21:37:25","modified_gmt":"2019-10-31T18:37:25","slug":"stroitelnye-bloki-raspredelennyh-prilozhenij-nulevoe-priblizhenie","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/stroitelnye-bloki-raspredelennyh-prilozhenij-nulevoe-priblizhenie","title":{"rendered":"Building blocks of distributed applications. Zero approximation","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Building blocks of distributed applications. Zero approximation\" src=\"\/wp-content\/uploads\/2019\/04\/88d4e363538bdfc67690c8339dca790b.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>The world is constantly changing. Progress creates new technological challenges. In accordance with the changing demands, the architecture of information systems must evolve. Today, we will discuss event-driven architecture, concurrency, parallelism, and asynchrony, as well as how to coexist peacefully with all of this in Erlang.<\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2 id=\"vvedenie\">Introduction<\/h2>\n<p><\/p>\n<p>Depending on the size of the designed system and its requirements, we developers choose a way to exchange information within the system. In most cases, a broker-based approach, for example using RabbitMQ or Kafka, is a practical option for organizing service interactions. However, sometimes the event stream, SLA, and level of system control are such that off-the-shelf messaging doesn\u2019t suit our needs. Of course, we can complicate the system a bit by taking on the responsibility for the transport layer and cluster formation, for example using ZeroMQ or nanomsg. But if the system has sufficient bandwidth and capabilities of a standard Erlang cluster, the question of adding an additional entity requires thorough examination and economic justification.<\/p>\n<p><\/p>\n<p>The topic of reactive distributed applications is quite broad. To fit the format of this article, today we will focus only on homogeneous environments built on Erlang\/Elixir. The Erlang\/OTP ecosystem allows for the implementation of a reactive architecture with minimal labor costs. However, we will need a messaging layer. <\/p>\n<p><\/p>\n<h2 id=\"teoreticheskiy-bazis\">Theoretical Basis<\/h2>\n<p><\/p>\n<p>The design process begins with defining goals and constraints. The primary goal does not lie in development for the sake of development. We need to create a secure and scalable tool that can support and, most importantly, develop modern applications at various levels: starting from single-server applications that serve a small audience, which can later evolve into clusters of 50-60 nodes, to federations of clusters. Thus, the main goal is to maximize profits by reducing the cost of development and ownership of the resulting system.<\/p>\n<p><\/p>\n<p>We will outline 4 main requirements for the final system:<\/p>\n<p><\/p>\n<ul>\n<li><strong>C<\/strong>event-driven orientation.<br \/>\nThe system is always ready to pass through a stream of events and take the necessary actions;<\/li>\n<li><strong>M<\/strong>Scalability.<br \/>\nIndividual blocks can scale both vertically and horizontally. The entire system should have the capability for infinite horizontal growth;<\/li>\n<li><strong>O<\/strong>fault tolerance.<br \/>\nAll levels and services must have the ability to automatically recover from failures;<\/li>\n<li><strong>G<\/strong>uaranteed response time.<br \/>\nTime is valuable, and users should not have to wait too long.<\/li>\n<\/ul>\n<p><\/p>\n<p>Remember the old story about 'The little engine that could'? To ensure that the designed system successfully transitions from prototype stage to being progressive, its foundation must meet minimal requirements. <strong>CAN<\/strong>.<\/p>\n<p><\/p>\n<p>The messaging as an infrastructural tool and basis for all services now includes another aspect: ease of use for programmers.<\/p>\n<p><\/p>\n<h3 id=\"orientirovannost-na-sobytiya\">Event-driven focus<\/h3>\n<p><\/p>\n<p>For the application to grow from a single instance <a class=\"wpil_keyword_link\" href=\"https:\/\/prohoster.info\/en\/server\/dts-los-angeles\/\"   title=\"server\" data-wpil-keyword-link=\"linked\"  data-wpil-monitor-id=\"3480\">server<\/a> to a cluster, its architecture must ensure loose coupling. This requirement is met by an asynchronous model. In this model, the sender and receiver take care of the information payload of the message and do not worry about the transmission and routing within the system.<\/p>\n<p><\/p>\n<h3 id=\"masshtabiruemost\">Scalability<\/h3>\n<p><\/p>\n<p>Scalability and efficiency of the system go hand in hand. Application components must be able to utilize all available resources. The more efficiently we can utilize our capabilities and the more optimized our processing methods are, the less money we spend on equipment. <\/p>\n<p><\/p>\n<p>Within a single machine, Erlang creates a highly concurrent environment. The balance between concurrency and parallelism can be set by the choice of the number of operating system threads available for the Erlang VM and the number of schedulers utilizing these threads.<br \/>\nErlang processes do not share state and operate in a non-blocking mode. This ensures relatively low latency and higher throughput compared to traditional applications built on blocking synchronization. The Erlang scheduler takes care of fair distribution of CPU and IO, and the absence of locks allows the application to respond even under peak loads or failures. <\/p>\n<p><\/p>\n<p>At the cluster level, there is also a problem with resource utilization. It is important that all machines in the cluster are evenly loaded and that the network is not overloaded. Imagine a situation where user traffic lands on incoming load balancers (haproxy, nginx, etc.), which distribute requests for processing as evenly as possible among a set of available backends. Within the application infrastructure, the service that implements the required interface is just the last mile, and it will need to request a number of other services in order to respond to the initial request. Internal requests also require routing and balancing.<br \/>\nTo effectively manage data streams, messaging must provide developers with an interface for managing routing and load distribution. This allows developers to solve both standard tasks and rare occurrences using microservice patterns (aggregator, proxy, chain, branch, etc.). <\/p>\n<p><\/p>\n<p>From a business perspective, scalability is one of the tools for managing risks. The key is to meet customer demands while optimizing equipment usage:<\/p>\n<p><\/p>\n<ul>\n<li>By increasing hardware capacity due to advancements, it will not sit idle due to software inadequacies. Erlang scales vertically very well and will always be able to utilize all CPU cores and available memory.<\/li>\n<li>In cloud environments, we can manage the amount of hardware based on current or predicted loads and ensure SLA adherence.<\/li>\n<\/ul>\n<p><\/p>\n<h3 id=\"otkazoustoychivost\">Fault tolerance<\/h3>\n<p><\/p>\n<p>Let\u2019s consider two axioms: 'Failures are unacceptable' and 'Failures will always occur.' For businesses, software failure means financial loss, and worse yet, loss of reputation. By balancing possible losses with the cost of developing fault-tolerant software, a compromise can often be found.<\/p>\n<p><\/p>\n<p>In the short term, an architecture that incorporates fault tolerance saves money on purchasing ready-made clustering solutions. They are expensive and also contain flaws.<br \/>\nIn the long term, a fault-tolerant architecture pays back multiple times on the investments made at all stages of development.<br \/>\nMessaging within the codebase is still in development, allowing for detailed interaction between components within the system. This simplifies the task of responding to and managing failures, as all responsible components handle failures, and the final system knows how to return to its operational state automatically after a failure by design.<\/p>\n<p><\/p>\n<h3 id=\"otzyvchivost\">Responsiveness<\/h3>\n<p><\/p>\n<p>Regardless of failures, the application must respond to requests and meet SLAs. The reality is that people do not want to wait, and businesses must adapt. Increasingly, high responsiveness is expected from applications.<br \/>\nResponsive applications operate in a mode close to real-time. The Erlang VM functions in soft real-time mode. For certain areas, such as stock trading, healthcare, and industrial equipment management, hard real-time mode is crucial.<br \/>\nResponsive systems enhance UX and are beneficial for business.<\/p>\n<p><\/p>\n<h2 id=\"predvaritelnyy-itog\">Preliminary Summary<\/h2>\n<p><\/p>\n<p>In planning this article, I wanted to share my experience of creating a message broker and building complex systems based on it. However, the theoretical and motivational part turned out to be quite extensive.<br \/>\nIn the second part of the article, I will discuss the nuances of implementing exchange points, message exchange patterns, and their applications.<br \/>\nIn the third part, we will cover general questions about service organization, routing, and load balancing. We will discuss the practical aspects of scalability and resilience of systems.<\/p>\n<p><\/p>\n<p>End of the first part.<\/p>\n<p><\/p>\n<p>Photo <noindex><a rel=\"nofollow\" href=\"https:\/\/unsplash.com\/photos\/SRjZtxsK3Os\">@lucabravo<\/a><\/noindex>.<\/p>\n<p>Source: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/446028\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041c\u0438\u0440 \u043d\u0435 \u0441\u0442\u043e\u0438\u0442 \u043d\u0430 \u043c\u0435\u0441\u0442\u0435. \u041f\u0440\u043e\u0433\u0440\u0435\u0441\u0441 \u0441\u043e\u0437\u0434\u0430\u0435\u0442 \u043d\u043e\u0432\u044b\u0435 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0435 \u0432\u044b\u0437\u043e\u0432\u044b. \u0412 \u0441\u043e\u043e\u0442\u0432\u0435\u0442\u0441\u0442\u0432\u0438\u0438 \u0441 \u0438\u0437\u043c\u0435\u043d\u0438\u0432\u0448\u0438\u043c\u0438\u0441\u044f \u0442\u0440\u0435\u0431\u043e\u0432\u0430\u043d\u0438\u044f\u043c\u0438, \u0434\u043e\u043b\u0436\u043d\u0430 \u044d\u0432\u043e\u043b\u044e\u0446\u0438\u043e\u043d\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u0438 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u0430 \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c. \u0421\u0435\u0433\u043e\u0434\u043d\u044f \u043c\u044b \u0431\u0443\u0434\u0435\u043c \u0433\u043e\u0432\u043e\u0440\u0438\u0442\u044c \u043e \u0441\u043e\u0431\u044b\u0442\u0438\u0439\u043d\u043e-\u043e\u0440\u0438\u0435\u043d\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0439 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u0435, \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442\u043d\u043e\u0441\u0442\u0438, \u043f\u0430\u0440\u0430\u043b\u043b\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438, \u0430\u0441\u0438\u043d\u0445\u0440\u043e\u043d\u043d\u043e\u0441\u0442\u0438 \u0438 \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u0432 Erlang \u043c\u043e\u0436\u043d\u043e \u0441\u043e \u0432\u0441\u0435\u043c \u044d\u0442\u0438\u043c \u0436\u0438\u0442\u044c \u043c\u0438\u0440\u043d\u043e. \u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0412 \u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e\u0441\u0442\u0438 \u043e\u0442 \u0440\u0430\u0437\u043c\u0435\u0440\u043e\u0432 \u043f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u0443\u0435\u043c\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u0438 \u0442\u0440\u0435\u0431\u043e\u0432\u0430\u043d\u0438\u0439 \u043a \u043d\u0435\u0439, \u043c\u044b, [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":22770,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-30785","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.1.1 - aioseo.com -->\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/stroitelnye-bloki-raspredelennyh-prilozhenij-nulevoe-priblizhenie\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.1.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0421\u0442\u0440\u043e\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0435 \u0431\u043b\u043e\u043a\u0438 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u043d\u044b\u0445 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u0439. \u041d\u0443\u043b\u0435\u0432\u043e\u0435 \u043f\u0440\u0438\u0431\u043b\u0438\u0436\u0435\u043d\u0438\u0435 | ProHoster\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/stroitelnye-bloki-raspredelennyh-prilozhenij-nulevoe-priblizhenie\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T18:37:25+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T18:37:25+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Building Blocks of Distributed Applications. Zero Approximation | ProHoster","description":"","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/stroitelnye-bloki-raspredelennyh-prilozhenij-nulevoe-priblizhenie","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0421\u0442\u0440\u043e\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0435 \u0431\u043b\u043e\u043a\u0438 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u043d\u044b\u0445 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u0439. \u041d\u0443\u043b\u0435\u0432\u043e\u0435 \u043f\u0440\u0438\u0431\u043b\u0438\u0436\u0435\u043d\u0438\u0435 | ProHoster","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/stroitelnye-bloki-raspredelennyh-prilozhenij-nulevoe-priblizhenie","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:37:25+00:00","article:modified_time":"2019-10-31T18:37:25+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"30785","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-02-22 15:31:13","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 03:29:56","updated":"2026-02-22 15:31:13","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/30785","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=30785"}],"version-history":[{"count":1,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/30785\/revisions"}],"predecessor-version":[{"id":162006,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/30785\/revisions\/162006"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/22770"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=30785"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=30785"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=30785"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}