{"id":79153,"date":"2020-04-24T19:42:43","date_gmt":"2020-04-24T17:42:43","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/istoriya-sozdaniya-oblachnogo-servisa-pripravlennaya-kiberpankom"},"modified":"2020-04-24T19:42:43","modified_gmt":"2020-04-24T17:42:43","slug":"istoriya-sozdaniya-oblachnogo-servisa-pripravlennaya-kiberpankom","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/istoriya-sozdaniya-oblachnogo-servisa-pripravlennaya-kiberpankom","title":{"rendered":"The history of the creation of a cloud service, spiced with cyberpunk","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/d1cad1dfa3513b5794c7be916a9dbb4b.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>As you gain experience in IT, you start to notice that systems have their own character. They can be accommodating, quiet, unpredictable, or stern. They can either attract or repel. Either way, you have to 'negotiate' with them, navigate through the 'pitfalls', and build chains of their interaction.<\/p>\n<p><\/p>\n<p>Thus, we had the honor of building a cloud platform, which required us to 'persuade' a couple of subsystems to work with us. Luckily, we have our 'API language', skilled hands, and a lot of enthusiasm. <\/p>\n<p><\/p>\n<p>This article won't dive into technical hardcore, but it will describe the problems we faced while building the cloud. I decided to outline our journey as a light technical fantasy about how we sought common ground with the systems and what came of it. <\/p>\n<p><\/p>\n<p>Welcome to the article.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p><\/p>\n<h3 id=\"nachalo-puti\">The Beginning of the Journey<\/h3>\n<p><\/p>\n<p>Some time ago, our team was tasked with launching a cloud platform for our clients. We had the support of management, resources, hardware stack, and the freedom to choose technologies for implementing the software part of the service.<\/p>\n<p><\/p>\n<p>There were also a number of requirements:<\/p>\n<p><\/p>\n<ul>\n<li>the service needs a user-friendly personal account;<\/li>\n<li>the platform must be integrated into the existing billing system;<\/li>\n<li>software and hardware part: OpenStack + Tungsten Fabric (Open Contrail), which our engineers have learned to handle quite well.<\/li>\n<\/ul>\n<p><\/p>\n<p>We will talk about how the team was gathered, how the personal account interface was developed, and how design decisions were made at another time, if the Habr community shows interest.<br \/>\nThe tools we decided to use:<\/p>\n<p><\/p>\n<ul>\n<li>Python + Flask + Swagger + SQLAlchemy \u2014 a fairly standard Python set;<\/li>\n<li>Vue.js for the frontend;<\/li>\n<li>we decided to handle interaction between components and services using Celery over AMQP.<\/li>\n<\/ul>\n<p><\/p>\n<p>Anticipating questions about the choice of Python, let me clarify. The language has carved out its niche in our company, and a small, but still significant culture has formed around it. Therefore, it was decided to start building the service on it. Moreover, development speed often plays a key role in such tasks. <\/p>\n<p><\/p>\n<p>So, let's begin our acquaintance.<\/p>\n<p><\/p>\n<h3 id=\"molchalivyy-bill--billing\">Silent Bill \u2014 billing<\/h3>\n<p><\/p>\n<p><em>We had known this guy for a long time. He always sat nearby and quietly calculated something. Sometimes he would forward user requests to us, generate client invoices, and manage services. A regular hardworking guy. However, there were complications. He was silent, sometimes pensive, and often kept to himself.<\/em><\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/262bb5c9d6234041d264f8e041e647fe.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Billing was the first system we tried to get along with. And the first difficulty we encountered was with service processing.<\/p>\n<p><\/p>\n<p>For example, when creating or deleting, a task goes into the internal billing queue. This is how the system for asynchronous service processing is implemented. To handle our service types, we needed to 'queue' our tasks in this queue. Here we ran into a problem: the lack of documentation. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/4d96251df014e315308c94f848bc6cd0.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>According to the API description, this task can still be solved, but we did not have time for reverse engineering, so we moved the logic outside and organized a task queue on top of RabbitMQ. The operation on the service is initiated by the client from their dashboard, wrapped in a 'task' on the backend using Celery, and executed on the billing side and OpenStack. Celery allows for fairly convenient task management, handling retries, and monitoring status. You can read more about 'Celery,' for example, <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/oleg-bunin\/blog\/433476\/\">here<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p>Also, billing did not stop the project when the funds ran out. While communicating with the developers, we found out that when counting based on statistics (which is the logic we needed to implement), there is a complex interrelationship of stopping rules. However, these models do not fit well with our realities. We also implemented this through tasks on Celery, transferring the service management logic to the backend. <\/p>\n<p><\/p>\n<p>Both of the above problems led to the code becoming a bit bloated, and in the future, we will need to refactor it to separate the task management logic into its own service. We also need to store part of the information about users and their services in our tables to maintain this logic.<\/p>\n<p><\/p>\n<p>Another problem is silence.<\/p>\n<p><\/p>\n<p>In response to some API requests, Billy silently replies 'Ok'. This was the case, for instance, when we credited promised payments during testing (more on that later). The requests were executed correctly, and we did not see any errors. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/d03f2c381fd39dbf33f2a0c3eeb1294d.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>I had to study the logs while working with the system through the UI. It turned out that the billing itself performs such requests by changing the scope to a specific user, for example, admin, passing it in the su parameter.<\/p>\n<p><\/p>\n<p>Overall, despite gaps in the documentation and minor API issues, everything went quite well. The logs can be read even under heavy load if you understand how they are structured and what you need to look for. The database structure is convoluted, but quite logical and even somewhat appealing in certain aspects.<\/p>\n<p><\/p>\n<p>In summary, the main issues we encountered during interaction were related to the specifics of the implementation of the particular system: <\/p>\n<p><\/p>\n<ul>\n<li>undocumented \u201cfeatures\u201d that affected us in one way or another;<\/li>\n<li>closed source code (the billing is written in C++), as a result \u2014 there was no way to solve issue 1 except for the \u2018trial and error method\u2019.<\/li>\n<\/ul>\n<p><\/p>\n<p>Fortunately, the product has a sufficiently extensive API, and we integrated the following subsystems into our personal account:<\/p>\n<p><\/p>\n<ul>\n<li>technical support module \u2014 requests from the personal account are \u2018proxied\u2019 to billing transparently for the clients of the service;<\/li>\n<li>financial module \u2014 allows for invoicing current clients, handling charges, and generating payment documents;<\/li>\n<li>service management module \u2014 for which we had to implement our own handler. The system's expandability worked in our favor, and we \u2018trained\u2019 Billy on a new type of service.<br \/>\nIt took some effort, but I believe we will get along with Billy.<\/li>\n<\/ul>\n<p><\/p>\n<h3 id=\"progulki-po-volframovym-polyam--tungsten-fabric\">Wandering through tungsten fields \u2014 Tungsten Fabric<\/h3>\n<p><\/p>\n<p><em>Tungsten fields, littered with hundreds of wires, running thousands of bits of information. The information is gathered into \u2018packets\u2019, dissected, and arranged into complex routes, like magic.<\/em><\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/9b6dd46d9f15e0e3393a8c1222abbf2f.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>This is the domain of the second system we had to befriend \u2014 Tungsten Fabric (TF), formerly OpenContrail. Its task is to manage networking equipment, providing software abstraction to us as users. TF is SDN, encapsulating complex logic for working with networking hardware. There\u2019s a decent article on the technology, for example, <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/muk\/blog\/251959\/\">here<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p>The system is integrated with OpenStack (which we will discuss below) through the Neutron plugin. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/5abe3992efaab3e016a0c51dc0070a94.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Interaction of OpenStack services.<\/em><\/p>\n<p><\/p>\n<p>We were introduced to this system by the guys from the operations department. We use the API of the system to manage the network stack of our services. So far, it hasn't caused us any serious problems or inconveniences (I won't speak for the guys from the operations department), but we have had some curious interactions.<\/p>\n<p><\/p>\n<p>The first issue looked like this: commands that required outputting a large amount of data to the instance console when connecting via SSH would simply hang the connection, while everything worked fine via VNC.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/02bc97d63a1299e1dda08fce0f10fb97.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>For those unfamiliar with the issue, it looks quite amusing: ls \/root works correctly, while, for example, top completely hangs. Fortunately, we had encountered similar problems before. It was resolved by tuning the MTU on the route from compute nodes to routers. By the way, this isn't a problem for TF.<\/p>\n<p><\/p>\n<p>The next problem awaited just around the corner. At one 'wonderful' moment, the magic of routing simply disappeared. TF stopped managing the routing on the equipment.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/db2a75ffdf2940b2ca90f73835f7a321.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>We worked with OpenStack from the admin level and then switched to the necessary user level. SDN seems to 'intercept' the user scope under which actions are performed. The thing is, this same admin account is used for communication between TF and OpenStack. When switching to the user, the 'magic' disappeared. It was decided to create a separate account for working with the system. This allowed us to work without breaking the integration functionality.<\/p>\n<p><\/p>\n<h3 id=\"silikonovye-formy-zhizni--openstack\">Silicon life forms \u2014 OpenStack<\/h3>\n<p><\/p>\n<p><em>The silicon creature, with its bizarre shape, inhabits the area near the tungsten fields. It most resembles a giant child who could crush us with a single swipe, but it shows no overt aggression. It doesn't invoke fear, but its size is intimidating, along with the complexity of what is happening around it.<\/em><\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/8622d43e0cb963bc9ef2a2ce59a32499.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>OpenStack is the core of our platform.<\/p>\n<p><\/p>\n<p>OpenStack has several subsystems, among which we are currently using Nova, Glance, and Cinder the most actively. Each of these has its own API. Nova is responsible for compute resources and the creation of instances, Cinder handles volume management and their snapshots, while Glance is the image service that manages OS templates and their metadata.<\/p>\n<p><\/p>\n<p>Each service runs in a container, with 'RabbitMQ' being the message broker.<\/p>\n<p><\/p>\n<p>This system has caused us the most unexpected troubles.<\/p>\n<p><\/p>\n<p>The first issue arose when we tried to connect an additional volume to the server. The Cinder API flatly refused to carry out this task. Specifically, according to OpenStack itself, the connection is established, but within the virtual server, the disk device is missing.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/b4755d893f3f8ad7d7aeab3af69f35cf.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>We decided to 'go around' and requested the same action from the Nova API. The result \u2014 the device connects correctly and is accessible inside the server. It seems that the problem arises when block-storage does not respond to Cinder.<\/p>\n<p><\/p>\n<p>Another challenge awaited us when working with disks. We were unable to detach the system volume from the server.<\/p>\n<p><\/p>\n<p>Again, OpenStack 'claims' that it has destroyed the connection and now we can work with the volume separately. However, the API categorically refused to perform operations on the disk. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/952fd7cd11181605c52a7df71cd4f69f.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Here, we decided not to fight too hard, but rather to change our perspective on the service's logic. Since there is an instance, there must be a system volume. Therefore, users cannot delete or detach the system 'disk' without deleting the 'server.'<\/p>\n<p><\/p>\n<p>OpenStack is a fairly complex system with its own interaction logic and intricate API. We're aided by quite detailed documentation and, of course, the trial and error method (where would we be without it?).<\/p>\n<p><\/p>\n<h3 id=\"testovyy-zapusk\">Test Launch<\/h3>\n<p><\/p>\n<p>We conducted the test launch in December last year. Our main goal was to check our project in a live environment from a technical and UX perspective. The audience was invited selectively, and the testing was closed. However, we also left the opportunity to request access to the testing on our website.<\/p>\n<p><\/p>\n<p>The test, of course, was not without humorous moments, as our adventures are just beginning. <\/p>\n<p><\/p>\n<p>Firstly, we somewhat incorrectly assessed the interest in the project and had to urgently add compute nodes during the test. A typical case for a cluster, but there were nuances here too. The documentation for this specific version of TF mentions a particular kernel version on which the interaction with vRouter was tested. We decided to run nodes with newer kernels. As a result, TF did not receive routes from the nodes. We had to urgently revert the kernels.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/377208246d5e396317fb934a35ed1227.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Another funny moment is related to the functionality of the 'change password' button in the personal account. <\/p>\n<p><\/p>\n<p>We decided to use JWT to manage access to the personal account, so we wouldn't have to deal with sessions. Since the systems are diverse and widely scattered, we manage our own token, in which we 'wrap' the billing sessions and the token from OpenStack. Of course, when the password is changed, the token 'expires' since the user's data is no longer valid and needs to be reissued. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/daa27256d0b888b2c59d4bc6129f600d.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>We overlooked this aspect, and we simply didn't have the resources to quickly complete this part. We had to cut functionality right before the testing launch.<br \/>\nAt this moment, we log out the user if the password has been changed.<\/p>\n<p><\/p>\n<p>Despite these nuances, the testing went well. Over a couple of weeks, about 300 people visited us. We were able to see the product through the eyes of users, test it in action, and gather valuable feedback. <\/p>\n<p><\/p>\n<h3 id=\"prodolzhenie-sleduet\">To be continued<\/h3>\n<p><\/p>\n<p>For many of us, this is the first project of this scale. We have learned several valuable lessons about working in a team, making architectural and design decisions. How to integrate complex systems with limited resources and deploy them into production. <\/p>\n<p><\/p>\n<p>Of course, there's still work to be done in terms of code and at the integration junctions of the systems. The project is still young, but we are full of ambition to grow it into a reliable and user-friendly service.<\/p>\n<p><\/p>\n<p>We have already managed to negotiate with the systems. Bill is dutifully handling calculations, invoicing, and user requests from his little office. The \"magic\" of tungsten fields provides us with stable connectivity. Only OpenStack occasionally acts up, exclaiming something like \"'WSREP has not yet prepared node for application use.\" But that is a whole different story...<\/p>\n<p><\/p>\n<p>We recently launched the service.<br \/>\nYou can find all the details on our <noindex><a rel=\"nofollow\" href=\"https:\/\/clo.ru?utm_source=habr&amp;utm_medium=article&amp;utm_campaign=product&amp;utm_term=first_clo\">the website<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"The history of the creation of a cloud service, spiced with cyberpunk\" src=\"\/wp-content\/uploads\/2020\/04\/32550980b529a5d7b395b3962218e53e.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>CLO Development Team<\/em><\/p>\n<p><\/p>\n<p>                        <b class=\"spoiler_title\">Useful links<\/b><\/p>\n<p><strong>OpenStack<\/strong><\/p>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/docs.openstack.org\/nova\/latest\/\">https:\/\/docs.openstack.org\/nova\/latest\/<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/docs.openstack.org\/keystone\/latest\/\">https:\/\/docs.openstack.org\/keystone\/latest\/<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/docs.openstack.org\/cinder\/latest\/\">https:\/\/docs.openstack.org\/cinder\/latest\/<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/docs.openstack.org\/glance\/latest\/\">https:\/\/docs.openstack.org\/glance\/latest\/<\/a><\/noindex><\/li>\n<\/ul>\n<p><\/p>\n<p><strong>Tungsten Fabric<\/strong><\/p>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"http:\/\/docs.tungsten.io\/en\/latest\/user\/getting-started\/index.html\">http:\/\/docs.tungsten.io\/en\/latest\/user\/getting-started\/index.html<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.juniper.net\/documentation\/en_US\/contrail-cloud10.0\/topics\/concept\/contrail-cloud-openstack-integration-overview.html\">https:\/\/www.juniper.net\/documentation\/en_US\/contrail-cloud10.0\/topics\/concept\/contrail-cloud-openstack-integration-overview.html<\/a><\/noindex><\/li>\n<\/ul>\n<p>Source: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/first\/blog\/493606\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0421 \u0440\u043e\u0441\u0442\u043e\u043c \u0441\u0442\u0430\u0436\u0430 \u0440\u0430\u0431\u043e\u0442\u044b \u0432 IT \u043d\u0430\u0447\u0438\u043d\u0430\u0435\u0448\u044c \u0437\u0430\u043c\u0435\u0447\u0430\u0442\u044c, \u0447\u0442\u043e \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u0438\u043c\u0435\u044e\u0442 \u0441\u0432\u043e\u0439 \u0445\u0430\u0440\u0430\u043a\u0442\u0435\u0440. \u041e\u043d\u0438 \u043c\u043e\u0433\u0443\u0442 \u0431\u044b\u0442\u044c \u043f\u043e\u043a\u043b\u0430\u0434\u0438\u0441\u0442\u044b\u043c\u0438, \u043c\u043e\u043b\u0447\u0430\u043b\u0438\u0432\u044b\u043c\u0438, \u0432\u0437\u0431\u0430\u043b\u043c\u043e\u0448\u043d\u044b\u043c\u0438, \u0441\u0443\u0440\u043e\u0432\u044b\u043c\u0438. \u041c\u043e\u0433\u0443\u0442 \u0440\u0430\u0441\u043f\u043e\u043b\u0430\u0433\u0430\u0442\u044c \u043a \u0441\u0435\u0431\u0435 \u0438\u043b\u0438 \u043e\u0442\u0442\u0430\u043b\u043a\u0438\u0432\u0430\u0442\u044c. \u0422\u0430\u043a \u0438\u043b\u0438 \u0438\u043d\u0430\u0447\u0435, \u043f\u0440\u0438\u0445\u043e\u0434\u0438\u0442\u0441\u044f \u00ab\u0434\u043e\u0433\u043e\u0432\u0430\u0440\u0438\u0432\u0430\u0442\u044c\u0441\u044f\u00bb \u0441 \u043d\u0438\u043c\u0438, \u043b\u0430\u0432\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043c\u0435\u0436\u0434\u0443 \u00ab\u043f\u043e\u0434\u0432\u043e\u0434\u043d\u044b\u043c\u0438 \u043a\u0430\u043c\u043d\u044f\u043c\u0438\u00bb \u0438 \u0432\u044b\u0441\u0442\u0440\u0430\u0438\u0432\u0430\u0442\u044c \u0446\u0435\u043f\u043e\u0447\u043a\u0438 \u0438\u0445 \u0432\u0437\u0430\u0438\u043c\u043e\u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f. \u0412\u043e\u0442 \u0438 \u043d\u0430\u043c \u0432\u044b\u043f\u0430\u043b\u0430 \u0447\u0435\u0441\u0442\u044c \u043f\u043e\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u043e\u0431\u043b\u0430\u0447\u043d\u0443\u044e \u043f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u0443, \u0430 \u0434\u043b\u044f \u044d\u0442\u043e\u0433\u043e \u043f\u043e\u0442\u0440\u0435\u0431\u043e\u0432\u0430\u043b\u043e\u0441\u044c \u00ab\u0443\u0433\u043e\u0432\u043e\u0440\u0438\u0442\u044c\u00bb [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":79154,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-79153","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.1.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0421 \u0440\u043e\u0441\u0442\u043e\u043c \u0441\u0442\u0430\u0436\u0430.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/istoriya-sozdaniya-oblachnogo-servisa-pripravlennaya-kiberpankom\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.1.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0418\u0441\u0442\u043e\u0440\u0438\u044f \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043e\u0431\u043b\u0430\u0447\u043d\u043e\u0433\u043e \u0441\u0435\u0440\u0432\u0438\u0441\u0430, \u043f\u0440\u0438\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043d\u0430\u044f \u043a\u0438\u0431\u0435\u0440\u043f\u0430\u043d\u043a\u043e\u043c | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0421 \u0440\u043e\u0441\u0442\u043e\u043c \u0441\u0442\u0430\u0436\u0430.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/istoriya-sozdaniya-oblachnogo-servisa-pripravlennaya-kiberpankom\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-04-24T17:42:43+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-04-24T17:42:43+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47The story of the creation of the cloud service, spiced up with cyberpunk | ProHoster","description":"As experience grows.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/istoriya-sozdaniya-oblachnogo-servisa-pripravlennaya-kiberpankom","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0418\u0441\u0442\u043e\u0440\u0438\u044f \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043e\u0431\u043b\u0430\u0447\u043d\u043e\u0433\u043e \u0441\u0435\u0440\u0432\u0438\u0441\u0430, \u043f\u0440\u0438\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u043d\u0430\u044f \u043a\u0438\u0431\u0435\u0440\u043f\u0430\u043d\u043a\u043e\u043c | ProHoster","og:description":"\u0421 \u0440\u043e\u0441\u0442\u043e\u043c \u0441\u0442\u0430\u0436\u0430.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/istoriya-sozdaniya-oblachnogo-servisa-pripravlennaya-kiberpankom","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-04-24T17:42:43+00:00","article:modified_time":"2020-04-24T17:42:43+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"79153","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 16:43:24","updated":"2022-10-02 03:25:48","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/79153","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=79153"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/79153\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/79154"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=79153"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=79153"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=79153"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}