{"id":35973,"date":"2019-10-31T22:08:54","date_gmt":"2019-10-31T19:08:54","guid":{"rendered":"https:\/\/prohoster.info\/blog\/avtomatizatsiya-dlya-samyh-malenkih-chast-pervaya-kotoraya-posle-nulevoj-virtualizatsiya-seti\/"},"modified":"2019-10-31T22:08:54","modified_gmt":"2019-10-31T19:08:54","slug":"avtomatizatsiya-dlya-samyh-malenkih-chast-pervaya-kotoraya-posle-nulevoj-virtualizatsiya-seti","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/avtomatizatsiya-dlya-samyh-malenkih-chast-pervaya-kotoraya-posle-nulevoj-virtualizatsiya-seti","title":{"rendered":"Automation for the Very Young. Part One (which comes after zero). Network Virtualization","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>In <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/453516\/\">the previous release<\/a><\/noindex> I outlined a network automation framework. Feedback from some people indicates that even this initial approach to the problem has clarified certain issues. This makes me very happy because our goal in this cycle is not to coat Ansible with Python scripts but to build a system.<\/p>\n<p>This framework establishes the order in which we will address the question.<br \/>\nAnd the network virtualization, which this issue is dedicated to, does not really fit into the theme of ADSM, where we discuss automation. <\/p>\n<p>But let's look at it from a different angle.<\/p>\n<p>For a long time, one network has been used by many services. In the case of a telecommunications operator, this includes 2G, 3G, LTE, broadband, and B2B, for example. For data centers: connectivity for different clients, the Internet, block storage, object storage.<\/p>\n<p>And all services require isolation from each other. That\u2019s how overlay networks came into being.<\/p>\n<p>And all services don\u2019t want to wait for a person to configure them manually. That\u2019s where orchestrators and SDN emerged.<\/p>\n<p>The first approach to systematic network automation, or rather a part of it, has long been undertaken and implemented in various places: VMWare, OpenStack, Google Compute Cloud, AWS, Facebook.<\/p>\n<p>That\u2019s what we will analyze today. <\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/fs.linkmeup.ru\/images\/adsm\/1\/kdpv.jpg\"><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/e0bf73b7b71c383c5526da48a6f6b776.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h1>Content<\/h1>\n<p><\/p>\n<ul>\n<li><b>Reasons<\/b><\/li>\n<li><b>Terminology<\/b><\/li>\n<li><b>Underlay \u2014 physical network<\/b><\/li>\n<li><b>Overlay \u2014 virtual network<\/b>\n<ul>\n<li>Overlay with ToR<\/li>\n<li>Overlay from the host<\/li>\n<li>Using Tungsten Fabric as an example\n<ul>\n<li>Communication within a single physical machine<\/li>\n<li>Communication between VMs located on different physical machines<\/li>\n<li>Access to the outside world<\/li>\n<\/ul>\n<p>\n <\/li>\n<\/ul>\n<p>\n <\/li>\n<li><b>FAQ<\/b><\/li>\n<li><b>Conclusion<\/b><\/li>\n<li><b>Useful links<\/b><\/li>\n<\/ul>\n<p><\/p>\n<h1>Reasons<\/h1>\n<p>\nAnd since we are on this topic, it is worth mentioning the prerequisites for network virtualization. In fact, this process didn\u2019t start yesterday. <\/p>\n<p>You have probably heard many times that the network has always been the most inert part of any system. And this is true in every sense. The network is the basis upon which everything relies, and making changes to it is quite difficult \u2014 services cannot tolerate if the network goes down. Often, taking one node offline can disrupt a significant portion of applications and impact many clients. Partly for this reason, the network team may resist any changes \u2014 because it\u2019s working somehow now (<i>we may not even know how<\/i>), and now we need to set something new up, and it\u2019s uncertain how it will affect the network.<\/p>\n<p>To avoid waiting for network operators to configure VLANs and to avoid having to set up services on every node in the network, people invented the use of overlays \u2014 layered networks \u2014 which come in a vast array: GRE, IPinIP, MPLS, MPLS L2\/L3VPN, VXLAN, GENEVE, MPLSoverUDP, MPLSoverGRE, etc.<\/p>\n<p>Their appeal lies in two simple things:<\/p>\n<ul>\n<li>Only the end nodes are configured \u2014 transit nodes need not be touched. This significantly speeds up the process and sometimes even allows the network infrastructure department to be excluded from the onboarding of new services.<\/li>\n<li>The load is hidden deep within the headers \u2014 transit nodes don't need to know anything about it, about the addressing of hosts, or the routes of the overlay network. This means less information has to be stored in the tables, allowing for simpler\/cheaper devices.<\/li>\n<\/ul>\n<p>\nIn this not entirely complete issue, I do not plan to cover all possible technologies but rather to describe the framework for how overlay networks function in data centers.<\/p>\n<p>The entire series will describe a data center consisting of rows of identical racks, in which identical server hardware is installed. <\/p>\n<p>On this hardware, virtual machines\/containers\/serverless are launched to implement services.<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/a6863eb2804a298304d1e93daecf90d9.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<h1>Terminology<\/h1>\n<p>\nIn the cycle <b>proxy server<\/b> I will refer to the program that implements the server side of client-server communication.<\/p>\n<p>Physical machines in racks will be called servers <b>do not<\/b> from now on.<\/p>\n<p><b>A physical machine<\/b> is an x86 computer installed in a rack. The most commonly used term <b>host<\/b>. We will call it \"<b>an application running on a physical machine that emulates physical resources where Virtual Machines run. Sometimes in literature and online, the term \"hypervisor\" is used as a synonym for \"host.\"<\/b>\" or <b>host<\/b>.<\/p>\n<p><b>Hypervisor<\/b> VM<\/p>\n<p><b>Virtual machine<\/b> \u2014 an operating system running on a physical machine above a hypervisor. For our purposes in this cycle, it is not so crucial whether this is actually a virtual machine or just a container. We will refer to it as \"<b>is a broad term that in this article I will define as a separate service or a specific client.<\/b>&lt;&lt;<\/p>\n<p><b>Multi-tenancy<\/b> or multi-tenant architecture refers to the use of the same application by different clients\/services. Here, the isolation of clients from each other is achieved through the application architecture rather than through separately launched instances.<\/p>\n<p><b>ToR \u2014 Top of the Rack switch<\/b> or multi-tenancy \u2014 the use of the same application by different clients\/services. In this case, client isolation is achieved through the application's architecture rather than through separately launched instances.<\/p>\n<p><b>ToR \u2014 Top of the Rack switch<\/b> \u2014 a rack-mounted switch to which all physical machines are connected.<\/p>\n<blockquote><p> In addition to ToR topology, various providers practice End of Row (EoR) or Middle of Row (though the latter is rarely used, and I haven't encountered the abbreviation MoR).\n<\/p><\/blockquote>\n<p> <b>Underlay network<\/b> or the underlay network is the physical network infrastructure: switches, routers, cables.<\/p>\n<p><b>Overlay network<\/b> or overlay network \u2014 a virtual network of tunnels operating on top of the physical one.<\/p>\n<p><b>L3 fabric or IP fabric<\/b> \u2014 a wonderful invention of humanity that allows one to avoid repeating STP and learning TRILL during interviews. A concept where the entire network, down to the access level, is entirely L3, without VLANs and, as a result, huge sprawling broadcast domains. We'll unpack the term 'fabric' in the next part.<\/p>\n<p><b>SDN<\/b> \u2014 Software Defined Network. Hardly needs an introduction. It's an approach to network management where changes to the network are performed not by a person but by a program. It usually means taking the Control Plane out of the end network devices and placing it on a controller.<\/p>\n<p><b>NFV<\/b> \u2014 Network Function Virtualization \u2014 virtualization of network devices, implying that some network functions can be run as virtual machines or containers to accelerate the deployment of new services, organize Service Chaining, and simplify horizontal scalability.<\/p>\n<p><b>VNF<\/b> \u2014 Virtual Network Function. A specific virtual device: router, switch, firewall, NAT, IPS\/IDS, etc.<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/971b6a28a9caab5f3a7067ce4d25749c.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<blockquote><p> I am currently intentionally simplifying the description to a specific implementation to avoid confusing the reader too much. For a more thoughtful read, I refer you to the section <noindex><a rel=\"nofollow\" href=\"#LINKS\">Links<\/a><\/noindex>. Additionally, Roma Gorgi, who criticizes this article for inaccuracies, promises to write a separate issue on server and network virtualization technologies that is more in-depth and detail-oriented.<\/p><\/blockquote>\n<p>Most networks today can clearly be divided into two parts: <\/p>\n<p><b>Underlay<\/b> \u2014 a physical network with a stable configuration.<br \/>\n<b>Overlay<\/b> \u2014 an abstraction over Underlay for tenant isolation. <\/p>\n<p>This is true for both the datacenter case (which we will discuss in this article) and for ISPs (which we will not discuss because it was already covered in <noindex><a rel=\"nofollow\" href=\"https:\/\/linkmeup.ru\/sdsm\">SDN<\/a><\/noindex>). With enterprise networks, of course, the situation is somewhat different. <\/p>\n<p>A diagram focusing on the network:<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/7b7dabc6d9f3b87c598346bf9195d568.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<h1>Underlay<\/h1>\n<p>\nUnderlay is a physical network: hardware switches and cables. Devices in the underlay know how to reach physical machines.<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/776a77a3821fe79a7323947e0bdf67e8.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nIt relies on standard protocols and technologies. Not least because hardware devices still operate on proprietary software that does not allow chip programming or the implementation of their own protocols; therefore, compatibility with other vendors and standardization are required.<\/p>\n<blockquote><p>However, companies like Google can afford the development of their own switches and the abandonment of commonly accepted protocols. But LAN_DC is not Google.\n<\/p><\/blockquote>\n<p> Underlay changes relatively rarely because its task is to provide basic IP connectivity between physical machines. Underlay knows nothing about the services, clients, or tenants running on top of it; it only needs to deliver packets from one machine to another.<br \/>\nAn example of an Underlay might be: <\/p>\n<ul>\n<li>IPv4+OSPF<\/li>\n<li>IPv6+ISIS+BGP+L3VPN<\/li>\n<li>L2+TRILL<\/li>\n<li>L2+STP<\/li>\n<\/ul>\n<p>\nThe Underlay network is configured in a classic way: CLI\/GUI\/NETCONF.<\/p>\n<p>Manually, with scripts, proprietary utilities.<\/p>\n<p>The next article in the series will delve into Underlay in more detail.<\/p>\n<p><\/p>\n<h1>Overlay<\/h1>\n<p>\nOverlay is a virtual network of tunnels stretched over Underlay, allowing one client\u2019s VMs to communicate with each other, while ensuring isolation from other clients.<\/p>\n<p>Client data is encapsulated in tunnel headers for transmission over a shared network.<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/3060bc4913c55c21b785083fbe0b603b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nThus, one client\u2019s VMs (from one service) can communicate with each other through Overlay, even unaware of the actual path the packet takes. <\/p>\n<p>An example of an Overlay might be, as I mentioned earlier:<\/p>\n<ul>\n<li>GRE tunnel<\/li>\n<li>VXLAN<\/li>\n<li>EVPN<\/li>\n<li>L3VPN<\/li>\n<li>GENEVE<\/li>\n<\/ul>\n<p>\nThe Overlay network is typically configured and maintained via a central controller. From there, the configuration, Control Plane, and Data Plane are delivered to the devices responsible for routing and encapsulating client traffic. A bit <noindex><a rel=\"nofollow\" href=\"#TF\">further down<\/a><\/noindex> we will explore this with examples.<\/p>\n<p><b>Yes, this is SDN in its purest form. <\/b><\/p>\n<p>There are two fundamentally different approaches to organizing an Overlay network:<\/p>\n<ol>\n<li>Overlay with ToR<\/li>\n<li>Overlay from the host<\/li>\n<\/ol>\n<h2>Overlay with ToR<\/h2>\n<p>\nOverlay can start at the access switch (ToR) located in the rack, as happens, for example, in the case of a VXLAN fabric. <\/p>\n<p>This is a time-tested mechanism used in ISP networks, and all networking vendors support it.<\/p>\n<p>However, in this case, the ToR switch must be able to separate different services accordingly, and the network administrator must collaborate to some extent with virtual machine administrators and make changes (even if automatically) to the device configuration.<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/9a5fe4d4a0cbb7e5c3bbac82527c6a6a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nHere I will refer the reader to the article on <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/344326\/\">VxLAN on Habr<\/a><\/noindex> our old friend <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/users\/bormoglotx\/\">@bormoglotx<\/a><\/noindex>.<br \/>\n, there will be a list of resources for preparation for the Azure Dev\/Infra\/Architecture exam. <noindex><a rel=\"nofollow\" href=\"https:\/\/www.enog.org\/wp-content\/uploads\/presentations\/enog-16\/18-Scaleway-P14-fabric-ENOG16.pdf\">presentation with ENOG<\/a><\/noindex> which describes in detail the approaches to building a DC network with an EVPN VXLAN fabric. <\/p>\n<p>And for a more comprehensive immersion into the realities, you can read Cisco's book <noindex><a rel=\"nofollow\" href=\"https:\/\/www.cisco.com\/c\/dam\/en\/us\/td\/docs\/switches\/datacenter\/nexus9000\/sw\/vxlan_evpn\/VXLAN_EVPN.pdf\">A Modern, Open, and Scalable Fabric: VXLAN EVPN<\/a><\/noindex>.<\/p>\n<blockquote><p> I should note that VXLAN is just a method of encapsulation and termination of tunnels may not occur on the ToR, but on the host, as is the case in OpenStack, for instance.<\/p>\n<p>However, a VXLAN fabric where the overlay starts on the ToR is one of the established designs for overlay networks.\n<\/p><\/blockquote>\n<p><\/p>\n<h2>Overlay from the host<\/h2>\n<p>\nAnother approach is to start and terminate tunnels on the end hosts.<br \/>\nIn this case, the underlay network remains as simple and static as possible.<br \/>\nAnd the host itself will handle all necessary encapsulations.<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/c9c2971369d1b3e5f026d6844afeb67f.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nTo do this, it will certainly be necessary to run a special application on the hosts, but it's worth it. <\/p>\n<p>Firstly, running a client on a Linux machine is easier or, let's say, even possible\u2014whereas on a switch, you will likely have to resort to proprietary SDN solutions, which undermines the idea of multivendor compatibility.<\/p>\n<p>Secondly, the ToR switch in this case can be kept as simple as possible, both from the perspective of the Control Plane and the Data Plane. Indeed, with an SDN controller it does not need to communicate, and storing networks\/ARP of all connected clients is also unnecessary\u2014it is sufficient to know the IP address of the physical machine, which significantly simplifies the switching\/routing tables.<\/p>\n<p>\nIn the ADSM series, I choose the overlay approach from the host\u2014further, we will only discuss this, and we will not return to the VXLAN fabric.<\/p>\n<p>\nIt is easiest to consider this with examples. As a test subject, we will take the OpenSource SDN platform OpenContrail, now known as <noindex><a rel=\"nofollow\" href=\"https:\/\/tungsten.io\">Tungsten Fabric<\/a><\/noindex>.<\/p>\n<blockquote><p> At the end of the article, I will present some thoughts on the analogy with OpenFlow and OpenvSwitch.\n<\/p><\/blockquote>\n<p><\/p>\n<h2>Using Tungsten Fabric as an example<\/h2>\n<p>\nOn each physical machine, there is <b>vRouter<\/b> \u2014 a virtual router that is aware of the networks connected to it and which clients belong to them\u2014essentially a PE router. For each client, it maintains an isolated routing table (read VRF). The vRouter itself handles the Overlay tunneling.<\/p>\n<p>A bit more about vRouter is at the end of the article.<\/p>\n<p>Each VM located on the hypervisor connects to the vRouter of that machine via <noindex><a rel=\"nofollow\" href=\"https:\/\/en.wikipedia.org\/wiki\/TUN\/TAP\">TAP interface<\/a><\/noindex>.<\/p>\n<p><b>TAP<\/b> \u2014 Terminal Access Point \u2014 a virtual interface in the Linux kernel that allows for network communication.<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/a6fa1b4370ba4e787303ae647a28f5ed.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nIf there are multiple networks behind the vRouter, a virtual interface is created for each of them, to which an IP address is assigned\u2014it will be the default gateway address.<br \/>\nAll networks of one client are placed in one <b>VRF<\/b> (one table), while different ones are in different tables.<br \/>\n<i>I want to note that it's not that simple, and I will send the curious reader to the end of the article.<\/i>.<\/p>\n<p>For vRouters to communicate with each other, and accordingly, with the VMs behind them, they exchange routing information through <b>SDN controller<\/b>.<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/fs.linkmeup.ru\/images\/adsm\/1\/sdn-controller.png\"><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/8b133e10582dbde9c0670ad98c1840de.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p>To connect to the outside world, there is an exit point from the matrix \u2014 the virtual network gateway <b>VNGW<\/b> \u2014 Virtual Network GateWay (<i>this term is mine<\/i>).<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/fs.linkmeup.ru\/images\/adsm\/1\/vngw.png\"><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/9ac4d084d9b58b7ed1bc2c0ed6d06059.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p>\nNow let's look at communication examples \u2014 this will clarify things.<\/p>\n<p><\/p>\n<h3>Communication within a single physical machine<\/h3>\n<p>\nVM0 wants to send a packet to VM2. For now, let's assume these VMs belong to the same client.<\/p>\n<h4>Data Plane<\/h4>\n<p><\/p>\n<ol>\n<li>VM-0 has a default route through its interface eth0. The packet is sent there.<br \/>\n This interface eth0 is actually virtually connected to the virtual router vRouter via the TAP interface tap0.<\/li>\n<li>The vRouter analyzes which interface the packet arrived on, i.e., which client (VRF) it belongs to, and checks the recipient's address against that client's routing table.<\/li>\n<li>Upon discovering that the recipient is on the same machine via another port, the vRouter simply sends the packet to it without any additional headers\u2014on this occasion, the vRouter already has an ARP entry. <\/li>\n<\/ol>\n<p>\n<img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/a42da3f48a3f538eb92c1908d9d66921.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nIn this case, the packet does not enter the physical network \u2014 it is routed within the vRouter.<\/p>\n<p><\/p>\n<h4>Control Plane<\/h4>\n<p>\nWhen starting the virtual machine, the hypervisor informs it:<\/p>\n<ul>\n<li>Its own IP address.<\/li>\n<li>The default route is through the IP address of the vRouter in this network.<\/li>\n<\/ul>\n<p>\nThe vRouter is informed by the hypervisor through a special API:<\/p>\n<ul>\n<li>That a virtual interface needs to be created.<\/li>\n<li>What Virtual Network it needs to create (for the VM).<\/li>\n<li>To which VRF to bind it (VN).<\/li>\n<li>The static ARP entry for this VM - which interface its IP address is associated with and to which MAC address it is tied.<\/li>\n<\/ul>\n<p><\/p>\n<blockquote><p> Once again, the real procedure of interaction is simplified for the sake of understanding the concept.\n<\/p><\/blockquote>\n<p> <img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/593d7ad41782e882a88732b475bf91cb.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nThus, all VMs of a single client on this machine see the vRouter as directly connected networks and can route between them independently.<\/p>\n<p>\nHowever, VM0 and VM1 belong to different clients and are thus located in different vRouter tables.<\/p>\n<p>Whether they can communicate directly depends on the vRouter settings and network design.<br \/>\nFor example, if both clients' VMs use public addresses, or if NAT occurs on the vRouter itself, direct routing to the vRouter can be established.<\/p>\n<p>In the opposite case, address space overlap may occur \u2013 a NAT server is needed to obtain a public address \u2013 this resembles accessing external networks, which will be discussed below.<\/p>\n<h3>Communication between VMs located on different physical machines<\/h3>\n<p><\/p>\n<h4>Data Plane<\/h4>\n<p><\/p>\n<ol>\n<li>The start is exactly the same: VM-0 sends a packet addressed to VM-7 (172.17.3.2) based on its default.<\/li>\n<li>The vRouter receives it and this time sees that the recipient is on a different machine and is accessible through Tunnel0.<\/li>\n<li>First, it attaches an MPLS label that identifies the remote interface so that on the other side, the vRouter can determine where to place this packet without additional lookups.\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/fs.linkmeup.ru\/images\/adsm\/1\/inter-hv-dp.png\"><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/1e159049376e2bf458b8c6cb5878cc17.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex>\n <\/li>\n<li>For Tunnel0, the source is 10.0.0.2, and the destination is 10.0.1.2.<br \/>\n The vRouter adds GRE (or UDP) headers and a new IP to the original packet.<\/li>\n<li>In the routing table, the vRouter has a default route through ToR1 10.0.0.1. It sends it there.\n<p> <img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/8cb98a8ec43d202259acf6a1fcaeb3ee.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n \n <\/li>\n<li>ToR1, as a participant in the underlay network, knows (for example, via OSPF) how to reach 10.0.1.2 and sends the packet along that route. Note that ECMP is involved here. In the illustration, there are two next hops, and different flows will be distributed among them based on hashes. In a real fabric, there would be more like 4 next hops.\n<p>At the same time, it does not need to know what is under the external IP header. So actually, under IP, it may be a sandwich of IPv6 over MPLS over Ethernet over MPLS over GRE over and over again.<\/li>\n<li>Consequently, on the receiving side, the vRouter strips off GRE and, by the MPLS label, understands to which interface this packet should be forwarded, unpacks it, and sends it to the recipient in its original form.<\/li>\n<\/ol>\n<h4>Control Plane<\/h4>\n<p>\nWhen the machine is powered up, everything described above occurs.<\/p>\n<p>And plus, the following happens:<\/p>\n<ul>\n<li>For each client, the vRouter allocates an MPLS label. This is a service label for L3VPN, which clients will use to be separated within the same physical machine.<br \/>\n<blockquote><p> In fact, the MPLS label is always assigned by the vRouter without exception \u2014 it is unknown in advance whether the machine will interact only with other machines behind the same vRouter, and this is likely not the case. \n <\/p><\/blockquote>\n<\/li>\n<li>The vRouter establishes a connection with the SDN controller using the BGP protocol (or a similar one\u2014in the case of TF, it\u2019s XMPP 0_o).<\/li>\n<li>Through this session, the vRouter informs the SDN controller of the routes to the connected networks:\n<ul>\n<li>Network address<\/li>\n<li>Encapsulation method (MPLSoGRE, MPLSoUDP, VXLAN)<\/li>\n<li>Client MPLS label<\/li>\n<li>Its IP address as the nexthop<\/li>\n<\/ul>\n<p>\n <\/li>\n<li>The SDN controller receives such routes from all connected vRouters and reflects them to others. In other words, it acts as a Route Reflector.<\/li>\n<\/ul>\n<p>\nThe same happens in the opposite direction.<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/fs.linkmeup.ru\/images\/adsm\/1\/inter-hv-cp.png\"><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/feef9181133f3a50a5cb21aa7ba16dc0.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p>The overlay can change every minute. This is roughly how it operates in public clouds, where clients regularly start and stop their virtual machines.<\/p>\n<p>The central controller takes on all the complexities of maintaining configuration and controlling switching\/routing tables on the vRouter.<\/p>\n<p>To put it simply, the controller connects to all vRouters via BGP (or a similar protocol) and simply passes on routing information. BGP, for example, already has an Address-Family for transmitting encapsulation methods. <noindex><a rel=\"nofollow\" href=\"https:\/\/tools.ietf.org\/html\/rfc4023\">MPLS-in-GRE<\/a><\/noindex> or <noindex><a rel=\"nofollow\" href=\"https:\/\/tools.ietf.org\/html\/rfc7510\">MPLS-in-UDP<\/a><\/noindex>.<\/p>\n<p>At the same time, the configuration of the Underlay network does not change in any way, which, by the way, is significantly more complex to automate, and can be broken with a careless move.<\/p>\n<h3>Access to the outside world<\/h3>\n<p>\nSomewhere, the simulation must end, and it's necessary to step out of the virtual world into the real one. A gateway for that is needed.<\/p>\n<p>Two approaches are practiced:<\/p>\n<ol>\n<li>A hardware router is installed.<\/li>\n<li>Some appliance is launched that implements router functions (yes, yes, after SDN, we also encountered VNF). Let's call it a virtual gateway.<\/li>\n<\/ol>\n<p><\/p>\n<blockquote><p> The advantage of the second approach is its cheap horizontal scalability\u2014if there's a lack of power, another virtual machine with a gateway is launched. On any physical machine, without the need to look for available racks, units, power output, buy the hardware itself, transport it, install it, connect it, configure it, and then also replace its faulty components.<\/p>\n<p>The downsides of a virtual gateway are that a unit of physical router is still exponentially more powerful than a multi-core virtual machine, and its software, tailored to its hardware base, operates significantly more stably (<i>none<\/i>). It\u2019s hard to deny the fact that the hardware-software complex simply works, requiring only configuration, whereas launching and maintaining a virtual gateway is a task for skilled engineers.\n<\/p><\/blockquote>\n<p> With one foot, the gateway looks into the Overlay virtual network, like a regular Virtual Machine, and can interact with all other VMs. At the same time, it can terminate networks of all clients and respectively perform routing between them.<\/p>\n<p>With the other foot, the gateway looks into the backbone network and knows how to reach the Internet.<\/p>\n<p><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/552fa0428ff8dd3aed76091a86a72109.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<h4>Data Plane<\/h4>\n<p>\nSo the process looks like this: <\/p>\n<ol>\n<li>VM-0, having a default vRouter, sends a packet with a destination in the external world (185.147.83.177) to the eth0 interface.<\/li>\n<li>The vRouter receives this packet and performs a destination address lookup in the routing table\u2014finding the default route through gateway VNGW1 via Tunnel 1. <br \/>\n It also sees that this is a GRE tunnel with SIP 10.0.0.2 and DIP 10.0.255.2, and it needs to first apply the MPLS label for this client, which VNGW1 expects.\n <\/li>\n<li>The vRouter wraps the initial packet in MPLS, GRE headers, and a new IP, and sends it to the address ToR1 10.0.0.1 by default.<\/li>\n<li>The underlay network delivers the packet to gateway VNGW1.<\/li>\n<li>Gateway VNGW1 strips the tunneling GRE and MPLS headers, sees the destination address, consults its routing table, and understands that it is directed towards the Internet\u2014thus through Full View or Default. It performs NAT translation if necessary.<\/li>\n<li>From VNGW to the border may be a regular IP network, which is unlikely.<br \/>\n It could be a classic MPLS network (IGP+LDP\/Rsvp TE), or it could be a reverse fabric with BGP LU or a GRE tunnel from VNGW to the border over an IP network.<br \/>\n In any case, VNGW1 performs the necessary encapsulations and sends the initial packet towards the border.<\/li>\n<\/ol>\n<p>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/fs.linkmeup.ru\/images\/adsm\/1\/outside-dp.png\"><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/a66e490c75f8220349398af814d4ebf2.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p>The traffic in the reverse direction goes through the same steps in the opposite order. <\/p>\n<ol>\n<li>The border delivers the packet to VNGW1.<\/li>\n<li>It breaks it down, looks at the destination address, and sees that it is accessible via Tunnel1 (MPLSoGRE or MPLSoUDP).<\/li>\n<li>Accordingly, it applies the MPLS label, the GRE\/UDP header, and a new IP, and sends it to its ToR3 10.0.255.1.<br \/>\n The destination address of the tunnel is the IP address of the vRouter behind which the target VM is located \u2014 10.0.0.2.<\/li>\n<li>The underlay network delivers the packet to the desired vRouter. <\/li>\n<li>The target vRouter removes GRE\/UDP, identifies the interface based on the MPLS label, and sends the raw IP packet to its TAP interface associated with the VM's eth0.<\/li>\n<\/ol>\n<p>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/fs.linkmeup.ru\/images\/adsm\/1\/outside-dp-reverse.png\"><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/ba6748694a822fc04f4e872658d1fbbb.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<p><\/p>\n<h4>Control Plane<\/h4>\n<p>\nVNGW1 establishes BGP peering with the SDN controller from which it receives all routing information about the clients: which IP address (vRouter) belongs to which client and what MPLS label it is identified by.<\/p>\n<p>Likewise, it informs the SDN controller about the default route with this client's label, indicating itself as the nexthop. From there, this default route arrives at the vRouters.<\/p>\n<p>Route aggregation or NAT translation usually occurs at VNGW.<\/p>\n<p>And in the other direction, it sends exactly this aggregated route to the session with the borders or Route Reflectors. From them, it receives either the default route or a Full-View, or something else.<\/p>\n<p>In terms of encapsulation and traffic exchange, VNGW is indistinguishable from vRouter. <br \/>\nIf we expand the scope a bit, we can add other network devices to VNGW and vRouters, such as firewalls, traffic cleansing or enrichment farms, IPS, and so on.<\/p>\n<p>Using the sequential creation of VRFs and proper route announcements, traffic can be made to loop as desired, which is called Service Chaining.<\/p>\n<p>Thus, the SDN controller also acts as a Route Reflector between VNGW, vRouters, and other network devices.<\/p>\n<p>In fact, the controller also passes down information about ACL and PBR (Policy Based Routing), causing individual traffic flows to follow paths different from what the route commands.<\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/fs.linkmeup.ru\/images\/adsm\/1\/outside-cp.png\"><img decoding=\"async\" alt=\"Automation for the Very Young. Part One (which comes after zero). Network Virtualization\" src=\"\/wp-content\/uploads\/2019\/07\/dbba5a93d64edf56dc9c2b82311c5a11.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/p>\n<h1>FAQ<\/h1>\n<p><b>Why do you always make a point of GRE\/UDP?<\/b><\/p>\n<p>Well, actually, this can be said to be specific to Tungsten Fabric \u2014 one can completely ignore it.<\/p>\n<p>However, if we take it, TF, even when it was OpenContrail, supported both encapsulations: MPLS in GRE and MPLS in UDP. <\/p>\n<p>UDP is advantageous because it's very easy to encode a hash function from the original IP+Proto+Port in its header's Source Port, which allows for load balancing. <\/p>\n<p>In the case of GRE, unfortunately, there are only the external IP and GRE headers, which are identical for all encapsulated traffic, making load balancing difficult \u2014 few can look that deeply inside the packet.<\/p>\n<p>Until recently, routers that could handle dynamic tunnels only worked with MPLSoGRE and have only just begun to support MPLSoUDP. Therefore, it's always necessary to make a note of the possibility of using two different encapsulations.<\/p>\n<p>Fairness dictates that we note TF also fully supports L2 connectivity through VXLAN.<\/p>\n<p>\n<b>You promised to draw parallels with OpenFlow.<\/b><br \/>\nThey really do seem obvious. The vSwitch in the same OpenStack does very similar things using VXLAN, which, by the way, also has a UDP header.<\/p>\n<p>In the Data Plane, they operate quite similarly, but the Control Plane differs significantly. Tungsten Fabric uses XMPP to deliver routing information to the vRouter, while OpenStack operates with OpenFlow.<\/p>\n<p>\n<b>Can you elaborate more on vRouter?<\/b><br \/>\nIt consists of two parts: vRouter Agent and vRouter Forwarder.<\/p>\n<p>The first runs in the User Space of the host OS and communicates with the SDN controller by exchanging information about routes, VRF, and ACL.<\/p>\n<p>The second implements the Data Plane \u2014 usually in Kernel Space, but it can also run on SmartNICs \u2014 network cards with a CPU and a separate programmable switching chip, which helps alleviate the load on the host machine's CPU and makes the network faster and more predictable. <\/p>\n<p>There is also a scenario where the vRouter is a DPDK application in User Space. <\/p>\n<p>The vRouter Agent transmits the settings to the vRouter Forwarder.<\/p>\n<p>\n<b>What is a Virtual Network?<\/b><br \/>\nI mentioned earlier in the article about VRF, stating that each tenant is associated with their own VRF. And while this was sufficient for a superficial understanding of how the overlay network works, further clarifications are necessary in the next iteration.<\/p>\n<p>Typically, in virtualization mechanisms, the entity Virtual Network (consider it a proper noun) is introduced separately from clients\/tenants\/virtual machines \u2014 it is quite a standalone entity. This Virtual Network can then be connected through interfaces to one tenant, another, or multiple ones, wherever needed. This is how Service Chaining is implemented, where traffic needs to pass through certain nodes in a required sequence, simply creating and binding Virtual Networks in the correct order.<\/p>\n<p>Therefore, there is no direct correspondence between a Virtual Network and a tenant.<\/p>\n<h1>Conclusion<\/h1>\n<p>\nThis is a rather superficial description of how an overlay virtual network works with a host and an SDN controller. However, no matter which virtualization platform you choose today, it operates in a similar manner, whether it\u2019s VMWare, ACI, OpenStack, CloudStack, Tungsten Fabric, or Juniper Contrail. They will differ in terms of encapsulation types and headers, as well as protocols for delivering information to end network devices, but the principle of a software-defined overlay network operating atop a comparatively simple and static underlay network remains unchanged.<br \/>\nIt can be said that, as of today, the domain of private cloud creation has been won by SDN based on overlay networks. However, this doesn\u2019t mean that OpenFlow has no place in the modern world\u2014it is used in OpenStack and in VMWare NSX, and as far as I know, Google employs it for configuring the underlay network.<\/p>\n<p>Below, I've provided links to more detailed materials if you wish to study the issue further. <\/p>\n<p>And what about our Underlay? <\/p>\n<p>In general, nothing really changes. It has remained the same all along. All it needs to do in the case of an overlay from the host is to update routes and ARPs as vRouter\/VNGW appear and disappear, and to carry packets between them.<\/p>\n<p>Let\u2019s formulate a list of requirements for the Underlay network.<\/p>\n<ol>\n<li>It must support a certain routing protocol, in our case\u2014BGP.<\/li>\n<li>It should have a wide bandwidth, preferably without oversubscription, so that packets aren\u2019t lost due to congestion.<\/li>\n<li>Supporting ECMP is an integral part of the fabric.<\/li>\n<li>It must be able to provide QoS, including advanced features like ECN.<\/li>\n<li>Support for NETCONF is a future consideration.<\/li>\n<\/ol>\n<p>\nI have devoted very little time to the operation of the Underlay network here. This is because I will focus on it in the following series, while we will only touch on Overlay in passing.<\/p>\n<p>Clearly, I am severely limiting all of us by using a data center network built on a Clos fabric with pure IP routing and an overlay from the host as an example.<\/p>\n<p>However, I am confident that any network with a design can be described in formal terms and automated. My aim here is simply to understand the approaches to automation, not to confuse everyone in general by solving the problem in broad terms.<\/p>\n<p>Within the framework of ADSM, Roman Gorgy and I plan to publish a separate issue about the virtualization of computing resources and its interaction with network virtualization. Stay tuned.<\/p>\n<p><\/p>\n<h1>Useful links<\/h1>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/tungstenfabric.github.io\/website\/\">Tungsten Fabric Architecture<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/youtu.be\/Kr6WIYPts8I?t=3157\">about:cloud<\/a><\/noindex>. 6 hours on Yandex.Cloud, which includes a discussion about the virtual network on TF.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/docs.openvswitch.org\/en\/latest\/intro\/what-is-ovs\/\">What Is Open vSwitch?<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/344326\/\">Introduction to VxLAN<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/tools.ietf.org\/html\/rfc7348\">RFC 7348. Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks.<br \/>\n <\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.enog.org\/wp-content\/uploads\/presentations\/enog-16\/18-Scaleway-P14-fabric-ENOG16.pdf\">Scaleway approach to VXLAN EVPN Fabric<\/a><\/noindex>. This covers the entire DC network, including Underlay, Overlay, approaches to multi-homing, and management.<\/li>\n<\/ul>\n<p><\/p>\n<h5>Thanks<\/h5>\n<p><\/p>\n<ul>\n<li><noindex>Roman Gorgiy<\/noindex> \u2014 former host of the linkmeup podcast, now an expert in cloud platforms. Thanks for the comments and edits. We look forward to his upcoming in-depth article on virtualization.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.alexander-shalimov.com\">Alexander Shalimov<\/a><\/noindex> \u2014 my colleague and an expert in developing virtual networks. Thanks for the comments and edits.<\/li>\n<li><noindex>Valentin Sinitsin<\/noindex> \u2014 my colleague and an expert in Tungsten Fabric. Thanks for the comments and edits.<\/li>\n<li><noindex>Artem Chernobay<\/noindex> \u2014 the illustrator of linkmeup. Thanks for the design.<\/li>\n<li>Alexander Limonov. For the meme \"automato\".<\/li>\n<\/ul>\n<p>Source: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/458622\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412 \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0435\u043c \u0432\u044b\u043f\u0443\u0441\u043a\u0435 \u044f \u043e\u043f\u0438\u0441\u0430\u043b \u0444\u0440\u0435\u0439\u043c\u0432\u043e\u0440\u043a \u0441\u0435\u0442\u0435\u0432\u043e\u0439 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0430\u0446\u0438\u0438. \u041f\u043e \u043e\u0442\u0437\u044b\u0432\u0430\u043c \u0443 \u043d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0445 \u043b\u044e\u0434\u0435\u0439 \u0434\u0430\u0436\u0435 \u044d\u0442\u043e\u0442 \u043f\u0435\u0440\u0432\u044b\u0439 \u043f\u043e\u0434\u0445\u043e\u0434 \u043a \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u0435 \u0443\u0436\u0435 \u0440\u0430\u0437\u043b\u043e\u0436\u0438\u043b \u043d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0432\u043e\u043f\u0440\u043e\u0441\u044b \u043f\u043e \u043f\u043e\u043b\u043e\u0447\u043a\u0430\u043c. \u0418 \u044d\u0442\u043e \u043e\u0447\u0435\u043d\u044c \u043c\u0435\u043d\u044f \u0440\u0430\u0434\u0443\u0435\u0442, \u043f\u043e\u0442\u043e\u043c\u0443 \u0447\u0442\u043e \u043d\u0430\u0448\u0430 \u0446\u0435\u043b\u044c \u0432 \u0446\u0438\u043a\u043b\u0435 \u2014 \u043d\u0435 \u043e\u0431\u043c\u0430\u0437\u0430\u0442\u044c \u043f\u0438\u0442\u043e\u043d\u043e\u0432\u0441\u043a\u0438\u043c\u0438 \u0441\u043a\u0440\u0438\u043f\u0442\u0430\u043c\u0438 \u0430\u043d\u0437\u0438\u0431\u043b\u044c, \u0430 \u0432\u044b\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0441\u0438\u0441\u0442\u0435\u043c\u0443. \u042d\u0442\u043e\u0442 \u0436\u0435 \u0444\u0440\u0435\u0439\u043c\u0432\u043e\u0440\u043a \u0437\u0430\u0434\u0430\u0451\u0442 \u043f\u043e\u0440\u044f\u0434\u043e\u043a, \u0432 \u043a\u043e\u0442\u043e\u0440\u043e\u043c \u043c\u044b \u0431\u0443\u0434\u0435\u043c \u0440\u0430\u0437\u0431\u0438\u0440\u0430\u0442\u044c\u0441\u044f [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":26892,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-35973","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412 \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0435\u043c \u0432\u044b\u043f\u0443\u0441\u043a\u0435 \u044f \u043e\u043f\u0438\u0441\u0430\u043b \u0444\u0440\u0435\u0439\u043c\u0432\u043e\u0440\u043a \u0441\u0435\u0442\u0435\u0432\u043e\u0439 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0430\u0446\u0438\u0438.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/avtomatizatsiya-dlya-samyh-malenkih-chast-pervaya-kotoraya-posle-nulevoj-virtualizatsiya-seti\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0410\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0430\u0446\u0438\u044f \u0414\u043b\u044f \u0421\u0430\u043c\u044b\u0445 \u041c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u0445. \u0427\u0430\u0441\u0442\u044c \u043f\u0435\u0440\u0432\u0430\u044f (\u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043f\u043e\u0441\u043b\u0435 \u043d\u0443\u043b\u0435\u0432\u043e\u0439). \u0412\u0438\u0440\u0442\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u0441\u0435\u0442\u0438 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412 \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0435\u043c \u0432\u044b\u043f\u0443\u0441\u043a\u0435 \u044f \u043e\u043f\u0438\u0441\u0430\u043b \u0444\u0440\u0435\u0439\u043c\u0432\u043e\u0440\u043a \u0441\u0435\u0442\u0435\u0432\u043e\u0439 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0430\u0446\u0438\u0438.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/avtomatizatsiya-dlya-samyh-malenkih-chast-pervaya-kotoraya-posle-nulevoj-virtualizatsiya-seti\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T19:08:54+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T19:08:54+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Automation For The Little Ones. Part one (which comes after the zero part). Network virtualization | ProHoster","description":"In the previous issue, I described the network automation framework.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/avtomatizatsiya-dlya-samyh-malenkih-chast-pervaya-kotoraya-posle-nulevoj-virtualizatsiya-seti","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0410\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0430\u0446\u0438\u044f \u0414\u043b\u044f \u0421\u0430\u043c\u044b\u0445 \u041c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u0445. \u0427\u0430\u0441\u0442\u044c \u043f\u0435\u0440\u0432\u0430\u044f (\u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043f\u043e\u0441\u043b\u0435 \u043d\u0443\u043b\u0435\u0432\u043e\u0439). \u0412\u0438\u0440\u0442\u0443\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f \u0441\u0435\u0442\u0438 | ProHoster","og:description":"\u0412 \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0435\u043c \u0432\u044b\u043f\u0443\u0441\u043a\u0435 \u044f \u043e\u043f\u0438\u0441\u0430\u043b \u0444\u0440\u0435\u0439\u043c\u0432\u043e\u0440\u043a \u0441\u0435\u0442\u0435\u0432\u043e\u0439 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0430\u0446\u0438\u0438.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/avtomatizatsiya-dlya-samyh-malenkih-chast-pervaya-kotoraya-posle-nulevoj-virtualizatsiya-seti","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:08:54+00:00","article:modified_time":"2019-10-31T19:08:54+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"35973","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 01:28:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 01:53:30","updated":"2026-01-22 01:28:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/35973","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=35973"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/35973\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/26892"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=35973"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=35973"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=35973"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}