{"id":35291,"date":"2019-10-31T22:03:28","date_gmt":"2019-10-31T19:03:28","guid":{"rendered":"https:\/\/prohoster.info\/blog\/setevaya-fabrika-dlya-tsoda-cisco-aci-v-pomoshh-adminu\/"},"modified":"2019-10-31T22:03:28","modified_gmt":"2019-10-31T19:03:28","slug":"setevaya-fabrika-dlya-tsoda-cisco-aci-v-pomoshh-adminu","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/setevaya-fabrika-dlya-tsoda-cisco-aci-v-pomoshh-adminu","title":{"rendered":"Network fabric for Cisco ACI Data Centers \u2014 a helping hand for the admin","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Network
\nWith this magical piece of Cisco ACI script, you can quickly configure your network.<\/i><\/p>\n

The network fabric for Data Centers Cisco ACI has existed for five years, but there has been little detailed discussion about it on Habr, so I decided to remedy that a bit. I will share from my experience what it is, what benefits it offers, and where its pitfalls lie. <\/p>\n

What is it and where did it come from?<\/h3>\n

\nAt the time of the ACI (Application Centric Infrastructure) announcement in 2013, traditional approaches to Data Center networking were under attack from competitors on three fronts. <\/p>\n

On one side, 'first-generation' SDN solutions based on OpenFlow promised to make networks more flexible and cheaper at the same time. The idea was to shift decision-making, traditionally handled by proprietary switch software, to a central controller. <\/p>\n

This controller would have a unified view of everything happening and, based on that, would program the hardware of all switches at the level of specific flow processing rules. <\/a><\/noindex>
\nOn the other hand, overlay networking solutions allowed for the implementation of necessary connectivity and security policies without changing the physical network at all, creating software tunnels between virtualized hosts. The most well-known example of this approach was the solution from Nicira, which had already been acquired by VMWare for $1.26 billion and laid the foundation for the current VMWare NSX. Adding a touch of intrigue to the situation was that the co-founders of Nicira were the same individuals who previously initiated OpenFlow, now stating that to build a Data Center fabric, OpenFlow is not suitable.<\/a><\/noindex>.<\/p>\n

Finally, the switching chips available on the open market (what is known as merchant silicon) reached a level of maturity where they became a real threat to traditional switch manufacturers. Previously, each vendor developed chips for their switches, but over time, chips from third-party manufacturers, primarily from Broadcom, started to close the gap with vendor-specific chips in terms of features and exceeded them in terms of price\/performance ratio. As a result, many believed that the days of switches with proprietary chips were numbered.<\/p>\n

ACI became Cisco's 'asymmetric response' (more precisely, a response from Insieme, a company founded by former Cisco employees) to all of the above. <\/p>\n

What is the difference with OpenFlow?<\/h3>\n

\nFrom the perspective of function distribution, ACI is essentially the opposite of OpenFlow.
\nIn the OpenFlow architecture, the controller is responsible for writing detailed rules (flows) in the hardware of all switches, which means that in a large network it can be responsible for maintaining and, most importantly, modifying tens of millions of records across hundreds of points in the network. Therefore, its performance and reliability in large deployments become a bottleneck.
\nIn ACI, a reverse approach is used: there is indeed a controller, but the switches receive high-level declarative policies from it, and the rendering of those into specific settings in the hardware is done by the switch itself. The controller can be rebooted or even turned off, and nothing bad will happen to the network, except, of course, the inability to manage it at that moment. Interestingly, there are situations in ACI where OpenFlow is still used, but locally within the host for programming Open vSwitch.<\/p>\n

ACI is entirely built on overlay transport based on VXLAN, but it also includes underlying IP transport within a single solution. Cisco termed this \u2018integrated overlay.\u2019 In most cases, the factory switches serve as the termination point for overlays (doing so at line rate). Hosts do not need to know about the factory, encapsulations, etc., although in some cases (for example, for connecting OpenStack hosts), VXLAN traffic may also be delivered to them.<\/p>\n

Overlays in ACI are used not only to provide flexible connectivity over the transport network, but also to transmit metadata (which is used, for instance, for applying security policies).<\/p>\n

Overlays are used in ACI not only to provide flexible connectivity through the transport network but also to transmit metadata (which is used, for example, for applying security policies). <\/p>\n

Broadcom chips were previously used by Cisco in the Nexus 3000 series switches. The Nexus 9000 family, specifically released to support ACI, initially implemented a hybrid model known as Merchant+. This switch utilized both the new Broadcom Trident 2 chip and a complementary chip developed by Cisco, which executed the ACI functionalities. This approach apparently allowed for a quicker product release and reduced the switch's price to a level similar to models based solely on Trident 2. This strategy sufficed for the first two to three years of ACI deliveries. During this time, Cisco developed and launched the next generation of Nexus 9000, now utilizing their own chips with increased performance and feature sets, yet maintaining the same price level. The external specifications for interaction within the fabric remained entirely intact. Meanwhile, the internal components were completely overhauled: something akin to refactoring, but for hardware.<\/p>\n

How the Cisco ACI Architecture Works <\/h3>\n

\nIn its simplest form, ACI is built on a Clos network topology, or, as it is often referred to, a Spine-Leaf topology. There can be anywhere from two (or one if fault tolerance is not a concern) to six Spine-level switches. Accordingly, the more switches there are, the higher the fault tolerance (less reduction in bandwidth and reliability in the event of a failure or maintenance of a single Spine) and overall performance. All external connections are routed through the Leaf-level switches: including servers, connections with external networks over L2 or L3, and connections to APIC controllers. Generally, both configuration and statistics collection, failure monitoring, and other tasks in ACI are managed through the controllers' interface, of which there are typically three in standard deployments. <\/p>\n

There is no need to connect to the switches via console, even to launch the network: the controller automatically discovers the switches and assembles the fabric, including the configuration of all service protocols. This is why it is crucial to record the serial numbers of the installed equipment during installation, so later there is no uncertainty about which switch is in which rack. For troubleshooting, it is possible to connect to the switches via SSH if necessary: they can adequately reproduce the standard Cisco show commands. <\/p>\n

Inside, the fabric uses IP transport, so there is no Spanning Tree and other horrors of the past: all links are utilized, and convergence during failures is very quick. Traffic in the fabric is transmitted through tunnels based on VXLAN. Specifically, Cisco refers to the encapsulation as iVXLAN, which differs from regular VXLAN in that the reserved fields in the network header are used to transmit control information, primarily regarding the traffic's relation to the EPG group. This allows for the implementation of interaction rules between groups in hardware, using their numbers just as addresses are used in regular access lists. <\/p>\n

Tunnels allow the stretching of both L2 segments and L3 (that is, VRF) over the internal IP transport. In this case, the default gateway is distributed. This means that each switch handles the routing of incoming traffic to the fabric. In terms of traffic delivery logic, ACI resembles a VXLAN\/EVPN-based fabric. <\/p>\n

If so, what are the differences? In every other respect! <\/h3>\n

\nThe first difference you encounter in ACI is how servers are connected to the network. In traditional networks, both physical servers and virtual machines connect to VLANs, and everything else \u2014 connectivity, security, etc. \u2014 operates around them. In ACI, however, there is a construct that Cisco calls an EPG (Endpoint Group), which is unavoidable. Can it be equated to a VLAN? Yes, but in that case, there's a chance to lose a significant part of what ACI offers. <\/p>\n

All access rules are formulated with respect to the EPG, and ACI uses a default \u2018whitelist\u2019 principle, meaning that only traffic explicitly permitted is allowed. This means we can create EPG groups like \u2018Web\u2019 and \u2018MySQL\u2019 and define a rule allowing interaction between them only on port 3306. This will work without being tied to network addresses and even within the same subnet! <\/p>\n

We have customers who chose ACI specifically because of this feature, as it allows restricting access between servers (whether virtual or physical \u2014 it doesn't matter), without moving them between subnets, thereby not touching the addressing. Yes, we know, no one manually writes configurations for applications, right? an IP address<\/a> Isn't that so?<\/p>\n

Traffic management rules in ACI are called contracts. In such a contract, one or several groups or levels in a multi-tier application become the service provider (for instance, a database service), while others become the consumer. The contract can either simply pass the traffic or do something more sophisticated, like directing it to a firewall or load balancer, and also change the QoS value.<\/p>\n

How do servers end up in these groups? If they are physical servers or something integrated into an existing network, for which we have created a VLAN trunk, then to place them in an EPG, it is necessary to specify the switch port and the VLAN used on it. As we can see, VLANs appear where they are indispensable.<\/p>\n

If the servers are virtual machines, then it is enough to refer to the connected virtualization environment, and everything else will happen automatically: a port group (to use VMWare terminology) will be created for connecting VMs, the necessary VLANs or VXLANs will be assigned, and will be configured on the required switch ports, etc. Thus, although ACI is built around a physical network, connecting virtual servers looks much simpler than for physical ones. ACI already has integrated support for VMWare and MS Hyper-V, as well as support for OpenStack and RedHat Virtualization. Recently, built-in support for container platforms has also emerged: Kubernetes, OpenShift, Cloud Foundry, which covers both policy application and monitoring, meaning the network administrator can immediately see which hosts are running which pods and to which groups they belong.<\/p>\n

In addition to inclusion in a particular port group, virtual servers have additional properties: names, attributes, etc., which can be used as criteria for moving them to another group, for example, when renaming a VM or when it gains an additional tag. Cisco refers to these as micro-segmentation groups, although, broadly speaking, the very structure allowing to create multiple security segments in the form of EPGs within the same subnet is also quite micro-segmentation. Well, the vendor knows best. <\/p>\n

The EPGs themselves are purely logical constructs, not tied to specific switches, servers, etc., so you can do things with them and the constructs based on them (applications and tenants) that are difficult to achieve in traditional networks, such as cloning. As a result, it is very easy to create a clone of a production environment to obtain a test environment that is guaranteed to be identical to production. This can be done manually, but it's better (and easier) through the API. <\/p>\n

In general, the control logic in ACI is very different from what you usually encounter.
\nIn traditional networks from the same Cisco: the software interface is primary, while the GUI or CLI is secondary, as they operate through the same API. Therefore, almost everyone who works with ACI eventually starts to navigate the object model used for management and automates something according to their needs. The easiest way to do this is from Python, as there are convenient ready-made tools available for it. <\/p>\n

The promised pitfalls.<\/h3>\n

\nThe main problem is that many things in ACI are done differently. To start working with it properly, you need to relearn. This is especially true for networking operations teams in large enterprises, where engineers have been engaged in 'defining VLANs' based on requests for years. The fact that VLANs are no longer just VLANs and that you don't actually need to create VLANs manually to interconnect new networks in virtualized hosts completely confounds traditional network engineers and makes them cling to familiar approaches. It's worth noting that Cisco has tried to sweeten the pill a bit and added an 'NXOS-like' CLI in the controller, allowing configuration from an interface similar to traditional switches. But still, to start using ACI properly, you'll need to understand how it works. <\/p>\n

From a pricing perspective, the costs of large and medium-scale ACI networks are practically comparable to traditional networks using Cisco equipment, as they utilize the same switches (Nexus 9000 can operate in both ACI and traditional modes and have become the primary \"workhorse\" for new data center projects). However, for data centers using a two-switch setup, the existence of controllers and Spine-Leaf architecture definitely makes a difference. Recently, a Mini ACI fabric was introduced, where two of the three controllers are replaced with virtual machines. This reduces the cost difference, but it still remains. Thus, for the client, the choice is dictated by how interested they are in security features, virtualization integration, a unified management point, and other factors.
\n
Source: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0421 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0432\u043e\u0442 \u044d\u0442\u043e\u0433\u043e \u0432\u043e\u043b\u0448\u0435\u0431\u043d\u043e\u0433\u043e \u043a\u0443\u0441\u043a\u0430 \u0441\u043a\u0440\u0438\u043f\u0442\u0430 Cisco ACI \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0441\u0435\u0442\u044c. \u0421\u0435\u0442\u0435\u0432\u0430\u044f \u0444\u0430\u0431\u0440\u0438\u043a\u0430 \u0434\u043b\u044f \u0426\u041e\u0414\u0430 Cisco ACI c\u0443\u0449\u0435\u0441\u0442\u0432\u0443\u0435\u0442 \u0443\u0436\u0435 \u043f\u044f\u0442\u044c \u043b\u0435\u0442, \u043d\u043e \u043d\u0430 \u0425\u0430\u0431\u0440\u0435 \u043f\u0440\u043e \u043d\u0435\u0451 \u0442\u043e\u043b\u043a\u043e\u043c \u043d\u0438\u0447\u0435\u0433\u043e \u043d\u0435 \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u0430\u043d\u043e, \u0432\u043e\u0442 \u0438 \u0440\u0435\u0448\u0438\u043b \u044d\u0442\u043e \u043d\u0435\u043c\u043d\u043e\u0433\u043e \u0438\u0441\u043f\u0440\u0430\u0432\u0438\u0442\u044c. \u0420\u0430\u0441\u0441\u043a\u0430\u0436\u0443 \u043d\u0430 \u0441\u0432\u043e\u0451\u043c \u043e\u043f\u044b\u0442\u0435, \u0447\u0442\u043e \u044d\u0442\u043e \u0442\u0430\u043a\u043e\u0435, \u043a\u0430\u043a\u0430\u044f \u043e\u0442 \u043d\u0435\u0451 \u043f\u043e\u043b\u044c\u0437\u0430 \u0438 \u0433\u0434\u0435 \u0443 \u043d\u0435\u0451 \u0433\u0440\u0430\u0431\u043b\u0438. \u0427\u0442\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-35291","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Cisco ACI Data Center Fabric \u2014 A Guide for Admins | ProHoster","description":"With the help of this magical piece of script.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/setevaya-fabrika-dlya-tsoda-cisco-aci-v-pomoshh-adminu","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0421\u0435\u0442\u0435\u0432\u0430\u044f \u0444\u0430\u0431\u0440\u0438\u043a\u0430 \u0434\u043b\u044f \u0426\u041e\u0414\u0430 Cisco ACI \u2014 \u0432 \u043f\u043e\u043c\u043e\u0449\u044c \u0430\u0434\u043c\u0438\u043d\u0443 | ProHoster","og:description":"\u0421 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0432\u043e\u0442 \u044d\u0442\u043e\u0433\u043e \u0432\u043e\u043b\u0448\u0435\u0431\u043d\u043e\u0433\u043e \u043a\u0443\u0441\u043a\u0430 \u0441\u043a\u0440\u0438\u043f\u0442\u0430.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/setevaya-fabrika-dlya-tsoda-cisco-aci-v-pomoshh-adminu","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:03:28+00:00","article:modified_time":"2019-10-31T19:03:28+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"35291","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-02-08 20:27:18","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 02:06:24","updated":"2026-02-08 20:27:18","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/35291","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=35291"}],"version-history":[{"count":1,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/35291\/revisions"}],"predecessor-version":[{"id":157819,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/35291\/revisions\/157819"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=35291"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=35291"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=35291"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}