With the help of this magical piece of Cisco ACI script, you can quickly set up a network.
The network factory for the Cisco ACI data center has existed for five years, but Habré didn’t really say anything about it, so I decided to fix it a little. I’ll tell you from my own experience what it is, what is the use of it and where it has a rake.
What is it and where did it come from?
By the time ACI (Application Centric Infrastructure) was announced in 2013, competitors were advancing on traditional approaches to data center networks from three sides at once.
On the one hand, "first generation" SDN solutions based on OpenFlow promised to make networks more flexible and cheaper at the same time. The idea was to move the decision making traditionally done by proprietary switch software to a central controller.
This controller would have a single vision of everything that happens and, based on this, would program the hardware of all switches at the level of rules for processing specific flows.
On the other hand, overlay network solutions made it possible to implement the necessary connectivity and security policies without any changes in the physical network at all, building software tunnels between virtualized hosts. The best-known example of this approach was Nicira, which by then had already been acquired by VMWare for $1,26 billion and gave rise to the current VMWare NSX. Some piquancy of the situation was added by the fact that the co-founders of Nicira were the same people who previously stood at the origins of OpenFlow, now saying that in order to build a data center factory .
And finally, switching chips available on the open market (what is called merchant silicon) have reached a stage of maturity where they have become a real threat to traditional switch manufacturers. If earlier each vendor independently developed chips for its switches, then over time, chips from third-party manufacturers, primarily from Broadcom, began to reduce the distance with vendor chips in terms of functions, and surpassed them in terms of price / performance ratio. Therefore, many believed that the days of switches on chips of their own design were numbered.
ACI has become Cisco's "asymmetric response" (more precisely, its Insieme company, founded by its former employees) to all of the above.
What is the difference with OpenFlow?
In terms of distribution of functions, ACI is actually the opposite of OpenFlow.
In the OpenFlow architecture, the controller is responsible for writing detailed rules (flows)
in the hardware of all switches, that is, in a large network, it may be responsible for maintaining and, most importantly, changing tens of millions of records at hundreds of points in the network, so its performance and reliability become a bottleneck in a large implementation.
ACI uses the reverse approach: of course, there is also a controller, but the switches receive high-level declarative policies from it, and the switch itself performs their rendering into details of specific settings in the hardware. The controller can be rebooted or turned off altogether, and nothing bad will happen to the network, except, of course, the lack of control at this moment. Interestingly, there are situations in ACI in which OpenFlow is still used, but locally within the host for Open vSwitch programming.
ACI is built entirely on VXLAN-based overlay transport, but includes the underlying IP transport as part of a single solution. Cisco called this the "integrated overlay" term. As a termination point for overlays in ACI, in most cases, factory switches are used (they do this at link speed). Hosts are not required to know anything about the factory, encapsulation, etc., however, in some cases (for example, to connect OpenStack hosts), VXLAN traffic can be brought to them.
Overlays are used in ACI not only to provide flexible connectivity through the transport network, but also to transfer metainformation (it is used, for example, to apply security policies).
Chips from Broadcom were previously used by Cisco in the Nexus 3000 series switches. In the Nexus 9000 family, specially released to support ACI, a hybrid model was originally implemented, which was called Merchant +. The switch simultaneously used both the new Broadcom Trident 2 chip and a complementary chip developed by Cisco, which implements all the magic of ACI. Apparently, this made it possible to speed up the release of the product and reduce the price tag of the switch to a level close to models simply based on Trident 2. This approach was enough for the first two or three years of ACI deliveries. During this time, Cisco has developed and launched the next generation Nexus 9000 on its own chips with more performance and feature set, but at the same price level. External specifications in terms of interaction in the factory are completely preserved. At the same time, the internal filling has completely changed: something like refactoring, but for hardware.
How the Cisco ACI Architecture Works
In the simplest case, ACI is built on the topology of the Klose network, or, as they often say, Spine-Leaf. Spine-level switches can be from two (or one, if we do not care about fault tolerance) to six. Accordingly, the more of them, the higher the fault tolerance (the lower the bandwidth and reliability reduction in case of an accident or maintenance of one Spine) and the overall performance. All external connections go to leaf-level switches: these are servers, and docking with external networks via L2 or L3, and connecting APIC controllers. In general, with ACI, not only configuration, but also statistics collection, failure monitoring, and so on - everything is done through the interface of controllers, of which there are three in standard-sized implementations.
You never have to connect to the switches with the console, even to start the network: the controller itself detects the switches and assembles a factory from them, including the settings of all service protocols, therefore, by the way, it is very important to write down the serial numbers of the equipment being installed during installation, so that later you do not have to guess which switch is in which rack is located. For troubleshooting, if necessary, you can connect to the switches via SSH: they reproduce the usual Cisco show commands quite carefully.
Internally, the factory uses IP transport, so there is no Spanning Tree and other horrors of the past in it: all links are involved, and convergence in case of failures is very fast. The traffic in the fabric is transmitted through tunnels based on VXLAN. More precisely, Cisco itself calls iVXLAN encapsulation, and it differs from regular VXLAN in that the reserved fields in the network header are used to transmit service information, primarily about the relationship of traffic to the EPG group. This allows you to implement the rules of interaction between groups in the equipment, using their numbers in the same way as addresses are used in ordinary access lists.
Tunnels allow both L2 segments and L3 segments (i.e. VRF) to be stretched through the internal IP transport. In this case, the default gateway is distributed. This means that each switch is responsible for routing the traffic entering the fabric. In terms of traffic flow logic, ACI is similar to a VXLAN/EVPN fabric.
If so, what are the differences? Everything else!
The number one difference you encounter with ACI is how servers are connected to the network. In traditional networks, the inclusion of both physical servers and virtual machines goes to VLANs, and everything else dances from them: connectivity, security, etc. In ACI, a design is used that Cisco calls EPG (End-point Group), from which there is nowhere get away. Whether it is possible to equate it to VLAN? Yes, but in this case there is a chance to lose most of what ACI gives.
With regard to EPG, all access rules are formulated, and in ACI, the “white list” principle is used by default, that is, only traffic is allowed, the passage of which is explicitly allowed. That is, we can create the "Web" and "MySQL" EPG groups and define a rule that allows communication between them only on port 3306. This will work without being tied to network addresses and even within the same subnet!
We have customers who have chosen ACI precisely because of this feature, since it allows you to restrict access between servers (virtual or physical - it doesn’t matter) without dragging them between subnets, which means without touching the addressing. Yes, yes, we know that no one prescribes IP addresses in application configurations by hand, right?
Traffic rules in ACI are called contracts. In such a contract, one or more groups or levels in a multi-tier application become a service provider (say, a database service), others become a consumer. The contract can simply pass traffic, or it can do something more tricky, for example, direct it to a firewall or balancer, and also change the QoS value.
How do servers get into these groups? If these are physical servers or something included in an existing network into which we created a VLAN trunk, then in order to place them in the EPG, you will need to point to the switch port and the VLAN used on it. As you can see, VLANs appear where you cannot do without them.
If the servers are virtual machines, then it is enough to refer to the connected virtualization environment, and then everything will happen by itself: a port group will be created (in terms of VMWare) to connect the VM, the necessary VLANs or VXLANs will be assigned, they will be registered on the necessary switch ports, etc. So, although ACI is built around a physical network, connections for virtual servers look much simpler than for physical ones. ACI already has built-in connectivity with VMWare and MS Hyper-V, as well as support for OpenStack and RedHat Virtualization. From some point on, built-in support for container platforms has also appeared: Kubernetes, OpenShift, Cloud Foundry, while it concerns both the application of policies and monitoring, that is, the network administrator can immediately see which hosts which pods work on and which groups they fall into.
In addition to being included in a particular port group, virtual servers have additional properties: name, attributes, etc., which can be used as criteria for transferring them to another group, say, when a VM is renamed or an additional tag appears in it. Cisco calls this micro-segmentation groups, although, by and large, the design itself with the ability to create many security segments in the form of EPGs on the same subnet is also quite a micro-segmentation. Well, the vendor knows better.
EPGs themselves are purely logical constructs, not tied to specific switches, servers, etc., so you can do things with them and constructs based on them (applications and tenants) that are difficult to do in ordinary networks, such as cloning. As a result, let's say it's very easy to clone a production environment in order to get a test environment that is guaranteed to be identical to the production environment. You can do it manually, but it's better (and easier) through the API.
In general, the control logic in ACI is not at all similar to what you usually meet
in traditional networks from the same Cisco: the software interface is primary, and the GUI or CLI are secondary, since they work through the same API. Therefore, almost everyone involved in ACI, after a while, begins to navigate the object model used for management and automate something to fit their needs. The easiest way to do this is from Python: there are convenient ready-made tools for it.
Promised rake
The main problem is that many things in ACI are done differently. To start working with it normally, you need to retrain. This is especially true for network operations teams in large customers, where engineers have been “prescribing VLANs” for years on request. The fact that now VLANs are no longer VLANs, and you don’t need to create VLANs by hand to lay new networks in virtualized hosts, completely blows the roof off traditional networkers and makes them cling to familiar approaches. It should be noted that Cisco tried to sweeten the pill a little and added an “NXOS-like” CLI to the controller, which allows you to do configuration from an interface similar to traditional switches. But still, in order to start using ACI normally, you have to understand how it works.
In terms of price, on large and medium scales, ACI networks do not actually differ from traditional networks on Cisco equipment, since the same switches are used to build them (Nexus 9000 can work in ACI and in traditional mode and have become now the main "workhorse" for new data center projects). But for data centers of two switches, the presence of controllers and Spine-Leaf architecture, of course, make themselves felt. Recently, a Mini ACI factory has appeared, in which two of the three controllers are replaced by virtual machines. This reduces the difference in cost, but it still remains. So for the customer, the choice is dictated by how much he is interested in security features, integration with virtualization, a single point of control, and so on.
Source: habr.com