🥇Will Cisco SD-WAN cut the branch it is sitting on, DMVPN?

Since August 2017, when Cisco acquired Viptela, the primary technology offered for building distributed corporate networks has become Cisco SD-WAN. Over the past three years, the SD-WAN technology has undergone numerous changes, both qualitative and quantitative. Its functionalities have significantly expanded, with support for classic routers such as Cisco ISR 1000, ISR 4000, ASR 1000, and the virtual CSR 1000v. At the same time, many Cisco customers and partners continue to wonder – what are the differences between Cisco SD-WAN and the familiar approaches based on such technologies as Cisco DMVPN and Cisco Performance Routing and how important are these differences?

It should be noted that before the arrival of SD-WAN in Cisco's portfolio, DMVPN along with PfR formed a key part of the architecture of Cisco IWAN (Intelligent WAN), which in turn was the predecessor of a full-fledged SD-WAN technology. Despite the general similarities in the tasks being solved and the methods of solving them, IWAN did not achieve the level of automation, flexibility, and scalability necessary for SD-WAN, and over time, the development of IWAN significantly declined. At the same time, the constituent technologies of IWAN have not disappeared, and many customers continue to successfully use them, including on modern equipment. As a result, an interesting situation has arisen – the same Cisco equipment allows the selection of the most suitable technology for building a WAN (classic, DMVPN+PfR, or SD-WAN) according to customers' requirements and expectations.

This article does not intend to delve into all the specifics of Cisco SD-WAN and DMVPN technologies (together or without Performance Routing) — there is an abundance of available documents and materials for that. The main task is to try to assess the key differences between these technologies. However, before moving on to discussing these differences, let’s briefly remind ourselves of the technologies themselves.

What is Cisco DMVPN and why is it needed?

Cisco DMVPN addresses the challenge of dynamic (i.e., scalable) connectivity of remote branch networks to the central office network using various types of communication links, including the Internet (with encrypted channels). Technically, this is achieved by creating a virtualized overlay network of class L3. VPN in point-to-multipoint mode with a logical star topology (Hub-n-Spoke). To accomplish this, DMVPN uses a combination of the following technologies:

IP routing
Multipoint GRE tunnels (mGRE)
Next Hop Resolution Protocol (NHRP)
IPSec Crypto profiles

What are the main advantages of Cisco DMVPN compared to traditional routing using MPLS VPN channels?

To create an inter-branch network, any communication channels can be used—all that can provide IP connectivity between branches, ensuring that traffic will be encrypted (where necessary) and balanced (where possible).
A fully connected topology is automatically formed between the branches. Static tunnels exist between the central and remote branches, while dynamic tunnels are formed on demand (when traffic is present) between the remote branches.
The central and remote branch routers maintain a uniform configuration, with precision down to (the key to connect to is specified, and iroh finds the associated host and establishes an encrypted connection using the QUIC protocol). Direct P2P connections are established whenever possible, but if not, it falls back to using relays, which are also employed for host discovery by keys. You can run your own relay or connect to public relays supported by the community. the interfaces. By using mGRE, there is no need for individual configuration of dozens, hundreds, or even thousands of tunnels. Consequently, there is substantial scalability with proper design.

What is Cisco Performance Routing and what is its purpose?

In a DMVPN inter-branch network, one extremely important question remains unresolved—how to dynamically assess the state of each DMVPN tunnel in order to meet the requirements of traffic critical to our organization and, based on such an assessment, dynamically make decisions on rerouting? The fact is, DMVPN in this regard is not significantly different from traditional routing—the best one can do is set up QoS mechanisms that prioritize outbound traffic but cannot account for the state of the entire path at any given moment.

What should you do if a channel degrades partially rather than completely—how can this be detected and assessed? DMVPN alone cannot handle this. Given that channels connecting branches can pass through entirely different service providers, using completely different technologies, this task becomes quite non-trivial. This is where Cisco Performance Routing comes into play, which by then had already gone through several stages of development.

The task of Cisco Performance Routing (PfR) is to measure the state of paths (tunnels) through which traffic passes based on key metrics that are important for network applications— latency, latency variation (jitter), and packet loss (in percentage). Additionally, the utilized bandwidth can be measured. These measurements happen as close to real-time as possible (to the extent that it is feasible and justified), and the results of these measurements allow the router utilizing PfR to dynamically make decisions about the need to change the routing of specific types of traffic.

Thus, the task of combining DMVPN/PfR can be summarized as follows:

Allow the customer to use any channels for WAN networks
Ensure the highest possible quality for critical applications over these channels

What is Cisco SD-WAN?

Cisco SD-WAN is a technology that employs an SDN approach to create and operate an organization's WAN. This specifically means using so-called controllers (software elements) that provide centralized orchestration and automated configuration of all solution components. Unlike canonical SDN (Clean Slate style), Cisco SD-WAN uses several types of controllers, each performing its own role—this is done intentionally to ensure better scalability and geo-redundancy.

In the case of SD-WAN, the goal of utilizing any type of channels and ensuring the operation of business applications remains, but the requirements for automation, scalability, security, and flexibility of such a network are expanded.

Discussion of differences

If we now begin analyzing the differences between these technologies, they will fall into one of the categories:

Architectural differences – how functions are distributed across different components of the solution, how these components interact, and how this affects the capabilities and flexibility of the technology?
Functional capabilities – what can one technology do that another cannot? And is that really important?

What are the architectural differences and are they significant?

Each of the mentioned technologies has many "moving parts" with not only different roles but also different interaction principles with each other. The scalability, fault tolerance, and overall efficiency of the solution directly depend on how well-thought-out these principles are and the general mechanics of the solution.

Let's take a closer look at various aspects of the architecture:

Data-plane – the part of the solution responsible for transmitting user traffic between the source and the recipient. In DMVPN and SD-WAN, this is generally implemented the same way on routers based on Multipoint GRE tunnels. The difference lies in how the necessary set of parameters for these tunnels is formed:

downward API support (simultaneously with this in DMVPN/PfR – this is an exclusively two-level hierarchy of nodes with a star-shaped or Hub-and-Spoke topology. Static configuration of Hub and static binding of Spoke to Hub are mandatory, as well as interaction via the NHRP protocol for forming data-plane connectivity. Consequently, changes on the Hub are significantly complicated, related, for example, to modifying/connecting new WAN channels or changing parameters of existing ones.
downward API support (simultaneously with this in SD-WAN – this is a fully dynamic model for discovering parameters of established tunnels, relying on the control-plane (OMP protocol) and orchestration-plane (interaction with the vBond controller for controller discovery and NAT traversal tasks). The imposed topologies can be any, including hierarchical ones. Within the established imposed topology of tunnels, flexible configuration of the logical topology is possible in each specific VPN (VRF).

Control-plane – functions for exchanging, filtering, and modifying routing and other information between components of the solution.

downward API support (simultaneously with this in DMVPN/PfR – is conducted only between Hub and Spoke routers. Direct exchange of routing information between Spoke is not possible. Consequently, without an active Hub, neither the control-plane nor the data-plane can function., which imposes additional high availability requirements on the Hub that are not always met.
downward API support (simultaneously with this in SD-WAN – control-plane communication does not occur directly between routers – interactions happen based on the OMP protocol and must go through a separate specialized type of vSmart controller, enabling load balancing, geo-redundancy, and centralized management of signaling traffic. Another feature of the OMP protocol is its significant resilience to loss and independence from the speed of the connection to the controllers (within reasonable limits, of course). This equally allows for the placement of SD-WAN controllers in public or private clouds with Internet access.

Policy-plane – part of the solution responsible for defining, distributing, and applying traffic management policies across the distributed network.

DMVPN – is essentially limited by the quality of service (QoS) policies, which are individually configured on each router via CLI or Prime Infrastructure templates.
DMVPN/PfR – PfR policies are created on the centralized Master Controller (MC) router through CLI and are then automatically distributed to branch MCs. The same policy distribution paths are used as for the data-plane. There is no capability to separate the exchange of policies, routing information, and user data. Policy distribution requires mandatory IP connectivity between the Hub and Spoke. Additionally, the MC function can be combined with a DMVPN router if necessary. The use of Prime Infrastructure templates for centralized policy formation is possible (but not required). A key feature is that the policy is formed globally across the entire network uniformly – individual policies for specific segments are not supported..
SD-WAN The traffic management and quality of service policies are centrally defined through the graphical interface of Cisco vManage, which is also accessible via the Internet if necessary. They are distributed over signaling channels directly or indirectly through vSmart controllers (depending on the type of policy). They are independent of the data-plane connectivity between routers, as they utilize all available traffic transmission paths between the controller and the router.
For different network segments, flexible formation of various policies is possible - the application scope of the policy is defined by numerous unique identifiers provided in the solution - branch number, application type, traffic direction, etc.

Orchestration-plane mechanisms that allow components to dynamically discover each other, configure, and coordinate subsequent interactions.

downward API support (simultaneously with this in DMVPN/PfR The mutual discovery of routers is based on the static configuration of Hub devices and the corresponding configuration of Spoke devices. Dynamic discovery occurs only for Spokes, which report their connection parameters to the Hub device, which, in turn, has been pre-entered into the configuration of the Spoke. Without IP connectivity between Spoke and at least one Hub, it is impossible to form either the data-plane or the control-plane.
downward API support (simultaneously with this in SD-WAN Orchestration of the solution components occurs using the vBond controller, with which every component (routers and vManage/vSmart controllers) must first establish IP connectivity.
Initially, components are unaware of each other's connection parameters - for this, they require the intermediary orchestrator vBond. The general principle is as follows - each component in the initial phase learns (automatically or statically) only the connection parameters to vBond, which then informs the router about the vManage and vSmart controllers (previously discovered), enabling the automatic establishment of all necessary signaling connections.
The next step is that the new router learns about the other routers in the network through OMP exchange with the vSmart controller. This way, the router, initially unaware of the network parameters, can fully automatically discover and connect to the controllers, and then also automatically discover and establish connectivity with other routers. Initially, the connection parameters of all components are unknown and may change during operation.

Management-plane – the part of the solution that provides centralized management and monitoring.

DMVPN/PfR – there is no specialized management-plane solution provided. For basic automation and monitoring, products such as Cisco Prime Infrastructure can be used. Each router can be managed via the command-line interface (CLI). Integrations with external systems via API are not provided.
SD-WAN – all standard interaction and monitoring are performed centrally through the graphical interface of the vManage controller. All capabilities of the solution, without exception, are available for configuration via vManage, as well as through a fully documented REST API library.
All configurations of the SD-WAN network in vManage boil down to two main constructs – creating device templates and formulating policies that define the logic of network operation and traffic processing. In this process, vManage, by translating the policy created by the administrator, automatically selects which changes need to be made and on which individual devices/controllers, significantly increasing the efficiency and scalability of the solution.
The vManage interface provides not only configuration of the Cisco SD-WAN solution but also comprehensive monitoring of the state of all solution components, down to the current status of metrics for individual tunnels and statistics on the usage of various applications based on DPI analysis.
Despite the centralization of interaction, all components (controllers and routers) also feature a fully functional CLI, which is necessary during the deployment phase or in case of emergencies for local diagnostics. In normal operation (when there is a signaling channel between components), the command line on routers is only accessible for diagnostics and is not available for making local changes, ensuring both local security and a single point of changes in such a network – vManage.

Integrated Security – this should concern not only the protection of user data during transmission over open channels but also the overall security of the WAN network based on the chosen technology.

downward API support (simultaneously with this in DMVPN/PfR The possibility of encrypting user data and signaling protocols is provided. Certain router models additionally offer firewall functionalities with traffic inspection, IPS/IDS. There is an option for segmenting branch networks using VRF. Authentication (single-factor) of control protocols is also available.
In this case, the remote router is considered a trusted element of the network by default – meaning that instances of physical compromise of individual devices and the possibility of unauthorized access to them are not assumed and not taken into account; there is no two-factor authentication for the components of the solution, which in the case of a geographically distributed network can pose serious additional risks.
downward API support (simultaneously with this in SD-WAN Similar to DMVPN, there is an option for encrypting user data, but with significantly expanded network security functions and L3/VRF segmentation (MC, IPS/IDS, URL filtering, DNS filtering, AMP/TG, SASE, TLS/SSL proxy, etc.). In this case, the exchange of encryption keys is carried out more efficiently through vSmart controllers (rather than directly), over pre-established signaling channels protected by DTLS/TLS encryption based on security certificates. This, in turn, guarantees the security of such exchanges and provides better scalability of the solution for up to tens of thousands of devices in a single network.
All signaling connections (controller-controller, controller-router) are also secured based on DTLS/TLS. Routers come equipped with security certificates during manufacturing, with the option for replacement/renewal. Two-factor authentication is achieved by requiring both conditions to be met simultaneously for the router/controller to function within the SD-WAN network:
- Active Security Certificate
- Explicit and conscious whitelisting by the administrator for each component of allowed devices.

Functional Differences Between SD-WAN and DMVPN/PfR

When discussing functional differences, it's important to note that many of these are extensions of architectural differences. It's no secret that developers draw from the capabilities they want to achieve when forming the architecture of a solution. Let's examine the most significant differences between the two technologies.

AppQ (Application Quality) – Functions that ensure quality traffic transmission for business applications

The key features of the technologies being discussed are aimed at improving the user experience as much as possible when using business-critical applications in a distributed network. This is especially important when part of the infrastructure is not controlled by IT or does not guarantee successful data transmission.

DMVPN does not provide such mechanisms on its own. The best that can be done in a classic DMVPN network is to classify outbound traffic by applications and prioritize it when transmitting toward the WAN channel. The choice of DMVPN tunnel is based solely on its availability and the performance of routing protocols, without considering the overall state of the path/tunnel and its potential partial degradation concerning key metrics important for network applications – latency, delay variation (jitter), and loss (%). Therefore, directly comparing classic DMVPN with SD-WAN in the context of solving AppQ tasks makes no sense – DMVPN cannot address this issue. However, with the addition of Cisco Performance Routing (PfR) technology into this context, the situation changes and comparison with Cisco SD-WAN becomes more relevant.

Before discussing the differences, let's briefly cover what the technologies have in common. So, both technologies:

feature a mechanism that allows for the dynamic assessment of the status of each established tunnel based on specific metrics – at a minimum, latency, latency variation, and packet loss (%).
utilize a specific set of tools for forming, distributing, and applying traffic management rules (policies) based on the results of measuring the status of key tunnel metrics.
classify application traffic at L3-L4 (DSCP) levels of the OSI model or by L7 application signatures based on integrated DPI mechanisms within the router.
enable significant applications to define acceptable threshold values for metrics, default traffic forwarding rules, and traffic rerouting rules when threshold values are exceeded.
when encapsulating traffic in GRE/IPSec, use an industry-established mechanism to transfer the internal DSCP marking to the external GRE/IPSec packet header, which allows synchronizing QoS policies between the organization and the service provider (if a corresponding SLA is available).

How do the mechanisms for evaluating end-to-end metrics differ between SD-WAN and DMVPN/PfR?

DMVPN/PfR

For assessing standard tunnel state metrics, both active and passive software sensors (Probes) are used. Active sensors rely on user traffic, while passive ones emulate such traffic (in its absence).
Fine-tuning of timers and conditions for detecting degradation is absent – the algorithm is fixed.
Additionally, measurement of bandwidth utilized in the outbound direction is available. This adds DMVPN/PfR additional flexibility in traffic management.
Some PfR mechanisms, when metrics are exceeded, rely on reverse signaling feedback in the form of specific TCA (Threshold Crossing Alert) messages, which must originate from the traffic recipient to the source. This, in turn, implies that the states of the measured channels must be at least sufficient to transmit such TCA messages. While this is usually not a problem, it clearly cannot be guaranteed.

SD-WAN

To comprehensively assess the standard metrics of tunnel status, the BFD protocol is used in echo mode. In this case, special feedback in the form of TCA or similar messages is unnecessary – isolation of failure domains is maintained. There is also no need for user traffic to evaluate the tunnel status.
There is an option for fine-tuning BFD timers to control the speed of activation and the algorithm's sensitivity to degradation in the communication channel, ranging from several seconds to minutes.
At the time of writing this article, each tunnel accommodates only one BFD session. Potentially, this creates lower granularity in tunnel status analysis. In reality, this may only become a limitation when using a WAN connection based on MPLS L2/L3 VPN with an agreed QoS SLA – if the DSCP marking of BFD traffic (after encapsulation in IPSec/GRE) matches the high-priority queue in the service provider's network, it may affect the accuracy and speed of degradation detection for low-priority traffic. However, there is an option to change the default BFD marking to reduce the risk of such situations. Future versions of Cisco SD-WAN are expected to provide more granular BFD settings and the ability to run multiple BFD sessions within a single tunnel with individual DSCP values (for different applications).
BFD also allows assessing the maximum packet size that can be transmitted through a particular tunnel without fragmentation. This enables SD-WAN to dynamically adjust parameters like MTU and TCP MSS Adjust to maximize the effective utilization of available bandwidth on each channel.
In SD-WAN, the option to synchronize QoS with service providers is also available, not only based on L3 DSCP fields but also on L2 CoS values, which can be automatically generated in the branch network by specialized devices, such as IP phones.

How do the capabilities, methods of identification, and application of AppQ policies differ?

DMVPN/PfR Policies:

Defined on the central branch router(s) through the CLI command line or CLI configuration templates. The formation of CLI templates requires preparation and knowledge of policy syntax.
Defined globally without the possibility of individual customization or modification to meet the needs of specific network segments.
Interactive policy formation in the graphical interface is not provided.
Change tracking, inheritance, and the creation of multiple policy versions for quick switching are not included.
Automatically distributed to the routers of remote branches. The same communication channels used for user data transmission are utilized. In the absence of a communication channel between the central and remote branches, policy distribution or modification is not possible.
Applied at each router and, if necessary, modify the output of standard routing protocols, having higher priority.
In cases where all WAN channels of the branch experience significant traffic loss, compensation mechanisms are not provided..

SD-WAN Policies:

Defined in the vManage graphical interface via an interactive template wizard.
Support the creation of multiple policies, copying, inheritance, and real-time switching between policies.
Support customization of policies for different segments (branches) of the network.
Distributed using any available signaling channel between the controller and the router and/or vSmart – not directly dependent on data-plane connectivity between the routers. However, IP connectivity between the router and the controllers is required.
In cases where all available channels of the branch experience significant data loss exceeding acceptable thresholds for critical applications, additional mechanisms to enhance transmission reliability may be utilized:
- FEC (Forward Error Correction) – uses a special redundancy coding algorithm. When transmitting critical traffic over channels with a significant loss percentage, FEC may be automatically activated and allows for the recovery of lost data if necessary. This slightly increases the bandwidth used but significantly enhances reliability.
- Duplication of data streams. In addition to FEC, the policy may provide for automatic duplication of traffic from selected applications in the event of an even more serious level of losses that cannot be compensated for by FEC. In this case, the selected data will be transmitted through all tunnels to the receiving branch with subsequent de-duplication (discarding redundant packet copies). This mechanism significantly increases channel utilization but also considerably enhances transmission reliability.

Cisco SD-WAN capabilities have no direct analogs in DMVPN/PfR.

The architecture of the Cisco SD-WAN solution, in some cases, allows for capabilities that are either extremely difficult to implement within DMVPN/PfR, impractical due to required labor costs, or entirely impossible. Let's consider the most interesting among them:

Traffic Engineering (TE)

TE includes mechanisms that allow traffic to be diverted from the standard path established by routing protocols. TE is often used to ensure high availability of network services, by being able to quickly and/or preemptively shift critical traffic to an alternative (non-overlapping) transmission path, aimed at providing better service quality or faster recovery in the event of a failure on the main path.

The complexity of implementing TE lies in the need to pre-calculate and reserve (check) an alternative path. In MPLS networks of telecommunications operators, this task is addressed using technologies such as MPLS Traffic Engineering with IGP protocol extensions and RSVP protocol. Recently, the technology of Segment Routing has also gained popularity, as it is more optimized for centralized configuration and orchestration. In classical WAN networks, these technologies are generally absent or reduced to the use of hop-by-hop mechanisms like Policy-Based Routing (PBR), which can divert traffic but do so on each router individually—without considering the overall state of the network or the results of PBR in previous or subsequent steps. The outcome of applying these TE options is unpromising—MPLS TE, due to its complexity in configuration and operation, is typically used only in the most critical part of the network (core), while PBR is used on individual routers without the ability to formulate a unified PBR policy across the entire network. This clearly also applies to networks based on DMVPN.

SD-WAN, in this regard, offers a much more elegant solution that is not only easy to configure but also significantly better scalable. This is a result of the architectures used in the control-plane and policy-plane. Implementing the policy-plane in SD-WAN allows for the centralized definition of the TE policy—what traffic is of interest? for which VPNs? through which nodes/tunnels must alternative routes be formed or, conversely, prohibited? In turn, centralizing control-plane management based on vSmart controllers allows modifying routing results without resorting to configurations on individual devices—the routers only see the results of the logic that was formulated in the vManage interface and transmitted for application to the vSmart.

Service-chaining

The formation of service chains is an even more labor-intensive task in classical routing than the previously described Traffic Engineering mechanism. In this case, it is necessary not only to create a special route for a specific network application but also to ensure the capability of routing traffic from the network at certain (or all) nodes of the SD-WAN for processing by a specific application or service (MPLS, Load Balancing, Caching, Traffic Inspection, etc.). It is also essential to be able to monitor the status of these external services to avoid black-holing situations, as well as to have mechanisms that allow such similar external services to be placed in various geo-locations, enabling the network to automatically select the most optimal service node for processing the traffic of each branch. In the case of Cisco SD-WAN, this can be easily achieved by creating a corresponding centralized policy that will

The ability to create geo-distributed processing of selected application types in a specific sequence on specialized (but not related to the SD-WAN network itself) equipment is perhaps the most vivid demonstration of the advantages of Cisco SD-WAN over classical technologies and even some alternative SD-WAN solutions from other manufacturers.

What’s the result?

It is obvious that both DMVPN (with or without Performance Routing) and Cisco SD-WAN ultimately solve very similar tasks in relation to an organization's distributed WAN network. However, the significant architectural and functional differences of Cisco SD-WAN technology elevate the process of solving these tasks to a different quality level. In summary, the following significant differences between SD-WAN and DMVPN/PfR technologies can be noted:

DMVPN/PfR generally uses proven technologies for creating overlay VPN networks and is similar in data-plane aspects to the more modern SD-WAN technology; however, it has a number of limitations such as mandatory static router configurations and a restricted choice of topologies limited to Hub-and-Spoke. On the other hand, DMVPN/PfR has some functional capabilities that are not yet available within SD-WAN (specifically regarding per-application BFD).
In terms of control-plane technology, they differ fundamentally. With centralized processing of signaling protocols, SD-WAN allows, among other things, to significantly narrow down failure domains and 'decouple' the process of user traffic transmission from signaling interactions – temporary unavailability of controllers does not impact the ability to transmit user traffic. At the same time, temporary unavailability of any branch (including the central one) does not affect the ability of the other branches to interact with each other and with the controllers.
The architecture for formulating and applying traffic management policies in SD-WAN also surpasses that of DMVPN/PfR – geo-redundancy is implemented much better, there is no binding to a Hub, and there are more options for fine-tuning policies, with a significantly larger list of implemented traffic management scenarios.
The orchestration process of the solution also differs significantly. DMVPN assumes the presence of predefined parameters that must somehow be reflected in the configuration, which somewhat restricts the flexibility of the solution and the opportunity for dynamic changes. In turn, SD-WAN operates under the paradigm that at the initial time of connection, the router 'knows nothing' about its controllers, but knows 'who to ask' – this is sufficient not only for the automatic establishment of connections with the controllers but also for automatically forming a fully connected data-plane topology, which can then be flexibly configured/altered using policies.
In terms of centralized management, automation, and monitoring, SD-WAN clearly surpasses the capabilities of DMVPN/PfR, which are a result of the evolution of traditional technologies and mainly rely on the CLI and the use of NMS systems based on templates.
In SD-WAN, compared to DMVPN, security requirements have reached a different level of quality. The main principles are zero trust, scalability, and two-factor authentication.

From these simple conclusions, one might mistakenly think that creating a network based on DMVPN/PfR has lost all relevance today. This is not entirely true. For example, in cases where a network utilizes a lot of outdated equipment and there is no opportunity to replace it, DMVPN can allow the integration of 'old' and 'new' devices into a single geo-distributed network with many of the aforementioned advantages.

On the other hand, it is important to remember that all current Cisco enterprise routers based on IOS XE (ISR 1000, ISR 4000, ASR 1000, CSR 1000v) today support any mode of operation – both classic routing and DMVPN and SD-WAN – the choice depends on current needs and the understanding that at any moment the same equipment can begin to transition to more advanced technology.

Source: habr.com