This article is the second in a series titled "How to Take Control of Your Network Infrastructure." You can find the content of all articles in the series and links here. .
Our goal at this stage is to bring order to the documentation and configuration.
By the end of this process, you should have the necessary documentation and a network configured according to it.
At this point, we won't discuss security audits – that will be covered in the third part.
The complexity of the task at this stage varies greatly from company to company.
The ideal situation is when
- your network was built according to the design and you have a complete set of documents
- your company has implemented for the network
- according to this process, you have documents (including all necessary diagrams) that provide complete information about the current state of affairs
In this case, your task is quite simple. You need to review the documents and look through all the changes that have been made.
In the worst-case scenario, you will have
- a network created without a design, without a plan, without approvals, by engineers lacking sufficient qualification,
- with chaotic, undocumented changes, full of 'junk' and suboptimal solutions.
It’s clear that your situation is somewhere in between, but unfortunately, on this scale, better or worse likely places you closer to the worse end.
In this case, you will also need the ability to read minds, because you will have to learn to understand what the 'designers' intended, restore their logic, finish what was left undone, and remove the 'junk.'
And of course, you will need to fix their mistakes, change (as minimally as possible at this stage) the design, and modify or recreate the diagrams.
This article does not claim to be exhaustive. Here, I will only describe general principles and touch on some common issues that need to be addressed.
Set of Documents
Let's start with an example.
Below are some documents that are typically created at Cisco Systems during the design process.
CR – Customer Requirements, client requirements (technical assignment).
Created collaboratively with the client, it defines the network requirements.HLD – High Level Design, high-level design based on network requirements (CR). This document explains and justifies the architectural decisions made (topology, protocols, equipment selection, etc.). HLD does not contain design details, such as the interfaces used and IP addresses. Specific equipment configuration is also not discussed here. This document is primarily intended to explain key design concepts to the client's technical management.
LLD – Low Level Design, low-level design based on high-level design (HLD).
It should include all the details necessary for project implementation, such as information on how to connect and configure the equipment. This is a complete guide for implementing the design. This document should provide sufficient information for its execution even by less skilled personnel.Certain elements, such as IP addresses, AS numbers, or physical cabling schemes, can be 'extracted' into separate documents, such as NIP (Network Implementation Plan).
Network construction begins after these documents are created and occurs strictly in accordance with them, and is then verified by the client (tests) for compliance with the design.
Of course, different integrators, different clients, and different countries may have varying requirements for project documentation. However, we would like to avoid formalities and focus on the substance. This stage is not about design but about establishing order, and we need a sufficient set of documents (diagrams, tables, descriptions, etc.) to accomplish our tasks.
In my opinion, there exists an absolute minimum without which effective network control is impossible.
These documents are as follows:
- a diagram (log) of physical cabling
- one or more network diagrams with significant L2/L3 information
Physical cabling diagram
In some small companies, tasks related to equipment installation and physical cabling are the responsibility of network engineers.
In this case, the problem is partially solved by the following approach.
- use the description on the interface to explain what is connected to it
- administratively shut down all unused ports of the networking equipment
This will allow you, even in case of a link issue (when CDP or LLDP is not working on this interface), to quickly identify what is connected to this port.
You will also easily see which ports are occupied and which are free, which is necessary for planning connections of new networking equipment, servers, or workstations.
However, it is clear that if you lose access to the equipment, you will also lose access to this information. Additionally, this way you won’t be able to log such important information as what equipment is there, its power consumption, the number of ports, what rack it is located in, what patch panels are there, and where (in which rack/patch panel) they are connected. Therefore, additional documentation (not just descriptions on the equipment) is very useful.
The ideal option is to use applications designed for working with such information. But you can also get by with simple tables (for example, in Excel) or display the information you consider necessary in L1/L2 diagrams.
Important!
A network engineer can certainly be well-versed in the intricacies and standards of structured cabling systems, types of racks, types of uninterruptible power supplies, what a cold and hot aisle is, and how to make proper grounding, just as they may know the physics of elementary particles or C++. But it must be understood that all of this is not their area of expertise.
Therefore, it is good practice to have either dedicated departments or designated individuals for tasks related to the installation, connection, support of the equipment's functionality, as well as physical patching. Usually, for data centers, this means data center engineers, and for offices — help desk.
If such departments are provided in your company, then the issue of maintaining a log of physical patching is not your responsibility, and you can limit yourself to just the description on the interface and the administrative shutdown of unused ports.
Network diagrams
There is no one-size-fits-all approach to drawing diagrams.
The most important thing is that the diagrams should provide an understanding of how traffic will flow through the logical and physical elements of your network.
By physical elements, we mean
- active equipment
- interfaces/ports of active equipment
By logical, we mean
- logical devices (N7K VDC, Palo Alto VSYS, …)
- VRF
- VLANs
- subinterfaces
- tunnels
- zones
- …
Moreover, if your network is not completely basic, it will consist of different segments.
For example
- data center
- internet
- WAN
- remote access
- office LAN
- DMZ
- …
It would be reasonable to have several diagrams that provide both a general picture (how traffic flows between all these segments) and a detailed explanation of each individual segment.
Since modern networks can have many logical layers, it might be a good (but not mandatory) approach to create different diagrams for different layers. For example, in the case of an overlay approach, these could be the following diagrams:
- overlay
- L1/L2 underlay
- L3 underlay
Of course, the most important diagram, without which it's impossible to understand the idea of your design, is the routing diagram.
Routing diagram
At a minimum, this diagram should reflect
- which routing protocols are used and where
- basic information on routing protocol configuration (area/AS number/router-id/…)
- on which devices redistribution occurs
- where route filtering and aggregation take place
- information about the default route
Additionally, an L2 diagram (OSI) is often useful.
L2 diagram (OSI)
This diagram can reflect the following information:
- which VLANs
- which ports are trunk ports
- which ports are aggregated in ether-channel (port channel), virtual port channel
- which STP protocols are used and on which devices
- main STP settings: root/root backup, STP cost, port priority
- additional STP settings: BPDU guard/filter, root guard…
Common mistakes in design
An example of a poor approach to building a network.
Let's take a simple example of building a basic office local area network.
Based on my experience teaching telecom to students, I can say that virtually any student by the middle of the second semester possesses the required knowledge (within the course I taught) to configure a simple office LAN.
What is so difficult about connecting switches to each other, configuring VLANs, SVI interfaces (in the case of L3 switches), and writing static routing?
Everything will work.
But there are still sidelined issues related to
- Mutual TLS, access control, etc.
- redundancy
- network scalability
- performance
- throughput
- Retrying requests, timeouts, canary approaches (traffic splitting/redirection), etc.
- …
Sometimes I hear the assertion that an office LAN is something very simple, and I usually hear this from engineers (and managers) who deal with anything but networks, and they say this with such confidence that you wouldn't be surprised if the LAN is set up by people with insufficient practice and knowledge, making them with the kinds of mistakes I will describe below.
Typical level L1 (OSI) design errors
- If you are indeed responsible for structured cabling, one of the most unpleasant legacies you may inherit is careless and poorly thought-out cabling.
Also, I would attribute errors of L1 type to issues related to the resources of the equipment used, such as
- insufficient bandwidth
- insufficient TCAM on the equipment (or its ineffective use)
- insufficient performance (often applies to firewalls)
Typical level L2 (OSI) design errors
Often, when there is a lack of good understanding of how STP works and what potential problems it poses, switches are connected chaotically, with default settings, without additional STP tuning.
As a result, we often have the following
- a large STP network diameter, which can lead to broadcast storms
- STP root will be determined randomly (based on MAC address) and the traffic path will be suboptimal
- ports connected to hosts will not be configured as edge (portfast), which will lead to STP recalculating when end stations are turned on/off
- the network will not be segmented at L1/L2, resulting in issues with any switch (for example, power overload) causing STP topology recalculation and stopping traffic across all VLANs on all switches (including in critical segments for service continuity)
Examples of L3 (OSI) design errors
Several characteristic mistakes made by novice networkers:
- frequent use (or reliance solely on) static routing
- the use of suboptimal routing protocols for this design
- suboptimal logical segmentation of the network
- suboptimal use of address space, preventing route aggregation
- absence of backup routes
- lack of redundancy for the default gateway
- asymmetric routing during route reconstructions (which can be critical in the case of NAT/PAT, stateful firewalls)
- MTU issues
- during route reconstructions, traffic passes through other security zones or even different firewalls, leading to dropped traffic
- poor scalability of the topology
Criteria for evaluating design quality
When we talk about optimality/suboptimality, we must understand from which criteria we can evaluate it. From my perspective, the most significant (but not all) criteria (and their interpretations concerning routing protocols) are:
- scalability
For example, you decided to add another data center. How easily can you do that? - manageability
How easy and safe are operational changes, such as announcing a new network or filtering routes - availability
What percentage of the time does your system provide the required level of service - security
How secure is the transmitted data - price
Changes
The main principle at this stage can be expressed by the formula 'do no harm'.
Therefore, even if you do not fully agree with the design and the chosen implementation (configuration), it is not always advisable to make changes. A reasonable approach is to prioritize all identified issues based on two parameters:
- how easily this problem can be fixed
- how great the risk it poses
First, eliminate anything that currently degrades the level of service below acceptable, such as problems causing packet loss. Then address what is easier and safer to fix in decreasing order of risk severity (from design or configuration issues posing the greatest risks to lesser ones).
Perfectionism at this stage can be harmful. Bring the design to a satisfactory state and synchronize the network configuration accordingly.
Source: habr.com
