This article is the fourth in a series of articles "How to take control of your network infrastructure." The content of all articles in the cycle and links can be found .
В In this chapter, we looked at some aspects of network security for the Data Center segment. This part will be dedicated to the "Internet Access" segment.
Internet access
The topic of security is undoubtedly one of the most complex topics in the world of data networks. As in previous cases, without claiming depth and completeness, I will consider here quite simple, but, in my opinion, important questions, the answers to which, I hope, will help increase the level of security of your network.
When auditing this segment, pay attention to the following aspects:
- design
- BGP settings
- DOS/DDOS protection
- firewall traffic filtering
Design
As an example of this segment design for an enterprise network, I would recommend from Cisco within .
Of course, you may find other vendors' solution more attractive (see below). ), but without urging you to follow this design in detail, I still find it useful to understand the principles and ideas behind it.
Comment
In SAFE, the "Remote Access" segment is part of "Internet Access". But in this series of articles we will consider it separately.
The standard set of equipment in this segment for the enterprise network are
- border routers
- firewalls
Note 1
In this series of articles, when I talk about firewalls, I mean .
Note 2
I omit consideration of various kinds of L2/L1 or overlay L2 over L3 solutions necessary to provide L1/L2 connectivity and limit myself to questions of the L3 level and above. In part, L1 / L2 issues were considered in the chapter “«.
If you have not found a firewall in this segment, then do not rush to conclusions.
Let's, as in , let's start with the question, is it necessary to use a firewall in this segment in your case?
I can say that this seems to be the most reasonable place to use firewalls and to apply complex traffic filtering algorithms. IN we mentioned 4 factors that can prevent the use of firewalls in the data center segment. But here they are no longer so significant.
Example 1. Delay
As far as the Internet is concerned, it makes no sense to talk about delays even of the order of 1 millisecond. Therefore, the delay in this segment cannot be a factor limiting the use of the firewall.
Example 2. Performance
In some cases, this factor can still be significant. Therefore, you may have to allow some traffic (for example, load balancer traffic) to bypass the firewall.
Example 3. Reliability
This factor still needs to be taken into account, but still, given the unreliability of the Internet itself, its significance for this segment is not as significant as for the data center.
So let's say your service lives on top of http/https (with short sessions). In this case, you can use two independent boxes (without HA) and if there is a problem with one of them, route all traffic to the second one.
Or you can use firewalls in a transperant fashion and, if they fail, allow traffic to bypass the firewalls while solving the problem.
Therefore, it is rather price may be the factor that will make you abandon the use of firewalls in this segment.
Important!
There is a temptation to combine this firewall with the data center firewall (use one firewall for these segments). The solution, in principle, is possible, but it must be understood that because The "Internet Access" firewall actually stands at the forefront of your defense and "takes over" at least part of the malicious traffic, then, of course, you need to take into account the increased risk that this firewall will be disabled. That is, using the same devices in these two segments, you will significantly reduce the availability of your data center segment.
As usual, you need to understand that depending on the service that the company provides, the design of this segment can be very different. You, as usual, can choose different approaches depending on the requirements.
Example
If you are a content provider with a CDN (see, for example, ), then you may not want to create infrastructure in dozens or even hundreds of points of presence using separate devices for routing and filtering traffic. It will be expensive, and it may simply be redundant.
For BGP, you do not need to have dedicated routers at all, you can use open-source tools, for example, . So perhaps all you need is a server or multiple servers, a switch, and BGP.
In this case, your server or several servers can play the role of not only a CDN server, but also a router. Of course, there are still a lot of details (for example, how to ensure balancing), but it is feasible, and we have successfully applied this approach for one of our partners.
You can have several data centers with full protection (firewalls, DDOS protection services provided by your ISPs) and dozens or hundreds of "simplified" points of presence with only L2 switches and servers.
But what about protection in this case?
Let's consider, for example, the recently popular . Its danger lies in the fact that a large amount of traffic is generated, which simply “clogs” all your uplinks by 100%.
What we have in case of our design.
- if you use AnyCast then traffic is distributed between your points of presence. If your total bandwidth is terabits, then this in itself actually (yet there have been several attacks with malicious traffic of the order of terabits lately) protects you from “overflowing” uplinks
- if, nevertheless, some uplinks are “clogged”, then you simply remove this site from maintenance (stop announcing the prefix)
- you can also increase the share of traffic sent from your “full-fledged” (and, accordingly, protected) data centers, thus removing a significant part of malicious traffic from unprotected points of presence
And one more note to this example. If you send enough traffic through IXs, then this also reduces your exposure to such attacks.
Configuring BGP
There are two topics here.
- Connectivity
- Configuring BGP
We have already talked a little about connectivity in . The bottom line is that traffic to your customers goes in the best way. While optimality isn't always just about latency, it's usually low latency that is the main indicator of optimality. For some companies this is more important, for others it is less. It all depends on the service you provide.
Example 1
If you are an exchange, and time intervals of less than milliseconds are important for your clients, then, of course, there can be no talk of any Internet at all.
Example 2
If you are a gaming company, and tens of milliseconds are important to you, then, of course, connectivity is very important to you.
Example 3
You also need to understand that, due to the properties of the TCP protocol, the data transfer rate within one TCP session also depends on RTT (Round Trip Time). CDN networks are built, among other things, to solve this problem by moving the content distribution server closer to the consumer of this content.
Connectivity research is a separate interesting topic worthy of a separate article or a series of articles and requires a good understanding of how the Internet “works”.
Useful resources:
Example
I will give just one small example.
Let's assume that your data center is located in Moscow, and you have a single uplink - Rostelecom (AS12389). In this case, (single homed) BGP is not needed, and you most likely use the address pool from Rostelecom as public addresses.
Suppose you provide a certain service and you have a sufficient number of clients from Ukraine, and they complain about large delays. In research, you found out that the IP addresses of some of them are in the 37.52.0.0/21 grid.
By running a traceroute, you saw that the traffic is going through AS1299 (Telia), and by running a ping, you got an average RTT of 70 - 80 milliseconds. You can see it also on .
Using the whois utility (on the ripe.net site or local utility) you can easily determine that block 37.52.0.0/21 belongs to AS6849 (Ukrtelecom).
Next, by going to you can see that AS6849 has no relationship with AS12389 (they are neither clients nor uplinks to each other, nor do they have peering). But if you look at for AS6849, you will see, for example, AS29226 (Mastertel) and AS31133 (Megafon).
By finding the looking glass of these providers, you can compare path and RTT. For example, for Mastertel RTT will be about 30 milliseconds.
So, if the difference between 80 and 30 milliseconds is significant for your service, then perhaps you need to think about connectivity, get your AS number, your address pool in RIPE and connect additional uplinks and / or create points of presence on IXs.
When using BGP, you not only have the opportunity to improve connectivity, but also reserve your Internet connection.
contains recommendations for configuring BGP. Even though these recommendations have been developed based on the "best practice" of providers, nevertheless (if your BGP settings are not quite elementary) they are undoubtedly useful and in fact should be part of the hardening that we discussed in .
DOS/DDOS protection
Now DOS/DDOS attacks have become a daily reality for many companies. In fact, in one form or another, you are attacked quite often. The fact that you do not notice this yet only means that a targeted attack against you has not yet been organized, and that the means of protection that you use, even, perhaps without suspecting it (various built-in protections of operating systems), sufficient to minimize the degradation of the service provided for you and your customers.
There are Internet resources that, based on equipment logs, draw beautiful attack maps in real time.
you can find links to them.
My lovely from checkpoint.
DDOS/DOS protection is usually layered. To understand why, you need to understand what types of DOS/DDOS attacks exist (see, for example, or )
That is, we have three types of attacks:
- volumetric attacks
- protocol attacks
- application attacks
If you can protect yourself from the last two types of attacks on your own using, for example, firewalls, then you yourself cannot protect yourself from attacks aimed at “overflowing” your uplinks (of course, if your total capacity of Internet channels is not calculated in terabits, but rather, dozens terabit).
Therefore, the first line of defense is protection against "volumetric" attacks and your provider or providers should provide this protection to you. If you haven't figured it out yet, you're just lucky for now.
Example
Suppose you have several uplinks, but only one of the providers can provide you with this protection. But if all traffic goes through one provider, then what about the connectivity that we briefly discussed a little earlier?
For the duration of the attack, you will have to partly sacrifice connectivity in this case. But
- this is only for the duration of the attack. In the event of an attack, you can manually or automatically reconfigure BGP in such a way that traffic only goes through the provider that provides you with an "umbrella". After the end of the attack, you can return the routing to its previous state
- it is not necessary to transfer all traffic. If, for example, you see that there is no attack through some uplinks or peering (or the traffic is not significant), you can continue to advertise prefixes with contention attributes towards these BGP neighbors.
You can also outsource protection against "protocol attacks" and "application attacks" to partners.
Here you can read a good study (). True, the article is two years old, but it will give you an idea of the approaches, how you can protect yourself from DDOS attacks.
In principle, you can limit yourself to this, completely outsourcing your protection. There are advantages to this solution, but there is also an obvious disadvantage. The fact is that it can be (again, depending on what your company does) about the survival of the business. And trust such things to third parties ...
Therefore, let's look at how to organize the second and third lines of defense (as an addition to protection from the provider).
So, the second line of defense is filtering and traffic limiters (policers) at the entrance to your network.
Example 1
Let's assume that you "closed the umbrella" from DDOS with the help of one of the providers. Let's assume that this provider uses Arbor for traffic filtering and filters at the edge of its network.
The bandwidth that Arbor can “process” is limited, and the provider, of course, cannot constantly pass the traffic of all its partners who ordered this service through filtering equipment. Therefore, under normal conditions, traffic is not filtered.
Let's assume that there is a SYN flood attack. Even if you ordered a service that automatically switches traffic to filtering in the event of an attack, this does not happen instantly. For a minute or more, you remain under attack. And this can lead to the failure of your equipment or the degradation of the service. In this case, limiting traffic on the edge routing, although it will cause some TCP sessions to not be established during this time, will save your infrastructure from larger problems.
Example 2
An abnormally high number of SYN packets may not only be the result of a SYN flood attack. Let's assume that you provide a service in which you can have about 100 thousand TCP connections at the same time (to one data center).
Let's say that as a result of a short-term problem with one of your main providers, you "kicked" half of the sessions. If your application is designed in such a way that it immediately (or after some time interval the same for all sessions) tries to reestablish the connection, then you will receive at least 50 SYN packets approximately simultaneously.
If, on top of these sessions, you, for example, have to work ssl / tls handshake, which involves the exchange of certificates, then in terms of resource exhaustion for your load balancer, this will be a much stronger “DDOS” than a simple SYN flood. It would seem that balancers should process such events, but ... unfortunately, we are faced with such a problem.
And, of course, the policer on the border router will save your equipment in this case too.
The third level of protection against DDOS/DOS is your firewall settings.
Here you can stop both attacks of the second and third types. In general, everything that reaches the firewall can be filtered here.
Council
Try to give the firewall as little work as possible by filtering out as much as possible on the first two lines of defense. And that's why.
Has it happened to you that by chance, when generating traffic to check, for example, how resistant the operating system of your servers is to DDOS attacks, you “killed” your firewall, loading it 100 percent, while traffic at the usual intensity? If not, then maybe just because you haven't tried?
In general, the firewall, as I said, is a complex thing, and it works well with known vulnerabilities and tested solutions, but if you send something unusual, just some garbage or packets with incorrect headers, then you are with some, not such a small (based on my experience) probability can lead to a stupor and top-end equipment. Therefore, at stage 2, using ordinary ACLs (at the L3 / L4 level), let only the traffic that should enter there into your network.
Firewall traffic filtering
We continue talking about the firewall. You need to understand that DOS / DDOS attacks are just one of the varieties of cyber attacks.
In addition to DOS/DDOS protection, we can also have something like the following list of features:
- application firewalling
- threat prevention (antivirus, anti-spyware, and vulnerability)
- URL filtering
- data filtering (content filtering)
- file blocking (file types blocking)
You decide what you need from this list.
To be continued
Source: habr.com