Well, okay, about “fell in love” is an exaggeration. Rather "could coexist with".
As you all know, since April 16, 2018, Roskomnadzor has been blocking access to resources on the network with extremely broad strokes, adding to the Unified Registry of domain names, pointers to pages of sites on the Internet and network addresses that allow you to identify sites on the Internet, containing information, the dissemination of which is prohibited in the Russian Federation” (in the text - just a register) /10 sometimes. As a result, the citizens of the Russian Federation and businesses suffer, having lost access to the absolutely legal resources they need.
After I said in the comments to one of the articles on Habré that I was ready to help the victims with setting up a bypass scheme, several people contacted me asking for such help. When everything worked for them, one of them recommended describing the technique in an article. On reflection, I decided to break my silence on the site and try for once to write something intermediate between a project and a post on Facebook, i.e. habrapost. The result is in front of you.
Disclaimer
Since it is not very legal to publish ways to bypass blocking access to information prohibited on the territory of the Russian Federation, the purpose of this article will be to talk about a method that allows you to automate access to resources that are allowed on the territory of the Russian Federation, but due to someone's actions inaccessible directly through your provider. And access to other resources, obtained as a result of actions from the article, is an unfortunate side effect and is by no means the purpose of the article.
Also, since I am primarily a network architect by profession, vocation and life path, programming and Linux are not my strengths. Therefore, of course, scripts can be written better, security issues in VPS can be worked out more deeply, etc. Your suggestions will be accepted with gratitude, if they are detailed enough - I will be happy to add them to the text of the article.
TL; DR
We automate access to resources through your existing tunnel using a copy of the registry and the BGP protocol. The goal is to remove all traffic addressed to blocked resources into the tunnel. Minimum explanation, mostly step by step instructions.
What do you need for this
Unfortunately, this post is not for everyone. In order to use this technique, you will need to put together a few elements:
- You must have a linux server somewhere outside of the blocking field. Or at least the desire to start such a server - since it now costs from $ 9 / year, and possibly less. The method is also suitable if you have a separate VPN tunnel, then the server can be located inside the block field.
- Your router must be smart enough to be able to
- any VPN client you like (I prefer OpenVPN, but it can be PPTP, L2TP, GRE+IPSec, and any other option that creates a tunnel interface);
- BGPv4 protocol. Which means that for SOHO it can be Mikrotik or any router with OpenWRT/LEDE/similar custom firmware that allows you to install Quagga or Bird. The use of a PC router is also not prohibited. For an enterprise, see the documentation for your border router for BGP support.
- You should be familiar with Linux usage and network technologies, including BGP. Or at least want to get that idea. Since I am not ready to embrace the immensity this time, you will have to study some points that are incomprehensible to you on your own. However, I will, of course, answer specific questions in the comments and I’m unlikely to be the only one answering, so feel free to ask.
What is used in the example
- Copy of the register
- Routing service -
- Router -
- Working folders - since we are working as root, most of everything will be placed in the root home folder. Respectively:
- /root/blacklist - working folder with compilation script
- /root/zi - a copy of the registry from github
- /etc/bird - standard bird service settings folder
- We accept 194.165.22.146, ASN 64998 as the external IP address of the VPS with the routing server and the tunnel termination point; external IP address of the router - 81.177.103.94, ASN 64999
- The IP addresses inside the tunnel are 172.30.1.1 and 172.30.1.2, respectively.
Of course, you can use any other routers, operating systems and software products, adjusting the solution to fit their logic.
Briefly - the logic of the solution
- Preparatory Actions
- Getting a VPS
- We raise the tunnel from the router to the VPS
- Obtaining and regularly updating a copy of the registry
- Installing and configuring the routing service
- Create a list of static routes for the routing service based on the registry
- We connect the router to the service and set up sending all traffic through the tunnel.
The actual decision
Preparatory Actions
In the vastness of the network there are many services that provide VPS for extremely reasonable money. So far, I have found and use the option for $9/year, but even if you don’t really bother, there are a lot of options for 1E/month on every corner. The question of choosing a VPS lies far beyond the scope of this article, so if something is not clear to someone about this, ask in the comments.
If you use VPS not only for the routing service, but also for terminating a tunnel on it, you need to raise this tunnel and, almost unequivocally, configure NAT for it. There are a large number of instructions on the network for these actions, I will not repeat them here. The main requirement for such a tunnel is that it must create a separate interface on your router that supports the tunnel towards the VPS. Most used VPN technologies meet this requirement - for example, OpenVPN in tun mode is fine.
Get a copy of the registry
As Jabrail said, "He who hinders us will help us." Since the RKN is creating a registry of prohibited resources, it would be a sin not to use this registry to solve our problem. We will receive a copy of the registry from github.
We go to your Linux server, fall into the context of root'a (sudo su -) and install git if it's not already installed.
apt install gitGo to your home directory and pull out a copy of the registry.
cd ~ && git clone --depth=1 https://github.com/zapret-info/z-i Set up a cron update (I have it every 20 minutes, but you can choose any interval that interests you). To do this, we run crontab -e and add the following line to it:
*/20 * * * * cd ~/z-i && git pull && git gcWe connect a hook that will create files for the routing service after updating the registry. To do this, we create a file /root/zi/.git/hooks/post-merge with the following content:
#!/usr/bin/env bash
changed_files="$(git diff-tree -r --name-only --no-commit-id ORIG_HEAD HEAD)"
check_run() {
echo "$changed_files" | grep --quiet "$1" && eval "$2"
}
check_run dump.csv "/root/blacklist/makebgp"and don't forget to make it executable
chmod +x /root/z-i/.git/hooks/post-mergeThe makebgp script referenced by the hook will be created later.
Installing and configuring the routing service
Install bird. Unfortunately, the currently published version of bird in the Ubuntu repositories is comparable in freshness to the feces of Archeopteryx, so we need to first add the official PPA of the software developers to the system.
add-apt-repository ppa:cz.nic-labs/bird
apt update
apt install birdAfter that, we immediately disable bird for IPv6 - in this installation we will not need it.
systemctl stop bird6
systemctl disable bird6Below is a minimalistic configuration file for the bird service (/etc/bird/bird.conf), which is quite enough for us (and once again I remind you that no one forbids developing and tuning the idea to suit your own needs)
log syslog all;
router id 172.30.1.1;
protocol kernel {
scan time 60;
import none;
# export all; # Actually insert routes into the kernel routing table
}
protocol device {
scan time 60;
}
protocol direct {
interface "venet*", "tun*"; # Restrict network interfaces it works with
}
protocol static static_bgp {
import all;
include "pfxlist.txt";
#include "iplist.txt";
}
protocol bgp OurRouter {
description "Our Router";
neighbor 81.177.103.94 as 64999;
import none;
export where proto = "static_bgp";
local as 64998;
passive off;
multihop;
}
router id - router identifier, visually looking like an IPv4 address, but it is not. In our case, it can be any 32-bit number in the IPv4 address format, but it's good practice to specify the IPv4 address of your device (in this case, VPS) there.
protocol direct determines which interfaces will work with the routing process. The example gives a couple of examples of names, you can add more. You can also simply delete the line, in which case the server will listen on all available interfaces with an IPv4 address.
protocol static is our magic that loads lists of prefixes and ip addresses (which are, of course, /32 prefixes) from files for later announcement. Where these lists come from will be discussed below. Please note that the loading of ip addresses is commented out by default, the reason for this is the large amount of uploading. For comparison, at the time of writing the article, there are 78 lines in the list of prefixes, and 85898 in the list of ip addresses. I strongly recommend that you start and debug only on the list of prefixes, and decide whether or not to enable ip loading in the future after experimenting with your router. Not every one of them can easily digest 85 thousand entries in the routing table.
protocol bgp actually sets up bgp peering with your router. ip-address is the address of the external interface of the router (or the address of the tunnel interface from the side of the router), 64998 and 64999 are the numbers of autonomous systems. In this case, they can be assigned in the form of any 16-bit numbers, but it is good practice to use AS numbers from the private range defined by RFC6996 - 64512-65534 inclusive (there is a 32-bit ASN format, but in our case this is definitely overkill). The described configuration uses eBGP peering, in which the autonomous system numbers of the routing service and the router must be different.
As you can see, the service needs to know the IP address of the router, so if you have a dynamic or non-routable private (RFC1918) or shared (RFC6598) address, you have no option to raise peering on the external interface, but the service will still work inside the tunnel.
It is also quite transparent that you can provide several different routers with routes from one service - just duplicate the settings for them by copying the protocol bgp section with changing the neighbor's IP address. That is why the example shows the settings for peering outside the tunnel as the most universal. It is not difficult to remove them into the tunnel by changing the IP addresses in the settings accordingly.
Registry Processing for the Routing Service
Now we need, in fact, to create lists of prefixes and ip-addresses, which are mentioned in the previous step in protocol static. To do this, we take the registry file and make the files we need out of it with the following script, located in /root/blacklist/makebgp
#!/bin/bash
cut -d";" -f1 /root/z-i/dump.csv| tr '|' 'n' | tr -d ' ' > /root/blacklist/tmpaddr.txt
cat /root/blacklist/tmpaddr.txt | grep / | sed 's_.*_route & reject;_' > /etc/bird/pfxlist.txt
cat /root/blacklist/tmpaddr.txt | sort | uniq | grep -Eo "([0-9]{1,3}[.]){3}[0-9]{1,3}" | sed 's_.*_route &/32 reject;_' > /etc/bird/iplist.txt
/etc/init.d/bird reload
logger 'bgp list compiled'Don't forget to make it executable
chmod +x /root/blacklist/makebgpNow you can run it manually and observe the appearance of files in /etc/bird.
Most likely, at this moment bird does not work for you, because at the previous stage you suggested that it search for files that did not exist yet. Therefore, we launch it and control that it starts:
systemctl start bird
birdc show routeThe output of the second command should show about 80 entries (this is at the moment, and when you set it up, everything will depend on the zealousness of the ILV in blocking networks) like this:
54.160.0.0/12 unreachable [static_bgp 2018-04-19] * (200)Team
birdc show protocolwill show the status of the protocols within the service. Until you configure the router (see the next paragraph), the OurRouter protocol will be in the start state (Connect or Active phases), and after a successful connection, it will go to the up state (Established phase). For example, on my system, the output of this command looks like this:
BIRD 1.6.3 ready.
name proto table state since info
kernel1 Kernel master up 2018-04-19
device1 Device master up 2018-04-19
static_bgp Static master up 2018-04-19
direct1 Direct master up 2018-04-19
RXXXXXx1 BGP master up 13:10:22 Established
RXXXXXx2 BGP master up 2018-04-24 Established
RXXXXXx3 BGP master start 2018-04-22 Connect Socket: Connection timed out
RXXXXXx4 BGP master up 2018-04-24 Established
RXXXXXx5 BGP master start 2018-04-24 Passive
Router connection
Everyone is probably already tired of reading this footcloth, but take heart - the end is near. Moreover, in this section I will not be able to give step-by-step instructions - it will be different for each manufacturer.
However, I can show you a couple of examples. The main logic is to raise BGP peering and attach nexthop to all received prefixes, pointing to our tunnel (if you need to output traffic through the p2p interface) or nexthop ip-address if the traffic goes to ethernet).
For example, on Mikrotik in RouterOS, this is solved as follows
/routing bgp instance set default as=64999 ignore-as-path-len=yes router-id=172.30.1.2
/routing bgp peer add in-filter=dynamic-in multihop=yes name=VPS remote-address=194.165.22.146 remote-as=64998 ttl=default
/routing filter add action=accept chain=dynamic-in protocol=bgp comment="Set nexthop" set-in-nexthop=172.30.1.1and in Cisco IOS - like this
router bgp 64999
neighbor 194.165.22.146 remote-as 64998
neighbor 194.165.22.146 route-map BGP_NEXT_HOP in
neighbor 194.165.22.146 ebgp-multihop 250
!
route-map BGP_NEXT_HOP permit 10
set ip next-hop 172.30.1.1In the event that the same tunnel is used both for BGP peering and for transmitting useful traffic, it is not necessary to set nexthop, it will be set correctly by means of the protocol. But if you set it manually, it won't get any worse either.
On other platforms, you will have to figure out the configuration yourself, but if you have any difficulties, write in the comments, I will try to help.
After your BGP session has risen, routes to large networks have arrived and are installed in the table, traffic to the addresses from them has gone and happiness is close, you can return to the bird service and try to uncomment the entry there that connects the list of ip addresses, execute after that
systemctl reload birdand see how your router transferred these 85 thousand routes. Get ready to turn it off and think about what to do with it 🙂
Total
Purely theoretically, after performing the above steps, you have a service that automatically redirects traffic to IP addresses banned in the Russian Federation past the filtering system.
It can, of course, be improved upon. For example, it is easy enough to sum up a list of ip addresses through perl or python solutions. A simple perl script doing this with Net::CIDR::Lite turns 85 thousand prefixes into 60 (not thousand), but naturally covers a much larger range of addresses than is blocked.
Since the service operates at the third level of the ISO / OSI model, it will not save you from site / page blocking if it does not resolve to the address that is recorded in the registry. But along with the registry from github, the nxdomain.txt file arrives, which with a few strokes of the script easily turns into a source of addresses for, for example, the SwitchyOmega plugin in Chrome.
It should also be mentioned that the solution requires additional sharpening if you are not just an Internet user, but also publish some resources from yourself (for example, a website or mail server runs on this connection). By means of the router, you need to hard-bind outgoing traffic from this service to your public address, otherwise you will lose connectivity with those resources that are covered by the list of prefixes received by the router.
If you have any questions - ask, ready to answer.
UPD. Thank you и for options for git to reduce download volumes.
UPD2. Colleagues, it seems that I made a mistake by not adding instructions for setting up a tunnel between the VPS and the router to the article. A lot of questions are caused by this.
Just in case, I note again - it is assumed that before starting the steps in this guide, you have already configured the VPN tunnel in the direction you need and checked its performance (for example, wrapping traffic there by default or static). If you have not completed this phase yet, it does not make much sense to follow the steps from the article. So far, I don’t have my own text on this subject, but if you google “OpenVPN server setup” along with the name of the operating system installed on the VPS, and “OpenVPN client setup” with the name of your router, most likely you will find a number of articles on this subject , including on Habré.
UPD3. wrote a code that makes the resulting file for bird from dump.csv with optional summation of ip addresses. Therefore, the "Registry processing for the routing service" section can be replaced with a call to its program.
UPD4. A little work on the errors (did not contribute in the text):
1) instead systemctl reload bird it makes sense to use the command birdc configure.
2) in the Mikrotik router, instead of changing the next-hop to the IP of the second side of the tunnel /routing filter add action=accept chain=dynamic-in protocol=bgp comment="Set nexthop" set-in-nexthop=172.30.1.1 it makes sense to specify the route directly to the tunnel interface, without the address /routing filter add action=accept chain=dynamic-in protocol=bgp comment="Set nexthop" set-in-nexthop-direct=<interface name>
UPD5. A new service has arrived , from where you can take ready-made lists of ip-addresses. Updated every half an hour. On the client side, all that remains is to frame the entries with the corresponding “route ... reject”.
And that's probably enough to shag my grandmother and update the article.
UPD6. A revised version of the article for those who do not want to understand, but want to start - .
Source: habr.com