ProHoster > Blog > Administration > Migration from OpenVPN to WireGuard to consolidate networks into one L2 network

Migration from OpenVPN to WireGuard to consolidate networks into one L2 network

Migration from OpenVPN to WireGuard to consolidate networks into one L2 network

I would like to share my experience of combining networks in three geographically distant apartments, each of which uses routers with OpenWRT as a gateway, into one common network. When choosing a method for combining networks between L3 with subnet routing and L2 with bridging, when all network nodes will be in the same subnet, preference was given to the second method, which is more difficult to configure, but provides more opportunities, since transparent use of technologies was planned in the created network Wake-on-Lan and DLNA.

Part 1: Background

OpenVPN was initially chosen as the protocol for implementing this task, since, firstly, it can create a tap device that can be added to the bridge without any problems, and secondly, OpenVPN supports operation over the TCP protocol, which was also important, because none of the apartments had a dedicated IP address, and I was unable to use STUN, because for some reason my ISP blocks incoming UDP connections from their networks, while the TCP protocol allowed me to forward the VPN server port on rented VPS using SSH. Yes, this approach gives a big load, since the data is encrypted twice, but I did not want to introduce the VPS into my private network, since there was still a risk of third parties gaining control over it, therefore, having such a device on the home network was extremely undesirable and it was decided pay for security with a big overhead.

To forward the port on the router on which it was planned to deploy the server, the sshtunnel program was used. I will not describe the intricacies of its configuration - this is done quite easily, I just note that its task was to forward TCP port 1194 from the router to the VPS. Next, the OpenVPN server was configured on the tap0 device, which was connected to the br-lan bridge. After checking the connection to the newly created server from the laptop, it became clear that the idea of ​​port forwarding justified itself and my laptop became a member of the router's network, although it was not physically in it.

The matter remained small: it was necessary to distribute IP addresses in different apartments so that they did not conflict and configure routers as OpenVPN clients.
The following router IP addresses and DHCP server ranges were selected:

  • 192.168.10.1 with range 192.168.10.2 — 192.168.10.80 for the server
  • 192.168.10.100 with range 192.168.10.101 — 192.168.10.149 for a router in apartment No. 2
  • 192.168.10.150 with range 192.168.10.151 — 192.168.10.199 for a router in apartment No. 3

It was also necessary to assign exactly these addresses to the client routers of the OpenVPN server by adding the line to its configuration:

ifconfig-pool-persist /etc/openvpn/ipp.txt 0

and adding the following lines to the /etc/openvpn/ipp.txt file:

flat1_id 192.168.10.100
flat2_id 192.168.10.150

where flat1_id and flat2_id are the device names specified when generating certificates for connecting to OpenVPN

Next, OpenVPN clients were configured on the routers, the tap0 devices on both were added to the br-lan bridge. At this stage, everything seemed to be in order, since all three networks see each other and work as a whole. However, a not very pleasant detail turned out: sometimes devices could get an IP address not from their router, with all the ensuing consequences. For some reason, the router in one of the apartments did not have time to respond to DHCPDISCOVER in time and the device received the wrong address. I realized that I need to filter such requests in tap0 on each of the routers, but as it turned out, iptables cannot work with a device if it is part of a bridge and ebtables should come to my rescue. To my regret, it was not in my firmware and I had to rebuild the images for each device. By doing this and adding these lines to /etc/rc.local of each router, the problem was solved:

ebtables -A INPUT --in-interface tap0 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
ebtables -A INPUT --in-interface tap0 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
ebtables -A FORWARD --out-interface tap0 --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
ebtables -A FORWARD --out-interface tap0 --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP

This configuration lasted for three years.

Part 2: Introducing WireGuard

Recently, the Internet has been increasingly talking about WireGuard, admiring the simplicity of its configuration, high transfer speed, low ping with comparable security. Searching for more information about it made it clear that neither work as a bridge member nor work on the TCP protocol is supported by it, which made me think that there are still no alternatives to OpenVPN for me. So I put off getting to know WireGuard.

A few days ago, the news spread through resources one way or another related to IT that WireGuard will finally be included in the Linux kernel, starting with version 5.6. News articles, as always, praised WireGuard. I again plunged into the search for ways to replace the good old OpenVPN. This time I ran into this article. It talked about creating an Ethernet tunnel over L3 using GRE. This article gave me hope. It remained unclear what to do with the UDP protocol. Searching led me to articles about using socat in conjunction with an SSH tunnel to forward a UDP port, however, they noted that this approach only works in single connection mode, which means that multiple VPN clients would be impossible. I came up with the idea to set up a VPN server on a VPS, and configure GRE for clients, but as it turned out, GRE does not support encryption, which will lead to the fact that if third parties gain access to the server, all traffic between my networks is in their hands which didn't suit me at all.

Again, the decision was made in favor of redundant encryption, by using VPN over VPN according to the following scheme:

Layer XNUMX VPN:
VPS is server with internal address 192.168.30.1
MS is client VPS with internal address 192.168.30.2
MK2 is client VPS with internal address 192.168.30.3
MK3 is client VPS with internal address 192.168.30.4

Layer XNUMX VPN:
MS is server with external address 192.168.30.2 and internal 192.168.31.1
MK2 is client MS with the address 192.168.30.2 and has an internal IP of 192.168.31.2
MK3 is client MS with the address 192.168.30.2 and has an internal IP of 192.168.31.3

* MS - router-server in apartment 1, MK2 - router in apartment 2, MK3 - router in apartment 3
* Device configurations are published in the spoiler at the end of the article.

And so, pings between the nodes of the network 192.168.31.0/24 go, it's time to move on to setting up the GRE tunnel. Before that, in order not to lose access to routers, it is worth setting up SSH tunnels to forward port 22 to the VPS, so that, for example, a router from apartment 10022 will be available on port 2 of the VPS, and a router from apartment 11122 will be available on port 3 of the VPS. router from apartment XNUMX. It is best to configure the forwarding with the same sshtunnel, since it will restore the tunnel in case it falls.

The tunnel is configured, you can connect to SSH through the forwarded port:

ssh root@МОЙ_VPS -p 10022

Next, disable OpenVPN:

/etc/init.d/openvpn stop

Now let's set up a GRE tunnel on the router from apartment 2:

ip link add grelan0 type gretap remote 192.168.31.1 local 192.168.31.2
ip link set grelan0 up

And add the created interface to the bridge:

brctl addif br-lan grelan0

Let's perform a similar procedure on the server router:

ip link add grelan0 type gretap remote 192.168.31.2 local 192.168.31.1
ip link set grelan0 up

And, also, add the created interface to the bridge:

brctl addif br-lan grelan0

starting from this moment, pings begin to successfully go to the new network and I, with satisfaction, go to drink coffee. Then, to see how the network on the other end of the wire works, I try to SSH into one of the computers in apartment 2, but the ssh client freezes without prompting me for a password. I try to connect to this computer via telnet on port 22 and see a line from which you can understand that the connection is being established, the SSH server is responding, but for some reason it does not offer me to enter.

$ telnet 192.168.10.110 22
SSH-2.0-OpenSSH_8.1

I'm trying to connect to it via VNC and I see a black screen. I convince myself that the matter is in the remote computer, because I can easily connect to the router from this apartment using the internal address. However, I decide to SSH into this computer through the router and am surprised to find that the connection succeeds and the remote computer works fine but fails to connect to my computer either.

I take the grelan0 device out of the bridge and start OpenVPN on the router in apartment 2 and make sure the network is working properly again and connections are not dropping. Searching I come across forums where people complain about the same problems, where they are advised to raise the MTU. No sooner said than done. However, until the MTU was set large enough to 7000 for gretap devices, either dropped TCP connections or slow transmissions were observed. Due to the high MTU for gretap, the MTUs for WireGuard connections of the first and second levels were set to 8000 and 7500, respectively.

I did a similar setup on the router from apartment 3, with the only difference being that a second gretap interface named grelan1 was added to the server router, which was also added to the br-lan bridge.

Everything is working. Now you can put the gretap assembly into autoload. For this:

Placed these lines in /etc/rc.local on the router in apartment 2:

ip link add grelan0 type gretap remote 192.168.31.1 local 192.168.31.2
ip link set dev grelan0 mtu 7000
ip link set grelan0 up
brctl addif br-lan grelan0

Added this to /etc/rc.local on the router in apartment 3:

ip link add grelan0 type gretap remote 192.168.31.1 local 192.168.31.3
ip link set dev grelan0 mtu 7000
ip link set grelan0 up
brctl addif br-lan grelan0

And on the server router:

ip link add grelan0 type gretap remote 192.168.31.2 local 192.168.31.1
ip link set dev grelan0 mtu 7000
ip link set grelan0 up
brctl addif br-lan grelan0

ip link add grelan1 type gretap remote 192.168.31.3 local 192.168.31.1
ip link set dev grelan1 mtu 7000
ip link set grelan1 up
brctl addif br-lan grelan1

After rebooting the client routers, I found that for some reason they did not connect to the server. Connecting to their SSH (fortunately, I had previously configured sshtunnel for this), it was discovered that WireGuard for some reason creates a route for the endpoint, while being incorrect. So, for 192.168.30.2, the route table was specified in the route table through the pppoe-wan interface, that is, through the Internet, although the route to it should have been directed through the wg0 interface. After deleting this route, the connection was restored. I could not find instructions anywhere on how to force WireGuard not to create these routes. Moreover, I didn’t even understand if this is a feature of OpenWRT, or of WireGuard itself. Without having to deal with this problem for a long time, I simply added to both routers in a script looped by a timer, a line that deleted this route: 

route del 192.168.30.2

Summing up

I have not yet achieved a complete rejection of OpenVPN, since I sometimes need to connect to a new network from a laptop or phone, and setting up a gretap device on them is generally impossible, but despite this, I got an advantage in data transfer speed between apartments and, for example, using VNC is no longer inconvenient. Ping decreased slightly, but became more stable:

When using OpenVPN:

[r0ck3r@desktop ~]$ ping -c 20 192.168.10.110
PING 192.168.10.110 (192.168.10.110) 56(84) bytes of data.
64 bytes from 192.168.10.110: icmp_seq=1 ttl=64 time=133 ms
...
64 bytes from 192.168.10.110: icmp_seq=20 ttl=64 time=125 ms

--- 192.168.10.110 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 19006ms
rtt min/avg/max/mdev = 124.722/126.152/136.907/3.065 ms

When using WireGuard:

[r0ck3r@desktop ~]$ ping -c 20 192.168.10.110
PING 192.168.10.110 (192.168.10.110) 56(84) bytes of data.
64 bytes from 192.168.10.110: icmp_seq=1 ttl=64 time=124 ms
...
64 bytes from 192.168.10.110: icmp_seq=20 ttl=64 time=124 ms
--- 192.168.10.110 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 19003ms
rtt min/avg/max/mdev = 123.954/124.423/126.708/0.675 ms

It is mostly affected by high ping to VPS which is approximately 61.5ms

However, the speed has increased significantly. So, in an apartment with a router-server, I have an Internet connection speed of 30 Mbps, and in other apartments, 5 Mbps. At the same time, while using OpenVPN, I could not achieve a data transfer rate between networks of more than 3,8 Mbps according to iperf, while WireGuard “pumped” it up to the same 5 Mbps.

WireGuard configuration on VPS[Interface] Address = 192.168.30.1/24
ListenPort = 51820
PrivateKey = <ЗАКРЫТЫЙ_КЛЮЧ_ДЛЯ_VPS>

[Peer] PublicKey = <ОТКРЫТЫЙ_КЛЮЧ_VPN_1_МС>
AllowedIPs = 192.168.30.2/32

[Peer] PublicKey = <ОТКРЫТЫЙ_КЛЮЧ_VPN_2_МК2>
AllowedIPs = 192.168.30.3/32

[Peer] PublicKey = <ОТКРЫТЫЙ_КЛЮЧ_VPN_2_МК3>
AllowedIPs = 192.168.30.4/32

WireGuard configuration on MS (added to /etc/config/network)

#VPN первого уровня - клиент
config interface 'wg0'
        option proto 'wireguard'
        list addresses '192.168.30.2/24'
        option private_key 'ЗАКРЫТЫЙ_КЛЮЧ_VPN_1_МС'
        option auto '1'
        option mtu '8000'

config wireguard_wg0
        option public_key 'ОТКРЫТЫЙ_КЛЮЧ_VPN_1_VPS'
        option endpoint_port '51820'
        option route_allowed_ips '1'
        option persistent_keepalive '25'
        list allowed_ips '192.168.30.0/24'
        option endpoint_host 'IP_АДРЕС_VPS'

#VPN второго уровня - сервер
config interface 'wg1'
        option proto 'wireguard'
        option private_key 'ЗАКРЫТЫЙ_КЛЮЧ_VPN_2_МС'
        option listen_port '51821'
        list addresses '192.168.31.1/24'
        option auto '1'
        option mtu '7500'

config wireguard_wg1
        option public_key 'ОТКРЫТЫЙ_КЛЮЧ_VPN_2_МК2'
        list allowed_ips '192.168.31.2'

config wireguard_wg1ip link add grelan0 type gretap remote 192.168.31.1 local 192.168.31.3

        option public_key 'ОТКРЫТЫЙ_КЛЮЧ_VPN_2_МК3'
        list allowed_ips '192.168.31.3'

WireGuard configuration on MK2 (added to /etc/config/network)

#VPN первого уровня - клиент
config interface 'wg0'
        option proto 'wireguard'
        list addresses '192.168.30.3/24'
        option private_key 'ЗАКРЫТЫЙ_КЛЮЧ_VPN_1_МК2'
        option auto '1'
        option mtu '8000'

config wireguard_wg0
        option public_key 'ОТКРЫТЫЙ_КЛЮЧ_VPN_1_VPS'
        option endpoint_port '51820'
        option persistent_keepalive '25'
        list allowed_ips '192.168.30.0/24'
        option endpoint_host 'IP_АДРЕС_VPS'

#VPN второго уровня - клиент
config interface 'wg1'
        option proto 'wireguard'
        option private_key 'ЗАКРЫТЫЙ_КЛЮЧ_VPN_2_МК2'
        list addresses '192.168.31.2/24'
        option auto '1'
        option listen_port '51821'
        option mtu '7500'

config wireguard_wg1
        option public_key 'ОТКРЫТЫЙ_КЛЮЧ_VPN_2_МС'
        option endpoint_host '192.168.30.2'
        option endpoint_port '51821'
        option persistent_keepalive '25'
        list allowed_ips '192.168.31.0/24'

WireGuard configuration on MK3 (added to /etc/config/network)

#VPN первого уровня - клиент
config interface 'wg0'
        option proto 'wireguard'
        list addresses '192.168.30.4/24'
        option private_key 'ЗАКРЫТЫЙ_КЛЮЧ_VPN_1_МК3'
        option auto '1'
        option mtu '8000'

config wireguard_wg0
        option public_key 'ОТКРЫТЫЙ_КЛЮЧ_VPN_1_VPS'
        option endpoint_port '51820'
        option persistent_keepalive '25'
        list allowed_ips '192.168.30.0/24'
        option endpoint_host 'IP_АДРЕС_VPS'

#VPN второго уровня - клиент
config interface 'wg1'
        option proto 'wireguard'
        option private_key 'ЗАКРЫТЫЙ_КЛЮЧ_VPN_2_МК3'
        list addresses '192.168.31.3/24'
        option auto '1'
        option listen_port '51821'
        option mtu '7500'

config wireguard_wg1
        option public_key 'ОТКРЫТЫЙ_КЛЮЧ_VPN_2_МС'
        option endpoint_host '192.168.30.2'
        option endpoint_port '51821'
        option persistent_keepalive '25'
        list allowed_ips '192.168.31.0/24'

In the described configurations for the second level VPN, I specify port 51821 to WireGuard clients. In theory, this is not necessary, since the client will establish a connection from any free non-privileged port, but I made it so that all incoming connections can be denied on the wg0 interfaces of all routers, except incoming UDP connections on port 51821.

I hope that the article will be useful to someone.

PS Also, I want to share my script that sends me a PUSH notification to my phone in the WirePusher application when a new device appears on my network. Here is a link to the script: github.com/r0ck3r/device_discover.

UPDATE: OpenVPN server and clients configuration

OpenVPN server

client-to-client

ca /etc/openvpn/server/ca.crt
cert /etc/openvpn/server/vpn-server.crt
dh /etc/openvpn/server/dh.pem
key /etc/openvpn/server/vpn-server.key

dev tap
ifconfig-pool-persist /etc/openvpn/ipp.txt 0
keepalive 10 60
proto tcp4
server-bridge 192.168.10.1 255.255.255.0 192.168.10.80 192.168.10.254
status /var/log/openvpn-status.log
verb 3
comp-lzo

OpenVPN client

client
tls-client
dev tap
proto tcp
remote VPS_IP 1194 # Change to your router's External IP
resolv-retry infinite
nobind

ca client/ca.crt
cert client/client.crt
key client/client.key
dh client/dh.pem

comp-lzo
persist-tun
persist-key
verb 3

I used easy-rsa to generate certificates.

Source: habr.com

Add a comment