Is it possible to combine several Internet channels into one? There are a lot of misconceptions and myths around this topic, even experienced network engineers often do not know that this is possible. In most cases, link aggregation is erroneously referred to as NAT balancing or failover. But real summation allows run one single TCP connection simultaneously over all Internet channels, for example, video broadcasting so that if any of the Internet channels is interrupted, the broadcast will not be interrupted.
There are expensive commercial solutions for video broadcasting, but such devices cost a lot of kilobucks. The article describes the configuration of the free, open package OpenMPTCPRouter, and deals with popular myths about channel summation.
Myths about summing channels
There are many home routers that support the Multi-WAN function. Sometimes manufacturers call this channel summing, which is not entirely true. Many networkers believe that in addition to and summation at the L2 layer, no other channel aggregation exists. I often heard that this is generally impossible from people who work in telecoms. Therefore, let's try to understand popular myths.
Balancing at the level of IP connections
This is the most affordable and popular way to utilize several Internet channels at the same time. For simplicity, let's imagine that you have three ISPs, each giving you a real IP address from their network. All these providers are connected to a router with support for the Multi-WAN function. This can be OpenWRT with the mwan3 package, mikrotik, ubiquiti, or any other household router, since now this option is no longer uncommon.
To simulate the situation, imagine that the providers gave us the following addresses:
WAN1 β 11.11.11.11
WAN2 β 22.22.22.22
WAN2 β 33.33.33.33
That is, connecting to a remote server example.com through each of the providers, the remote server will see three independent source ip of the client. Balancing allows you to divide the load across channels and use them all three at the same time. For simplicity, let's imagine that we divide the load between all channels equally. As a result, when a client opens a site with three images conditionally, he downloads each image through a separate provider. On the site side, it looks like connections from three different IPs.
When balancing at the connection level, each TCP connection goes through a separate provider.
This balancing mode often causes problems for users. For example, many sites hard-wire cookies and tokens to the client's IP address, and if it suddenly changes, the request is dropped or the client logs out on the site. This is often reproduced in client-bank systems and other sites with strict user session rules. Here is a simple illustrative example: music files in VK.com are available only with a valid session key that is bound to IP, and clients using such balancing often do not play audio, because the request did not go through the provider to which the session is bound.
When downloading torrents, balancing at the connection level sums the bandwidth of all channels
Such balancing allows you to get the summation of the speed of the Internet channel when using multiple connections. For example, if each of the three providers has a speed of 100 megabits, then when downloading torrents we will get 300 megabits. Because the torrent opens a lot of connections that are distributed among all providers and eventually utilize the entire channel.
It is important to understand that one single TCP connection will always go through only one provider. That is, if we download one large file via HTTP, then this connection will be made through one of the providers, and if the connection with this provider breaks, then the download will also break.
One connection will always use only one Internet channel
This is also true for video broadcasts. If you broadcast streaming video on some conditional Twitch, then balancing at the level of IP connections will not give any particular benefit, since the video stream will be broadcast within one IP connection. In this case, if the WAN 3 provider starts having communication problems, such as packet loss or slowdown, then you will not be able to immediately switch to another provider. The broadcast will have to be stopped and reconnected.
True channel summing
The real summation of channels makes it possible to start one connection to the conditional Twitch through all providers at once in such a way that if any of the providers breaks, the connection will not be interrupted. This is a surprisingly difficult problem, which still does not have an optimal solution. Many do not even know that this is possible!
From the previous illustrations, we remember that the conditional Twitch server can receive a video stream from us from only one source IP address, which means it must always be constant with us, regardless of which providers have fallen off and which ones work. To achieve this, we need a summation server that will terminate all our connections and merge them into one.
The summation server aggregates all channels into one tunnel. All connections originate from the address of the summation server
This scheme uses all providers, and disabling any of them will not cause a loss of communication with the Twitch server. In fact, this is a special VPN tunnel, under the hood of which there are several Internet channels at once. The main task of such a scheme is to obtain the highest quality communication channel. If problems start at one of the providers, packet loss, increase in delays, then this should not affect the quality of communication in any way, since the load will automatically be distributed over other, better channels that are available.
Commercial Solutions
This problem has long been a concern for those who broadcast events live and do not have access to high-quality Internet. For such tasks, there are several commercial solutions, for example, Teradek makes such monstrous routers into which packs of USB modems are inserted:
Broadcast video router with channel summing function
Such devices usually have the ability to capture video via HDMI or SDI. Together with the router, a subscription to the channel summation service is sold, as well as processing the video stream, transcoding it and retransmitting it further. The price of such devices starts from $ 2k with a set of modems, plus a separate subscription to the service.
Sometimes it looks quite intimidating:
Setting up OpenMPTCPRouter
Protocol (MultiPath TCP) was invented for the ability to connect on several channels at once. For example, his and can simultaneously connect to a remote server via WiFi and via a cellular network. It is important to understand that these are not two separate TCP connections, but one connection established over two channels at once. For this to work, the remote server must support MPTCP too.
is an open source software router project that allows you to truly sum channels. The authors state that the project is in the alpha version status, but it can already be used. It consists of two parts - a summing server, which is located on the Internet and a router, to which several Internet providers and client devices themselves are connected: computers, telephones. A custom router can be a Raspberry Pi, some WiFi routers, or a regular computer. There are ready-made assemblies for various platforms, which is very convenient.
How OpenMPTCPRouter works
Summarization server setup
The summing server is located on the Internet and terminates connections from all channels of the client router into one. The IP address of this server will be the external address when accessing the Internet through OpenMPTCPRouter.
For this task, we will use a VPS server on Debian 10.
Summation server requirements:
- MPTCP not working on OpenVZ virtualization
- It should be possible to install your own Linux kernel
The server is deployed by executing one command. The script will install the mptcp-enabled kernel and all required packages. Install scripts are available for Ubuntu and Debian.
wget -O - http://www.openmptcprouter.com/server/debian10-x86_64.sh | sh
The result of a successful server installation.
We save the passwords, we will need them to configure the client router, and reboot. It is important to keep in mind that after installation, SSH will be available on port 65222. After rebooting, we need to make sure that we boot with the new kernel
uname -a
Linux test-server.local 4.19.67-mptcp
We see the inscription mptcp next to the version number, which means the kernel was installed correctly.
Setting up a client router
On the Ready-made builds are available for some platforms, such as Raspberry Pi, Banana Pi, Lynksys routers, and virtual machines.
This part of openmptcprouter is based on OpenWRT, using LuCI as an interface, familiar to everyone who has ever encountered OpenWRT. The distribution kit weighs about 50Mb!
As a test bench, I will use Raspberry Pi and several USB modems with different operators: MTS and Megafon. How to write an image to an SD card, I guess, no need to tell.
Initially, the Ethernet port in the Raspberry Pi is configured as lan with a static IP address. 192.168.100.1. In order not to mess with the wires on the table, I connected the Raspberry Pi to a WiFi access point and set a static address on the computer's WiFi adapter 192.168.100.2. The DHCP server is not enabled by default, so static addresses must be used.
Now you can go to the web interface
When you first log in, the system will ask you to set the root password, SSH will be available with the same password.
In the LAN settings, you can set the desired subnet and enable the DHCP server.
I use modems that are defined as USB ethernet interfaces with a separate DHCP server, so this required installation . The procedure is identical to configuring modems in regular OpenWRT, so I won't cover it here.
Next, you need to configure the WAN interfaces. Initially, two virtual interfaces WAN1 and WAN2 were created in the system. They need to assign a physical device, in my case, these are the names of USB modem interfaces.
In order not to get confused in the interface names, I advise you to watch dmesg messages while connected via SSH.
Since my modems act as routers themselves and have a DHCP server themselves, I had to change the settings of their internal network ranges and disable the DHCP server, because initially both modems issue addresses from the same network, and this causes a conflict.
OpenMPTCPRouter requires WAN interface addresses to be static, so we come up with subnets for modems and configure them in the system β openmptcprouter β interface settings menu. Here you also need to specify the IP address and server key obtained during the installation of the summation server.
In case of successful setup, a similar picture should appear on the status page. It can be seen that the router was able to reach the summing server and both channels are working properly.
The default mode is shadowsocks + mptcp. This is such a proxy that wraps all connections in itself. Initially, it is configured to handle only TCP, but you can enable UDP as well.
If there are no errors on the status page, the setup can be considered complete.
With some providers, a situation may arise when the mptcp flag is truncated along the traffic path, then there will be such an error:
In this case, you can use another mode of operation, without using MPTCP, more about this .
Conclusion
The OpenMPTCPRouter project is very interesting and important, since it is perhaps the only open complex solution to the channel summing problem. Everything else is either tightly closed and proprietary, or just separate modules that an ordinary person cannot deal with. At the current stage of development, the project is still quite raw, extremely poor documentation, many things are simply not described. But at the same time, it still works. I hope that it will continue to develop, and we will get household routers that will be able to combine channels normally out of the box.
Follow our developer on Instagram
Source: habr.com