Is it possible to combine multiple internet connections into one? There are many misconceptions and myths surrounding this topic, even experienced network engineers often do not realize that it is possible. In most cases, channel aggregation is incorrectly referred to as NAT level load balancing or failover. However, true aggregation allows one single TCP connection to be utilized simultaneously across all internet connections, such as for video streaming, so that if any of the internet connections drop, the broadcast does not stop.
There are expensive commercial solutions for video streaming, but such devices cost thousands of dollars. This article describes the setup of the free, open package OpenMPTCPRouter, addressing popular myths about channel aggregation.
Myths about channel aggregation
There are many consumer routers that support the Multi-WAN function. Sometimes manufacturers refer to it as channel aggregation, which is not entirely accurate. Many network professionals believe that besides and aggregation at L2 level, no other form of channel bonding exists. I often hear from telecom workers that it is completely impossible. So let’s try to clarify popular myths.
IP connection load balancing
This is the most accessible and popular method to utilize multiple internet connections simultaneously. For simplicity, let’s assume you have three internet providers, each giving you a real IP address from their own network. All these providers are connected to a router that supports the Multi-WAN function. This could be OpenWRT with the mwan3 package, MikroTik, Ubiquiti, or any other consumer router, as this option is no longer rare these days.
To illustrate the situation, let’s say the providers issued us the following addresses:
WAN1 — 11.11.11.11
WAN2 — 22.22.22.22
WAN3 — 33.33.33.33
So, connecting to a remote server example.com Through each of the providers, the remote server will see three independent source IPs of the client. Load balancing allows the distribution of traffic across the channels and the simultaneous use of all three. For simplicity, let's assume we distribute the load evenly among all channels. As a result, when a client opens a website that hypothetically has three images, each image is loaded through a separate provider. From the website's perspective, this appears as connections from three different IPs.
With connection-level balancing, each TCP connection goes through a separate provider.
This type of balancing often causes issues for users. For example, many websites tightly bind cookies and tokens to the client's IP address, and if it suddenly changes, the request is rejected or the user is logged out of the site. This is often reproduced in online banking systems and other sites with strict user session rules. Here’s a simple illustrative example: audio files on VK.com are only available with a valid session key tied to the IP, and clients using such balancing often cannot play audio because the request is not routed through the provider to which the session is tied.
When downloading torrents, connection-level balancing sums the bandwidth of all channels.
Such balancing allows for the aggregation of internet channel speed when using multiple connections. For example, if each of the three providers has a speed of 100 Megabits, we will achieve 300 Megabits when downloading torrents. This is because torrents open multiple connections that are distributed among all providers, ultimately utilizing the entire channel.
It is important to understand that a single TCP connection will always go through only one provider. That is, if we are downloading one large file via HTTP, that connection will be made through one of the providers, and if the connection to that provider breaks, the download will also fail.
One connection will always use only one internet channel.
This is also true for video broadcasts. If you are streaming video on a platform like Twitch, IP-level connection balancing will not provide any significant benefit, as the video stream will be transmitted within a single IP connection. In this case, if the WAN provider experiences issues, such as packet loss or reduced speed, you won't be able to instantly switch to another provider. You will need to stop the stream and reconnect.
Channel aggregation
Real channel aggregation allows a single connection to a platform like Twitch to be routed through all providers. This way, if any provider fails, the connection will not be interrupted. It is surprisingly complex and still lacks an optimal solution. Many people don't even know this is possible!
From the previous illustrations, we remember that a typical Twitch server can only accept the video stream from one source IP address, meaning it must remain constant regardless of which providers fail and which are operational. To achieve this, we need an aggregation server that will terminate all our connections and combine them into one.
The aggregation server consolidates all channels into a single tunnel. All connections are made from the address of the aggregation server.
In this scheme, all providers are utilized, and disconnection of any of them will not cause a loss of connection with the Twitch server. Essentially, this is a special VPN tunnel that simultaneously uses several internet connections. The main goal of such a scheme is to achieve the highest quality connection. If one of the providers experiences issues, such as packet loss or increased latency, it should not affect the connection quality, as the load will automatically shift to other, higher-quality channels available.
Commercial solutions
This issue has long been a concern for those who conduct live streaming of events and do not have access to high-quality internet. There are several commercial solutions for such tasks; for example, the company Teradek makes monstrous routers that accommodate stacks of USB modems:
Router for video streaming with channel bonding functionality
Such devices typically come with the capability to capture video signals via HDMI or SDI. A subscription to the channel bonding service, along with the video stream processing, transcoding, and rebroadcasting service is sold with the router. Prices for these devices start at $2,000 with a modem kit, plus a separate subscription to the service.
Sometimes this looks quite daunting:
Setting up OpenMPTCPRouter
Protocol (MultiPath TCP) was designed to allow simultaneous connections over multiple channels. For example, it and can connect to a remote server via WiFi and through a cellular network at the same time. It's important to understand that this is not two separate TCP connections, but one connection established over two channels. For this to work, the remote server must also support MPTCP.
is an open-source software router project that truly enables channel bonding. The authors claim that the project is in alpha status, but it is already usable. It consists of two parts: a bonding server that is hosted on the internet, and a router that connects several internet service providers and client devices such as computers and phones. A Raspberry Pi, some WiFi routers, or a regular computer can serve as a user router. There are ready-made builds available for different platforms, which is very convenient.
How OpenMPTCPRouter Works
Setting up the bonding server
The bonding server is located on the internet and terminates connections from all channels of the client router into one. The IP address of this server will be the external address when accessing the internet via OpenMPTCPRouter.
For this task, we will use a VPS server running Debian 10.
Requirements for the bonding server:
- MPTCP does not work on OpenVZ virtualization
- There must be an option to install a custom Linux kernel
The server can be set up by executing a single command. The script will install a kernel supporting mptcp and all necessary packages. Installation scripts are available for Ubuntu and Debian.
wget -O - http://www.openmptcprouter.com/server/debian10-x86_64.sh | sh
Result of the successful server installation.
We save the passwords as they will be needed for configuring the client router and reboot. It's important to note that after installation, SSH will be accessible on port 65222. After rebooting, make sure we have booted with the new kernel.
uname -a
Linux test-server.local 4.19.67-mptcp
We see the label mptcp next to the version number, which means the kernel was installed correctly.
Configuring the client router
At Ready builds are available for some platforms, such as Raspberry Pi, Banana Pi, Lynksys routers, and virtual machines.
This part of openmptcprouter is based on OpenWRT, using LuCI as the interface, familiar to anyone who has ever dealt with OpenWRT. The distribution is about 50MB!
For the test setup, I will use a Raspberry Pi and several USB modems from different operators: MTS and MegaFon. I believe no explanation is needed on how to write an image to an SD card.
Initially, the Ethernet port on the Raspberry Pi is configured as lan with a static IP address. 192.168.100.1. To avoid dealing with wires on the desk, I connected the Raspberry Pi to a WiFi hotspot and assigned a static address to the computer's WiFi adapter. 192.168.100.2. The DHCP server is not enabled by default, so static addresses must be used.
Now we can access the web interface.
Upon the first login, the system will prompt you to set the root password, which will also be used for SSH access.
In the LAN settings, you can specify the desired subnet and enable the DHCP server.
I am using modems that are recognized as USB Ethernet interfaces with separate DHCP servers, so this required the installation of . The procedure is identical to configuring modems in regular OpenWRT, so I won’t cover it here.
Next, you need to configure the WAN interfaces. Initially, two virtual WAN1 and WAN2 interfaces are created in the system. They need to be assigned a physical device, in my case, the names of the USB modem interfaces.
To avoid confusion with interface names, I recommend checking the dmesg messages while connected via SSH.
Since my modems act as routers themselves and have their own DHCP servers, I had to change their internal network range settings and disable their DHCP servers, because both modems initially assigned addresses from the same network, causing a conflict.
OpenMPTCPRouter requires WAN interface addresses to be static, so we assign subnets to the modems and configure them in the menu system → openmptcprouter → interface settings. Here we also need to specify the IP address and the server key obtained during the installation of the aggregating server.
If the configuration is successful, the status page should show a similar picture. It shows that the router was able to communicate with the aggregating server and both channels are functioning normally.
By default, the shadowsocks + mptcp mode is used. This is a proxy that encapsulates all connections. Initially, it is configured to handle only TCP, but UDP can also be enabled.
If there are no errors on the status page, the configuration can be considered complete.
With some providers, there may be a situation where the mptcp flag is stripped along the traffic path, leading to such an error:
In this case, you can use a different operating mode without MPTCP; more details about this .
Conclusion
The OpenMPTCPRouter project is very interesting and important, as it is perhaps the only open comprehensive solution to the channel aggregation problem. Everything else is either completely closed and proprietary or just separate modules that an ordinary person cannot handle. At the current stage of development, the project is still quite raw, with extremely poor documentation, and many things are simply not described. However, it still works. I hope it continues to evolve, and we will get consumer routers that can aggregate channels properly out of the box.
Follow our developer on Instagram
Source: habr.com
