When faced with a question and break a large amount of documentation, try to systematize and write down what you learned in order to remember better. And also make instructions on this issue, so as not to go all the way again.
Source documentation is plentiful at
Formulation of the problem
The client wants to combine several rented servers into one network in order to get rid of the need to pay for several additional subnets, hang his entire household behind a router, assign local addresses to them inside, and protect himself with a firewall. So that all service traffic runs inside the VLAN. Plus, transfer virtual machines from one old server to a new one and refuse it, upgrade the used old hardware and at the same time move to fresh Proxmox.
Initially, the client has 5 servers, each with an additional subnet, the first address from the allocated subnet is assigned to an additional bridge on Proxmox
At the same time, VMs run on Windows and have the address 85.xx177/29 configured with a gate 85.xx176
And in a similar vein, all 5 servers are configured with their virtual machines.
It's funny that this configuration is wrong in setting up the network in principle, use the network address for the first node and it is also for the gateway. If you try to start such a configuration on a virtual machine in Ubuntu, the network does not work.
β
implementation
- We create a vSwitch in the interface, assign a VlanID to it, add this vSwitch to all the servers we need.
- We are making a test server so that you can set up and move without problems.
We raise the first virtual machine chr by .
If you use the above script, please note that the -d /root/temp directory is checked at the beginning, and if it is not there, the /home/root/temp directory is created, but further work is still carried out with the /root/temp directory. The script needs to be corrected to create the appropriate directory.
- Setting up a network for Proxmox.
We add a subinterface with the VLAN number, indicate that the address settings will occur on the bridges using the inet manual. IMPORTANT. You cannot configure IP addresses on interfaces that you will then include in the bridge, how it will work and whether no one will know at all.
Next, we create a vmbr0 bridge - and we hang on it the first address of the server itself, given to us by Hetzner providers, specify the bridge port - the first physical interface without VLAN, and also specify with an additional command to add a route to our additional network ordered from Hetzner for this server through this bridge. Adding a route will work when the interface comes up.
The second bridge will be our interface for local traffic, add an address to it to get connectivity between different Proxmox servers over a local network without access to the Internet and specify the eno1.4000 subinterface, which is allocated for our VlanID, as the port.
During the initial setup, there are tips that you can install an additional ifupdown2 package for Proxmox and you can not restart the entire server when changes in network interfaces. However, this is typical only for the initial setup, and when using bridges and setting up virtual machines, you encounter network failure problems in virtual machines. Despite the fact that you ruled, for example, the vmbr2 interface, and when applying the configuration, the network falls off already on all internal interfaces and does not rise until the server is completely restarted. ifdown&&ifup don't help. If someone has a solution, I will be grateful.
The first configured interface on the server itself remains operational and available.
-
Address allocation for CHR so as not to lose addresses from the pool
The address pool that Hetzner gives out looks very strange for a networker, something like this:
The strange thing is that the gate is proposed to use its own address of the physical server.
The classic version proposed by Hetzner himself is indicated in the problem statement and was implemented by the client independently. In this option, the client loses the first address to the network address, the second address to the proxmox bridge and it will also be the gateway, and the last address for broadcasting. IPv4 addresses are never redundant. If you directly try to register the CHR IP address 136.Ρ .Ρ .177/29 and the gateway for 0.0.0.0/0 148.Ρ .Ρ .165, then you can do it, but the gateway will not be Direct Connected and therefore will be unreachable.
You can get out of the situation if you use 32 networks for each address and specify the address we need as the network name, which can be anything. It turns out an analogue of a point-to-point connection.
In this case, the gateway will of course be available, and everything will work as we need.
Keep in mind that in such a configuration it is not recommended to use the SRC-NAT masquerade rule, because the output address will be indefinitely different, but it is more correct to specify action: src-NAT and the specific address from which you will release the client.
- And finally.
To block access to Proxmox itself from the Internet, use the built-in tools: there is an excellent firewall.
You should not use the firewall offered by hetzner, so as not to get confused about the location of the settings. Hetzner will also act on all networks, including those established on CHR, and in order to open and forward ports, it will also be necessary to open it in the provider's web interface.
Source: habr.com