Letters to Tucha's technical support often turn into real articles. Recently, a client reached out to us asking to clarify what happens with connections within the VPN tunnel between the user's office and the cloud environment, as well as with connections outside the VPN tunnel. Therefore, the text provided below is an actual letter we sent to one of the clients in response to their question. Of course, we changed the IP addresses to anonymize the client. But yes, Tucha's technical support is indeed renowned for its detailed responses and informative letters. 🙂
Of course, we understand that for many, this article may not be groundbreaking. However, since articles for beginner administrators occasionally appear on Habr, and given that this article originated from a real letter to a real client, we still want to share this information here. There's a good chance it will be useful to someone.
So, we will explain in detail what happens between the cloud server and the office if they are connected via a site-to-site network. Note that some services are only available from the office, while others can be accessed from anywhere on the Internet.
First, we will clarify that our client requested the server 192.168.A.1 to allow connections via RDP from anywhere, connecting to A.A.A.2:13389, while the other services are accessible only from the office (192.168.B.0/24), connected through VPN. The client also initially set it up so that the machine 192.168.B.2 in the office could also be accessed via RDP from anywhere, connecting to B.B.B.1:11111. We assisted in organizing IPSec connections between the cloud and the office, and the client's IT specialist began to ask questions about what would happen in various scenarios. To answer all these questions, we wrote out everything you can read below.
Now, let's examine these processes in more detail.
Position One
When something is sent from 192.168.B.0/24 downward API support (simultaneously with this in 192.168.A.0/24 or from 192.168.A.0/24 downward API support (simultaneously with this in 192.168.B.0/24, it enters the VPN. This means that the packet is additionally encrypted and transmitted between B.B.B.1 and A.A.A.1, but 192.168.A.1 sees the packet specifically from 192.168.B.1. They can communicate with each other using any protocols. The responses are sent back through the VPN in the same way, thus the packet from 192.168.A.1 for 192.168.B.1 will be sent as an ESP datagram from A.A.A.1 to B.B.B.1, which the router on the other side will unwrap, extract the packet from, and deliver to 192.168.B.1 how a packet from 192.168.A.1.
A specific example:
1) 192.168.B.1 calls 192.168.A.1, wants to establish a TCP connection with 192.168.A.1:3389;
2) 192.168.B.1 sends a request to establish a connection from 192.168.B.1:55555 (the port number for feedback is chosen by it, here and thereafter we will use number 55555 as an example of such a port number that the system selects during the formation of the TCP connection) to 192.168.A.1:3389;
3) the operating system running on the computer with the address 192.168.B.1, decides to send this packet to the gateway address of the router (192.168.B.254 in our case), because it does not have other, more specific routes for 192.168.A.1, so it sends the packet via the default route (0.0.0.0/0);
4) for this, it tries to find the MAC address for the IP address 192.168.B.254 in the ARP protocol cache table. If not found, it sends from the address 192.168.B.1 a broadcast who-has request to the network 192.168.B.0/24When 192.168.B.254 responds by sending its MAC address, the system sends the Ethernet packet to it and records this information in its cache table;
5) the router receives this packet and determines where to send it: it has a policy that states it must route all packets between 192.168.B.0/24 and 192.168.A.0/24 over a VPN connection between B.B.B.1 and A.A.A.1;
6) the router forms an ESP datagram from B.B.B.1 to A.A.A.1;
7) the router decides where to send this packet, it sends it to, for example, B.B.B.254 (ISP gateway), because it does not have more specific routes to A.A.A.1, than 0.0.0.0/0;
8) just like mentioned before, it finds the MAC address for B.B.B.254 and sends the packet to the ISP gateway;
9) ISPs transmit the ESP datagram through their networks from B.B.B.1 to A.A.A.1;
10) the virtual router at A.A.A.1 receives this datagram, decrypts it, and obtains the packet from 192.168.B.1:55555 for 192.168.A.1:3389;
11) the virtual router checks where to send it, finds the network in the routing table 192.168.A.0/24 and sends it directly to 192.168.A.1, since it has an interface 192.168.A.254/24;
12) for this, the virtual router finds the MAC address for 192.168.A.1 and sends this packet to it over a virtual Ethernet network;
13) 192.168.A.1 receives this packet on port 3389, agrees to establish a connection and forms a response packet from 192.168.A.1:3389 to 192.168.B.1:55555;
14) its system sends this packet to the gateway address of the virtual router (192.168.A.254 in our case), because it does not have other, more specific routes for 192.168.B.1, it does not have, so it must send the packet via the default route (0.0.0.0/0);
15) just as in the previous cases, the system running on the server with the address 192.168.A.1, finds the MAC address 192.168.A.254, since it is in the same network as its interface 192.168.A.1/24;
16) the virtual router receives this packet and decides where to forward it: it has a policy that dictates it must process all packets between 192.168.A.0/24 and 192.168.B.0/24 over a VPN connection between A.A.A.1 and B.B.B.1;
17) the virtual router creates an ESP datagram from A.A.A.1 for B.B.B.1;
18) the virtual router decides whom to send this packet to, forwarding it to A.A.A.254 (the internet service provider's gateway, which, in this case, is also us), because there are no more specific routes to B.B.B.1, than 0.0.0.0/0;
19) internet providers transmit the ESP datagram across their networks A.A.A.1 to B.B.B.1;
20) the router at B.B.B.1 receives this datagram, decrypts it, and obtains the packet from 192.168.A.1:3389 for 192.168.B.1:55555;
21) it understands that it should forward it specifically to 192.168.B.1, since it is in the same network, therefore, there is a corresponding entry in the routing table that compels it to send packets for all 192.168.B.0/24 directly;
22) the router finds the MAC address for 192.168.B.1 and forwards this packet to it;
23) the operating system on the computer with the address 192.168.B.1 receives the packet from 192.168.A.1:3389 for 192.168.B.1:55555 and initiates the next steps to establish a TCP connection.
This example concisely and simply describes what happens at layers 2-4 (many more details could be recalled here). Layers 1, 5-7 are not covered.
Position two
If something is sent specifically to 192.168.B.0/24 A.A.A.2 , it goes not through the VPN, but directly. This means that if a user from address, this packet is NAT-ed from address 192.168.B.1 calls A.A.A.2:13389, passing to B.B.B.1, and there the router receives it and forwards it to , it goes not through the VPN, but directly. This means that if a user from addressknows nothing about 192.168.A.1. 192.168.A.1 , it sees the packet from 192.168.B.1, since it is NAT-ed. Therefore, the response to this request goes through the common route, it is NAT-ed from address B.B.B.1and heads to , it goes not through the VPN, but directly. This means that if a user from address , and that router returns this response to B.B.B.1, which sees the response from 192.168.B.1, the one it was addressing. , it goes not through the VPN, but directly. This means that if a user from address(this number, just like in the previous example, may be different) at
A specific example:
1) 192.168.B.1 calls , it goes not through the VPN, but directly. This means that if a user from address, wants to establish a TCP connection with A.A.A.2:13389;
2) 192.168.B.1 sends a request to establish a connection from 192.168.B.1:55555 , it does not exist, which means it forwards the packet via the default route (0.0.0.0/0); A.A.A.2:13389;
3) the operating system running on the computer with the address 192.168.B.1, decides to send this packet to the gateway address of the router (192.168.B.254 in our case), because it does not have other, more specific routes for , it goes not through the VPN, but directly. This means that if a user from address4) for this, as we mentioned in the previous example, it tries to find the MAC address for the IP address
5) the router receives this packet and decides where to forward it: it has a policy that dictates it must NAT (substituting the source address) all packets from 192.168.B.254 in the ARP protocol cache table. If not found, it sends from the address 192.168.B.1 a broadcast who-has request to the network 192.168.B.0/24When 192.168.B.254 responds by sending its MAC address, the system sends the Ethernet packet to it and records this information in its cache table;
to other nodes in the internet; 192.168.B.0/24 to other nodes of the Internet;
6) Since this policy implies that the return address must match the lower address on the interface through which this packet will be sent, the router first determines to whom exactly it should send this packet, as in the previous example, it must send it to B.B.B.254 (ISP gateway), because it does not have more specific routes to , it goes not through the VPN, but directly. This means that if a user from address, than 0.0.0.0/0;
7) Therefore, the router replaces the return address of the packet, from now on the packet from B.B.B.1:44444 (the port number, of course, can be different) to A.A.A.2:13389;
8) The router remembers what it did, so when the response from A.A.A.2:13389 to B.B.B.1:44444 comes in, it will know that it should change the address and port of the recipient to 192.168.B.1:55555.
9) Now, the router must send it to the internet service provider through B.B.B.254, therefore, just as we mentioned earlier, it finds the MAC address for B.B.B.254 and sends the packet to the ISP gateway;
10) Internet service providers transmit the packet through their networks from B.B.B.1 to , it goes not through the VPN, but directly. This means that if a user from address;
11) The virtual router at , it goes not through the VPN, but directly. This means that if a user from address receives this packet on port 13389;
12) The virtual router has a rule that stipulates that packets received from any sender on this port should be sent to 192.168.A.1:3389;
13) The virtual router finds the network in the routing table 192.168.A.0/24 and sends it directly to 192.168.A.1, as it has the interface 192.168.A.254/24;
14) For this, the virtual router finds the MAC address for 192.168.A.1 and sends this packet to it over a virtual Ethernet network;
15) 192.168.A.1 receives this packet on port 3389, agrees to establish a connection and forms a response packet from 192.168.A.1:3389 to B.B.B.1:44444;
16) Its system forwards this packet to the gateway address of the virtual router (192.168.A.254 in our case), because it does not have other, more specific routes for B.B.B.1, it does not have, so it must send the packet via the default route (0.0.0.0/0);
17) Just like in the previous cases, the system running on the server with the address 192.168.A.1, finds the MAC address 192.168.A.254, since it is in the same network as its interface 192.168.A.1/24;
18) The virtual router receives this packet. It should be noted that it remembers having received the packet from A.A.A.2:13389 and changed its address and recipient port to B.B.B.1:44444 , therefore, for the packet from 192.168.A.1:3389it changes the sender's address to 192.168.A.1:3389 for B.B.B.1:44444 19) The virtual router decides to whom to send this packet, it sends it to A.A.A.2:13389;
20) Internet service providers transmit the packet with A.A.A.254 (the internet service provider's gateway, which, in this case, is also us), because there are no more specific routes to B.B.B.1, than 0.0.0.0/0;
21) The router at , it goes not through the VPN, but directly. This means that if a user from address to B.B.B.1;
receives this packet and recalls that when it forwarded the packet from B.B.B.1 , it changed its sender address and port to 192.168.B.1:55555 for A.A.A.2:13389, so this is a response that needs to be sent to B.B.B.1:44444(In reality, there are several more checks, but we will not delve into that); 192.168.B.1:55555 22) It realizes that it should be sent directly to
, since it is in the same network, therefore, it has a corresponding entry in the routing table that directs packets for all 192.168.B.1. 192.168.B.0/24 directly;
23) the router finds the MAC address for 192.168.B.1 and forwards this packet to it;
24) the operating system on the computer with the address 192.168.B.1 receives the packet from A.A.A.2:13389 for 192.168.B.1:55555 and initiates the next steps to establish a TCP connection.
It should be noted that in this case, the computer with the address 192.168.B.1 knows nothing about the server with the address 192.168.A.1, it only communicates with , it goes not through the VPN, but directly. This means that if a user from address. Similarly, the server with the address 192.168.A.1 knows nothing about the computer with the address 192.168.B.1. It believes that it has been contacted from the address B.B.B.1, and it does not know anything else, so to speak.
It’s also worth noting that if this computer accesses A.A.A.2:1540, the connection will not be established because port 1540 is not configured for connection forwarding on the virtual router, even if there are any servers in the virtual network 192.168.A.0/24 (for example, on the server with the address 192.168.A.1) and there are services waiting for connections on that port. If the user of the computer with the address 192.168.B.1 urgently needs to establish a connection with this service, they must use VPN, that is, connect directly to 192.168.A.1:1540.
It should be emphasized that any attempts to establish a connection with A.A.A.1 (except for IPSec connections from B.B.B.1 ) will not be successful. Any attempts to establish connections with , it goes not through the VPN, but directly. This means that if a user from address, except for connections to port 13389, will also not be successful.
It should also be noted that in the event that someone else (for example, C.C.C.C) accesses , it goes not through the VPN, but directly. This means that if a user from address , everything mentioned in points 10-20 will apply to them as well. What happens before and after depends on what exactly is behind this C.C.C.C. We do not have such information, therefore we recommend consulting the administrators of the node with the address C.C.C.C.
Third position
And conversely, if from 192.168.A.1 anything is sent to any port that is configured for forwarding inside to B.B.B.1 (for example, 11111), it also does not go into the VPN but simply goes through A.A.A.1 and arrives at B.B.B.1, and that one forwards it somewhere to, say, 192.168.B.2:3389. That one sees this packet not from 192.168.A.1, but from A.A.A.1. And, when 192.168.B.2 responds, the packet goes from B.B.B.1 to A.A.A.1, and later reaches the connection initiator — 192.168.A.1.
A specific example:
1) 192.168.A.1 calls B.B.B.1, wants to establish a TCP connection with B.B.B.1:11111;
2) 192.168.A.1 sends a request to establish a connection from 192.168.A.1:55555 , it does not exist, which means it forwards the packet via the default route (0.0.0.0/0); B.B.B.1:11111;
3) the operating system that runs on the server with the address 192.168.A.1, decides to send this packet to the gateway address of the router (192.168.A.254 in our case), because it does not have other, more specific routes for B.B.B.1, so it sends the packet via the default route (0.0.0.0/0);
4) for this, as we mentioned in previous examples, it tries to find the MAC address for the IP address 192.168.A.254 in the ARP protocol cache table. If not found, it sends from the address 192.168.A.1 a broadcast who-has request to the network 192.168.A.0/24When 192.168.A.254 in response, it sends its MAC address to it, the system sends an Ethernet packet for it, and records this information in its cache table;
5) The virtual router receives this packet and determines where to send it: it has a policy that requires it to NAT (modifying the source address) all packets from 192.168.A.0/24 to other nodes of the Internet;
6) since this policy requires the source address to match the lowest address on the interface through which this packet will be sent, the virtual router first decides to whom exactly to send this packet, and, as in the previous example, it must send it to A.A.A.254 (the internet service provider's gateway, which, in this case, is also us), because there are no more specific routes to B.B.B.1, than 0.0.0.0/0;
7) thus, the virtual router replaces the source address of the packet, from now on it is a packet from A.A.A.1:44444 (the port number, of course, can be different) to B.B.B.1:11111;
8) The virtual router remembers what it did, thus, when from B.B.B.1:11111 for A.A.A.1:44444 comes in, it will know that it should change the address and port of the recipient to 192.168.A.1:55555.
9) now the virtual router must send it to the ISP's network through A.A.A.254, therefore, just as we mentioned before, it finds the MAC address for A.A.A.254 and sends the packet to the ISP gateway;
10) Internet service providers transmit the packet through their networks from A.A.A.1 on B.B.B.1;
11) The router at B.B.B.1 receives this packet on port 11111;
12) on the virtual router, there exists a rule that states that packets received from any sender on this port should be forwarded to 192.168.B.2:3389;
13) The router finds in the routing table the network 192.168.B.0/24 and sends it directly to 192.168.B.2, since it has an interface 192.168.B.254/24;
14) For this, the virtual router finds the MAC address for 192.168.B.2 and sends this packet to it over a virtual Ethernet network;
15) 192.168.B.2 receives this packet on port 3389, agrees to establish a connection and forms a response packet from 192.168.B.2:3389 to A.A.A.1:44444;
16) its system forwards this packet to the gateway address of the router (192.168.B.254 in our case), because it does not have other, more specific routes for A.A.A.1, it does not have, so it must send the packet via the default route (0.0.0.0/0);
17) just as in previous cases, the system operating on the computer with the address 192.168.B.2, finds the MAC address 192.168.B.254, since it is in the same network as its interface 192.168.B.2/24;
18) The router receives this packet. It is worth noting that it remembers what it received at B.B.B.1:11111 and changed its address and recipient port to A.A.A.1 , therefore, for the packet from 192.168.B.2:3389it changes the sender's address to 192.168.B.2:3389 for A.A.A.1:44444 19) The virtual router decides to whom to send this packet, it sends it to B.B.B.1:11111;
19) The router determines to whom to forward this packet. It sends it to, let’s say, B.B.B.254 (the gateway of the Internet provider, the exact address of which we do not know), because there are no more specific routes to A.A.A.1, than 0.0.0.0/0;
21) The router at B.B.B.1 to A.A.A.1;
21) the virtual router at A.A.A.1 , it changed its sender address and port to 192.168.A.1:55555 for B.B.B.1:11111, so this is a response that needs to be sent to A.A.A.1:44444. Thus, this is the response that needs to be forwarded to 192.168.A.1:55555 (in fact, as we mentioned in the previous example, there are also several checks there, but this time we will not delve into them);
22) it understands that it should send it directly to 192.168.A.1, since it is on the same network, therefore, it has a corresponding entry in the routing table that causes it to send packets for all 192.168.A.0/24 directly;
23) the router finds the MAC address for 192.168.A.1 and forwards this packet to it;
24) the operating system on the server with the address 192.168.A.1 receives the packet from B.B.B.1:11111 for 192.168.A.1:55555 and initiates the next steps to establish a TCP connection.
Just as in the previous case, in this case the server with the address 192.168.A.1 knows nothing about the computer with the address 192.168.B.1, it only communicates with B.B.B.1. The computer with the address 192.168.B.1 also knows nothing about the server with the address 192.168.A.1. It believes that it has been contacted from the address A.A.A.1, and the rest is hidden from it.
Output
This is how everything occurs during connections inside the VPN tunnel between the client's office and the cloud environment, as well as during connections outside the VPN tunnel. If you have any questions or need our help with cloud solutions,
Source: habr.com
