In this article, I would like to reveal the possibilities of transparent proxying, which allows you to completely unnoticed by clients to redirect all or part of the traffic through external proxy servers.
When I started solving this problem, I encountered the fact that its implementation has one significant problem - the HTTPS protocol. In the good old days, there were no particular problems with transparent HTTP proxying, but with HTTPS proxying, browsers report protocol interference and that's where the happiness ends.
In common instructions for the Squid proxy server, they even offer to generate their own certificate and install it on clients, which is complete nonsense at least irrational and looks like a MITM attack. I know that Squid already knows how to do something similar, but this article is about a proven and working method using 3proxy from the respected 3APA3A.
Next, we will take a detailed look at the process of building 3proxy from source, its configuration, full and selective proxying using NAT, distribution of the channel to several external proxy servers, as well as the use of a router and static routes. We use Debian 9 x64 as OS. Begin!
Installing 3proxy and running a normal proxy
1. Install ifconfig (from the net-tools package)
apt-get install net-tools
2. Install Midnigth Commander
apt-get install mc
3. We now have 2 interfaces:
enp0s3 - external, looks at the Internet
enp0s8 - internal, must look into the local network
In other Debian-based distributions, the interfaces are usually named eth0 and eth1.
ifconfig -a
Interfacesenp0s3: flags=4163 mtu 1500
inet 192.168.23.11 netmask 255.255.255.0 broadcast 192.168.23.255
inet6 fe80::a00:27ff:fec2:bae4 prefixlen 64 scopeid 0x20 ether 08:00:27:c2:ba:e4 txqueuelen 1000 (Ethernet)
RX packets 6412 bytes 8676619 (8.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1726 bytes 289128 (282.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s8: flags=4098 mtu 1500
ether 08:00:27:79:a7:e3 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
The enp0s8 interface is currently not used, we will enable it when we want to use the NAT or NAT Proxy configuration. It is then that it would be logical to assign a static ip to it.
4. Let's start installing 3proxy
4.1 Installing base packages for compiling 3proxy from source
root@debian9:~# apt-get install build-essential libevent-dev libssl-dev -y
4.2. Create a folder for downloading the archive with sources
root@debian9:~# mkdir -p /opt/proxy
4.3. Let's go to this folder
root@debian9:~# cd /opt/proxy
4.4. Now let's download the latest 3proxy package. At the time of this writing, the latest stable version was 0.8.12 (18/04/2018) Download it from the official 3proxy website
root@debian9:/opt/proxy# wget https://github.com/z3APA3A/3proxy/archive/0.8.12.tar.gz
4.5. Unpack the downloaded archive
root@debian9:/opt/proxy# tar zxvf 0.8.12.tar.gz
4.6. Go to the unpacked directory to build the program
root@debian9:/opt/proxy# cd 3proxy-0.8.12
4.7. Next, you need to add a line to the header file so that our server is completely anonymous (it really works, everything is checked, client ips are hidden)
root@debian9:/opt/proxy/3proxy-0.8.12# nano +29 src/proxy.h
Adding a line
#define ANONYMOUS 1
Press Ctrl+x and Enter to save changes.
4.8. Let's build the program
root@debian9:/opt/proxy/3proxy-0.8.12# make -f Makefile.Linux
makelogmake[2]: Leaving directory '/opt/proxy/3proxy-0.8.12/src/plugins/TransparentPlugin'
make[1]: Leaving directory '/opt/proxy/3proxy-0.8.12/src'
No errors, continue.
4.9. Install the program on the system
root@debian9:/opt/proxy/3proxy-0.8.12# make -f Makefile.Linux install
4.10. Go to the root directory and check where the program was installed
root@debian9:/opt/proxy/3proxy-0.8.12# cd ~/
root@debian9:~# whereis 3proxy
3proxy: /usr/local/bin/3proxy /usr/local/etc/3proxy
4.11. Let's create a folder for configuration files and logs in the user's home directory
root@debian9:~# mkdir -p /home/joke/proxy/logs
4.12. Go to the directory where the config should be
root@debian9:~# cd /home/joke/proxy/
4.13. Create an empty file and copy the config there
root@debian9:/home/joke/proxy# cat > 3proxy.conf
3proxy.confdaemon
pidfile /home/joke/proxy/3proxy.pid
server 8.8.8.8
nscache 65536
user tester:CL:1234
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- +_L%t.%. %N.%p %E %U %C:%c %R:%r %O %I %h %T"
rotate 3
auth strong
flush
allow tester
socks -p3128
proxy -p8080
To save, press Ctrl + Z
4.14. Let's create a pid file so that there are no startup errors.
root@debian9:/home/joke/proxy# cat > 3proxy.pid
To save, press Ctrl + Z
4.15. Let's start the proxy server!
root@debian9:/home/joke/proxy# 3proxy /home/joke/proxy/3proxy.conf
4.16. Let's see if the server is listening on ports
root@debian9:~/home/joke/proxy# netstat -nlp
netstat logActive Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID / Program name
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 504/3proxy
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 338/sshd
tcp 0 0 0.0.0.0:3128 0.0.0.0:* LISTEN 504/3proxy
tcp6 0 0 :::22 :::* LISTEN 338/sshd
udp 0 0 0.0.0.0:68 0.0.0.0:* 352/dhclient
As it was written in the config, the web proxy listens on port 8080, Socks5 proxy - 3128.
4.17. To autostart the proxy service after a reboot, you need to add it to cron.
root@debian9:/home/joke/proxy# crontab -e
Adding a line
@reboot /usr/local/bin/3proxy /home/joke/proxy/3proxy.conf
We press Enter, since cron should see the end-of-line character and save the file.
There should be a message about installing a new crontab.
crontab: installing new crontab
4.18. Let's reboot the system and try to connect through the browser to the proxy. For verification, we use the Firefox browser (for a web proxy) and the FoxyProxy add-on for socks5 with authentication.
root@debian9:/home/joke/proxy# reboot
4.19. After checking the work of the proxy after the reboot, you can see the logs. This completes the proxy server setup.
3 proxy log1542573996.018 PROXY.8080 00000 tester 192.168.23.10:50915 217.12.15.54:443 1193 6939 0 CONNECT_ads.yahoo.com:443_HTTP/1.1
1542574289.634 SOCK5.3128 00000 tester 192.168.23.10:51193 54.192.13.69:443 0 0 0 CONNECT_normandy.cdn.mozilla.net:443
Setting up and running the Transparent Proxy NAT configuration
In this configuration, all devices on the internal network will transparently work on the Internet through a remote proxy server. Absolutely all tcp connections will be redirected to one or several (really expands the channel width, configuration example No. 2!) proxy servers. The DNS service will use 3proxy (dnspr) capabilities. UDP will not βgoβ outside, since we do not yet use the forward mechanism (disabled by default in the Linux kernel).
1. It's time to enable the enp0s8 interface
root@debian9:~# nano /etc/network/interfaces
/etc/network/interfaces file# This file describes the network interfaces available on your system
# And how to activate Them. For more information, see interfaces (5).
source /etc/network/interfaces.d/*
# The loopback network interface
the car
iface lo inet loopback
# The primary network interface
allow-hotplug enp0s3
iface enp0s3 inet dhcp
# The secondary network interface
allow-hotplug enp0s8
iface enp0s8 inet static
address 192.168.201.254
netmask 255.255.255.0
Here we assigned the enp0s8 interface a static address 192.168.201.254 and a mask 255.255.255.0
Save config Ctrl+X and reboot
root@debian9:~# reboot
2. Checking interfaces
root@debian9:~# ifconfig
ifconfig logenp0s3: flags=4163 mtu 1500
inet 192.168.23.11 netmask 255.255.255.0 broadcast 192.168.23.255
inet6 fe80::a00:27ff:fec2:bae4 prefixlen 64 scopeid 0x20 ether 08:00:27:c2:ba:e4 txqueuelen 1000 (Ethernet)
RX packets 61 bytes 7873 (7.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 65 bytes 10917 (10.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s8: flags=4163 mtu 1500
inet 192.168.201.254 netmask 255.255.255.0 broadcast 192.168.201.255
inet6 fe80::a00:27ff:fe79:a7e3 prefixlen 64 scopeid 0x20 ether 08:00:27:79:a7:e3 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8 bytes 648 (648.0B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
3. Everything worked out, now you need to configure 3proxy for transparent proxying.
root@debian9:~# cd /home/joke/proxy/
root@debian9:/home/joke/proxy# cat > 3proxytransp.conf
Transparent Proxy Configuration Example #1daemon
pidfile /home/joke/proxy/3proxy.pid
server 8.8.8.8
nscache 65536
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- +_L%t.%. %N.%p %E %U %C:%c %R:%r %O %I %h %T"
rotate 3
flush
auth uniquely
dnspr
allow *
parent 1000 socks5 IP_ADDRESS_EXTERNAL_PROXY 3128 tester 1234
plugin /opt/proxy/3proxy-0.8.12/src/TransparentPlugin.ld.so transparent_plugin
tcppm -i0.0.0.0 888 127.0.0.1 11111
4. Now run 3proxy with new config
root@debian9:/home/joke/proxy# /usr/local/bin/3proxy /home/joke/proxy/3proxytransp.conf
5. Add to crontab again
root@debian9:/home/joke/proxy# crontab -e
@reboot /usr/local/bin/3proxy /home/joke/proxy/3proxytransp.conf
6. Let's see what our proxy is listening to now
root@debian9:~# netstat -nlp
netstat logActive Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID / Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 349/sshd
tcp 0 0 0.0.0.0:888 0.0.0.0:* LISTEN 354/3proxy
tcp6 0 0 :::22 :::* LISTEN 349/sshd
udp 0 0 0.0.0.0:53 0.0.0.0:* 354/3proxy
udp 0 0 0.0.0.0:68 0.0.0.0:* 367/dhclient
7. Now the proxy is ready to accept any TCP connections on port 888, DNS on port 53, in order to redirect them to remote socks5 - proxy and DNS Google 8.8.8.8. It remains for us to configure the netfilter (iptables) and DHCP rules for issuing addresses.
8. Install the iptables-persistent and dhcpd package
root@debian9:~# apt-get install iptables-persistent isc-dhcp-server
9. Edit the dhcpd startup file
root@debian9:~# nano /etc/dhcp/dhcpd.conf
dhcpd.conf#dhcpd.conf
#
# Sample configuration file for ISC dhcpd
#
# option definitions common to all supported networksβ¦
option domain-name "example.org";
option domain-name-servers ns1.example.org, ns2.example.org;
default-lease-time 600;
max-lease-time 7200;
ddns-update-style none;
# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;
# A slightly different configuration for an internal subnet.
subnet 192.168.201.0 netmask 255.255.255.0 {
range 192.168.201.10 192.168.201.250;
option domain-name-servers 192.168.201.254;
option routers 192.168.201.254;
option broadcast-address 192.168.201.255;
default-lease-time 600;
max-lease-time 7200;
}
11. Reboot and check the service on port 67
root@debian9:~# reboot
root@debian9:~# netstat -nlp
netstat logActive Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID / Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 389/sshd
tcp 0 0 0.0.0.0:888 0.0.0.0:* LISTEN 310/3proxy
tcp6 0 0 :::22 :::* LISTEN 389/sshd
udp 0 0 0.0.0.0:20364 0.0.0.0:* 393/dhcpd
udp 0 0 0.0.0.0:53 0.0.0.0:* 310/3proxy
udp 0 0 0.0.0.0:67 0.0.0.0:* 393/dhcpd
udp 0 0 0.0.0.0:68 0.0.0.0:* 405/dhclient
udp6 0 0 :::31728 :::* 393/dhcpd
raw 0 0 0.0.0.0:1 0.0.0.0:* 393/dhcpd
12. It remains to redirect all tcp requests to port 888 and save the rule in iptables
root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.201.0/24 -p tcp -j REDIRECT --to-ports 888
root@debian9:~# iptables-save > /etc/iptables/rules.v4
13. To expand the bandwidth of the channel, you can use several proxy servers at once. The total amount should be 1000. New connections are established with a probability of 0.2, 0.2, 0.2, 0.2, 0,1, 0,1 to the specified proxy servers.
Note: if we have a web proxy, then instead of socks5 you need to write connect, if socks4, then socks4 (socks4 DOES NOT SUPPORT LOGIN / PASSWORD AUTHORIZATION!)
Transparent Proxy Configuration Example #2daemon
pidfile /home/joke/proxy/3proxy.pid
server 8.8.8.8
nscache 65536
maxconn 500
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- +_L%t.%. %N.%p %E %U %C:%c %R:%r %O %I %h %T"
rotate 3
flush
auth uniquely
dnspr
allow *
parent 200 socks5 IP_ADDRESS_EXTERNAL_PROXY#1 3128 tester 1234
parent 200 socks5 IP_ADDRESS_EXTERNAL_PROXY#2 3128 tester 1234
parent 200 socks5 IP_ADDRESS_EXTERNAL_PROXY#3 3128 tester 1234
parent 200 socks5 IP_ADDRESS_EXTERNAL_PROXY#4 3128 tester 1234
parent 100 socks5 IP_ADDRESS_EXTERNAL_PROXY#5 3128 tester 1234
parent 100 socks5 IP_ADDRESS_EXTERNAL_PROXY#6 3128 tester 1234
plugin /opt/proxy/3proxy-0.8.12/src/TransparentPlugin.ld.so transparent_plugin
tcppm -i0.0.0.0 888 127.0.0.1 11111
Setting up and running the NAT + Transparent Proxy configuration
In this configuration, we will use the usual NAT mechanism with selective or full transparent proxying of individual addresses or subnets. Users of the internal network will work with certain services / subnets without even realizing that they are working through a proxy. All https connections work fine, no certificates need to be generated / replaced.
First, let's decide which subnets / services we want to proxy. Let's assume that external proxies are located where a service like pandora.com is running. Now it remains to determine its subnets / addresses.
1. Ping
root@debian9:~# ping pandora.com
PING pandora.com (208.85.40.20) 56(84) bytes of data.
2. We type in Google BGP 208.85.40.20
Go to website
It can be seen that the subnet I am looking for is AS40428 Pandora Media, Inc.
Opening prefixes v4
Here are the required subnets!
199.116.161.0/24
199.116.162.0/24
199.116.164.0/23
199.116.164.0/24
199.116.165.0/24
208.85.40.0/24
208.85.41.0/24
208.85.42.0/23
208.85.42.0/24
208.85.43.0/24
208.85.44.0/24
208.85.46.0/23
208.85.46.0/24
208.85.47.0/24
3. To reduce the number of subnets, you need to perform aggregation. Let's go to the site and copy our list there. As a result - 6 subnets instead of 14.
199.116.161.0/24
199.116.162.0/24
199.116.164.0/23
208.85.40.0/22
208.85.44.0/24
208.85.46.0/23
4. Clean up iptables rules
root@debian9:~# iptables -F
root@debian9:~# iptables -X
root@debian9:~# iptables -t nat -F
root@debian9:~# iptables -t nat -X
Enable the forward mechanism and NAT
root@debian9:~# echo 1 > /proc/sys/net/ipv4/ip_forward
root@debian9:~# iptables -A FORWARD -i enp0s3 -o enp0s8 -j ACCEPT
root@debian9:~# iptables -A FORWARD -i enp0s8 -o enp0s3 -j ACCEPT
root@debian9:~# iptables -t nat -A POSTROUTING -o enp0s3 -s 192.168.201.0/24 -j MASQUERADE
In order for forward to be enabled permanently after reboot, we will change the file
root@debian9:~# nano /etc/sysctl.conf
And uncomment the line
net.ipv4.ip_forward = 1
Ctrl+X to save file
5. Wrap pandora.com subnets in a proxy
root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.201.0/24 -d 199.116.161.0/24,199.116.162.0/24,199.116.164.0/23,208.85.40.0/22,208.85.44.0/24,208.85.46.0/23 -p tcp -j REDIRECT --to-ports 888
6. Save the rules
root@debian9:~# iptables-save > /etc/iptables/rules.v4
Setting up and running the Transparent Proxy via router configuration
In this configuration, the transparent proxy server can be a separate PC or a virtual machine behind a home/corporate router. It is enough to register static routes on the router or devices, and the entire subnet will use the proxy without the need for any additional settings.
IMPORTANT! It is necessary that our gateway receive a static IP from the router, or be configured for static itself.
1. Set up a static gateway address (adapter enp0s3)
root@debian9:~# nano /etc/network/interfaces
/etc/network/interfaces file# This file describes the network interfaces available on your system
# And how to activate Them. For more information, see interfaces (5).
source /etc/network/interfaces.d/*
# The loopback network interface
the car
iface lo inet loopback
# The primary network interface
allow-hotplug enp0s3
iface enp0s3 inet static
address 192.168.23.2
netmask 255.255.255.0
192.168.23.254 gateway
# The secondary network interface
allow-hotplug enp0s8
iface enp0s8 inet static
address 192.168.201.254
netmask 255.255.255.0
2. Allow devices from the 192.168.23.0/24 subnet to use proxying
root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.23.0/24 -d 199.116.161.0/24,199.116.162.0/24,199.116.164.0/23,208.85.40.0/22,208.85.44.0/24,208.85.46.0/23 -p tcp -j REDIRECT --to-ports 888
3. Save the rules
root@debian9:~# iptables-save > /etc/iptables/rules.v4
4. Let's write subnets on the router
Router network list199.116.161.0 255.255.255.0 192.168.23.2
199.116.162.0 255.255.255.0 192.168.23.2
199.116.164.0 255.255.254.0 192.168.23.2
208.85.40.0 255.255.252.0 192.168.23.2
208.85.44.0 255.255.255.0 192.168.23.2
208.85.46.0 255.255.254.0 192.168.23.2
Used materials/resources
1. Official website of the 3proxy program
2. Instructions for installing 3proxy from sources
3. 3proxy developer branch on GitHub
Source: habr.com