🥇Basics of transparent proxying using 3proxy and iptables/netfilter or how to "route everything through a proxy"

This article aims to unveil the possibilities of transparent proxying, which allows redirecting all or part of the traffic through external proxy servers without any noticeable impact on clients.

When I began tackling this task, I encountered a significant problem — the HTTPS protocol. In the good old days, there weren't many issues with transparent HTTP proxying, but when proxying HTTPS, browsers report tampering with the protocol, and that’s where the fun ends.

Common instructions for the Squid proxy server even suggest generating your own certificate and installing it for clients, which is completely ridiculous and looks like a MITM attack at best. I know that Squid is capable of doing something similar, but this article discusses a proven and working method using 3proxy by the respected 3APA3A.

Next, we will take a detailed look at the process of building 3proxy from source, its configuration, complete and selective proxying using NAT, distributing bandwidth across multiple external proxy servers, as well as the use of a router and static routes. We will use Debian 9 x64 as our OS. Let's get started!

Installing 3proxy and starting a standard proxy server

1. Install ifconfig (from the net-tools package)
apt-get install net-tools
2. Install Midnight Commander
apt-get install mc
3. We now have 2 interfaces:
enp0s3 — external, looks to the internet
enp0s8 — internal, should look to the local network
In other Debian-based distributions, interfaces are usually named eth0 and eth1.
ifconfig -a

Interfacesenp0s3: flags=4163 mtu 1500
inet 192.168.23.11 netmask 255.255.255.0 broadcast 192.168.23.255
inet6 fe80::a00:27ff:fec2:bae4 prefixlen 64 scopeid 0x20 ether 08:00:27:c2:ba:e4 txqueuelen 1000 (Ethernet)
RX packets 6412 bytes 8676619 (8.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1726 bytes 289128 (282.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

enp0s8: flags=4098 mtu 1500
ether 08:00:27:79:a7:e3 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

The enp0s8 interface is currently unused; we will activate it when we want to use the Proxy NAT or NAT configuration. It is then logical to assign it a static IP.

4. Let's begin the installation of 3proxy

4.1 Installing essential packages to compile 3proxy from source

root@debian9:~# apt-get install build-essential libevent-dev libssl-dev -y

4.2. Create a folder to download the source archive

root@debian9:~# mkdir -p /opt/proxy

4.3. Navigate to this folder

root@debian9:~# cd /opt/proxy

4.4. Now let's download the latest 3proxy package. At the time of writing, the latest stable version was 0.8.12 (04/18/2018). We will download it from the official 3proxy site.

root@debian9:/opt/proxy# wget https://github.com/z3APA3A/3proxy/archive/0.8.12.tar.gz

4.5. Extract the downloaded archive

root@debian9:/opt/proxy# tar zxvf 0.8.12.tar.gz

4.6. Enter the unpacked directory to build the program

root@debian9:/opt/proxy# cd 3proxy-0.8.12

4.7. Next, we need to add a line to the header file to ensure our server is completely anonymous (this really works, everything has been verified, client IPs are hidden)

root@debian9:/opt/proxy/3proxy-0.8.12# nano +29 src/proxy.h

Add the line

#define ANONYMOUS 1
Press Ctrl+x and Enter to save the changes.

4.8. Let's start building the program

root@debian9:/opt/proxy/3proxy-0.8.12# make -f Makefile.Linux

Makelogmake[2]: Leaving directory '/opt/proxy/3proxy-0.8.12/src/plugins/TransparentPlugin'
make[1]: Leaving directory '/opt/proxy/3proxy-0.8.12/src'

No errors, let's continue.

4.9. Install the program in the system

root@debian9:/opt/proxy/3proxy-0.8.12# make -f Makefile.Linux install

4.10. Navigate to the root directory and check where the program was installed

root@debian9:/opt/proxy/3proxy-0.8.12# cd ~/ root@debian9:~# whereis 3proxy
3proxy: /usr/local/bin/3proxy /usr/local/etc/3proxy

4.11. Create a folder for configuration files and logs in the user's home directory

root@debian9:~# mkdir -p /home/joke/proxy/logs

4.12. Navigate to the directory where the config should be

root@debian9:~# cd /home/joke/proxy/

4.13. Create an empty file and copy the config there

root@debian9:/home/joke/proxy# cat > 3proxy.conf

3proxy.confdaemon
pidfile /home/joke/proxy/3proxy.pid
nserver 8.8.8.8
nscache 65536
users tester:CL:1234
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- +_L%t.%. %N.%p %E %U %C:%c %R:%r %O %I %h %T"
rotate 3
auth strong
flush
allow tester
socks -p3128
proxy -p8080

To save, press Ctrl + Z

4.14. Create a pid file to avoid errors on startup.

root@debian9:/home/joke/proxy# cat > 3proxy.pid

To save, press Ctrl + Z

4.15. Start the proxy server!

root@debian9:/home/joke/proxy# 3proxy /home/joke/proxy/3proxy.conf

4.16. Let's check if the server is listening on the ports

root@debian9:~/home/joke/proxy# netstat -nlp

netstat logActive Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 504/3proxy
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 338/sshd
tcp 0 0 0.0.0.0:3128 0.0.0.0:* LISTEN 504/3proxy
tcp6 0 0 :::22 :::* LISTEN 338/sshd
udp 0 0 0.0.0.0:68 0.0.0.0:* 352/dhclient

As mentioned in the config, the web proxy listens on port 8080, and the Socks5 proxy listens on 3128.

4.17. To enable automatic start of the proxy service after reboot, add it to cron.

root@debian9:/home/joke/proxy# crontab -e

Add the line

@reboot /usr/local/bin/3proxy /home/joke/proxy/3proxy.conf

Press Enter, as cron needs to see the end-of-line character, and save the file.

There should be a message indicating a new crontab installation.

crontab: installing new crontab

4.18. We'll reboot the system and try connecting through the browser to the proxy. For testing, we will use Firefox (for web proxy) and the FoxyProxy add-on for socks5 with authentication.

root@debian9:/home/joke/proxy# reboot

4.19. After checking the proxy functionality post-reboot, we can review the logs. This completes the proxy server setup.

3 proxy log1542573996.018 PROXY.8080 00000 tester 192.168.23.10:50915 217.12.15.54:443 1193 6939 0 CONNECT_ads.yahoo.com:443_HTTP/1.1
1542574289.634 SOCK5.3128 00000 tester 192.168.23.10:51193 54.192.13.69:443 0 0 0 CONNECT_normandy.cdn.mozilla.net:443

Setting up and launching Transparent Proxy NAT configuration

In this configuration, all devices in the internal network will transparently operate on the internet through a remote proxy server. All TCP connections will be redirected to one or several proxy servers (which effectively expands bandwidth, refer to configuration example #2!). The DNS service will utilize the capabilities of 3proxy (dnspr). UDP will not go outside as we are not using the forward mechanism (which is disabled by default in the Linux kernel).

1. It’s time to enable the enp0s8 interface.

root@debian9:~# nano /etc/network/interfaces

/etc/network/interfaces file# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug enp0s3
iface enp0s3 inet dhcp

# The secondary network interface
allow-hotplug enp0s8
iface enp0s8 inet static
address 192.168.201.254
netmask 255.255.255.0

Here we assigned a static address 192.168.201.254 and a mask 255.255.255.0 to the enp0s8 interface.
Save the config with Ctrl+X and reboot.

root@debian9:~# reboot

2. Checking the interfaces

root@debian9:~# ifconfig

ifconfig logenp0s3: flags=4163 mtu 1500
inet 192.168.23.11 netmask 255.255.255.0 broadcast 192.168.23.255
inet6 fe80::a00:27ff:fec2:bae4 prefixlen 64 scopeid 0x20 ether 08:00:27:c2:ba:e4 txqueuelen 1000 (Ethernet)
RX packets 61 bytes 7873 (7.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 65 bytes 10917 (10.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

enp0s8: flags=4163 mtu 1500
inet 192.168.201.254 netmask 255.255.255.0 broadcast 192.168.201.255
inet6 fe80::a00:27ff:fe79:a7e3 prefixlen 64 scopeid 0x20 ether 08:00:27:79:a7:e3 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8 bytes 648 (648.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

3. Everything worked, now we need to configure 3proxy for transparent proxying.

root@debian9:~# cd /home/joke/proxy/ root@debian9:/home/joke/proxy# cat > 3proxytransp.conf

Example configuration of transparent proxy server #1daemon
pidfile /home/joke/proxy/3proxy.pid
nserver 8.8.8.8
nscache 65536
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- +_L%t.%. %N.%p %E %U %C:%c %R:%r %O %I %h %T"
rotate 3
flush
auth iponly
dnspr
allow *
parent 1000 socks5 EXTERNAL_PROXY_IP 3128 tester 1234
plugin /opt/proxy/3proxy-0.8.12/src/TransparentPlugin.ld.so transparent_plugin
tcppm -i0.0.0.0 888 127.0.0.1 11111

4. Now we start 3proxy with the new config.
root@debian9:/home/joke/proxy# /usr/local/bin/3proxy /home/joke/proxy/3proxytransp.conf

5. We will add to crontab again.
root@debian9:/home/joke/proxy# crontab -e @reboot /usr/local/bin/3proxy /home/joke/proxy/3proxytransp.conf

6. Let's see what our proxy is listening to now.
root@debian9:~# netstat -nlp

netstat logActive Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 349/sshd
tcp 0 0 0.0.0.0:888 0.0.0.0:* LISTEN 354/3proxy
tcp6 0 0 :::22 :::* LISTEN 349/sshd
udp 0 0 0.0.0.0:53 0.0.0.0:* 354/3proxy
udp 0 0 0.0.0.0:68 0.0.0.0:* 367/dhclient

7. The proxy is now ready to accept any TCP connections on port 888 and DNS on port 53, to then redirect them to a remote socks5 proxy and Google's DNS 8.8.8.8. We just need to configure the netfilter (iptables) rules and DHCP for assigning addresses.

8. We'll install the iptables-persistent and dhcpd packages.

root@debian9:~# apt-get install iptables-persistent isc-dhcp-server

9. Let's edit the dhcpd startup file.
root@debian9:~# nano /etc/dhcp/dhcpd.conf

dhcpd.conf# dhcpd.conf
#
# Sample configuration file for ISC dhcpd
#

# option definitions common to all supported networks…
option domain-name "example.org";
option domain-name-servers ns1.example.org, ns2.example.org;

default-lease-time 600;
max-lease-time 7200;

ddns-update-style none;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.

authoritative;

# A slightly different configuration for an internal subnet.
subnet 192.168.201.0 netmask 255.255.255.0 {
range 192.168.201.10 192.168.201.250;
option domain-name-servers 192.168.201.254;
option routers 192.168.201.254;
option broadcast-address 192.168.201.255;
default-lease-time 600;
max-lease-time 7200;
}

11. We restart and check the service on port 67.
root@debian9:~# reboot root@debian9:~# netstat -nlp

netstat logActive Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 389/sshd
tcp 0 0 0.0.0.0:888 0.0.0.0:* LISTEN 310/3proxy
tcp6 0 0 :::22 :::* LISTEN 389/sshd
udp 0 0 0.0.0.0:20364 0.0.0.0:* 393/dhcpd
udp 0 0 0.0.0.0:53 0.0.0.0:* 310/3proxy
udp 0 0 0.0.0.0:67 0.0.0.0:* 393/dhcpd
udp 0 0 0.0.0.0:68 0.0.0.0:* 405/dhclient
udp6 0 0 :::31728 :::* 393/dhcpd
raw 0 0 0.0.0.0:1 0.0.0.0:* 393/dhcpd

12. We need to redirect all TCP requests to port 888 and save the rule in iptables.

root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.201.0/24 -p tcp -j REDIRECT --to-ports 888

root@debian9:~# iptables-save > /etc/iptables/rules.v4

13. To increase bandwidth, multiple proxy servers can be used simultaneously. The total should be 1000. New connections are established with probabilities of 0.2, 0.2, 0.2, 0.2, 0.1, 0.1 to the specified proxy servers.

Note: if we have a web proxy, then instead of socks5, we need to write connect; if it's socks4, we need socks4 (socks4 DOES NOT SUPPORT LOGIN/PASSWORD AUTHORIZATION!).

Example configuration of transparent proxy server #2.daemon
pidfile /home/joke/proxy/3proxy.pid
nserver 8.8.8.8
nscache 65536
maxconn 500
timeouts 1 5 30 60 180 1800 16 60
log /home/joke/proxy/logs/3proxy.log D
logformat "- +_L%t.%. %N.%p %E %U %C:%c %R:%r %O %I %h %T"
rotate 3
flush
auth iponly
dnspr
allow *

parent 200 socks5 EXTERNAL_PROXY_IP_1 3128 tester 1234
parent 200 socks5 EXTERNAL_PROXY_IP_2 3128 tester 1234
parent 200 socks5 EXTERNAL_PROXY_IP_3 3128 tester 1234
parent 200 socks5 EXTERNAL_PROXY_IP_4 3128 tester 1234
parent 100 socks5 EXTERNAL_PROXY_IP_5 3128 tester 1234
parent 100 socks5 EXTERNAL_PROXY_IP_6 3128 tester 1234

plugin /opt/proxy/3proxy-0.8.12/src/TransparentPlugin.ld.so transparent_plugin
tcppm -i0.0.0.0 888 127.0.0.1 11111

Configuration and startup of NAT + Transparent Proxy.

In this configuration, we will use the standard NAT mechanism with selective or full transparent proxying for specific addresses or subnets. Users of the internal network will interact with certain services/subnets without even realizing they are doing so through a proxy. All HTTPS connections work perfectly; no certificates need to be generated or replaced.

First, let's determine which subnets/services we want to proxy. Let's assume that the external proxy servers are located where a service like pandora.com operates. Now we need to identify its subnets/addresses.

1. We ping

root@debian9:~# ping pandora.com
PING pandora.com (208.85.40.20) 56(84) bytes of data.

2. We search in Google BGP 208.85.40.20

Go to the website bgp.he.net/net/208.85.40.0/24#_netinfo
It is clear that the subnet we are looking for is AS40428 Pandora Media, Inc

bgp.he.net/net/208.85.40.0/24#_netinfo

We open the v4 prefixes

bgp.he.net/AS40428#_prefixes

Here are the desired subnets!

199.116.161.0/24
199.116.162.0/24
199.116.164.0/23
199.116.164.0/24
199.116.165.0/24
208.85.40.0/24
208.85.41.0/24
208.85.42.0/23
208.85.42.0/24
208.85.43.0/24
208.85.44.0/24
208.85.46.0/23
208.85.46.0/24
208.85.47.0/24

3. To reduce the number of subnets, we need to perform aggregation. We go to the site ip-calculator.ru/aggregate and paste our list there. As a result — 6 subnets instead of 14.

199.116.161.0/24
199.116.162.0/24
199.116.164.0/23
208.85.40.0/22
208.85.44.0/24
208.85.46.0/23

4. Clear iptables rules

root@debian9:~# iptables -F root@debian9:~# iptables -X root@debian9:~# iptables -t nat -F root@debian9:~# iptables -t nat -X

Enable the forward and NAT mechanisms

root@debian9:~# echo 1 > /proc/sys/net/ipv4/ip_forward root@debian9:~# iptables -A FORWARD -i enp0s3 -o enp0s8 -j ACCEPT root@debian9:~# iptables -A FORWARD -i enp0s8 -o enp0s3 -j ACCEPT root@debian9:~# iptables -t nat -A POSTROUTING -o enp0s3 -s 192.168.201.0/24 -j MASQUERADE

To ensure that forwarding is enabled permanently after a reboot, we need to edit the file

root@debian9:~# nano /etc/sysctl.conf

And uncomment the line

net.ipv4.ip_forward = 1

Press Ctrl+X to save the file

5. We wrap the subnets of pandora.com in a proxy

root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.201.0/24 -d 199.116.161.0/24,199.116.162.0/24,199.116.164.0/23,208.85.40.0/22,208.85.44.0/24,208.85.46.0/23 -p tcp -j REDIRECT --to-ports 888

6. Save the rules

root@debian9:~# iptables-save > /etc/iptables/rules.v4

Configuration and launch of Transparent Proxy via router

In this configuration, the transparent proxy server can be a separate PC or a virtual machine behind a home/corporate router. It is enough to set static routes on the router or devices, and the entire subnet will use the proxy without the need for any additional settings.

IMPORTANT! It is necessary for our gateway to receive a static IP from the router, or to be configured for static itself.

1. Set up a static address for the gateway (adapter enp0s3)

root@debian9:~# nano /etc/network/interfaces

/etc/network/interfaces file# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug enp0s3
iface enp0s3 inet static
address 192.168.23.2
netmask 255.255.255.0
gateway 192.168.23.254

# The secondary network interface
allow-hotplug enp0s8
iface enp0s8 inet static
address 192.168.201.254
netmask 255.255.255.0

2. Allow devices from the 192.168.23.0/24 subnet to use proxying

root@debian9:~# iptables -t nat -A PREROUTING -s 192.168.23.0/24 -d 199.116.161.0/24,199.116.162.0/24,199.116.164.0/23,208.85.40.0/22,208.85.44.0/24,208.85.46.0/23 -p tcp -j REDIRECT --to-ports 888

3. Save the rules
root@debian9:~# iptables-save > /etc/iptables/rules.v4

4. Write down the subnets on the router

Router network list199.116.161.0 255.255.255.0 192.168.23.2
199.116.162.0 255.255.255.0 192.168.23.2
199.116.164.0 255.255.254.0 192.168.23.2
208.85.40.0 255.255.252.0 192.168.23.2
208.85.44.0 255.255.255.0 192.168.23.2
208.85.46.0 255.255.254.0 192.168.23.2