The eXpress Data Path (XDP) technology allows arbitrary traffic processing on Linux interfaces before packets reach the network stack of the kernel. XDP applications include protection against DDoS attacks (CloudFlare), complex filtering, and statistics collection (Netflix). XDP programs run in the eBPF virtual machine, thus they have limitations on their code and the kernel functions available depending on the type of filter.
This article aims to address the shortcomings of numerous materials on XDP. Firstly, they provide ready-made code that immediately bypasses the features of XDP: prepared for verification or too simple to cause problems. When trying to write your own code from scratch later, there's no understanding of how to handle characteristic errors. Secondly, it does not cover ways to locally test XDP without VMs and hardware, despite having their own pitfalls. The text is aimed at programmers familiar with networks and Linux, who are interested in XDP and eBPF.
In this section, we will detail how to assemble an XDP filter and how to test it, after which we will write a simple version of the well-known SYN cookies mechanism at the packet processing level. For now, we will not form a 'whitelist' of
verified clients, keep counters, and manage the filter ā logs will be sufficient.
We will write in C ā it's not trendy, but practical. All the code is available on GitHub at the link at the end and is broken down into commits based on the stages described in the article.
Disclaimer. Throughout the article, a mini-solution will be developed to mitigate DDoS attacks, as this is a realistic task for XDP and my area of expertise. However, the main goal is to understand the technology; this is not a guide to creating ready-made protection. The educational code is not optimized and skips some nuances.
Brief overview of XDP
I will only outline the key points so as not to duplicate the documentation and existing articles.
Thus, a filter code is loaded into the kernel. Incoming packets are passed to the filter. Ultimately, the filter must decide whether to pass the packet into the kernel (XDP_PASS), drop the packet (XDP_DROP), or send it back (XDP_TX). The filter can modify the packet, which is especially relevant forĀ XDP_TX. It is also possible to abort the program (XDP_ABORTED) and drop the packet, but this is analogous to assert(0)Ā ā for debugging.
The eBPF (extended Berkley Packet Filter) virtual machine is designed to be simple, allowing the kernel to verify that the code does not loop endlessly or damage other memory. Aggregate limitations and checks:
- Backtracking is prohibited.
- There is a stack for data, but no functions (all C functions must be inlined).
- Accessing memory outside of the stack and packet buffer is prohibited.
- Code size is limited, but this is not very significant in practice.
- Only calls to special kernel functions (eBPF helpers) are allowed.
The development and installation of a filter looks like this:
- The source code (for example,
kernel.c) is compiled into an object file (kernel.o) for the architecture of the eBPF virtual machine. As of October 2019, compiling to eBPF is supported by Clang and promised in GCC 10.1. - If this object code contains accesses to kernel structures (like tables and counters), zeros are placed instead of their IDs, meaning such code cannot be executed. Before loading into the kernel, these zeros must be replaced with the IDs of specific objects created via kernel calls (linking the code). This can be done with external utilities, or a program can be written to link and load a specific filter.
- The kernel verifies the program being loaded. It checks for the absence of loops and out-of-bounds accesses to the packet and stack. If the verifier cannot prove that the code is correct, the program is rejected ā one must be able to please it.
- After successful verification, the kernel compiles the object code of the eBPF architecture into machine code for the system architecture (just-in-time).
- The program is attached to the interface and starts processing packets.
Since XDP operates in the kernel, debugging is done through trace logs and, in fact, through the packets that the program filters or generates. Nevertheless, eBPF ensures the safety of the loaded code for the system, so one can experiment with XDP directly on local Linux.
Preparing the Environment
Building
Clang cannot directly produce object code for the eBPF architecture, so the process consists of two steps:
- Compile the C code to LLVM bytecode (
clang -emit-llvm). - ) Convert the bytecode to eBPF object code (
llc -march=bpf -filetype=obj).
When writing a filter, a couple of files with helper functions and macros from the kernel tests are useful. KVER). We download them tohelpers/Ā helpers/:
export KVER=v5.3.7
export BASE=https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/plain/tools/testing/selftests/bpf
wget -P helpers --content-disposition "${BASE}/bpf_helpers.h?h=${KVER}" "${BASE}/bpf_endian.h?h=${KVER}"
unset KVER BASEMakefile for Arch Linux (kernel 5.3.7):
CLANG ?= clang
LLC ?= llc
KDIR ?= /lib/modules/$(shell uname -r)/build
ARCH ?= $(subst x86_64,x86,$(shell uname -m))
CFLAGS =
-Ihelpers
-I$(KDIR)/include
-I$(KDIR)/include/uapi
-I$(KDIR)/include/generated/uapi
-I$(KDIR)/arch/$(ARCH)/include
-I$(KDIR)/arch/$(ARCH)/include/generated
-I$(KDIR)/arch/$(ARCH)/include/uapi
-I$(KDIR)/arch/$(ARCH)/include/generated/uapi
-D__KERNEL__
-fno-stack-protector -O2 -g
xdp_%.o: xdp_%.c Makefile
$(CLANG) -c -emit-llvm $(CFLAGS) $< -o - |
$(LLC) -march=bpf -filetype=obj -o $@
.PHONY: all clean
all: xdp_filter.o
clean:
rm -f ./*.oKDIR contains the path to the kernel headers, ARCHĀ ā system architecture. Paths and tools may vary slightly between distributions.
Example differences for Debian 10 (kernel 4.19.67)
# Š“ŃŃŠ³Š°Ń ŠŗŠ¾Š¼Š°Š½Š“Š°
CLANG ?= clang
LLC ?= llc-7
# Š“ŃŃŠ³Š¾Š¹ ŠŗŠ°ŃŠ°Š»Š¾Š³
KDIR ?= /usr/src/linux-headers-$(shell uname -r)
ARCH ?= $(subst x86_64,x86,$(shell uname -m))
# Š“Š²Š° Š“Š¾ŠæŠ¾Š»Š½ŠøŃŠµŠ»ŃŠ½ŃŃ
ŠŗŠ°ŃŠ°Š»Š¾Š³Š° -I
CFLAGS =
-Ihelpers
-I/usr/src/linux-headers-4.19.0-6-common/include
-I/usr/src/linux-headers-4.19.0-6-common/arch/$(ARCH)/include
# Š“Š°Š»ŠµŠµ Š±ŠµŠ· ŠøŠ·Š¼ŠµŠ½ŠµŠ½ŠøŠ¹CFLAGS include the directory with auxiliary headers and several directories with kernel headers. The symbol __KERNEL__ indicates that UAPI (userspace API) headers are defined for kernel code, as the filter runs in the kernel.
Stack protection can be disabled (-fno-stack-protector), because the eBPF code verifier checks for stack out-of-bounds access anyway. Optimizations should be enabled immediately, as the size of the eBPF bytecode is limited.
Let's start with a filter that passes all packets and does nothing:
#include <uapi/linux/bpf.h>
#include <bpf_helpers.h>
SEC("prog")
int xdp_main(struct xdp_md* ctx) {
return XDP_PASS;
}
char _license[] SEC("license") = "GPL";The command make data about the window subsystem ('Wayland', 'Wayland/drm', 'x11') xdp_filter.o. Where should we test it now?
Test Stand
The setup should include two interfaces: one where the filter will be and from which packets will be sent. These must be full-fledged Linux devices with their own IPs to check how regular applications work with our filter.
Devices like veth (virtual Ethernet) will work for us: this is a pair of virtual network interfaces directly 'connected' to each other. They can be created like this (in this section, all commands ip are performed fromĀ root):
ip link add xdp-remote type veth peer name xdp-localHere xdp-remote andĀ xdp-localĀ ā the names of the devices. OnĀ xdp-local (192.0.2.1/24) the filter will be attached, whileĀ xdp-remote (192.0.2.2/24) will send incoming traffic. However, there's a problem: the interfaces are on the same machine, and Linux won't send traffic from one to the other. This can be solved using clever rules iptables, but they would have to modify packets, which is inconvenient for debugging. It's better to use network namespaces (netns).
The network namespace contains a set of interfaces, routing tables, and NetFilter rules, isolated from similar objects in other netns. Each process operates in some namespace and has access only to the objects of that netns. By default, there is a single network namespace for all objects in the system, so one can work in Linux without knowing about netns.
Let's create a new namespace xdp-test and move it there xdp-remote.
ip netns add xdp-test
ip link set dev xdp-remote netns xdp-testThen the process running inĀ xdp-test, will not "see" xdp-local (it will remain in the default netns) and when sending a packet to 192.0.2.1, it will pass throughĀ xdp-remote, because this is the only interface in 192.0.2.0/24 available to this process. This applies in the opposite direction as well.
When moving between netns, the interface is dropped and loses its address. To configure the interface in netns, you need to run ip ... in this namespace command ip netns exec:
ip netns exec xdp-test
ip address add 192.0.2.2/24 dev xdp-remote
ip netns exec xdp-test
ip link set xdp-remote upAs you can see, this is no different from configuring xdp-local in the default namespace:
ip address add 192.0.2.1/24 dev xdp-local
ip link set xdp-local upIf you run tcpdump -tnevi xdp-local, you can see that the packets sent fromĀ xdp-test, are delivered to this interface:
ip netns exec xdp-test ping 192.0.2.1It's convenient to launch a shell inĀ xdp-test. There is a script in the repository that automates working with the environment, for example, you can set up the environment with the command sudo ./stand up and take it down sudo ./stand down.
Tracing
The filter is attached to the device as follows:
ip -force link set dev xdp-local xdp object xdp_filter.o verboseā some characteristic of the node (for example, a number). The key is needed to identify the element of the tree corresponding to this key. Example of a binary search tree: -force is needed to bind a new program if another is already bound. "No news is good news" does not apply to this command, output is bulky in any case. Specifying verbose is optional, but with it comes a report from the code verifier with the assembler listing:
Verifier analysis:
0: (b7) r0 = 2
1: (95) exitUnbinding the program from the interface:
ip link set dev xdp-local xdp offIn the script, these commands are sudo ./stand attach andĀ sudo ./stand detach.
Once the filter is attached, you can ensure thatĀ ping continues to work, but does the program? Let's add logs. The function is similar toĀ printf(), but it supports only up to three arguments besides the template and a limited list of specifiers. The macro bpf_printk() simplifies the call.
SEC("prog")
int xdp_main(struct xdp_md* ctx) {
+ bpf_printk("got packet: %pn", ctx);
return XDP_PASS;
}The output goes to the kernel tracing channel that needs to be enabled:
echo -n 1 | sudo tee /sys/kernel/debug/tracing/options/trace_printkViewing the message stream:
cat /sys/kernel/debug/tracing/trace_pipeBoth of these commands make a call sudo ./stand log.
Ping should now generate such messages in it:
-110930 [004] ..s1 78803.244967: 0: got packet: 00000000ac510377If you look closely at the verifier output, you can notice strange calculations:
0: (bf) r3 = r1
1: (18) r1 = 0xa7025203a7465
3: (7b) *(u64 *)(r10 -8) = r1
4: (18) r1 = 0x6b63617020746f67
6: (7b) *(u64 *)(r10 -16) = r1
7: (bf) r1 = r10
8: (07) r1 += -16
9: (b7) r2 = 16
10: (85) call bpf_trace_printk#6The thing is that eBPF programs donāt have a data section, so the only way to encode the format string is through immediate arguments of VM commands:
$ python -c "import binascii; print(bytes(reversed(binascii.unhexlify('0a7025203a74656b63617020746f67'))))"
b'got packet: %pn'For this reason, the debug output greatly inflates the final code.
Sending XDP packets
Let's change the filter: let it send all incoming packets back. This is incorrect from a networking perspective, as one would need to change the addresses in the headers, but for now, the main thing is just getting it to work.
bpf_printk("got packet: %pn", ctx);
- return XDP_PASS;
+ return XDP_TX;
}Launch tcpdump toĀ xdp-remote. It should show identical outgoing and incoming ICMP Echo Request and stop showing ICMP Echo Reply. But it does not show. It turns out that for it to work XDP_TX in the program on xdp-local , the paired interface xdp-remote must also have a program assigned, even if itās empty, and it must be up.
How did I find this out?
allows the perf events mechanism, which incidentally uses the same virtual machine, so eBPF is applied for debugging eBPF.
You must do good out of evil, as there is no other way to do it.
$ sudo perf trace --call-graph dwarf -e 'xdp:*'
0.000 ping/123455 xdp:xdp_bulk_tx:ifindex=19 action=TX sent=0 drops=1 err=-6
veth_xdp_flush_bq ([veth])
veth_xdp_flush_bq ([veth])
veth_poll ([veth])What is error code 6?
$ errno 6
ENXIO 6 No such device or addressFunction veth_xdp_flush_bq() gets the error code fromĀ veth_xdp_xmit(), where we search forĀ ENXIO and find the comment.
We will restore the minimal filter (XDP_PASS) in the file xdp_dummy.c, add it to the Makefile, bind it toĀ xdp-remote:
ip netns exec remote
ip link set dev int xdp object dummy.oNow tcpdump shows what is expected:
62:57:8e:70:44:64 > 26:0e:25:37:8f:96, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 13762, offset 0, flags [DF], proto ICMP (1), length 84)
192.0.2.2 > 192.0.2.1: ICMP echo request, id 46966, seq 1, length 64
62:57:8e:70:44:64 > 26:0e:25:37:8f:96, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 13762, offset 0, flags [DF], proto ICMP (1), length 84)
192.0.2.2 > 192.0.2.1: ICMP echo request, id 46966, seq 1, length 64If only ARP is displayed instead, you need to remove the filters (this does sudo ./stand detach), let ping, then set the filters and try again. The problem is that the filter XDP_TX also applies to ARP, and if the namespace
has āforgottenā the MAC address 192.0.2.1, it will not be able to resolve this IP. xdp-test Let's move on to the task at hand: to write an XDP mechanism for SYN cookies.
Task Definition
The SYN flood remains a popular DDoS attack, the essence of which is as follows. When establishing a connection (TCP handshake), the server receives a SYN, allocates resources for the future connection, responds with a SYNACK packet, and waits for an ACK. The attacker simply sends SYN packets from spoofed addresses in the thousands per second, from each host in a botnet of thousands. The server is forced to allocate resources immediately upon packet arrival, and frees them after a long timeout, resulting in exhausted memory or limits, and new connections are not accepted, rendering the service unavailable.
If resources are not allocated per SYN packet, but only a SYNACK packet is sent in response, how can the server know that the ACK packet, which arrives later, corresponds to the SYN packet that was not saved? After all, the attacker can also generate fake ACKs. The essence of SYN cookies is to encode in
seqnumĀ connection parameters as a hash of the addresses, ports, and changing salt. If the ACK arrives before the salt change, it can recalculate the hash and compare it to acknumĀ . The attacker cannot spoof this, as the salt includes a secret, and they won't be able to brute-force it due to the limited bandwidth.SYN cookies have long been implemented in the Linux kernel and may even be automatically activated if SYN packets arrive too quickly and in large volumes. . The attacker cannot spoof this, as the salt includes a secret, and they won't be able to brute-force it due to the limited bandwidth. A brief overview of TCP handshake
TCP ensures the transmission of data as a stream of bytes; for example, HTTP requests are sent over TCP. The stream is sent in chunks within packets. All TCP packets have logical flags and 32-bit sequence numbers:
A Guide to TCP Handshake
TCP ensures data transmission as a stream of bytes; for example, HTTP requests are transmitted over TCP. The stream is sent in chunks as packets. All TCP packets have logical flags and 32-bit sequence numbers:
The combination of flags determines the role of a specific packet. The SYN flag indicates that this is the sender's first packet in the connection. The ACK flag indicates that the sender has received all connection data up to the byte.
. The attacker cannot spoof this, as the salt includes a secret, and they won't be able to brute-force it due to the limited bandwidth.. A packet may have multiple flags and is named based on their combination, for example, a SYNACK packet.The sequence number (seqnum) defines the offset in the data stream for the first byte transmitted in this packet. For instance, if the first packet with X bytes of data had this number as N, in the next packet with new data it will be N+X. At the beginning of the connection, each side chooses this number randomly.
The acknowledgment number (acknum) is the same offset as seqnum, but it defines not the number of the transmitted byte, but the number of the first byte from the recipient that the sender has not seen.
At the beginning of the connection, the parties must agree connection parameters as a hash of the addresses, ports, and changing salt. If the ACK arrives before the salt change, it can recalculate the hash and compare it to andĀ . The attacker cannot spoof this, as the salt includes a secret, and they won't be able to brute-force it due to the limited bandwidth.. The client sends a SYN packet with its seqnum = X. The server responds with a SYNACK packet, where it writes its seqnum = Y and sets acknum = X + 1. The client responds to the SYNACK with an ACK packet, whereĀ seqnum = X + 1, acknum = Y + 1. After that, data transmission begins.
If the peer does not acknowledge the receipt of the packet, TCP will resend it after a timeout.
Why are SYN cookies not used all the time?
Firstly, if the SYNACK or ACK is lost, it will be necessary to wait for a retransmission, slowing down the connection establishment. Secondly, the SYN packetā and only it!ācarries a number of options that affect the further operation of the connection. By not remembering incoming SYN packets, the server thus ignores these options, and in subsequent packets, the client will not send them. TCP can still operate in this case, but at least at the initial stage, the quality of the connection will decrease.
From the point of view of packets, the XDP program must do the following:
- respond to SYN with a SYNACK containing a cookie;
- respond to ACK with RST (terminate the connection);
- drop other packets.
Pseudocode of the algorithm along with packet parsing:
If this is not Ethernet,
skip the packet.
If this is not IPv4,
skip the packet.
If the address is in the verification table, (*)
decrease the remaining checks counter,
skip the packet.
If this is not TCP,
drop the packet. (**)
If this is SYN,
respond with SYN-ACK and cookie.
If this is ACK,
if the acknum does not contain a cookie,
drop the packet.
Record the address with N remaining checks. (*)
Respond with RST. (**)
In other cases, drop the packet.One (*) the points marked where system state must be managed ā at the first stage, we can do without them, simply implementing the TCP handshake with the generation of SYN cookie as seqnum.
At the location (**), as long as we do not have a table, we will skip the packet.
Implementation of TCP handshake
Packet parsing and code verification
We will need network header structures: Ethernet (uapi/linux/if_ether.h), IPv4 (uapi/linux/ip.h) and TCP (uapi/linux/tcp.h). I couldn't connect the last one due to errors related toĀ atomic64_t, so I had to copy the necessary definitions into the code.
All functions that are allocated in C for readability should be inlined at the call site, as the eBPF verifier in the kernel prohibits backward jumps, meaning, effectively, loops and function calls.
#define INTERNAL static __attribute__((always_inline))Macro LOG() disables printing in the release build.
The program represents a pipeline of functions. Each takes a packet, in which the corresponding level header is allocated, for example, process_ether() expects that ether. Based on the analysis results of the fields, the function may pass the packet to the next level. The result of the function's work is the XDP action. For now, the SYN and ACK handlers are passing all packets.
struct Packet {
struct xdp_md* ctx;
struct ethhdr* ether;
struct iphdr* ip;
struct tcphdr* tcp;
};
INTERNAL int process_tcp_syn(struct Packet* packet) { return XDP_PASS; }
INTERNAL int process_tcp_ack(struct Packet* packet) { return XDP_PASS; }
INTERNAL int process_tcp(struct Packet* packet) { ... }
INTERNAL int process_ip(struct Packet* packet) { ... }
INTERNAL int
process_ether(struct Packet* packet) {
struct ethhdr* ether = packet->ether;
LOG("Ether(proto=0x%x)", bpf_ntohs(ether->h_proto));
if (ether->h_proto != bpf_ntohs(ETH_P_IP)) {
return XDP_PASS;
}
// B
struct iphdr* ip = (struct iphdr*)(ether + 1);
if ((void*)(ip + 1) > (void*)packet->ctx->data_end) {
return XDP_DROP; /* malformed packet */
}
packet->ip = ip;
return process_ip(packet);
}
SEC("prog")
int xdp_main(struct xdp_md* ctx) {
struct Packet packet;
packet.ctx = ctx;
// A
struct ethhdr* ether = (struct ethhdr*)(void*)ctx->data;
if ((void*)(ether + 1) > (void*)ctx->data_end) {
return XDP_PASS;
}
packet.ether = ether;
return process_ether(&packet);
}I want to highlight the checks marked A and B. If A is commented out, the program will compile, but there will be a verification error upon loading:
Verifier analysis:
11: (7b) *(u64 *)(r10 -48) = r1
12: (71) r3 = *(u8 *)(r7 +13)
invalid access to packet, off=13 size=1, R7(id=0,off=0,r=0)
R7 offset is outside of the packet
processed 11 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
Error fetching program/map!Key line invalid access to packet, off=13 size=1, R7(id=0,off=0,r=0): there are execution paths where the thirteenth byte from the start of the buffer is outside the packet. The listing makes it somewhat difficult to understand which line is being referenced, but there is an instruction number (12) and a disassembler showing lines of the source code:
llvm-objdump -S xdp_filter.o | lessIn this case, it points to the line
LOG("Ether(proto=0x%x)", bpf_ntohs(ether->h_proto));indicating that the problem lies inĀ ether. It should always be like that.
Response to SYN
The goal at this stage is to form a correct SYNACK packet with a fixed connection parameters as a hash of the addresses, ports, and changing salt. If the ACK arrives before the salt change, it can recalculate the hash and compare it to, which will later be replaced by a SYN cookie. All changes occur inĀ process_tcp_syn() and its surroundings.
Packet check
Strangely enough, here is the most notable line, or rather the comment about it:
/* Required to verify checksum calculation */
const void* data_end = (const void*)ctx->data_end;When writing the first version of the code, a kernel 5.1 was used, for which there was a difference between data_end andĀ (const void*)ctx->data_end. When writing the article, kernel 5.3.1 had no such issue. Possibly, the compiler treated a local variable differently than a field. The moral isāat high nesting levels, simplifying code may help.
Next, routine length checks for the sake of the verifier; aboutĀ MAX_CSUM_BYTES below.
const u32 ip_len = ip->ihl * 4;
if ((void*)ip + ip_len > data_end) {
return XDP_DROP; /* malformed packet */
}
if (ip_len > MAX_CSUM_BYTES) {
return XDP_ABORTED; /* implementation limitation */
}
const u32 tcp_len = tcp->doff * 4;
if ((void*)tcp + tcp_len > (void*)ctx->data_end) {
return XDP_DROP; /* malformed packet */
}
if (tcp_len > MAX_CSUM_BYTES) {
return XDP_ABORTED; /* implementation limitation */
}Packet reversal
Filling connection parameters as a hash of the addresses, ports, and changing salt. If the ACK arrives before the salt change, it can recalculate the hash and compare it to and . The attacker cannot spoof this, as the salt includes a secret, and they won't be able to brute-force it due to the limited bandwidth., setting ACK (SYN is already set):
const u32 cookie = 42;
tcp->ack_seq = bpf_htonl(bpf_ntohl(tcp->seq) + 1);
tcp->seq = bpf_htonl(cookie);
tcp->ack = 1;Switching TCP ports, IP address, and MAC addresses. The standard library is not available from the XDP program, so memcpy()Ā is a macro that hides the Clang intrinsic.
const u16 temp_port = tcp->source;
tcp->source = tcp->dest;
tcp->dest = temp_port;
const u32 temp_ip = ip->saddr;
ip->saddr = ip->daddr;
ip->daddr = temp_ip;
struct ethhdr temp_ether = *ether;
memcpy(ether->h_dest, temp_ether.h_source, ETH_ALEN);
memcpy(ether->h_source, temp_ether.h_dest, ETH_ALEN);Checksum recalculation
IPv4 and TCP checksums require adding all 16-bit words in the headers, and the header sizes are recorded in them, meaning they are unknown at compile time. This is an issue because the verifier will not pass a regular loop to the variable boundary. However, the header size is limited: up to 64 bytes each. A loop with a fixed number of iterations can be made, which may end prematurely.
I note that there is on how to recalculate the checksum partially if only fixed words of packets have changed. However, this method is not universal, and its implementation would be harder to maintain.
Checksum calculation function:
#define MAX_CSUM_WORDS 32
#define MAX_CSUM_BYTES (MAX_CSUM_WORDS * 2)
INTERNAL u32
sum16(const void* data, u32 size, const void* data_end) {
u32 s = 0;
#pragma unroll
for (u32 i = 0; i < MAX_CSUM_WORDS; i++) {
if (2*i >= size) {
return s; /* normal exit */
}
if (data + 2*i + 1 + 1 > data_end) {
return 0; /* should be unreachable */
}
s += ((const u16*)data)[i];
}
return s;
}Despite the fact thatĀ size it is checked by the calling code, the second exit condition is necessary for the verifier to prove the completion of the loop.
For 32-bit words, a simpler version is implemented:
INTERNAL u32
sum16_32(u32 v) {
return (v >> 16) + (v & 0xffff);
}The actual recalculation of checksums and sending the packet back:
ip->check = 0;
ip->check = carry(sum16(ip, ip_len, data_end));
u32 tcp_csum = 0;
tcp_csum += sum16_32(ip->saddr);
tcp_csum += sum16_32(ip->daddr);
tcp_csum += 0x0600;
tcp_csum += tcp_len <check = 0;
tcp_csum += sum16(tcp, tcp_len, data_end);
tcp->check = carry(tcp_csum);
return XDP_TX;Function carry() makes a 32-bit sum of 16-bit words into a checksum, according to RFC 791.
TCP handshake check
The filter correctly establishes a connection withĀ netcat, skipping the final ACK, which Linux responded to with a RST packet, as the network stack did not receive SYNāit was converted to SYNACK and sent backāand from the OS point of view, the packet arrived that was not related to open connections.
$ sudo ip netns exec xdp-test nc -nv 192.0.2.1 6666
192.0.2.1 6666: Connection reset by peerIt is important to check with full-fledged applications and observe tcpdump toĀ xdp-remote because, for example, hping3 does not respond to incorrect checksums.
SYN cookie
From the XDP point of view, the check itself is trivial. The calculation algorithm is primitive and likely vulnerable to a sophisticated attacker. The Linux kernel, for example, uses the cryptographic SipHash, but its implementation for XDP clearly goes beyond the scope of this article.
New TODOs have emerged concerning external interaction:
The XDP program cannot store
cookie_seed(the secret part of the salt) in a global variable; it needs storage in the kernel, the value of which will be periodically updated from a reliable generator.When a SYN cookie matches in the ACK packet, it is necessary not to print a message but to remember the IP of the verified client, so that subsequent packets from them can be allowed.
Verification by a legitimate client:
$ sudo ip netns exec xdp-test nc -nv 192.0.2.1 6666
192.0.2.1 6666: Connection reset by peerThe logs recorded the verification passage (flags=0x2Ā ā this is SYN, flags=0x10Ā ā this is ACK):
Ether(proto=0x800)
IP(src=0x20e6e11a dst=0x20e6e11e proto=6)
TCP(sport=50836 dport=6666 flags=0x2)
Ether(proto=0x800)
IP(src=0xfe2cb11a dst=0xfe2cb11e proto=6)
TCP(sport=50836 dport=6666 flags=0x10)
cookie matches for client 20200c0While there is no list of verified IPs, there will be no protection against SYN flood itself, but here is the reaction to an ACK flood, initiated by a command like this:
sudo ip netns exec xdp-test hping3 --flood -A -s 1111 -p 2222 192.0.2.1Log entries:
Ether(proto=0x800)
IP(src=0x15bd11a dst=0x15bd11e proto=6)
TCP(sport=3236 dport=2222 flags=0x10)
cookie mismatchConclusion
Sometimes eBPF, and XDP in particular, is seen more as a tool for advanced administrators than as a platform for development. Indeed, XDP is an intervention tool for packet processing by the kernel, rather than an alternative to the kernel stack like DPDK and other kernel bypass options. On the other hand, XDP allows for implementing quite complex logic, which can easily be updated without a pause in traffic processing. The verifier does not create major problems; personally, I wouldn't mind having something like that for parts of userspace code.
In the second part, if the topic is of interest, we will complete the table of verified clients and connection breaks, implement counters, and write a userspace utility for managing the filter.
Links:
Source: habr.com
