HTTP ãªã¯ãšã¹ãã®åŠçã§æ倧ã®ããã©ãŒãã³ã¹ãéæããããã® Linux ç°å¢ã®ãã¥ãŒãã³ã°ã«é¢ãã詳现ãªã¬ã€ããå ¬éãããŠããŸãã ææ¡ãããæ¹æ³ã«ãããAmazon EC2 ç°å¢ (4 vCPU) ã® libreactor ã©ã€ãã©ãªã«åºã¥ã JSON ããã»ããµã®ããã©ãŒãã³ã¹ããã«ãŒãã« 224 ã䜿çšãã Amazon Linux 2 ã®æšæºèšå®ã§ 4.14 ç§ããã 1.2 API ãªã¯ãšã¹ããã 436 äžãªã¯ãšã¹ã/ç§ã«åäžãããããšãã§ããŸãããæé©ååŸ 79 ç§ (XNUMX% å¢å )ããªã¯ãšã¹ãåŠçã®é 延ã XNUMX% æžå°ããŸããã ææ¡ãããæ¹æ³ã¯ libreactor ã«åºæã®ãã®ã§ã¯ãªããnginxãActixãNettyãNode.js ãªã©ã®ä»ã® http ãµãŒããŒã䜿çšããå Žåã«ãæ©èœããŸã (ãã¹ãã§ã¯ libreactor ãããŒã¹ã«ãããœãªã¥ãŒã·ã§ã³ã®æ¹ãããã©ãŒãã³ã¹ãåªããŠãããããlibreactor ã䜿çšãããŸãã)ã
åºæ¬çãªæé©å:
- ãªãã¬ã¢ã¯ã¿ãŒã³ãŒãã®æé©åã Techempower ãããã® R18 ãªãã·ã§ã³ãããŒã¹ãšããŠäœ¿çšããã䜿çšããã CPU ã³ã¢ã®æ°ãå¶éããã³ãŒããåé€ã㊠(æé©åã«ããäœæ¥ã 25 ïœ 27% é«éåãããŸãã)ãã-O3ã ãªãã·ã§ã³ã䜿çšã㊠GCC ã§ã¢ã»ã³ãã«ããããšã§æ¹åãããŸããã (5-10% ã®å¢å ) ããã³ "-march-native" (5-10%)ãèªã¿åã/æžã蟌ã¿åŒã³åºãã recv/send ã«çœ®ãæã (5-10%)ãpthread 䜿çšæã®ãªãŒããŒããããåæž (2-3%) ã ã³ãŒãæé©ååŸã®å šäœçãªããã©ãŒãã³ã¹ã¯ 55% åäžããã¹ã«ãŒããã㯠224k req/s ãã 347k req/s ã«å¢å ããŸããã
- ææ©çå®è¡ã®è匱æ§ã«å¯Ÿããä¿è·ãç¡å¹ã«ããŸãã ã«ãŒãã«ãããŒããããšãã«ãã©ã¡ãŒã¿ãŒãnospectre_v1 nospectre_v2 pti=off mds=off tsx_async_abort=offãã䜿çšãããšãããã©ãŒãã³ã¹ã 28% åäžããã¹ã«ãŒãããã 347k req/s ãã 446k req/s ã«å¢å ããŸããã ãããšã¯å¥ã«ããã©ã¡ãŒã¿ãnospectre_v1ãïŒSpectre v1 + SWAPGS ããã®ä¿è·ïŒããã®å¢å 㯠1 ïœ 2%ããnospectre_v2ãïŒSpectre v2 ããã®ä¿è·ïŒ - 15 ïœ 20%ããpti=offãïŒSpectre v3/MeltdownïŒã§ããã - 6 %ããmds=off tsx_async_abort=offã (MDS/Zombieload ããã³ TSX éåæäžæ¢) - 6%ã L1TF/ãã©ã¢ã·ã£ã㊠(l1tf=flush)ãiTLB ãã«ãããããææ©çã¹ã㢠ãã€ãã¹ãããã³ SRBDS æ»æã«å¯Ÿããä¿è·ã®èšå®ã¯å€æŽãããããã¹ããããæ§æ (KVM åºæããã¹ããããæ§æãªã©) ãšäº€å·®ããªãã£ããããããã©ãŒãã³ã¹ã«ã¯åœ±é¿ããŸããã§ãããä»®æ³åãªã©ã® CPU ã¢ãã«)ã
- ãauditctl -a Never,taskãã³ãã³ãã䜿çšããDocker ã³ã³ãããŒã®èµ·åæã«ã--security-opt seccomp=unconfinedããªãã·ã§ã³ãæå®ããŠãç£æ»ããã³ã·ã¹ãã ã³ãŒã« ããã㯠ã¡ã«ããºã ãç¡å¹ã«ããŸãã å šäœçãªããã©ãŒãã³ã¹ã¯ 11% åäžããã¹ã«ãŒããã㯠446k req/s ãã 495k req/s ã«å¢å ããŸããã
- é¢é£ããã«ãŒãã« ã¢ãžã¥ãŒã«ãã¢ã³ããŒãããŠãiptables/netfilter ãç¡å¹ã«ããŸãã ç¹å®ã®ãµãŒã㌠ãœãªã¥ãŒã·ã§ã³ã§äœ¿çšãããŠããªããã¡ã€ã¢ãŠã©ãŒã«ãç¡å¹ã«ãããšããèãã¯ãnf_hook_slow é¢æ°ã®å®è¡ã« 18% ã®æéãããã£ããšãããããã¡ã€ãªã³ã°çµæããå€æãããŸããã nftables 㯠iptables ãããå¹ççã«åäœããŸãããAmazon Linux ã¯åŒãç¶ã iptables ã䜿çšããããšã«æ³šæããŠãã ããã iptables ãç¡å¹ã«ããåŸãããã©ãŒãã³ã¹ã¯ 22% åäžããã¹ã«ãŒããã㯠495k req/s ãã 603k req/s ã«å¢å ããŸããã
- ç°ãªã CPU ã³ã¢éã§ã®ãã³ãã©ãŒã®ç§»è¡ãåæžããããã»ããµãŒ ãã£ãã·ã¥ã®äœ¿çšå¹çãåäžãããŸããã æé©åã¯ãlibreactor ããã»ã¹ã CPU ã³ã¢ã«ãã€ã³ãããã¬ãã« (CPU ãã³ãã³ã°) ãšãã«ãŒãã« ãããã¯ãŒã¯ ãã³ãã©ãŒã®ãã³ãã³ã° (åä¿¡åŽã¹ã±ãŒãªã³ã°) ã®äž¡æ¹ã®ã¬ãã«ã§å®è¡ãããŸããã ããšãã°ãirqbalance ãç¡å¹ã«ãªããCPU ãžã®ãã¥ãŒ ã¢ãã£ããã£ã /proc/irq/$IRQ/smp_affinity_list ã§æ瀺çã«èšå®ãããŸããã åã CPU ã³ã¢ã䜿çšããŠãlibreactor ããã»ã¹ãšåä¿¡ãã±ããã®ãããã¯ãŒã¯ ãã¥ãŒãåŠçããã«ã¯ãã«ã¹ã¿ã BPF ãã³ãã©ãŒã䜿çšããããœã±ããã®äœææã« SO_ATTACH_REUSEPORT_CBPF ãã©ã°ãèšå®ããããšã§æ¥ç¶ãããŸãã éä¿¡ãã±ããã®ãã¥ãŒã CPU ã«ãã€ã³ãããããã«ãèšå® /sys/class/net/eth0/queues/tx- ãå€æŽãããŸããã /xps_cpusã å šäœçãªããã©ãŒãã³ã¹ã¯ 38% åäžããã¹ã«ãŒããã㯠603k req/s ãã 834k req/s ã«å¢å ããŸããã
- å²ã蟌ã¿åŠçãšããŒãªã³ã°ã®äœ¿çšã®æé©åã ENA ãã©ã€ããŒã§adaptive-rx ã¢ãŒããæå¹ã«ããsysctl net.core.busy_read ãæäœãããšãããã©ãŒãã³ã¹ã 28% åäžããŸãã (ã¹ã«ãŒããã㯠834k req/s ãã 1.06M req/s ã«å¢å ããã¬ã€ãã³ã·ãŒã¯ 361ÎŒs ãã 292ÎŒs ã«æžå°ããŸãã)ã
- ãããã¯ãŒã¯ ã¹ã¿ãã¯å ã§äžå¿ èŠãªãããã¯ãåŒãèµ·ããã·ã¹ãã ãµãŒãã¹ãç¡å¹ã«ããŸãã dhclient ãç¡å¹ã«ã㊠IP ã¢ãã¬ã¹ãæåã§èšå®ãããšãããã©ãŒãã³ã¹ã 6% åäžããã¹ã«ãŒãããã 1.06M req/s ãã 1.12M req/s ã«å¢å ããŸããã dhclient ãããã©ãŒãã³ã¹ã«åœ±é¿ãäžããçç±ã¯ãçã®ãœã±ããã䜿çšãããã©ãã£ãã¯åæã«ãããŸãã
- ã¹ãã³ããã¯ãšã®æŠãã sysctlãnet.core.default_qdisc=noqueueãããã³ãtc qdisc replace dev eth0 root mqããä»ããŠãããã¯ãŒã¯ ã¹ã¿ãã¯ããnoqueueãã¢ãŒãã«åãæ¿ãããšãããã©ãŒãã³ã¹ã 2% åäžããã¹ã«ãŒãããã 1.12M req/s ãã 1.15M req/s ã«å¢å ããŸãããèŠæ±/ç§
- ã³ãã³ããethtool -K eth0 gro offãã䜿çšã㊠GRO (Generic Receive Offload) ãç¡å¹ã«ããsysctlãnet.ipv4.tcp_congestion_control=renoãã䜿çšããŠãã¥ãŒããã¯èŒ»èŒ³å¶åŸ¡ã¢ã«ãŽãªãºã ã reno ã«çœ®ãæãããªã©ãæåŸã®ãã€ããŒãªæé©åã å šäœçãªçç£æ§ã®åäžã¯ 4% ã§ããã ã¹ã«ãŒããã㯠1.15 äžèŠæ±/ç§ãã 1.2 äžèŠæ±/ç§ã«å¢å ããŸããã
ãã®èšäºã§ã¯ãå¹æããã£ãæé©åã«å ããŠãæåŸ ãããããã©ãŒãã³ã¹ã®åäžã«ã€ãªãããªãã£ãæ¹æ³ã«ã€ããŠã説æããŸãã ããšãã°ã次ã®ãããªããšã¯å¹æããªãããšãå€æããŸããã
- libreactor ãåå¥ã«å®è¡ããŠããã³ã³ãããŒå ã§å®è¡ããå Žåãšããã©ãŒãã³ã¹ã«éãã¯ãããŸããã§ããã writev ã send ã«çœ®ãæããepoll_wait ã® maxevents ãå¢ãããGCC ã®ããŒãžã§ã³ãšãã©ã°ãè©ŠããŠãå¹æã¯ãããŸããã§ãã (å¹æã¯ã-O3ãããã³ã-march-nativeããã©ã°ã§ã®ã¿é¡èã§ãã)ã
- Linux ã«ãŒãã«ãããŒãžã§ã³ 4.19 ããã³ 5.4 ã«ã¢ããã°ã¬ãŒãããSCHED_FIFO ããã³ SCHED_RR ã¹ã±ãžã¥ãŒã©ãŒã䜿çšããsysctl kernel.sched_min_granularity_nsãkernel.sched_wakeup_granularity_nsãtransparent_hugepages=neverãskew_tick=1ãããã³ Clocksource=tsc ãæäœããŠããããã©ãŒãã³ã¹ã«ã¯åœ±é¿ããŸããã§ããã
- ENA ãã©ã€ããŒã§ã¯ããªãããŒã ã¢ãŒã (ã»ã°ã¡ã³ããŒã·ã§ã³ãã¹ãã£ãã¿ãŒã®ã£ã¶ãŒãrx/tx ãã§ãã¯ãµã ) ãæå¹ã«ããã-O3ããã©ã°ã䜿çšããŠãã«ãããena.rx_queue_size ããã³ ena.force_large_llq_header ãã©ã¡ãŒã¿ãŒã䜿çšããŠãå¹æã¯ãããŸããã§ããã
- ãããã¯ãŒã¯ ã¹ã¿ãã¯ãå€æŽããŠãããã©ãŒãã³ã¹ã¯åäžããŸããã§ããã
- IPv6 ãç¡å¹ã«ãã: ipv6.disable=1
- VLAN ãç¡å¹ã«ãã: modprobe -rv 8021q
- ããã±ãŒãžã®ãœãŒã¹ãã§ãã¯ãç¡å¹ã«ãã
- net.ipv4.conf.all.rp_filter=0
- net.ipv4.conf.eth0.rp_filter=0
- net.ipv4.conf.all.accept_local=1 (ãã€ãã¹ã®åœ±é¿)
- net.ipv4.tcp_sack = 0
- net.ipv4.tcp_dsack=0
- net.ipv4.tcp_mem/tcp_wmem/tcp_rmem
- net.core.netdev_budget
- net.core.dev_weight
- net.core.netdev_max_backlog
- net.ipv4.tcp_slow_start_after_idle=0
- net.ipv4.tcp_moderate_rcvbuf=0
- net.ipv4.tcp_timestamps=0
- net.ipv4.tcp_low_latency = 1
- SO_PRIORITY
- TCP_NODELAY
åºæïŒ ãªãŒãã³ããã.ru