ืืืขื ืึท ืืึทืืึทืืืืก ืกืขืจืืืขืจ ืงืืืืฅ ืึทื ืืงืกืคึผืขืงืืืืื ืืื ืืื ืืงืก, ืืืจ ืืึทืจืคึฟื ืฆื ืืขืคึฟืื ืขื ืื ืกืืื. ืขืก ืงืขื ืืืื ืขืืืขืืข ืกืืืืช. ืืืฉื, SIGSEGV - ืืืจืืคืึทื ืจืขืื ืฆื ืึท ืืฉืืง ืืื ืื ืืึทืงืขื ื ืกืขืจืืืขืจ. ืืืขืจ ืืึธืก ืืื ืืขืืื. ืจืืึฟ ืึธืคื ืืืจ ื ืึธืจ ืืืืคื ืืืืก ืคืื ืืืกืง ืคึผืืึทืฅ ืึธืืขืจ ืืึผืจืื. ืืืื ืืืจ ืืืืคื ืืืืก ืคืื ืืืกืง ืคึผืืึทืฅ, ืขืก ืืื ืืืืื ืืืื ืืืขื ืืืืก - ืคืจืื ืคึผืืึทืฅ ืืื ืจืืกืืึทืจื ืื ืืึทืืึทืืืืก.
ืืืืก-ืคืื-ืืืงืึธืจื ืงืืืืขืจ
ืืืขื ืึท ืกืขืจืืืขืจ ืึธืืขืจ ืคึผืจืึธืฆืขืก ืืืืคื ืืืืก ืคืื ืืึผืจืื, ืืื ืืงืก ืึธืคืคืขืจืก 2 ืกืึทืืืฉืึทื ื: ืงืจืึทื ืื ืืื ืฆืข ืกืืกืืขื ืึธืืขืจ ืคืึทืจืขื ืืืงื ืืขื ืคึผืจืึธืฆืขืก (ืึทืคึผืืึทืงืืืฉืึทื) ืืืึธืก ืืื ืขืกื ืืึผืจืื. ืขืก ืืื ืืขืกืขืจ, ืคืื ืงืืจืก, ืฆื ืคืึทืจืขื ืืืงื ืืขื ืคึผืจืึธืฆืขืก ืืื ืจืึทืืขืืืขื ืื ืึทืก ืคืื ืงืจืึทืฉืื ื. ืืื ืึท ื ืึทืืฉืขื, Out-Of-Memory Killer ืืื ืึท ืคึผืจืึธืฆืขืก ืืืึธืก ืงืืื ืึท ืึทืคึผืืึทืงืืืฉืึทื ืฆื ืจืึทืืขืืืขื ืื ืงืขืจื ืคืื ืงืจืึทืฉืื ื. ืขืก ืงืจืื ืื ืึทืคึผืืึทืงืืืฉืึทื ืฆื ืืึทืืื ืื ืึทืก ืคืืืกื ืืืง. ืืื ืก ืขืจืฉืืขืจ ืืืกืงืืืืจื ืืื OOM ืึทืจืืขื ืืื ืืื ืฆื ืงืึธื ืืจืึธืืืจื ืขืก, ืืื ืืขืจื ืึธื ืืขื ืืื OOM Killer ืืืกืืืื ืืืึธืก ืึทืคึผืืึทืงืืืฉืึทื ืฆื ืคืึทืจืขื ืืืงื.
ืืืื ืขืจ ืคืื ืื ืืืืคึผื ืืึทืกืงืก ืคืื ืืื ืืงืก ืืื ืฆื ืึทืืึทืงืืื ืืืงืึธืจื ืฆื ืคึผืจืึทืกืขืกืึทื ืืืขื ืืื ืืขืื ืขืก. ืืืคึผืืงืึทืืื, ืึท ืคึผืจืึธืฆืขืก ืึธืืขืจ ืึทืคึผืืึทืงืืืฉืึทื ืจืืงืืืขืก ืืืงืึธืจื ืคืื ืื ืึทืก, ืึธืืขืจ ืืื ื ืืฉื ืืึธืจ ื ืืฆื ืขืก. ืืืื ืื ืึทืก ืืื ืืืืก ืืืงืึธืจื ืฆื ืึทืืขืืขื ืืืืก ืืขื ืคึฟืึทืจ ืขืก ืึธืืขืจ ืืื ืงืืื ืคึผืืึทื ื ืฆื ื ืืฆื ืขืก, ืืืืขืจ ืืึทืื ืืขืจ ืืึผืจืื ืืืขื ืืืืคื ืืืืก ืืื ืื ืกืืกืืขื ืืืขื ืคืึทืจืืึธืื. ืฆื ืืืกืืืืื ืืขื, ืื ืึทืก ืจืืืขืจืืื ืืืงืึธืจื ืคึฟืึทืจ ืืขื ืคึผืจืึธืฆืขืก, ืึธืืขืจ ืืื ื ืืฉื ืึทืงืฉืื ืืขืืืื ื ืขืก. ืืืงืึธืจื ืืื ืึทืืึทืงืืืืื ืืืืื ืืืขื ืึท ืคึผืจืึธืฆืขืก ืืื ืคืืงืืืฉ ืืขืืื ืืขื ืฆื ื ืืฆื ืขืก. ืขืก ืืึทืคึผืึทื ื ืึทื ืื ืึทืก ืืื ื ืืฉื ืืึธืื ืคืจืื ืืึผืจืื, ืึธืืขืจ ืขืก ืึทืกืืื ื ืืึผืจืื ืฆื ืึท ืคึผืจืึธืฆืขืก, ืืื ืืืขื ืึท ืคึผืจืึธืฆืขืก ืืึทืจืฃ ืขืก, ืื ืึทืก ืึทืืึทืงืืืฅ ืขืก ืืืื ืขืก ืงืขื ืขื. ืื ืืึทืื ืกืืื ืืื ืึทื ืืื ืื ืึทืก ืจืืืขืจืืื ืืึผืจืื, ืึธืืขืจ ืืื ืื ืจืขืื ืฆืืื ืขืก ืืื ืงืืื ืคืจืื ืืึผืจืื, ืืื ืื ืกืืกืืขื ืงืจืึทืฉืื. OOM ืคืืขืกืขืก ืึท ืืืืืืืง ืจืึธืืข ืืื ืืขื ืกืฆืขื ืึทืจ ืืื ืืขืจืืึทื ืืืฅ ืคึผืจืึทืกืขืกืึทื ืฆื ืคืึทืจืืืึทืื ืื ืงืขืจื ืคืื ืคึผืึทื ืืง. ืืืขื ืึท PostgreSQL ืคึผืจืึธืฆืขืก ืืื ืืขืฆืืืื ืืขื ืฆื ืคืึทืจืขื ืืืงื, ืึท ืึธื ืืึธื ืืืืก ืืื ืื ืงืืึธืฅ:
Out of Memory: Killed process 12345 (postgres).ืืืื ืื ืกืืกืืขื ืืื ื ืืืขืจืืง ืืืืฃ ืืึผืจืื ืืื ืขืก ืงืขื ื ืืฉื ืืืื ืคืจืื, ืื ืคึฟืื ืงืฆืืข ืืื ืืขืจืืคื out_of_memory. ืืื ืืขื ืืื ืข, ืื ืืื ืืืืื ืืืื ืืึทื ืฆื ืืึธื - ืคืึทืจืขื ืืืงื ืืืื ืึธืืขืจ ืืขืจ ืคึผืจืึทืกืขืกืึทื. ืืึธื OOM-killer ืคืึทืจืขื ืืืงื ืืขื ืคึผืจืึธืฆืขืก ืืืืื ืึธืืขืจ ืงืขื ืขืก ืืืึทืจืื? ืืึธื, ืืืขื ืึธืื_ืคืื_ืืขืืึธืจื ืืื ืืขืจืืคื, ืืึธืก ืืื ืจืขืื ืฆื ืืืืจืื ืคึฟืึทืจ ืึทื I/O ืึธืคึผืขืจืึทืฆืืข ืึธืืขืจ ืคึผืืืืืฉืื ื ืฆื ืืืกืง. ืืขืจืืืขืจ, ืืขืจ OOM ืงืืืืขืจ ืืืื ืขืจืฉืืขืจ ืืืจืืคืืจื ืืฉืขืงืก ืืื, ืืืืืจื ืืืืฃ ืืื, ืืึทืฉืืืกื ืึทื ืืขืจ ืคึผืจืึธืฆืขืก ืืึทืจืฃ ืืืื ืืขืจืืึทื ืืืืื. ืืืื ืึทืืข ืื ืืฉืขืงืก ืืื ืื ืืขื ืขื positive, OOM ืืืขื ืคืึทืจืขื ืืืงื ืืขื ืคึผืจืึธืฆืขืก.
ืคึผืจืึธืฆืขืก ืกืขืืขืงืฆืืข
ืืืขื ืืึผืจืื ืืืืคื ืืืืก, ืื ืคึฟืื ืงืฆืืข ืืื ืืขืจืืคื out_of_memory(). ืขืก ืืื ืึท ืคึฟืื ืงืฆืืข select_bad_process(), ืืืึธืก ืืืงืืื ืึทื ืืคืฉืืฆืื ื ืคืื ืื ืคึฟืื ืงืฆืืข badness(). ืืขืจ "ืขืจืืกื" ืคึผืจืึธืฆืขืก ืืืขื ืืืื ืืึทืจืืขืืขื. ืคึฟืื ืงืฆืืข badness() ืกืึทืืขืงืฅ ืึท ืคึผืจืึธืฆืขืก ืืืื ืืืืขืจ ืึผืืืื.
- ืืขืจ ืงืขืจื ืืึทืจืฃ ืขืืืขืืข ืืื ืืืื ืืึผืจืื ืคึฟืึทืจ ืืื.
- ืืืจ ืืึทืจืคึฟื ืฆื ืืึทืคืจืืึทืขื ืึท ืคึผืืึทืฅ ืคืื ืืึผืจืื.
- ืขืก ืืื ื ืื ืืึทืจืคึฟื ืฆื ืคืึทืจืขื ืืืงื ืคึผืจืึทืกืขืกืึทื ืืืึธืก ื ืืฆื ืงืืืื ืืึผืจืื.
- ืืื ืืืื ืคึผืจืึทืกืขืกืึทื ืืึทืจืคึฟื ืฆื ืืืื ืืขืขื ืืืงื.
- ืงืึธืืคึผืืขืงืก ืึทืืืขืจืืืึทืื ืืืึธืก ืคืึทืจืืจืขืกืขืจื ืื ืืืืขืจ ืคืื ืงืึทืืคึผืืืฉืึทื ืคึฟืึทืจ ืื ืคึผืจืึทืกืขืกืึทื ืืืึธืก ืืขืจ ืืึทื ืืฆืขืจ ืืื ืืืื ืฆื ืคืึทืจืขื ืืืงื.
ื ืึธื ืคืึทืจืขื ืืืงื ืึทืืข ืื ืืฉืขืงืก, OOM ืืืืึทืืึทื ื ืื ืืขืืฉืื (oom_score). OOM ืึทืคึผืืื ืืืึทื ืฅ oom_score ืืขืืขืจ ืคึผืจืึธืฆืขืก, ืืื ืืขืืึธืื ืืึทืืืึทืคึผืืืื ืืขื ืืืขืจื ืืืจื ืื ืกืืืข ืคืื โโืืึผืจืื. ืคึผืจืึทืกืขืกืึทื ืืื ืืจืขืกืขืจืข ืืืึทืืืขืก ืืขื ืขื ืืขืจ ืืกืชึผืื ืฆื ืคืึทืื ืงืึธืจืื ืฆื ืื OOM Killer. ืคึผืจืึทืกืขืกืึทื ืคึฟืึทืจืืื ืื ืืื ืืขืจ ืืืึธืจืฆื ืืึทื ืืฆืขืจ ืืึธืื ืึท ื ืืืขืจืืงืขืจ ืืขืืฉืื ืืื ืืขื ืขื ืืืืื ืืงืขืจ ืืกืชึผืื ืฆื ืืืื ืืขืฆืืืื ืืขื ืฆื ืคืึทืจืขื ืืืงื.
postgres=# SELECT pg_backend_pid();
pg_backend_pid
----------------
3813
(1 row)ืื Postgres ืคึผืจืึธืฆืขืก ืฉืืึทื ืืื 3813, ืึทืืื ืืื ืื ืื ืืขืจ ืฉืึธื ืขืก ืืื ืืขืืืขื ืฆื ืืึทืงืืืขื ืื ืืขืืฉืื ืืื ืืขื ืงืขืจื ืคึผืึทืจืึทืืขืืขืจ oom_score:
vagrant@vagrant:~$ sudo cat /proc/3813/oom_score
2ืืืื ืืืจ ืืึธื ื ืื ืืืขืื OOM-Killer ืฆื ืืืืื ืืขื ืคึผืจืึธืฆืขืก ืืื ืึทืืข, ืขืก ืืื ืื ืื ืืขืจ ืงืขืจื ืึธืคึผืฆืืข: oom_score_adj. ืืืื ืึท ืืจืืืก ื ืขืืึทืืืื ืืืขืจื ืฆื ืจืขืืืฆืืจื ืื ืืืืขืจ ืคืื ืงืึทืืคึผืืืืื ื ืึท ืคึผืจืึธืฆืขืก ืืืจ ืืืขืจื.
sudo echo -100 > /proc/3813/oom_score_adjืฆื ืฉืืขืื ืึท ืืืขืจื oom_score_adj, ืฉืืขืื OOMScoreAdjust ืืื ืื ืกืขืจืืืืก ืืืึธืง:
[Service]
OOMScoreAdjust=-1000ืึธืืขืจ ื ืืฆื oomprotect ืืื ืึท ืงืึธืืขืงืืืื rcctl.
rcctl set <i>servicename</i> oomprotect -1000ืงืจืึทืคื ืืขืจืืึทื ืืืฉืึทื ืคืื ืึท ืคึผืจืึธืฆืขืก
ืืืขื ืืืื ืขืจ ืึธืืขืจ ืืขืจ ืคึผืจืึทืกืขืกืึทื ืืขื ืขื ืฉืืื ืืืืกืืขืงืืืื, OOM-Killer ืจืืคื ืื ืคึฟืื ืงืฆืืข oom_kill_task(). ืืขื ืคึฟืื ืงืฆืืข ืกืขื ืื ืึท ืืขืจืืึทื ืืืฉืึทื ืกืืื ืึทื ืฆื ืืขื ืคึผืจืึธืฆืขืก. ืืื ืคืึทื ืคืื ืืืงืึธืจื ืืืืง oom_kill() ืจืืคื ืืขื ืคึฟืื ืงืฆืืข ืฆื ืฉืืงื ืึท SIGKILL ืกืืื ืึทื ืฆื ืืขื ืคึผืจืึธืฆืขืก. ื ืึธื ืืึธื ืืื ืืขืฉืจืืื ืฆื ืื ืงืขืจื ืงืืึธืฅ.
Out of Memory: Killed process [pid] [name].ืืื ืฆื ืงืึธื ืืจืึธืืืจื OOM-Killer
ืืืืฃ ืืื ืืงืก, ืืืจ ืงืขื ืขื ืืขืื ืึธืืขืจ ืืืกืืืืึทื OOM-Killer (ืืึธืืฉ ืื ืืขื ืขืจ ืืื ื ืืฉื ืจืขืงืึทืืขื ืืื). ืฆื ืืขืื ืึธืืขืจ ืืืกืืืืึทื ื ืืฆื ืืขื ืคึผืึทืจืึทืืขืืขืจ vm.oom-kill. ืฆื ืืขืื OOM-Killer ืืื ืจืื ืืืืข, ืืืืคื ืื ืืึทืคึฟืขื sysctl.
sudo -s sysctl -w vm.oom-kill = 1ืฆื ืืืกืืืืึทื OOM-Killer, ืกืคึผืขืฆืืคืืฆืืจื ืื ืืืขืจื 0 ืืื ืืขืจ ืืขืืืืงืขืจ ืืึทืคึฟืขื:
sudo -s sysctl -w vm.oom-kill = 0ืืขืจ ืจืขืืืืืึทื ืคืื ืืขื ืืึทืคึฟืขื ืืืขื ื ืื ืืืื ืืขืจืืืขืืืขื ืืืืฃ ืืืืืืง, ืึธืืขืจ ืืืืื ืืื ืืขืจ ืขืจืฉืืขืจ ืจืขืืึธืึธื. ืืืื ืืืจ ืืึทืจืคึฟื ืืขืจ ืคึผืขืจืกืืกืืึทื ืก, ืืืืื ืืขื ืฉืืจื ืฆื ืืขืจ ืืขืงืข /etc/sysctl.conf:
echo vm.oom-kill = 1 >>/etc/sysctl.confืื ืื ืืขืจ ืืืขื ืฆื ืืขืื ืืื ืืืกืืืืึทื ืืื ืฆื ืฉืจืืึทืื ืึท ืืืึทืืขืืืืืง panic_on_oom. ืื ืืืขืจื ืงืขื ืขื ืฉืืขื ืืืง ืืืื ืึธืคึผืืขืฉืืขืื /proc.
$ cat /proc/sys/vm/panic_on_oom
0ืืืื ืืืจ ืฉืืขืื ืื ืืืขืจื ืฆื 0, ืืืขื ืื ืืึผืจืื ืืืืคื ืืืืก, ืขืก ืืืขื ืืืื ืงืืื ืงืขืจื ืขื ืคึผืึทื ืืง.
$ echo 0 > /proc/sys/vm/panic_on_oomืืืื ืืืจ ืฉืืขืื ืื ืืืขืจื ืฆื 1, ืืืขื ืืขืจ ืืึผืจืื ืืืืคื ืืืืก, ืึท ืงืขืจื ืคึผืึทื ืืง ืืืขื ืคึผืึทืกืืจื.
echo 1 > /proc/sys/vm/panic_on_oomOOM-Killer ืงืขื ืขื ื ืื ืืืืื ืืืื ืืืืกืืขืืจืืื ืืืืฃ ืืื ืึทืืืขืง. ืืืจ ืืึธืื ืฉืืื ืืขืืึธืื ืึทื ืืื ืืงืก ืงืขื ืขื ืจืขืืขืจืืืืจื ืืขืจ ืืืงืึธืจื ืคึฟืึทืจ ืคึผืจืึทืกืขืกืึทื ืืื ืืื ืื ืืืฆื ืึธื ืึทืงืฉืื ืึทืืึทืงืืืืื ื ืขืก, ืืื ืืขื ื ืึทืืืจ ืืื ืงืึทื ืืจืึธืืื ืืืจื ืึท ืืื ืืงืก ืงืขืจื ืคึผืึทืจืึทืืขืืขืจ. ืืขืจ ืืืึทืจืืึทื ื ืืื ืคืึทืจืึทื ืืืืึธืจืืืขื ืคึฟืึทืจ ืืขื vm.overcommit_memory.
ืืืจ ืงืขื ืขื ืกืคึผืขืฆืืคืืฆืืจื ืื ืคืืืืขื ืืข ืืืึทืืืขืก ืคึฟืึทืจ ืขืก:
0: ืืขืจ ืงืขืจื ืขื ืืื ืืืฉืืืกื ืฆื ืฆื ืจืขืืขืจืืืืจื ืฆื ืคืื ืืืงืึธืจื. ืืึธืก ืืื ืื ืคืขืืืงืืึทื ืืื ืจืืึฟ ืืืขืจืกืืขืก ืคืื ืืื ืืงืก.
1: ืืขืจ ืงืขืจื ืืืขื ืฉืืขื ืืืง ืจืขืืขืจืืืืจื ืขืงืกืืจืข ืืึผืจืื. ืืึธืก ืืื ืจืืืืงืึทืืืฉ, ืืืืึทื ืืขืจ ืืึผืจืื ืงืขื ืืืืคื ืืืืก, ืืืืึทื, ืจืืึฟ ืืกืชึผืื, ืืืื ืืึธื ืื ืคึผืจืึทืกืขืกืึทื ืืืขื ืืึทืจืคื ืขืก.
2: ืืขืจ ืงืขืจื ืืืขื ื ืืฉื ืจืขืืขืจืืืืจื ืืขืจ ืืึผืจืื ืืื ืกืคึผืขืกืืคืืขื ืืื ืืขื ืคึผืึทืจืึทืืขืืขืจ overcommit_ratio.
ืืื ืืขื ืคึผืึทืจืึทืืขืืขืจ, ืืืจ ืกืคึผืขืฆืืคืืฆืืจื ืื ืคึผืจืึธืฆืขื ื ืคืื ืืึผืจืื ืืืึธืก ืืื ืขืจืืืืื ืฆื ืืืื ืืืืขืจ-ืจืขืืขืจืืืืจื. ืืืื ืขืก ืืื ืงืืื ืคึผืืึทืฅ ืคึฟืึทืจ ืืื, ืงืืื ืืึผืจืื ืืื ืึทืืึทืงืืืืื, ืืื ืื ืจืขืืขืจืืืึทืฆืืข ืืืขื ืืืื ืืขืืืืงื ื. ืืึธืก ืืื ืื ืกืืืคืึทืกื ืึธืคึผืฆืืข ืจืขืงืึทืืขื ืืื ืคึฟืึทืจ PostgreSQL. OOM-Killer ืืื ืึทืคืขืงืืึทื ืืืจื ืื ืื ืืขืจ ืขืืขืืขื ื - ืื ืกืืืึทืคึผืื ื ืคืืืืงืืื, ืืืึธืก ืืื ืงืึทื ืืจืึธืืื ืืืจื ืื ืืืึทืืขืืืืืง cat /proc/sys/vm/swappiness. ืื ืืืึทืืืขืก ืืึธืื ืื ืงืขืจื ืืื ืฆื ืฉืขืคึผื ืคึผืืืืืฉืื ื. ืื ืืขืืขืจ ืื ืืืขืจื, ืื ืืืืื ืืงืขืจ ืืกืชึผืื ืขืก ืืื ืึทื OOM ืืืขื ืคืึทืจืขื ืืืงื ืืขื ืคึผืจืึธืฆืขืก, ืึธืืขืจ ืจืขืื ืฆื I / O ืึทืคึผืขืจืืืฉืึทื ื, ืขืก ืืื ืึท ื ืขืืึทืืืื ืคึผืจืึทื ืืืืฃ ืื ืืึทืืึทืืืืก. ืืื ืืืืฆืข ืืืขืจืกืึท - ืื ื ืืืขืจืืงืขืจ ืื ืืืขืจื, ืื ืืขืืขืจ ืื ืืืงืขืืืืึธืึธื ืคืื OOM-Killer ืื ืืขืจืืืขื ืืืึธื, ืึธืืขืจ ืื ืืึทืืึทืืืืก ืคืึธืจืฉืืขืืื ื ืืื ืืืื ืืขืืขืจ. ืื ืคืขืืืงืืึทื ืืืขืจื ืืื 60, ืึธืืขืจ ืืืื ืื ืืื ืฆืข ืืึทืืึทืืืืก ืคืืฅ ืืื ืืึผืจืื, ืขืก ืืื ืืขืกืขืจ ืฆื ืฉืืขืื ืื ืืืขืจื ืฆื 1.
ืจืขืืืืืึทืื ืคืื
ืื ืืืืกื ื ืืฉื ืืึธืื ืื "ืงืืืขืจ" ืืื OOM-Killer ืืืขืจืฉืจืขืงื ืืืจ. ืืื ืืขื ืคืึทื, ืืขืจ ืจืขืฆืืืขื ืืืขื ืืืื ืืขืจ ืืืื ืคืื ืืืื ืกืืกืืขื. ืขืก "ืงืืื" ืื ืขืจืืกื ืคึผืจืึทืกืขืกืึทื ืืื ืกืึทืืืขืก ืื ืกืืกืืขื ืคืื ืงืจืึทืฉืื ื. ืฆื ืืืกืืืืื ืฆื ื ืืฆื OOM-Killer ืฆื ืคืึทืจืขื ืืืงื PostgreSQL, ืฉืืขืื ืฆื vm.overcommit_memory ืืืขืจื 2. ืืึธืก ืงืขื ื ืืฉื ืืึทืจืึทื ืืืจื ืึทื OOM-Killer ืืืขื ื ืืฉื ืืึธืื ืฆื ืื ืขืจืืืื, ืึธืืขืจ ืขืก ืืืขื ืจืขืืืฆืืจื ืื ืืืงืขืืืืึธืึธื ืคืื ืคืึธืจืกืื ื ืื PostgreSQL ืคึผืจืึธืฆืขืก ืฆื ืคืึทืจืขื ืืืงื.
ืืงืืจ: www.habr.com