Kukhazikitsa Out-Of-Memory Killer ku Linux kwa PostgreSQL

Seva ya database ikasiya mosayembekezereka ku Linux, muyenera kupeza chifukwa. Pakhoza kukhala zifukwa zingapo. Mwachitsanzo, Mtengo wa SIGSEGV - kulephera chifukwa cha cholakwika mu seva yakumbuyo. Koma izi ndizosowa. Nthawi zambiri, mumangosowa malo a disk kapena kukumbukira. Ngati mulibe malo a disk, pali njira imodzi yokha yotulukira - kumasula malo ndikuyambitsanso database.

Out-Of-Memory Killer

Seva kapena ndondomeko ikatha kukumbukira, Linux imapereka mayankho a 2: kusokoneza dongosolo lonse kapena kuthetsa ndondomeko (ntchito) yomwe ikudya kukumbukira. Ndibwino, ndithudi, kuthetseratu ndondomekoyi ndikupulumutsa OS kuti isawonongeke. Mwachidule, Out-Of-Memory Killer ndi njira yomwe imapha pulogalamu yopulumutsa kernel kuti isagwe. Imasiya ntchito kuti isunge OS ikuyenda. Tiyeni tikambirane kaye momwe OOM imagwirira ntchito komanso momwe mungayithandizire, kenako tiwone momwe OOM Killer amasankhira ntchito yomwe angayimitse.

Imodzi mwa ntchito zazikulu za Linux ndikugawa kukumbukira kumachitidwe akafuna. Nthawi zambiri, njira kapena ntchito imapempha kukumbukira kuchokera ku OS, koma saigwiritsa ntchito mokwanira. Ngati OS ikupereka kukumbukira kwa aliyense amene wapempha koma alibe malingaliro oigwiritsa ntchito, posachedwa kukumbukira kudzatha ndipo dongosolo lidzalephera. Kuti mupewe izi, OS imasunga kukumbukira kwa njirayi, koma sikumamasula. Memory imaperekedwa pokhapokha ngati njira ikugwiritsira ntchito. Zimachitika kuti OS ilibe kukumbukira kwaulere, koma imapatsa kukumbukira njira, ndipo ikafunika, OS imagawa ngati ingathe. Choyipa ndichakuti nthawi zina OS imasunga kukumbukira, koma pa nthawi yoyenera palibe kukumbukira kwaulere, ndipo dongosolo limawonongeka. OOM imatenga gawo lofunikira pankhaniyi ndikuyimitsa njira kuti kernel isachite mantha. Njira ya PostgreSQL ikakakamizika kuyimitsa, uthenga umawonekera mu chipika:

Out of Memory: Killed process 12345 (postgres).

Ngati dongosololi ndi lochepa pamtima ndipo silingathe kumasulidwa, ntchitoyi imatchedwa out_of_memory. Pakadali pano, ali ndi chinthu chimodzi chokha choti achite - kumaliza njira imodzi kapena zingapo. Kodi OOM-wakupha athetse ntchitoyi nthawi yomweyo kapena adikire? Mwachiwonekere, pamene out_of_memory imatchedwa, ndi chifukwa chodikira ntchito ya I / O kapena paging ku disk. Chifukwa chake, wakupha wa OOM ayenera kuchita macheke ndipo, potengera iwo, asankhe kuti ntchitoyi iyenera kuthetsedwa. Ngati macheke onse omwe ali pansipa ali abwino, OOM ithetsa ntchitoyi.

Kusankha njira

Pamene kukumbukira kutha, ntchitoyo imatchedwa out_of_memory(). Ili ndi ntchito select_bad_process(), yomwe imalandira kuwunika kuchokera ku ntchitoyi badness(). Njira "yoipitsitsa" idzayang'aniridwa. Ntchito badness() amasankha njira motsatira malamulo ena.

1. Kernel imafunika kukumbukira pang'ono yokha.
2. Muyenera kumasula zokumbukira zambiri.
3. Palibe chifukwa chothetsa njira zomwe zikugwiritsa ntchito kukumbukira pang'ono.
4. Njira zochepa ziyenera kumalizidwa.
5. Ma algorithms ovuta omwe amawonjezera mwayi womaliza panjira zomwe wogwiritsa ntchitoyo akufuna kumaliza.

Atamaliza macheke onsewa, OOM imayang'ana mphambu (oom_score). OOM amasankha oom_score ndondomeko iliyonse, ndiyeno kuchulukitsa mtengo uwu ndi kuchuluka kwa kukumbukira. Njira zokhala ndi zikhalidwe zazikulu ndizosavuta kugwidwa ndi OOM Killer. Njira zogwirizanirana ndi wogwiritsa ntchito mizu zimakhala zocheperako ndipo sizingakakamizidwe kuthetsedwa.

postgres=# SELECT pg_backend_pid();
pg_backend_pid 
----------------
    3813
(1 row)

ID ya Postgres process ndi 3813, kotero mu chipolopolo china ndizotheka kupeza mphambu pogwiritsa ntchito kernel parameter. oom_score:

vagrant@vagrant:~$ sudo cat /proc/3813/oom_score
2

Ngati simukufuna kuti OOM-Killer aphe njirayi, pali njira inanso: oom_score_adj. Onjezani mtengo woyipa waukulu kuti muchepetse mwayi womaliza ntchito yomwe mumaikonda.

sudo echo -100 > /proc/3813/oom_score_adj

Kukhazikitsa mtengo oom_score_adj, ikani OOMScoreAdjust mu block block:

[Service]
OOMScoreAdjust=-1000

Kapena ntchito oomprotect mu timu rcctl.

rcctl set <i>servicename</i> oomprotect -1000

Kukakamiza kuthetsa ndondomeko

Njira imodzi kapena zingapo zikasankhidwa kale, OOM-Killer imayimbira ntchitoyi oom_kill_task(). Ntchitoyi imatumiza chizindikiro chothetsa ntchito. Ngati kukumbukira kukumbukira oom_kill() Imayimbira ntchitoyi kuti itumize chizindikiro cha SIGKILL pakuchitapo kanthu. Uthenga umalembedwa ku kernel log.

Out of Memory: Killed process [pid] [name].

Momwe mungaletsere OOM-Killer

Pa Linux, mutha kuloleza kapena kuletsa OOM-Killer (ngakhale yomalizayo siyikulimbikitsidwa). Kuti muyambitse kapena kuletsa ntchito parameter vm.oom-kill. Kuti mutsegule OOM-Killer panthawi yothamanga, yesani lamulo sysctl.

sudo -s sysctl -w vm.oom-kill = 1

Kuti mulepheretse OOM-Killer, tchulani mtengo 0 mu lamulo lomwelo:

sudo -s sysctl -w vm.oom-kill = 0

Zotsatira za lamuloli sizidzapulumutsidwa mpaka kalekale, koma mpaka kuyambiranso koyamba. Ngati mukufuna kulimbikira kwambiri, onjezani mzerewu pafayilo /etc/sysctl.conf:

echo vm.oom-kill = 1 >>/etc/sysctl.conf

Njira ina yothandizira ndikuyimitsa ndikulemba kusintha panic_on_oom. Mtengo ukhoza kuwonedwa nthawi zonse /proc.

$ cat /proc/sys/vm/panic_on_oom
0

Ngati muyika mtengo kukhala 0, ndiye kuti kukumbukira kukatha, sipadzakhala mantha a kernel.

$ echo 0 > /proc/sys/vm/panic_on_oom

Ngati muyika mtengo ku 1, ndiye kuti kukumbukira kukatha, mantha a kernel adzachitika.

echo 1 > /proc/sys/vm/panic_on_oom

OOM-Killer sangangoyatsidwa ndikuzimitsa. Tanena kale kuti Linux ikhoza kusunga zokumbukira zambiri kuposa zomwe zilipo popanda kugawa, ndipo izi zimayendetsedwa ndi Linux kernel parameter. The variable ndiye amachititsa izi vm.overcommit_memory.

Mutha kufotokozera za izi:

0: Kernel yokha imasankha kusungira kukumbukira kwambiri. Izi ndizosakhazikika pamatembenuzidwe ambiri a Linux.
1: Kernel nthawi zonse imasunga kukumbukira kowonjezera. Izi ndizowopsa, chifukwa kukumbukira kumatha kutha, chifukwa, mwina, tsiku lina njira zidzafunika.
2: kernel sichisunga kukumbukira zambiri kuposa zomwe zafotokozedwa mu parameter overcommit_ratio.

Ndi parameter iyi, mumatchula kuchuluka kwa kukumbukira komwe kumaloledwa kusungidwa mopitirira muyeso. Ngati palibe malo, palibe kukumbukira komwe kwaperekedwa, ndipo kusungitsako kudzakanidwa. Iyi ndiye njira yotetezeka kwambiri yomwe ikulimbikitsidwa ku PostgreSQL. OOM-Killer imakhudzidwa ndi chinthu china - kusinthasintha, komwe kumayendetsedwa ndi kusinthika cat /proc/sys/vm/swappiness. Makhalidwe awa amauza kernel momwe angagwiritsire ntchito paging. Kukwera kwamtengo wapatali, sikutheka kuti OOM idzathetsa ndondomekoyi, koma chifukwa cha ntchito za I / O zimakhala ndi zotsatira zoipa pa database. Ndipo mosemphanitsa - kutsika kwa mtengo, kumapangitsanso mwayi woti OOM-Killer alowererepo, koma magwiridwe antchito a database ndi apamwamba. Mtengo wokhazikika ndi 60, koma ngati nkhokwe yonse ikugwirizana ndi kukumbukira, ndi bwino kuyika mtengo kukhala 1.

Zotsatira

Musalole "wakupha" mu OOM-Killer akuwopsezeni. Pankhaniyi, wakuphayo adzakhala mpulumutsi wa dongosolo lanu. "Imapha" njira zoyipa kwambiri ndikupulumutsa dongosolo kuti lisawonongeke. Kupewa kugwiritsa ntchito OOM-Killer kuthetsa PostgreSQL, yambitsani vm.overcommit_memory mtengo 2. Izi sizikutanthauza kuti OOM-Killer sadzayenera kulowererapo, koma zidzachepetsa mwayi wokakamiza ndondomeko ya PostgreSQL kuti ithetse.

Source: www.habr.com