ProHoster > Блог > Kev tswj hwm > Falling Down the Luav Qhov: Zaj Dab Neeg ntawm ib qho kua roj vanish reboot yuam kev - Part 1

Falling Down the Luav Qhov: Zaj Dab Neeg ntawm ib qho kua roj vanish reboot yuam kev - Part 1

ghostinushanka, tau hammering tam sim ntawd ntawm cov nyees khawm rau yav dhau los 20 feeb raws li yog hais tias nws lub neej nyob ntawm nws, tig los rau kuv nrog ib tug semi-kub saib nyob rau hauv nws ob lub qhov muag thiab ib tug luag ntxhi - "Tus txiv neej, kuv xav tias kuv tau txais."

"Saib ntawm no," nws hais tias, taw tes rau ib qho ntawm cov cim ntawm lub vijtsam, "Kuv twv kuv lub kaus mom liab yog tias peb ntxiv ntawm no qhov kuv nyuam qhuav xa tuaj rau koj," taw rau lwm ntu ntawm txoj cai, "qhov yuam kev yuav tsis muaj lawm. yuav tshwm sim."

me ntsis puzzled thiab nkees, Kuv hloov kho cov lus qhia peb tau ua hauj lwm rau ib pliag, txuag cov ntaub ntawv thiab khiav systemctl varnish reload. Cov lus yuam kev tau ploj lawm ...

"Cov emails kuv tau pauv nrog tus neeg sib tw," kuv cov npoj yaig txuas ntxiv, thaum nws lub ntsej muag loj hlob rau hauv kev luag nyav ntawm kev xyiv fab tiag tiag, "Nws mam li pom kuv tias qhov no yog qhov teeb meem tiag tiag!"

Yuav ua li cas nws tag nrho pib

Kab lus xav tias kev nkag siab txog yuav ua li cas bash, awk, sed thiab systemd ua haujlwm. Kev paub txog cov kua roj vanish yog nyiam, tab sis tsis tas yuav tsum tau.
Cov ntawv teev sijhawm nyob rau hauv snippet tau hloov pauv.
Sau nrog ghostinushanka.
Cov ntawv no yog ib qho kev txhais ntawm tus thawj luam tawm ua lus Askiv ob lub lis piam dhau los; txhais lus boikoden ua.

Lub hnub ci ntsa iab los ntawm lub qhov rais panoramic ntawm lwm lub caij nplooj zeeg sov thaum sawv ntxov, ib khob ntawm cov dej haus uas muaj caffeine-nplua nuj nyob deb ntawm cov keyboard, koj nyiam symphony ntawm lub suab nrov hauv koj lub mloog pob ntseg, poob dej tawm ntawm cov keyboards txhua yam, thiab thawj zaug nkag. nyob rau hauv daim ntawv teev npe ntawm daim pib rov qab ntawm lub rooj tsavxwm Kanban playfully glows nrog txoj hmoo lub npe "Nrhiav varnishreload" sh: ncha: I / O yuam kev nyob rau hauv staging" (Nrhiav "varnishreload sh: echo: I / O yuam kev" nyob rau hauv staging). Thaum nws los txog rau cov kua roj vanish, muaj thiab tsis tuaj yeem ua ib qho chaw rau qhov yuam kev, txawm tias lawv tsis ua rau muaj teeb meem li qhov no.

Rau cov uas tsis paub varnishreload, qhov no yog ib daim ntawv plhaub yooj yim siv los rov ua qhov kev teeb tsa kua roj vanish - kuj hu ua VCL.

Raws li lub npe ntawm daim pib qhia, qhov yuam kev tau tshwm sim ntawm ib qho ntawm cov servers ntawm theem, thiab txij li kuv paub tseeb tias cov kua roj vanish nyob rau theem tau ua haujlwm zoo, kuv xav tias qhov no yuav yog qhov yuam kev me. Yog li, tsuas yog cov lus uas tau xaus rau hauv qhov kev tso tawm twb kaw lawm. Kuv nqa daim pib rau kuv tus kheej, nyob rau hauv kev ntseeg siab tias kuv yuav kos nws nyob rau hauv tsawg tshaj li 30 feeb, pat kuv tus kheej rau sab nraum qab rau clearing lub rooj tsavxwm ntawm tsis tau lwm cov khib nyiab thiab rov qab mus rau ntau yam tseem ceeb.

Sib tsoo rau hauv phab ntsa ntawm 200 km / h

Qhib cov ntaub ntawv varnishreload, ntawm ib qho ntawm cov servers khiav Debian Stretch, kuv pom lub plhaub ntawv tsawg dua 200 kab ntev.

Tau dhau los ntawm tsab ntawv, kuv tsis pom ib yam dab tsi uas tuaj yeem ua rau muaj teeb meem thaum khiav nws ntau zaus ncaj qha los ntawm lub davhlau ya nyob twg.

Tom qab tag nrho, qhov no yog theem, txawm tias nws tawg, tsis muaj leej twg yuav yws, zoo ... tsis ntau dhau. Kuv khiav tsab ntawv thiab pom dab tsi yuav sau rau lub davhlau ya nyob twg, tab sis qhov yuam kev tsis pom lawm.

Ob peb zaug ntxiv kom paub tseeb tias kuv tsis tuaj yeem rov tsim dua qhov yuam kev yam tsis muaj kev siv dag zog ntxiv, thiab kuv tab tom pib txiav txim siab yuav hloov li cas tsab ntawv no thiab ua kom nws tseem cuam tshuam qhov yuam kev.

Cov ntawv tuaj yeem hla STDOUT (siv > &-)? Los yog STDERR? Ob leeg ntawm no tsis ua haujlwm thaum kawg.

Thaj systemd ua li cas hloov qhov chaw pib, tab sis yuav ua li cas, thiab vim li cas?
Kuv qhib vim thiab kho varnishreload, ntxiv set -x txoj cai nyob rau hauv lub shebang, vam tias qhov debug tso tawm ntawm tsab ntawv yuav tso ib co teeb.

Cov ntaub ntawv raug kho, yog li kuv rov ua cov kua roj vanish thiab pom tias qhov kev hloov pauv tau tawg tag nrho ... Cov pa tawm yog qhov ua kom tiav, uas muaj ntau tons ntawm C-zoo li code. Txawm tias scrolling hauv lub davhlau ya nyob twg tsis txaus los nrhiav qhov twg nws pib. Kuv tsis meej pem kiag li. Hom kev debugging puas tuaj yeem cuam tshuam rau kev ua haujlwm ntawm cov kev pabcuam tau pib hauv tsab ntawv? Tsis yog, nws tsis muaj tseeb. Kab mob hauv lub plhaub? Ntau qhov xwm txheej tau sib tw los ntawm kuv lub taub hau zoo li kab laum nyob rau hauv ntau qhov kev taw qhia. Lub khob ntawm cov dej haus caffeinated tam sim ntawd emptied, ib tug ceev mus rau lub chav ua noj kom ntxiv cov Tshuag thiab ... tawm peb mus. Kuv qhib tsab ntawv thiab ua tib zoo saib ntawm shebang: #!/bin/sh.

/bin/sh - qhov no tsuas yog symlink rau bash, yog li tsab ntawv tau txhais hauv POSIX-tshaj hom, txoj cai? Tsis yog li ntawd! Lub plhaub qub ntawm Debian yog khiav ceev, thiab qhov ntawd yog qhov nws zoo li. hais txog /bin/sh.

# ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Jan 24  2017 /bin/sh -> dash

Raws li kev xeem, kuv hloov lub shebang rau #!/bin/bash, tshem tawm set -x thiab sim dua. Thaum kawg, tom qab rov pib dua ntawm cov kua roj vanish, ib qho kev ua yuam kev tuaj yeem tshwm sim hauv cov zis:

Jan 01 12:00:00 hostname varnishreload[32604]: /usr/sbin/varnishreload: line 124: echo: write error: Broken pipe
Jan 01 12:00:00 hostname varnishreload[32604]: VCL 'reload_20190101_120000_32604' compiled

Kab 124, ntawm no yog!

114 find_vcl_file() {
115         VCL_SHOW=$(varnishadm vcl.show -v "$VCL_NAME" 2>&1) || :
116         VCL_FILE=$(
117                 echo "$VCL_SHOW" |
118                 awk '$1 == "//" && $2 == "VCL.SHOW" {print; exit}' | {
119                         # all this ceremony to handle blanks in FILE
120                         read -r DELIM VCL_SHOW INDEX SIZE FILE
121                         echo "$FILE"
122                 }
123         ) || :
124
125         if [ -z "$VCL_FILE" ]
126         then
127                 echo "$VCL_SHOW" >&2
128                 fail "failed to get the VCL file name"
129         fi
130
131         echo "$VCL_FILE"
132 }

Tab sis raws li nws hloov tawm, kab 124 yog qhov khoob khoob thiab tsis muaj kev txaus siab. Kuv tsuas tuaj yeem xav tias qhov yuam kev tshwm sim ua ib feem ntawm cov kab ntau kab pib ntawm kab 116.
Dab tsi yog qhov kawg sau rau qhov sib txawv? VCL_FILE Raws li qhov tshwm sim ntawm kev ua tiav cov sub-plhaub saum toj no?

Thaum pib, nws xa cov ntsiab lus ntawm qhov sib txawv VLC_SHOW, tsim ntawm kab 115, ua raws li cov lus txib los ntawm cov yeeb nkab. Thiab tom qab ntawd ua li cas rau qhov ntawd?

Ua ntej, nws yog siv nyob rau ntawd varnishadm, uas yog ib feem ntawm cov kua roj vanish installation pob, rau kev teeb tsa cov kua roj vanish tsis rov pib dua.

Pab neeg sub vcl.show -v siv los tso tawm tag nrho VCL configuration teev nyob rau hauv ${VCL_NAME}, rau STDOUT.

Txhawm rau tso saib qhov kev teeb tsa VCL tam sim no, nrog rau ntau yam dhau los ntawm cov kua roj vanish routing configurations uas tseem nyob hauv lub cim xeeb, koj tuaj yeem siv cov lus txib varnishadm vcl.list, cov zis ntawm uas yuav zoo ib yam li cov hauv qab no:

discarded   cold/busy       1 reload_20190101_120000_11903
discarded   cold/busy       2 reload_20190101_120000_12068
discarded   cold/busy       16 reload_20190101_120000_12259
discarded   cold/busy       16 reload_20190101_120000_12299
discarded   cold/busy       28 reload_20190101_120000_12357
active      auto/warm       32 reload_20190101_120000_12397
available   auto/warm       0 reload_20190101_120000_12587

Tus nqi sib txawv ${VCL_NAME} yog ntsia rau hauv lwm qhov ntawm tsab ntawv varnishreload mus rau lub npe ntawm tam sim no active VCL, yog tias muaj. Hauv qhov no nws yuav yog "reload_20190101_120000_12397".

Zoo heev, sib txawv ${VCL_SHOW} muaj tag nrho configuration rau cov kua roj vanish, meej rau tam sim no. Tam sim no kuv thaum kawg nkag siab tias yog vim li cas lub dash tso zis yog set -x muab tawm kom tawg - nws suav nrog cov ntsiab lus ntawm qhov kev teeb tsa tshwm sim.

Nws yog ib qho tseem ceeb kom nkag siab tias qhov ua tiav VCL teeb tsa feem ntau tuaj yeem sib koom ua ke los ntawm ntau cov ntaub ntawv. C-style cov lus tau siv los txheeb xyuas qhov twg qee cov ntaub ntawv teeb tsa tau suav nrog rau lwm tus, thiab qhov ntawd yog cov kab hauv qab no ntawm cov lej snippet yog txhua yam hais txog.
Cov syntax rau cov lus piav qhia cov ntaub ntawv suav nrog yog nyob rau hauv hom hauv qab no:

// VCL.SHOW <NUM> <NUM> <FILENAME>

Cov lej tsis tseem ceeb hauv cov ntsiab lus no, peb txaus siab rau lub npe ntawm cov ntaub ntawv.

Dab tsi thaum kawg tshwm sim hauv swamp ntawm cov lus txib pib ntawm kab 116?
Cia peb paub tseeb.
Pab neeg no muaj plaub ntu:

  1. Yooj yim echo, uas luam tawm tus nqi ntawm qhov sib txawv ${VCL_SHOW} 
    echo "$VCL_SHOW"
  2. awk, uas saib rau ib kab (cov ntaub ntawv) qhov twg thawj daim teb, tom qab rhuav cov ntawv nyeem, yog "//", thiab qhov thib ob yog "VCL.SHOW".
    Awk yuav sau thawj kab uas phim cov qauv no thiab tom qab ntawd nres ua haujlwm tam sim ntawd.

    awk '$1 == "//" && $2 == "VCL.SHOW" {print; exit}'
  3. Ib qho thaiv ntawm cov lej uas khaws cov nqi teb rau hauv tsib qhov sib txawv, sib cais los ntawm qhov chaw. Qhov thib tsib FILE hloov pauv tau txais lwm txoj kab. Thaum kawg, qhov kawg ncha sau tawm cov ntsiab lus ntawm qhov sib txawv ${FILE}. 
    { read -r DELIM VCL_SHOW INDEX SIZE FILE; echo "$FILE" }
  4. Txij li thaum tag nrho cov kauj ruam 1 txog 3 yog nyob rau hauv ib tug subshell, outputting tus nqi $FILE yuav sau mus rau qhov sib txawv VCL_FILE.

Raws li cov lus pom ntawm kab 119 qhia, qhov no yog lub hom phiaj ntawm kev txhim kho cov teeb meem uas VCL yuav siv cov ntaub ntawv nrog qhov chaw hauv lawv cov npe.

Kuv tau tawm tswv yim tawm qhov kev ua tiav thawj zaug rau ${VCL_FILE} thiab sim hloov cov lus txib, tab sis nws tsis ua dab tsi. Txhua yam ua haujlwm zoo rau kuv, tab sis thaum kuv pib qhov kev pabcuam nws tau ua yuam kev.

Nws zoo nkaus li tias qhov yuam kev tsuas yog tsis tuaj yeem rov ua dua thaum khiav cov ntawv manually, thaum lub sijhawm 30 feeb tau tas sijhawm rau lub sijhawm thiab, ntxiv rau, txoj haujlwm tseem ceeb dua tau tshwm sim, thawb lwm yam teeb meem ib sab. Cov so ntawm lub lim tiam tau sau nrog ntau yam haujlwm thiab tsuas yog me ntsis diluted los ntawm tsab ntawv ceeb toom ntawm sed thiab kev xam phaj nrog tus neeg sib tw. Teeb meem nrog yuam kev hauv varnishreload yog irretrievably poob nyob rau hauv cov xuab zeb ntawm lub sij hawm.

Koj lub npe hu ua sed-fu...yog tiag...rubbish

Lub lim tiam tom ntej kuv muaj ib hnub dawb dawb, yog li kuv txiav txim siab los daws daim pib no dua. Kuv vam tias hauv kuv lub hlwb, qee cov txheej txheem keeb kwm yav dhau tau tab tom nrhiav kev daws teeb meem rau lub sijhawm no, thiab lub sijhawm no kuv yuav nkag siab tias yuav ua li cas.

Txij li thaum hloov txoj cai yooj yim tsis tau pab lub sijhawm dhau los, kuv tsuas yog txiav txim siab rov sau nws pib los ntawm kab 116. Txawm li cas los xij, cov cai uas twb muaj lawm yog ruam. Thiab tsis tas yuav tsum siv nws kiag li read.

Saib ntawm qhov yuam kev dua:
sh: echo: broken pipe - ncha tshwm nyob rau hauv ob qhov chaw hauv cov lus txib no, tab sis kuv xav tias thawj zaug yog tus neeg ua txhaum ntau dua (lossis tsawg kawg yog tus neeg ua txhaum). Awk tsis txhawb kev ntseeg siab thiab. Thiab nyob rau hauv cov ntaub ntawv nws yog tiag tiag awk | {read; echo} tus tsim ua rau tag nrho cov teeb meem no, vim li cas ho tsis hloov nws? Qhov kev hais kom ua ib kab no tsis siv tag nrho cov yam ntxwv ntawm awk, thiab txawm tias qhov no ntxiv read ntxiv rau.

Txij li lub lim tiam dhau los muaj ib tsab ntawv ceeb toom rau sed, Kuv xav sim kuv qhov kev txawj ntse tshiab thiab ua kom yooj yim echo | awk | { read; echo} mus rau hauv ib tug ntau to taub echo | sed. Txawm hais tias qhov no tsis yog qhov zoo tshaj plaws rau kev txheeb xyuas tus kab mob, kuv xav tias kuv yuav tsum sim kuv sed-fu thiab tej zaum yuav kawm qee yam tshiab txog qhov teeb meem. Nyob rau hauv txoj kev, kuv hais kom kuv cov npoj yaig, tus sau ntawm sed tham, pab kuv tuaj nrog sed tsab ntawv ua tau zoo dua.

Kuv poob cov ntsiab lus varnishadm vcl.show -v "$VCL_NAME" rau ib cov ntaub ntawv, yog li kuv tuaj yeem tsom mus rau kev sau ntawv sed yam tsis muaj kev cuam tshuam ntawm kev pabcuam reboots.

Ib qho lus piav qhia luv luv ntawm qhov tseeb li cas cov txheej txheem sed tuaj yeem pom hauv nws GNU phau ntawv. Nyob rau hauv lub sed qhov chaw lub cim n qhia meej meej raws li kab cais.

Hauv ntau qhov kev hla dhau thiab nrog cov lus pom zoo ntawm kuv cov npoj yaig, peb tau sau ib tsab ntawv sed uas tau muab cov txiaj ntsig zoo ib yam li tag nrho cov thawj kab 116.

Hauv qab no yog cov qauv ntaub ntawv nrog cov ntaub ntawv nkag:

> cat vcl-example.vcl
Text
// VCL.SHOW 0 1578 file with 3 spaces.vcl
More text
// VCL.SHOW 0 1578 file.vcl
Even more text
// VCL.SHOW 0 1578 file with TWOspaces.vcl
Final text

Qhov no yuav tsis pom tseeb los ntawm cov lus piav qhia saum toj no, tab sis peb tsuas yog txaus siab rau thawj cov lus pom // VCL.SHOW, thiab tej zaum yuav muaj ob peb ntawm lawv hauv cov ntaub ntawv nkag. Qhov no yog vim li cas tus thawj awk xaus tom qab thawj qhov sib tw.

# шаг первый, вывести только строки с комментариями
# используя возможности sed, определяется символ-разделитель с помощью конструкции '#' вместо обычно используемого '/', за счёт этого не придётся экранировать косые в искомом комментарии
# определяется регулярное выражение “// VCL.SHOW”, для поиска строк с определенным шаблоном
# флаг -n позаботится о том, чтобы sed не выводил все входные данные, как он это делает по умолчанию (см. ссылку выше)
# -E позволяет использовать расширенные регулярные выражения
> cat vcl-processor-1.sed
#// VCL.SHOW#p
> sed -En -f vcl-processor-1.sed vcl-example.vcl
// VCL.SHOW 0 1578 file with 3 spaces.vcl
// VCL.SHOW 0 1578 file.vcl
// VCL.SHOW 0 1578 file with TWOspaces.vcl

# шаг второй, вывести только имя файла
# используя команду “substitute”, с группами внутри регулярных выражений, отображается только нужная группa
# и это делается только для совпадений, ранее описанного поиска
> cat vcl-processor-2.sed
#// VCL.SHOW# {
    s#.* [0-9]+ [0-9]+ (.*)$#1#
    p
}
> sed -En -f vcl-processor-2.sed vcl-example.vcl
file with 3 spaces.vcl
file.vcl
file with TWOspaces.vcl

# шаг третий, получить только первый из результатов
# как и в случае с awk, добавляется немедленное завершения после печати первого найденного совпадения
> cat vcl-processor-3.sed
#// VCL.SHOW# {
    s#.* [0-9]+ [0-9]+ (.*)$#1#
    p
    q
}
> sed -En -f vcl-processor-3.sed vcl-example.vcl
file with 3 spaces.vcl

# шаг четвертый, схлопнуть всё в однострочник, используя двоеточия для разделения команд
> sed -En -e '#// VCL.SHOW#{s#.* [0-9]+ [0-9]+ (.*)$#1#p;q;}' vcl-example.vcl
file with 3 spaces.vcl

Yog li, cov ntsiab lus ntawm cov ntawv varnishreload yuav zoo li no:

VCL_FILE="$(echo "$VCL_SHOW" | sed -En '#// VCL.SHOW#{s#.*[0-9]+ [0-9]+ (.*)$#1#p;q;};')"

Cov logic saum toj no tuaj yeem hais luv luv raws li hauv qab no:
Yog txoj hlua phim ib qho kev qhia tsis tu ncua // VCL.SHOW, tom qab ntawd greedily devour cov ntawv nyeem uas suav nrog ob tus lej hauv kab no, thiab txuag txhua yam uas tseem tshuav tom qab ua haujlwm no. Tshaj tawm tus nqi khaws cia thiab xaus qhov kev pab cuam.

Yooj yim, tsis yog nws?

Peb zoo siab nrog tsab ntawv sed thiab qhov tseeb tias nws hloov tag nrho cov cai qub. Tag nrho kuv cov kev ntsuam xyuas tau muab cov txiaj ntsig xav tau, yog li kuv hloov "varnishreload" ntawm lub server thiab khiav nws dua systemctl reload varnish. yuam kev phem echo: write error: Broken pipe luag ntawm peb lub ntsej muag dua. Tus cursor winking tau tos rau qhov kev hais kom ua tshiab nkag rau hauv qhov tsaus nti emptiness ntawm lub davhlau ya nyob twg ...

Tau qhov twg los: www.hab.com

Ntxiv ib saib