Txhua leej txhua tus tham txog cov txheej txheem ntawm kev txhim kho thiab kev sim, kev cob qhia cov neeg ua haujlwm, txhawb kev txhawb zog, tab sis cov txheej txheem no tsis txaus thaum ib feeb ntawm kev pabcuam downtime raug nqi ntau ntawm cov nyiaj. Yuav ua li cas thaum koj ua lag luam nyiaj txiag raws li SLA nruj? Yuav ua li cas txhawm rau txhim kho kev ntseeg siab thiab kev ua txhaum ntawm koj lub tshuab, kev txhim kho thiab kev sim tawm ntawm qhov sib npaug?
Lub rooj sib tham HighLoad++ tom ntej no yuav muaj rau lub Plaub Hlis 6 thiab 7, 2020 hauv St. Petersburg. Paub meej thiab daim pib rau . Peb 9, 18:00. HighLoad++ Moscow 2018, Delhi + Kolkata Hall. Cov no thiab .
Evgeniy Kuzovlev (tom qab no - EC): - Cov phooj ywg, nyob zoo! Kuv lub npe yog Kuzovlev Evgeniy. Kuv yog los ntawm EcommPay tuam txhab, ib qho kev faib tshwj xeeb yog EcommPay IT, IT division ntawm pawg tuam txhab. Thiab niaj hnub no peb yuav tham txog downtimes - txog yuav ua li cas zam lawv, hais txog yuav ua li cas txo lawv qhov tshwm sim yog tias nws tsis tuaj yeem zam tau. Lub ntsiab lus tau hais raws li nram no: "Yuav ua li cas thaum ib feeb ntawm downtime nqi $100"? Saib tom ntej, peb cov lej sib piv.
EcommPay IT ua dab tsi?
Peb yog leej twg? Vim li cas kuv thiaj sawv ntawm koj xub ntiag? Vim li cas kuv thiaj muaj cai qhia koj ib yam ntawm no? Thiab peb yuav tham txog dab tsi ntawm no kom ntxaws ntxiv?
EcommPay pab pawg ntawm cov tuam txhab yog cov khoom lag luam thoob ntiaj teb. Peb cov txheej txheem them nyiaj thoob plaws ntiaj teb - hauv Russia, Europe, Southeast Asia (Tag nrho lub ntiaj teb). Peb muaj 9 lub chaw ua haujlwm, 500 tus neeg ua haujlwm nyob rau hauv tag nrho, thiab kwv yees li tsawg dua ib nrab ntawm lawv yog cov kws tshaj lij IT. Txhua yam peb ua, txhua yam peb khwv tau los ntawm, peb ua peb tus kheej.
Peb tau sau tag nrho peb cov khoom (thiab peb muaj ntau heev ntawm lawv - nyob rau hauv peb cov kab ntawm cov khoom loj IT peb muaj txog 16 yam sib txawv) peb tus kheej; Peb sau peb tus kheej, peb txhim kho peb tus kheej. Thiab tam sim no peb ua tiav txog ib lab kev lag luam hauv ib hnub (million yog tej zaum yog txoj hauv kev los hais nws). Peb yog ib lub tuam txhab me me - peb tsuas yog muaj rau xyoo xwb.
6 xyoo dhau los nws yog qhov pib zoo li no thaum cov txiv neej tuaj nrog kev lag luam. Lawv tau koom ua ke los ntawm ib lub tswv yim (tsis muaj dab tsi ntxiv tab sis lub tswv yim), thiab peb tau khiav. Zoo li txhua qhov pib, peb khiav nrawm dua... Rau peb, kev ceev yog qhov tseem ceeb tshaj qhov zoo.
Qee lub sij hawm peb nres: peb pom tau tias peb tsis tuaj yeem nyob ntawm qhov nrawm thiab nrog qhov zoo thiab peb yuav tsum tsom mus rau qhov zoo ua ntej. Lub sijhawm no, peb tau txiav txim siab sau lub platform tshiab uas yuav yog qhov tseeb, ntsuas tau, thiab txhim khu kev qha. Lawv pib sau lub platform no (lawv pib nqis peev, tsim kev loj hlob, sim), tab sis qee lub sijhawm lawv pom tau tias kev txhim kho thiab kev sim tsis pub peb mus txog qib tshiab ntawm kev pabcuam zoo.
Koj ua ib yam khoom tshiab, koj muab tso rau hauv ntau lawm, tab sis tseem ib yam dab tsi yuav mus tsis ncaj ncees lawm qhov chaw. Thiab hnub no peb yuav tham txog yuav ua li cas kom ncav cuag qib tshiab zoo (peb tau ua li cas, txog peb qhov kev paub dhau los), kev txhim kho thiab kev sim tawm ntawm qhov sib npaug; peb yuav tham txog dab tsi muaj rau kev ua haujlwm - dab tsi ua haujlwm tuaj yeem ua nws tus kheej, nws tuaj yeem muab rau kev kuaj kom muaj txiaj ntsig zoo.
Kev poob qis. Cov lus txib ntawm kev ua haujlwm.
Ib txwm yog lub hauv paus tseem ceeb, qhov peb yuav tham txog hnub no yog lub sijhawm poob qis. Ib lo lus txaus ntshai. Yog tias peb muaj sijhawm poob, txhua yam tsis zoo rau peb. Peb tab tom khiav los tsa nws, cov thawj tswj hwm tuav lub server - Vajtswv txwv nws tsis poob, raws li lawv hais hauv zaj nkauj ntawd. Nov yog qhov peb yuav tham txog hnub no.
Thaum peb pib hloov peb txoj hauv kev, peb tsim 4 lus txib. Kuv muaj lawv nthuav tawm ntawm cov slides:
Cov lus txib no yooj yim heev:
- Ceev nrooj txheeb xyuas qhov teeb meem.
- Tshem tawm sai dua.
- Pab nkag siab qhov laj thawj (tom qab ntawd, rau cov neeg tsim khoom).
- Thiab standardize mus kom ze.
Kuv xav kom koj xav txog qhov point No. Kev txiav txim siab yog qhov thib ob. Rau peb, qhov tseem ceeb tshaj plaws yog tias tus neeg siv tau txais kev tiv thaiv los ntawm qhov teeb meem no. Nws yuav muaj nyob hauv qee qhov chaw nyob ib puag ncig, tab sis qhov chaw no yuav tsis muaj kev sib cuag nrog nws. Qhov tseeb, peb yuav dhau los ntawm plaub pawg ntawm cov teeb meem no (qee qhov nthuav dav ntxiv, qee qhov tsis meej), Kuv yuav qhia koj tias peb siv dab tsi, qhov kev paub zoo uas peb muaj hauv kev daws teeb meem.
Kev daws teeb meem: Thaum twg lawv tshwm sim thiab yuav ua li cas txog lawv?
Tab sis peb yuav pib tawm ntawm kev txiav txim, peb yuav pib nrog point No. 2 - yuav ua li cas kom tshem tau qhov teeb meem sai sai? Muaj ib qho teeb meem - peb yuav tsum kho nws. "Peb yuav tsum ua li cas txog qhov no?" - lub ntsiab lus nug. Thiab thaum peb pib xav txog yuav ua li cas kho qhov teeb meem, peb tsim rau peb tus kheej qee qhov kev xav tau uas kev daws teeb meem yuav tsum ua raws.
Txhawm rau tsim cov cai no, peb txiav txim siab nug peb tus kheej cov lus nug: "Peb muaj teeb meem thaum twg"? Thiab cov teeb meem, raws li nws tau muab tawm, tshwm sim hauv plaub qhov xwm txheej:
- Kho vajtse tsis ua hauj lwm.
- Kev pabcuam sab nraud ua tsis tau.
- Hloov cov software version (ib yam kev xa tawm).
- Explosive load loj hlob.
Peb yuav tsis tham txog thawj ob. Lub hardware malfunction yuav daws tau yooj yim heev: koj yuav tsum muaj txhua yam duplicated. Yog tias cov no yog cov disks, cov disks yuav tsum tau sib sau ua ke hauv RAID; yog tias qhov no yog server, tus neeg rau zaub mov yuav tsum tau muab luam tawm; yog tias koj muaj lub network infrastructure, koj yuav tsum muab daim ntawv thib ob ntawm lub network infrastructure, uas yog, koj coj nws thiab duplicate nws. Thiab yog tias ib yam dab tsi ua tsis tiav, koj hloov mus rau lub zog cia. Nws nyuaj rau hais dab tsi ntxiv ntawm no.
Qhov thib ob yog qhov tsis ua haujlwm ntawm kev pabcuam sab nraud. Rau feem ntau, qhov system tsis muaj teeb meem txhua, tab sis tsis yog rau peb. Txij li thaum peb ua cov nyiaj them poob haujlwm, peb yog ib qho kev sib sau ua ke ntawm cov neeg siv (uas nkag mus rau nws daim npav cov ntaub ntawv) thiab cov tsev txhab nyiaj, cov txheej txheem them nyiaj (Visa, MasterCard, Mira, thiab lwm yam). Peb cov kev pabcuam sab nraud (payment systems, banks) yuav ua tsis tau. Tsis yog peb lossis koj (yog tias koj muaj cov kev pabcuam) tuaj yeem cuam tshuam qhov no.
Yuav ua li cas tom qab ntawd? Muaj ob txoj kev xaiv ntawm no. Ua ntej, yog tias koj tuaj yeem ua tau, koj yuav tsum luam qhov kev pabcuam no hauv qee txoj kev. Piv txwv li, yog tias peb tuaj yeem ua tau, peb hloov tsheb los ntawm ib qho kev pabcuam mus rau lwm qhov: piv txwv li, daim npav tau ua tiav los ntawm Sberbank, Sberbank muaj teeb meem - peb hloov tsheb mus rau Raiffeisen. Qhov thib ob peb tuaj yeem ua tau yog kom pom qhov tsis ua haujlwm ntawm cov kev pabcuam sab nraud sai heev, thiab yog li peb yuav tham txog cov lus teb ceev hauv ntu tom ntej ntawm tsab ntawv tshaj tawm.
Qhov tseeb, ntawm plaub qhov no, peb tuaj yeem cuam tshuam tshwj xeeb rau kev hloov pauv ntawm software versions - ua cov haujlwm uas yuav ua rau muaj kev txhim kho hauv cov xwm txheej hauv cov ntsiab lus ntawm kev xa mus thiab hauv cov ntsiab lus ntawm kev loj hlob tawg hauv kev thauj khoom. Qhov tseeb, qhov ntawd yog qhov peb tau ua. Ntawm no, dua, ib daim ntawv me me ...
Ntawm plaub qhov teeb meem no, ob peb yuav daws tau tam sim yog tias koj muaj huab. Yog tias koj nyob hauv Microsoft Azhur, Ozone huab, lossis siv peb cov huab, los ntawm Yandex lossis Mail, ces yam tsawg kawg ntawm kev ua haujlwm kho vajtse dhau los ua lawv qhov teeb meem thiab txhua yam yuav zoo rau koj tam sim ntawd hauv cov ntsiab lus ntawm kev kho vajtse ua haujlwm tsis zoo.
Peb yog ib tug me ntsis unconventional tuam txhab. Ntawm no txhua tus tab tom tham txog "Kubernet", hais txog huab - peb tsis muaj "Kubernet" lossis huab. Tab sis peb muaj racks ntawm hardware nyob rau hauv ntau cov ntaub ntawv chaw, thiab peb raug yuam kom nyob rau hauv no hardware, peb raug yuam kom ua lub luag hauj lwm rau tag nrho cov. Yog li ntawd, peb yuav tham hauv cov ntsiab lus no. Yog li, hais txog cov teeb meem. Thawj ob tau muab tshem tawm ntawm cov kab tuav.
Hloov cov software version. Bases
Peb cov neeg tsim khoom tsis muaj kev nkag mus rau kev tsim khoom. Yog vim li cas? Nws tsuas yog tias peb yog PCI DSS tau ntawv pov thawj, thiab peb cov neeg tsim khoom tsuas yog tsis muaj cai nkag mus rau "cov khoom lag luam". Qhov ntawd yog nws, lub sijhawm. Tag nrho. Yog li ntawd, txoj kev loj hlob lub luag hauj lwm xaus raws nraim lub sij hawm thaum txoj kev loj hlob xa cov tsim rau kev tso tawm.
Peb lub hauv paus thib ob uas peb muaj, uas tseem pab tau peb ntau, yog qhov tsis muaj kev paub tsis muaj ntaub ntawv tshwj xeeb. Kuv vam tias nws yuav zoo ib yam rau koj. Vim yog qhov no tsis yog, koj yuav muaj teeb meem. Teeb meem yuav tshwm sim thaum qhov kev paub tshwj xeeb, tsis muaj ntaub ntawv pov thawj tsis muaj nyob rau lub sijhawm nyob hauv qhov chaw raug. Cia peb hais tias koj muaj ib tus neeg uas paub yuav ua li cas siv cov khoom tshwj xeeb - tus neeg tsis nyob ntawd, nws so haujlwm lossis mob - qhov ntawd yog nws, koj muaj teeb meem.
Thiab lub hauv paus thib peb uas peb tau los. Peb tuaj rau nws los ntawm kev mob, ntshav, kua muag - peb tuaj txog qhov xaus tias ib qho ntawm peb qhov kev tsim muaj qhov yuam kev, txawm tias nws tsis muaj qhov yuam kev. Peb txiav txim siab qhov no rau peb tus kheej: thaum peb xa ib yam dab tsi, thaum peb dov ib yam dab tsi rau hauv kev tsim khoom, peb muaj kev tsim nrog qhov yuam kev. Peb tau tsim cov kev cai uas peb qhov system yuav tsum ua kom txaus siab.
Yuav tsum tau hloov lub software version
Muaj peb qhov yuav tsum tau ua:
- Peb yuav tsum ceev nrooj rov qab xa mus.
- Peb yuav tsum txo qhov cuam tshuam ntawm kev xa tawm tsis tiav.
- Thiab peb yuav tsum muaj peev xwm xa mus sai sai rau qhov sib npaug.
Raws nraim li qhov kev txiav txim! Vim li cas? Vim hais tias, ua ntej ntawm tag nrho cov, thaum deploy ib tug tshiab version, ceev tsis yog ib qho tseem ceeb, tab sis nws yog ib qho tseem ceeb rau koj, yog hais tias ib yam dab tsi mus tsis ncaj ncees lawm, kom ceev ceev rov qab thiab muaj kev cuam tshuam tsawg heev. Tab sis yog tias koj muaj cov txheej txheem ntawm kev tsim khoom, uas nws hloov tawm tias muaj qhov yuam kev (tawm ntawm xiav, tsis muaj kev xa tawm, tab sis muaj qhov yuam kev) - qhov ceev ntawm kev xa mus tom ntej yog qhov tseem ceeb rau koj. Peb tau ua dab tsi los ua kom tau raws li cov kev xav tau no? Peb tau ua raws li cov txheej txheem hauv qab no:
Nws yog qhov paub zoo heev, peb tsis tau tsim nws - qhov no yog Xiav / Ntsuab xa mus. Nws yog dab tsi? Koj yuav tsum muaj daim ntawv theej rau txhua pab pawg ntawm cov servers uas koj daim ntawv thov raug teeb tsa. Daim ntawv luam yog "sov": tsis muaj tsheb khiav ntawm nws, tab sis thaum twg los tau lub tsheb no tuaj yeem xa mus rau daim ntawv no. Daim ntawv no muaj cov version dhau los. Thiab thaum lub sij hawm xa mus, koj yob tawm cov cai rau ib daim ntawv tsis siv. Tom qab ntawd koj hloov ib feem ntawm cov tsheb khiav (lossis tag nrho) mus rau qhov tshiab version. Yog li, txhawm rau hloov cov kev khiav tsheb khiav los ntawm cov qub version mus rau qhov tshiab, koj yuav tsum ua ib qho nkaus xwb: koj yuav tsum tau hloov qhov sib npaug ntawm cov dej ntws, hloov cov kev taw qhia - los ntawm ib qho dej mus rau lwm qhov. Qhov no yooj yim heev thiab daws qhov teeb meem ntawm kev hloov pauv sai thiab nrawm nrawm.Ntawm no cov kev daws teeb meem rau lo lus nug thib ob yog kev txo qis: koj tuaj yeem xa ib feem ntawm koj txoj kev tsheb mus rau ib txoj kab tshiab, mus rau ib txoj kab nrog tus lej tshiab (cia nws ua, piv txwv li, 2%). Thiab 2% no tsis yog 100%! Yog tias koj poob 100% ntawm koj txoj kev khiav tsheb vim qhov kev xa tawm tsis tiav, qhov ntawd txaus ntshai; yog tias koj poob 2% ntawm koj txoj kev khiav tsheb, qhov ntawd tsis txaus siab, tab sis nws tsis txaus ntshai. Ntxiv mus, cov neeg siv feem ntau yuav tsis pom qhov no, vim tias qee qhov (tsis yog tag nrho) tib tus neeg siv, nias F5, yuav raug coj mus rau lwm qhov, ua haujlwm version.
Xiav/Green deploy. Txoj kev
Txawm li cas los xij, tsis yog txhua yam yooj yim heev "Blue / Green deploy" ... Tag nrho peb cov khoom tuaj yeem muab faib ua peb pawg:
- qhov no yog frontend (cov nplooj ntawv them nyiaj uas peb cov neeg siv pom);
- kev ua haujlwm tseem ceeb;
- adapter rau kev ua hauj lwm nrog them nyiaj systems (banks, MasterCard, Visa ...).
Thiab muaj ib tug nuance ntawm no - lub nuance nyob rau hauv lub routing ntawm kab. Yog tias koj tsuas yog hloov 100% ntawm cov tsheb khiav, koj tsis muaj teeb meem no. Tab sis yog tias koj xav hloov 2%, koj pib nug cov lus nug: "Yuav ua li cas ua qhov no?" Qhov yooj yim tshaj plaws yog ncaj nraim rau pem hauv ntej: koj tuaj yeem teeb tsa Round Robin hauv nginx los ntawm kev xaiv random, thiab koj muaj 2% rau sab laug, 98% rau sab xis. Tab sis qhov no tsis yog ib txwm haum.
Piv txwv li, nyob rau hauv peb rooj plaub, tus neeg siv cuam tshuam nrog lub system nrog ntau tshaj ib qho kev thov. Qhov no yog ib txwm: 2, 3, 4, 5 thov - koj lub tshuab yuav zoo ib yam. Thiab yog tias nws yog ib qho tseem ceeb rau koj tias txhua tus neeg siv cov lus thov tuaj rau tib txoj kab uas thawj qhov kev thov tuaj, lossis (qhov thib ob) tag nrho cov neeg siv cov lus thov tuaj rau kab tshiab tom qab hloov pauv (nws tuaj yeem pib ua haujlwm ua ntej nrog cov system, ua ntej hloov), - ces qhov no random faib tsis haum rau koj. Tom qab ntawd muaj cov kev xaiv hauv qab no:
Thawj qhov kev xaiv, qhov yooj yim tshaj plaws, yog ua raws li cov txheej txheem ntawm tus neeg siv khoom (IP Hash). Koj muaj tus IP, thiab koj faib nws los ntawm sab xis mus rau sab laug los ntawm tus IP chaw nyob. Tom qab ntawd cov ntaub ntawv thib ob uas kuv tau piav qhia yuav ua haujlwm rau koj, thaum qhov kev xa tawm tshwm sim, tus neeg siv tuaj yeem pib ua haujlwm nrog koj lub cev, thiab txij li lub sijhawm xa tawm tag nrho cov kev thov yuav mus rau kab tshiab (rau tib qho, hais).Yog tias vim li cas qhov no tsis haum rau koj thiab koj yuav tsum xa cov lus thov mus rau kab uas tus neeg siv thawj zaug, kev thov thawj zaug tuaj, ces koj muaj ob txoj kev xaiv ...
Thawj qhov kev xaiv: koj tuaj yeem yuav them nginx +. Nws muaj ib qho kev sib tham sib tham, uas, raws li tus neeg siv qhov kev thov thawj zaug, muab qhov kev sib tham rau tus neeg siv thiab khi nws mus rau ib qho lossis lwm qhov dej ntws. Tag nrho cov kev thov tom ntej ntawm cov neeg siv nyob rau hauv qhov kev sib kho lub neej yuav raug xa mus rau tib lub nce toj uas qhov kev sib kho tau tshaj tawm.Qhov no tsis haum rau peb vim peb twb muaj nginx tsis tu ncua. Hloov mus rau nginx + tsis yog nws kim, nws tsuas yog tias nws mob me ntsis rau peb thiab tsis yog. Piv txwv li, "Sticks Sessions", tsis ua haujlwm rau peb rau qhov yooj yim vim li cas "Sticks Sessions" tsis tso cai routing raws li "Ib leeg-los yog". Nyob ntawd koj tuaj yeem qhia meej tias peb "Tshaj Sib Tham" ua li cas, piv txwv li, los ntawm IP chaw nyob lossis los ntawm IP chaw nyob thiab ncuav qab zib lossis los ntawm kev xa ntawv, tab sis "Txawm li cas los xij" yog qhov nyuaj dua nyob rau ntawd.
Yog li ntawd, peb tuaj rau qhov kev xaiv thib plaub. Peb coj nginx ntawm steroids (qhov no yog openresty) - qhov no yog tib nginx, uas ntxiv txhawb kev suav nrog cov ntawv kawg. Koj tuaj yeem sau tsab ntawv kawg, muab nws "qhib so", thiab tsab ntawv kawg no yuav raug ua tiav thaum tus neeg siv thov tuaj.
Thiab peb tau sau, qhov tseeb, xws li tsab ntawv, teem peb tus kheej "openresti" thiab hauv tsab ntawv no peb txheeb los ntawm 6 qhov sib txawv los ntawm concatenation "Los yog". Nyob ntawm seb muaj ib lossis lwm qhov parameter, peb paub tias tus neeg siv tuaj rau ib nplooj ntawv lossis lwm qhov, ib kab lossis lwm qhov.
Xiav/Green deploy. Qhov zoo thiab qhov tsis zoo
Tau kawg, tej zaum nws yuav ua tau kom yooj yim me ntsis (siv tib "Sticky Sessions"), tab sis peb kuj muaj xws li ib tug nuance uas tsis tsuas yog tus neeg siv cuam tshuam nrog peb nyob rau hauv lub moj khaum ntawm ib tug ua ntawm ib tug lw ... Tab sis cov txheej txheem them nyiaj kuj cuam tshuam nrog peb: Tom qab peb ua cov kev hloov pauv (los ntawm kev xa ib daim ntawv thov mus rau qhov kev them nyiaj), peb tau txais ib qho coolback.
Thiab cia peb hais, yog tias hauv peb lub voj voog peb tuaj yeem xa tus neeg siv tus IP chaw nyob hauv txhua qhov kev thov thiab faib cov neeg siv raws li qhov chaw nyob IP, ces peb yuav tsis qhia tib "Visa": "Tus txiv neej, peb yog lub tuam txhab retro, peb zoo li. kom muaj thoob ntiaj teb (ntawm lub vev xaib thiab hauv Russia) ... Thov muab peb tus neeg siv tus IP chaw nyob hauv ib qho ntxiv, koj cov txheej txheem yog tus qauv"! Nws yog qhov tseeb tias lawv yuav tsis pom zoo.
Yog li ntawd, qhov no tsis ua hauj lwm rau peb - peb tau ua openresty. Raws li, nrog routing peb tau txais ib yam dab tsi zoo li no:Blue/Green Deployment muaj, raws li, qhov zoo uas kuv tau hais thiab qhov tsis zoo.
Ob qhov tsis zoo:
- koj yuav tsum thab nrog routing;
- qhov tsis zoo thib ob yog qhov kev siv nyiaj.
Koj xav tau ob zaug ntau dua servers, koj xav tau ob zaug ntau li ntau qhov kev ua haujlwm, koj yuav tsum tau siv ob zaug ntau npaum li kev siv zog los tswj tag nrho lub vaj tsiaj.
Los ntawm txoj kev, ntawm qhov zoo muaj ib qho ntxiv uas kuv tsis tau hais ua ntej: koj muaj peev xwm khaws cia hauv cov ntaub ntawv ntawm kev loj hlob. Yog tias koj muaj kev loj hlob tawg hauv kev thauj khoom, koj muaj cov neeg siv ntau, ces koj tsuas yog suav nrog kab thib ob hauv 50 txog 50 kev faib tawm - thiab koj tam sim ntawd muaj x2 servers hauv koj pawg kom txog thaum koj daws qhov teeb meem ntawm muaj ntau servers.
Yuav ua li cas kom xa mus sai?
Peb tau tham txog yuav ua li cas los daws qhov teeb meem ntawm kev txo qis thiab rov qab sai, tab sis cov lus nug tseem nyob: "Yuav ua li cas xa sai sai?"
Nws yog luv thiab yooj yim ntawm no.- Koj yuav tsum muaj CD system (Kev xa mus tas li) - koj tsis tuaj yeem nyob yam tsis muaj nws. Yog tias koj muaj ib lub server, koj tuaj yeem siv manually. Peb muaj txog ib thiab ib nrab txhiab tus servers thiab ib thiab ib nrab txhiab tus tes tuav, tau kawg - peb tuaj yeem cog ib chav haujlwm loj ntawm chav tsev no tsuas yog siv.
- Kev xa tawm yuav tsum yog sib npaug. Yog tias koj qhov kev xa tawm ua ntu zus, ces txhua yam tsis zoo. Ib tus neeg rau zaub mov yog qhov qub, koj yuav tau xa ib thiab ib nrab txhiab servers txhua hnub.
- Ntxiv dua thiab, rau acceleration, qhov no tej zaum tsis tsim nyog. Thaum lub sij hawm xa mus, qhov project feem ntau yog tsim. Koj muaj lub vev xaib qhov project, muaj qhov ua ntej-kawg (koj ua lub vev xaib muaj, koj sau npm - ib yam dab tsi zoo li ntawd), thiab cov txheej txheem no, hauv txoj cai, luv luv - 5 feeb, tab sis cov 5 feeb no tuaj yeem ua qhov tseem ceeb. Yog vim li cas, piv txwv li, peb tsis ua li ntawd: peb tshem tawm cov 5 feeb no, peb xa cov khoom cuav.
Dab tsi yog artifact? Ib qho khoom cuav yog ib qho kev sib sau ua ke uas tag nrho cov khoom sib dhos tau ua tiav lawm. Peb khaws cov khoom cuav no rau hauv qhov khoom cuav. Nyob rau tib lub sij hawm peb tau siv ob qhov chaw khaws cia - nws yog Nexus thiab tam sim no jFrog Artifactory). Tom qab ntawd lawv muab qee cov ntawv sau npe hauv PHP rau hauv; thiab "Nexus" twb tsis haum lawm, thiab yog li ntawd peb xaiv jFrog Artefactory, uas muaj peev xwm artifactorize yuav luag txhua yam. Peb tau los txog rau lub ntsiab lus uas nyob rau hauv cov khoom pov thawj no peb khaws peb tus kheej binary pob uas peb sau rau servers.
Explosive load loj hlob
Peb tham txog kev hloov cov software version. Tom ntej no peb muaj yog ib tug tawg nce nyob rau hauv load. Ntawm no, kuv tej zaum txhais tau hais tias los ntawm kev loj hlob tawg ntawm lub load tsis yog qhov zoo ...
Peb tau sau cov txheej txheem tshiab - nws yog kev pabcuam-taw qhia, zam, zoo nkauj, neeg ua haujlwm nyob txhua qhov chaw, queues txhua qhov chaw, asynchrony txhua qhov chaw. Thiab hauv cov tshuab no, cov ntaub ntawv tuaj yeem ntws los ntawm kev sib txawv. Rau kev hloov pauv thawj zaug, 1st, 3rd, 10th tus neeg ua haujlwm tuaj yeem siv, rau kev hloov pauv thib ob - thib 2, 4, 5th. Thiab niaj hnub no, cia peb hais, thaum sawv ntxov koj muaj cov ntaub ntawv ntws uas siv thawj peb tus neeg ua haujlwm, thiab thaum yav tsaus ntuj nws hloov pauv, thiab txhua yam siv peb tus neeg ua haujlwm.
Thiab ntawm no nws hloov tawm tias koj yuav tsum tau ua qee yam ntsuas cov neeg ua haujlwm, koj yuav tsum ntsuas koj cov kev pabcuam, tab sis tib lub sijhawm tiv thaiv cov peev txheej bloat.
Peb tau txhais peb cov cai. Cov kev cai no yog qhov yooj yim heev: tias muaj Kev Pabcuam Tshawb Nrhiav, parameterization - txhua yam yog tus qauv rau kev tsim cov txheej txheem scalable, tshwj tsis yog rau ib qho - cov peev txheej depreciation. Peb tau hais tias peb tsis tau npaj kom txo cov peev txheej kom cov servers cua sov. Peb coj "Consul", peb coj "Nomad", uas tswj peb cov neeg ua haujlwm.Vim li cas qhov no yog qhov teeb meem rau peb? Cia peb rov qab me ntsis. Tam sim no peb muaj nyob ib ncig ntawm 70 lub tshuab them nyiaj tom qab peb. Thaum sawv ntxov, kev khiav mus los ntawm Sberbank, ces Sberbank poob, piv txwv li, thiab peb hloov mus rau lwm qhov kev them nyiaj. Peb muaj 100 tus neeg ua haujlwm ua ntej Sberbank, thiab tom qab ntawd peb yuav tsum tau nce 100 tus neeg ua haujlwm rau lwm qhov kev them nyiaj. Thiab nws yog qhov tsim nyog rau txhua qhov no tshwm sim yam tsis muaj tib neeg kev koom tes. Vim hais tias yog muaj tib neeg kev koom tes, yuav tsum muaj ib tug engineer zaum ntawd 24/7, leej twg yuav tsum tsuas yog ua qhov no, vim hais tias xws li tsis ua hauj lwm, thaum 70 systems nyob tom qab koj, tshwm sim tsis tu ncua.
Yog li ntawd, peb saib Nomad, uas muaj qhov qhib IP, thiab sau peb tus kheej li, Scale-Nomad - ScaleNo, uas ua kwv yees li hauv qab no: nws saib xyuas kev loj hlob ntawm cov kab thiab txo qis lossis nce cov neeg ua haujlwm nyob ntawm qhov muaj zog. ntawm cov kab. Thaum peb ua nws, peb xav tias: "Tej zaum peb tuaj yeem qhib qhov chaw nws?" Ces lawv ntsia nws - nws yooj yim li ob kopecks.
Txog tam sim no peb tsis tau qhib qhov chaw nws, tab sis yog tias mam li nco dheev tom qab daim ntawv tshaj tawm, tom qab paub tias koj xav tau ib yam dab tsi, koj xav tau, kuv cov neeg nyob hauv qhov kawg swb - thov sau ntawv rau kuv. Yog tias muaj tsawg kawg yog 3-5 tus neeg, peb yuav txhawb nqa.
Nws ua haujlwm li cas? Cia peb saib! Saib tom ntej: ntawm sab laug muaj ib daim ntawm peb kev saib xyuas: qhov no yog ib kab, nyob rau sab saum toj yog lub sijhawm ua haujlwm, hauv nruab nrab yog tus lej ntawm kev hloov pauv, hauv qab yog cov neeg ua haujlwm.Yog tias koj saib, muaj glitch hauv daim duab no. Nyob rau hauv daim ntawv qhia saum toj kawg nkaus, ib qho ntawm cov kab kos poob hauv 45 vib nas this - ib qho ntawm cov kev them nyiaj poob qis. Tam sim ntawd, kev thauj mus los hauv 2 feeb thiab cov kab pib loj hlob ntawm lwm qhov kev them nyiaj, uas tsis muaj neeg ua haujlwm (peb tsis siv cov peev txheej - ntawm qhov tsis sib xws, peb muab cov peev txheej pov tseg kom raug). Peb tsis xav kom sov - muaj tsawg kawg nkaus, kwv yees li 5-10 tus neeg ua haujlwm, tab sis lawv tsis tuaj yeem tiv taus.
Daim duab kawg qhia tau hais tias "hump", uas tsuas yog txhais tau tias "Skaleno" ob npaug ntawm qhov nyiaj no. Thiab tom qab ntawd, thaum daim duab poob me ntsis, nws txo qis me ntsis - tus naj npawb ntawm cov neeg ua haujlwm tau hloov pauv. Qhov no ua haujlwm li cas. Peb tham txog tus lej 2 - "Yuav ua li cas kom tshem tau cov laj thawj sai sai."
Saib xyuas. Yuav ua li cas kom paub qhov teeb meem sai sai?
Tam sim no thawj lub ntsiab lus yog "Yuav ua li cas kom paub qhov teeb meem sai sai?" Saib xyuas! Peb yuav tsum to taub tej yam sai sai. Peb yuav tsum to taub sai npaum li cas?
Peb yam!- Peb yuav tsum nkag siab thiab nkag siab sai sai ntawm kev ua haujlwm ntawm peb tus kheej cov peev txheej.
- Peb yuav tsum nkag siab sai sai rau qhov ua tsis tiav thiab saib xyuas kev ua haujlwm ntawm cov tshuab uas sab nraud rau peb.
- Lub ntsiab lus thib peb yog txheeb xyuas cov laj thawj tsis raug. Qhov no yog thaum lub kaw lus ua haujlwm rau koj, txhua yam yog ib txwm ua raws li txhua qhov ntsuas, tab sis qee yam mus tsis ncaj ncees lawm.
Tej zaum kuv yuav tsis qhia koj ib yam dab tsi uas txias ntawm no. Kuv yuav yog Captain Obvious. Peb tab tom nrhiav dab tsi hauv khw. Peb muaj "kev lom zem zoo". Nov yog hom tsiaj uas peb muaj tam sim no:
Peb siv Zabbix los saib xyuas kho vajtse, los saib xyuas cov cim tseem ceeb ntawm cov servers. Peb siv Okmeter rau databases. Peb siv "Grafana" thiab "Prometheus" rau tag nrho lwm cov ntsuas uas tsis haum rau thawj ob, ib txhia nrog "Grafana" thiab "Prometheus", thiab ib co nrog "Grafana" nrog "Influx" thiab Telegraf.Ib xyoo dhau los peb xav siv New Relic. Qhov txias, nws tuaj yeem ua tau txhua yam. Tab sis ntau npaum li nws ua tau txhua yam, nws kim heev. Thaum peb loj hlob mus rau qhov ntim ntawm 1,5 txhiab servers, ib tus neeg muag khoom tuaj rau peb thiab hais tias: "Cia peb xaus qhov kev pom zoo rau xyoo tom ntej." Peb saib tus nqi thiab hais tias tsis yog, peb yuav tsis ua li ntawd. Tam sim no peb tau tso tseg New Relic, peb muaj txog 15 servers sab laug nyob rau hauv kev saib xyuas ntawm New Relic. Tus nqi muab tawm los yog qus kiag li.
Thiab muaj ib lub cuab yeej uas peb siv peb tus kheej - qhov no yog Debugger. Thaum xub thawj peb hu nws "Bagger," tab sis tom qab ntawd tus kws qhia lus Askiv hla dhau, luag nyav, thiab hloov npe nws "Debagger." Nws yog dab tsi? Qhov no yog ib qho cuab yeej uas, qhov tseeb, nyob rau hauv 15-30 vib nas this ntawm txhua yam, xws li "lub thawv dub" ntawm lub kaw lus, khiav kev ntsuam xyuas ntawm tag nrho cov kev ua tau zoo ntawm cov khoom.
Piv txwv li, yog tias muaj nplooj ntawv sab nraud ( nplooj ntawv them nyiaj), nws tsuas qhib nws thiab saib seb nws yuav tsum zoo li cas. Yog tias qhov no ua tiav, nws xa qhov kev xeem "kev lag luam" thiab ua kom paub tseeb tias qhov "kev lag luam" tuaj txog. Yog tias qhov no yog kev sib txuas nrog cov txheej txheem them nyiaj, peb tua cov ntawv thov raws li qhov peb tuaj yeem ua tau, thiab pom tias txhua yam zoo nrog peb.
Cov ntsuas dab tsi tseem ceeb rau kev saib xyuas?
Peb saib xyuas dab tsi? Cov ntsuas twg tseem ceeb rau peb?
- Lub sij hawm teb / RPS ntawm pem hauv ntej yog qhov taw qhia tseem ceeb heev. Nws teb tam sim ntawd tias muaj ib yam dab tsi tsis ncaj ncees lawm rau koj.
- Tus naj npawb ntawm cov lus ua tiav hauv txhua kab.
- Tus naj npawb ntawm cov neeg ua haujlwm.
- Basic correctness metrics.
Lub ntsiab lus kawg yog "kev lag luam", "kev lag luam" metric. Yog tias koj xav saib xyuas tib yam, koj yuav tsum tau txhais ib lossis ob qhov kev ntsuas uas yog qhov ntsuas tseem ceeb rau koj. Peb metric yog throughput (qhov no yog qhov piv ntawm tus naj npawb ntawm kev vam meej kev lag luam mus rau tag nrho cov kev khiav hauj lwm khiav). Yog tias muaj qee yam hloov pauv hauv nws lub sijhawm 5-10-15 feeb, nws txhais tau tias peb muaj teeb meem (yog tias nws hloov pauv).
Yuav ua li cas nws zoo li rau peb yog ib qho piv txwv ntawm peb lub rooj tsavxwm:
Nyob rau sab laug muaj 6 daim duab, qhov no yog raws li cov kab - tus naj npawb ntawm cov neeg ua haujlwm thiab cov xov tooj ntawm cov kab. Ntawm sab xis - RPS, RTS. Hauv qab no yog tib qhov "kev lag luam" metric. Thiab nyob rau hauv "kev lag luam" metric peb tuaj yeem pom tam sim ntawd tias qee yam ua tsis raug hauv ob daim duab nruab nrab ... Qhov no tsuas yog lwm qhov system uas sawv tom qab peb uas tau poob.Qhov thib ob peb yuav tsum tau ua yog saib xyuas lub caij nplooj zeeg ntawm kev them nyiaj sab nraud. Ntawm no peb coj OpenTracing - lub tshuab, tus qauv, qauv uas tso cai rau koj taug qab cov kab ke faib; thiab nws tau hloov me ntsis. Tus qauv OpenTracing paradigm hais tias peb tsim ib txoj hauv kev rau txhua tus neeg thov. Peb tsis xav tau qhov no, thiab peb qhwv nws hauv cov ntsiab lus, sib sau ua ke. Peb tau tsim ib lub cuab yeej uas tso cai rau peb taug qab qhov ceev ntawm lub tshuab tom qab peb.
Daim duab qhia peb tias ib qho ntawm cov kev them nyiaj pib teb hauv 3 vib nas this - peb muaj teeb meem. Ntxiv mus, qhov no yuav tshwm sim thaum muaj teeb meem pib, ntawm ib ntus ntawm 20-30 vib nas this.Thiab chav kawm thib peb ntawm kev saib xyuas qhov tsis raug uas muaj nyob yog kev soj ntsuam xyuas qhov tseeb.
Yuav kom ncaj ncees, kuv tsis paub yuav ua li cas kos rau ntawm daim slide no, vim hais tias peb tau nrhiav ntev ntev ntawm kev ua lag luam rau tej yam uas yuav haum rau peb. Peb tsis pom dab tsi, yog li peb yuav tsum ua nws tus kheej.
Kuv txhais li cas los ntawm kev soj ntsuam xyuas? Zoo, xav txog: koj ua koj tus kheej ib qho system (piv txwv li, Tinder clone); koj ua nws, launched nws. Tus thawj coj ua tau zoo Vasya Pupkin muab tso rau hauv nws lub xov tooj, pom ib tug ntxhais nyob ntawd, nyiam nws ... thiab qhov zoo li tsis mus rau tus ntxhais - qhov zoo li mus rau tus neeg saib xyuas kev ruaj ntseg Mikhalych los ntawm tib lub chaw lag luam. Tus thawj coj mus rau pem hauv ntej, thiab ces xav tias: "Vim li cas tus neeg saib xyuas kev ruaj ntseg Mikhalych luag ntxhi rau nws?"Nyob rau hauv cov xwm txheej zoo li no... Rau peb, qhov xwm txheej no suab txawv me ntsis, vim (Kuv tau sau) qhov no yog kev poob lub koob npe nrov uas tsis ncaj qha ua rau kev poob nyiaj txiag. Peb qhov xwm txheej yog qhov tsis sib xws: peb yuav raug kev poob nyiaj ncaj qha - piv txwv li, yog tias peb tau ua tiav kev lag luam, tab sis nws ua tsis tiav (lossis rov ua dua). Kuv yuav tsum tau sau kuv tus kheej cov cuab yeej uas taug qab cov naj npawb ntawm kev ua tiav kev lag luam nyob rau lub sijhawm siv cov ntsuas kev lag luam. Tsis pom dab tsi hauv khw! Qhov no yog raws nraim lub tswv yim kuv xav qhia. Tsis muaj ib yam dab tsi hauv khw los daws qhov teeb meem no.
Qhov no yog hais txog yuav ua li cas kom paub qhov teeb meem sai sai.
Yuav ua li cas txiav txim siab vim li cas rau kev xa tawm
Pab pawg thib peb ntawm cov teeb meem uas peb daws tau yog tom qab peb tau txheeb xyuas qhov teeb meem, tom qab peb tshem tawm nws, nws yuav zoo rau kev nkag siab txog qhov laj thawj ntawm kev txhim kho, kev sim, thiab ua ib yam dab tsi txog nws. Raws li, peb yuav tsum tau soj ntsuam, peb yuav tsum tau tsa lub cav.
Yog tias peb tab tom tham txog cov cav ntoo (qhov laj thawj tseem ceeb yog cov cav), feem ntau ntawm peb cov cav yog nyob rau hauv ELK Stack - yuav luag txhua tus muaj tib yam. Rau qee qhov, nws yuav tsis nyob hauv ELK, tab sis yog tias koj sau cov ntawv teev lus hauv gigabytes, tom qab ntawd sai lossis tom qab koj yuav tuaj rau ELK. Peb sau lawv hauv terabytes.
Muaj teeb meem ntawm no. Peb kho nws, kho qhov yuam kev rau tus neeg siv, pib khawb tawm qhov muaj nyob ntawd, nce mus rau Kibana, nkag mus rau hauv tus lej pauv nyob rau ntawd thiab tau txais ib daim ntaub zoo li no (qhia ntau). Thiab tsis muaj dab tsi yog qhov tseeb hauv daim ntaub no. Vim li cas? Yog, vim nws tsis paub meej tias qhov twg yog tus neeg ua haujlwm, qhov twg yog cov khoom siv. Thiab lub sijhawm ntawd peb pom tau tias peb xav tau kev taug qab - tib yam OpenTracing uas kuv tau hais txog.Peb xav qhov no ib xyoos dhau los, tig peb lub siab rau kev ua lag luam, thiab muaj ob lub cuab yeej nyob ntawd - "Zipkin" thiab "Jaeger". "Jager" yog nyob rau hauv qhov tseeb xws li ib tug ideological txais cuab tam, ib tug ideological successor ntawm "Zipkin". Txhua yam zoo hauv Zipkin, tshwj tsis yog tias nws tsis paub yuav ua li cas sib sau ua ke, nws tsis paub yuav ua li cas suav nrog cov cav hauv cov kab, tsuas yog lub sijhawm taug qab. Thiab "Jager" txhawb qhov no.
Peb saib ntawm "Jager": koj tuaj yeem ntsuas cov ntawv thov, koj tuaj yeem sau hauv Api (tus qauv Api rau PHP thaum lub sijhawm ntawd, txawm li cas los xij, tsis pom zoo - qhov no yog ib xyoos dhau los, tab sis tam sim no nws twb tau pom zoo lawm), tab sis muaj yog tsis muaj neeg siv kiag li. "Okay," peb xav, thiab sau peb tus kheej cov neeg siv khoom. Peb tau txais dab tsi? Qhov no yog kwv yees li nws zoo li:
Hauv Jaeger, spans yog tsim rau txhua cov lus. Ntawd yog, thaum tus neeg siv qhib lub kaw lus, nws pom ib lossis ob qhov thaiv rau txhua qhov kev thov tuaj (1-2-3 - tus naj npawb ntawm cov neeg thov tuaj, tus naj npawb ntawm cov blocks). Txhawm rau ua kom yooj yim rau cov neeg siv, peb ntxiv cov cim npe rau cov cav thiab lub sijhawm taug qab. Raws li, nyob rau hauv cov ntaub ntawv ntawm qhov yuam kev, peb daim ntawv thov yuav kos lub cav nrog rau qhov tsim nyog yuam kev. Koj tuaj yeem lim los ntawm qhov yuam kev lossis tsuas yog spans uas muaj qhov thaiv no nrog qhov yuam kev yuav tshwm sim. Nov yog qhov nws zoo li yog tias peb nthuav dav:
Nyob rau hauv lub span muaj ib co kab. Hauv qhov no, cov no yog peb qhov kev sim, thiab qhov thib peb kab qhia qhia peb tias muaj qhov yuam kev tshwm sim. Nyob rau tib lub sijhawm, ntawm no peb pom ib lub sijhawm taug qab: peb muaj lub sijhawm teev nyob rau saum toj, thiab peb pom lub sijhawm li cas lub sijhawm no lossis lub log ntawd tau kaw.Yog li ntawd, tej yam mus zoo rau peb. Peb sau peb tus kheej txuas ntxiv thiab peb qhib qhov chaw nws. Yog tias koj xav ua haujlwm nrog kev taug qab, yog tias koj xav ua haujlwm nrog "Jager" hauv PHP, muaj peb qhov txuas ntxiv, txais tos siv, raws li lawv hais:
Peb muaj qhov txuas ntxiv no - nws yog tus neeg siv khoom rau OpenTracing Api, nws tau ua raws li php-extention, uas yog, koj yuav tsum tau sib sau ua ke thiab nruab rau hauv lub system. Ib xyoos dhau los tsis muaj dab tsi txawv. Tam sim no muaj lwm cov neeg siv khoom zoo li cov khoom siv. Ntawm no nws yog nyob ntawm koj: tog twg los koj tso tawm cov khoom nrog tus kws sau ntawv, lossis koj siv txuas ntxiv rau koj.Cov qauv kev lag luam
Peb tham txog peb nqe lus txib. Nqe lus txib thib plaub yog kom ua raws li cov txheej txheem. Qhov no yog dab tsi? Nws yog hais txog qhov no:
Vim li cas lo lus "corporate" nyob ntawm no? Tsis yog vim peb yog ib lub tuam txhab loj lossis bureaucratic, tsis yog! Kuv xav siv lo lus "kev lag luam" ntawm no hauv cov ntsiab lus uas txhua lub tuam txhab, txhua yam khoom yuav tsum muaj nws tus qauv, suav nrog koj. Peb muaj cov qauv dab tsi?
- Peb muaj cov kev cai xa tawm. Peb tsis tsiv mus nyob qhov twg yog tsis muaj nws, peb ua tsis tau. Peb xa tawm txog 60 zaug hauv ib lub lis piam, uas yog, peb xa yuav luag tas li. Nyob rau tib lub sijhawm, peb muaj, piv txwv li, hauv kev xa tawm cov cai txwv tsis pub tshaj tawm rau hnub Friday - hauv txoj cai, peb tsis xa mus.
- Peb xav tau cov ntaub ntawv. Tsis yog ib qho kev tivthaiv tshiab tau nkag mus rau hauv kev tsim khoom yog tias tsis muaj ntaub ntawv rau nws, txawm tias nws tau yug los nyob rau hauv tus cwj mem ntawm peb cov kws tshwj xeeb RnD. Peb xav tau los ntawm lawv cov lus qhia xa mus, daim ntawv qhia kev soj ntsuam thiab cov lus piav qhia ntxhib (zoo, raws li cov programmers tuaj yeem sau tau) ntawm cov khoom siv no ua haujlwm li cas, yuav daws teeb meem li cas.
- Peb daws tsis yog qhov ua rau ntawm qhov teeb meem, tab sis qhov teeb meem - qhov kuv twb tau hais lawm. Nws yog ib qho tseem ceeb rau peb los tiv thaiv tus neeg siv los ntawm cov teeb meem.
- Peb muaj clearances. Piv txwv li, peb tsis xav txog nws downtime yog tias peb poob 2% ntawm cov tsheb khiav hauv ob feeb. Qhov no yeej tsis suav nrog hauv peb cov txheeb cais. Yog tias nws ntau dua nyob rau hauv feem pua los yog ib ntus, peb twb suav.
- Thiab peb ib txwm sau postmortems. Txawm li cas los xij tshwm sim rau peb, txhua qhov xwm txheej uas ib tus neeg coj txawv txawv hauv kev tsim khoom yuav raug cuam tshuam hauv kev tuag tom qab tuag. Postmortem yog ib daim ntawv uas koj sau dab tsi tshwm sim rau koj, lub sij hawm kom ntxaws, koj tau ua dab tsi los kho nws thiab (qhov no yog qhov yuav tsum tau thaiv!) koj yuav ua dab tsi los tiv thaiv qhov no tsis tshwm sim yav tom ntej. Qhov no yog qhov yuav tsum tau ua thiab tsim nyog rau kev txheeb xyuas tom ntej.
Dab tsi yog suav tias yog downtime?
Txhua yam no ua rau dab tsi?Qhov no ua rau qhov tseeb tias (peb muaj qee yam teeb meem nrog kev ruaj ntseg, qhov no tsis haum rau cov neeg siv khoom lossis peb) dhau 6 lub hlis peb qhov ntsuas kev ruaj ntseg yog 99,97. Peb tuaj yeem hais tias qhov no tsis yog ntau heev. Yog, peb muaj ib yam dab tsi los siv zog rau. Ntawm qhov ntsuas no, kwv yees li ib nrab ntawm kev ruaj ntseg, zoo li nws yog, tsis yog ntawm peb, tab sis ntawm peb lub vev xaib daim ntawv thov firewall, uas sawv ntawm peb sab xub ntiag thiab siv los ua kev pabcuam, tab sis cov neeg siv khoom tsis quav ntsej txog qhov no.
Peb kawm pw hmo ntuj. Thaum kawg! Rau lub hlis dhau los peb ua tsis tau. Thiab ntawm daim ntawv no nrog cov txiaj ntsig, kuv xav ua ib daim ntawv. Nag hmo muaj ib daim ntawv qhia zoo heev txog kev tswj hwm rau lub tshuab hluav taws xob nuclear. Yog tias cov neeg uas sau cov kab ke no tuaj yeem hnov kuv, thov tsis nco qab txog qhov kuv tau hais txog "2% tsis yog lub sijhawm poob." Rau koj, 2% yog downtime, txawm tias ob feeb!
Yog tag nrho! Koj cov lus nug.
Hais txog balancers thiab database migration
Cov lus nug los ntawm cov neeg tuaj saib (tom qab no – B): – Nyob zoo hmo ntuj. Ua tsaug ntau rau cov ntawv qhia admin! Ib nqe lus nug luv luv txog koj cov khoom sib npaug. Koj tau hais tias koj muaj WAF, uas yog, raws li kuv nkag siab nws, koj siv qee yam khoom siv sab nraud ...
EK: - Tsis yog, peb siv peb cov kev pabcuam raws li kev sib npaug. Hauv qhov no, WAF tsuas yog ib qho kev tiv thaiv DDoS rau peb.
Hauv: - Koj puas tuaj yeem hais ob peb lo lus hais txog kev sib npaug?
EK: - Raws li kuv twb tau hais lawm, qhov no yog pab pawg ntawm servers hauv openresty. Peb tam sim no muaj 5 pab pawg tshwj xeeb uas teb tshwj xeeb ... uas yog, lub server uas ua haujlwm tshwj xeeb openresty, nws tsuas yog tso npe nkag mus. Raws li, kom nkag siab ntau npaum li cas peb tuav: tam sim no peb muaj cov tsheb khiav tsis tu ncua ntawm ntau pua megabits. Lawv tiv taus, lawv zoo siab, lawv tsis strain lawv tus kheej.
Hauv: – Kuj yog ib lo lus nug yooj yim. Ntawm no yog Blue/Green deployment. Koj ua li cas, piv txwv li, nrog database migrations?
EK: - Cov lus nug zoo! Saib, hauv Blue/Green deployment peb muaj nyias queues rau txhua kab. Ntawd yog, yog tias peb tab tom tham txog cov xwm txheej queues uas tau kis los ntawm tus neeg ua haujlwm mus rau tus neeg ua haujlwm, muaj cais cov kab xiav thiab rau kab ntsuab. Yog tias peb tab tom tham txog cov ntaub ntawv nws tus kheej, ces peb txhob txwm txo qis nws ntau npaum li peb ua tau, txav txhua yam mus rau hauv kab; hauv cov ntaub ntawv peb tsuas yog khaws cov khoom lag luam. Thiab peb cov khoom sib pauv yog tib yam rau txhua kab. Nrog rau cov ntaub ntawv hauv cov ntsiab lus no: peb tsis faib rau hauv xiav thiab ntsuab, vim hais tias ob qho tib si ntawm cov cai yuav tsum paub tias muaj dab tsi tshwm sim nrog kev hloov pauv.
Cov phooj ywg, kuv kuj muaj me ntsis nqi zog los txhawb koj ntawm - ib phau ntawv. Thiab kuv yuav tsum tau txais nws rau cov lus nug zoo tshaj plaws.
Hauv: - Nyob zoo. Ua tsaug rau daim ntawv qhia. Lo lus nug yog qhov no. Koj saib xyuas kev them nyiaj, koj saib xyuas cov kev pabcuam uas koj sib txuas lus ... Tab sis koj yuav ua li cas saib xyuas kom ib tus neeg tuaj rau koj nplooj ntawv them nyiaj, them nyiaj, thiab qhov project tau muab nyiaj rau nws? Qhov ntawd yog, koj yuav saib xyuas li cas tias lub marchant muaj thiab tau txais koj qhov kev hu rov qab?
EK: - "Me tub lag luam" rau peb nyob rau hauv cov ntaub ntawv no yog raws nraim tib yam kev pab cuam sab nraud raws li cov kev them nyiaj system. Peb saib xyuas cov tub lag luam cov lus teb ceev.
Hais txog database encryption
Hauv: - Nyob zoo. Kuv muaj lus nug me ntsis txog. Koj muaj PCI DSS cov ntaub ntawv rhiab heev. Kuv xav paub yuav ua li cas koj khaws PANs hauv cov kab uas koj xav tau hloov mus rau hauv? Koj puas siv ib qho encryption? Thiab qhov no ua rau cov lus nug thib ob: raws li PCI DSS, nws yog ib qho tsim nyog yuav tsum tau rov encrypt cov ntaub ntawv ib ntus thaum muaj kev hloov pauv (kev tshem tawm ntawm cov thawj coj, thiab lwm yam) - yuav ua li cas rau kev nkag mus rau qhov no?
EK: - Cov lus nug zoo kawg! Ua ntej, peb tsis khaws PANs hauv cov kab. Peb tsis muaj cai khaws PAN nyob txhua qhov chaw hauv daim ntawv ntshiab, raws li txoj cai, yog li peb siv qhov kev pabcuam tshwj xeeb (peb hu ua "Kademon") - qhov no yog ib qho kev pabcuam uas ua ib yam nkaus xwb: nws tau txais cov lus raws li kev nkag thiab xa tawm cov lus encrypted. Thiab peb khaws txhua yam nrog cov lus encrypted no. Raws li, peb qhov tseem ceeb ntev yog nyob rau hauv ib kilobyte, yog li ntawd qhov no yog qhov hnyav thiab txhim khu kev qha.Hauv: - Koj puas xav tau 2 kilobytes tam sim no?
EK: – Nws zoo li nag hmo nws yog 256... Zoo, qhov twg ntxiv?!
Raws li, qhov no yog thawj. Thiab qhov thib ob, qhov kev daws teeb meem uas muaj, nws txhawb cov txheej txheem rov encryption - muaj ob khub ntawm "keks" (cov yawm sij), uas muab "decks" uas encrypt (tus yuam sij yog cov yuam sij, dek yog cov txiaj ntsig ntawm cov yuam sij uas encrypt) . Thiab yog hais tias tus txheej txheem yog pib (nws tshwm sim tsis tu ncua, los ntawm 3 lub hlis mus rau ± ib co), peb download tau ib tug tshiab khub "khoom qab zib", thiab peb rov encrypt cov ntaub ntawv. Peb muaj nyias cov kev pab cuam uas rhais tawm tag nrho cov ntaub ntawv thiab encrypt nws nyob rau hauv txoj kev tshiab; Cov ntaub ntawv khaws cia ntawm ib sab ntawm tus cim ntawm tus yuam sij uas nws tau encrypted. Raws li, sai li sai tau thaum peb encrypt cov ntaub ntawv nrog cov yuam sij tshiab, peb tshem tawm tus yuam sij qub.
Qee zaum kev them nyiaj yuav tsum tau ua manually ...
Hauv: - Qhov ntawd yog, yog tias cov nyiaj rov qab tuaj txog rau qee qhov haujlwm, koj puas tseem yuav decrypt nws nrog tus yuam sij qub?
EK: - Yog.
Hauv: – Ces ib lo lus nug me me ntxiv. Thaum qee yam tsis ua haujlwm, poob, lossis qhov xwm txheej tshwm sim, nws yog qhov yuav tsum tau thawb los ntawm kev hloov pauv manually. Muaj qhov xwm txheej zoo li no.
EK: - Yog, qee zaum.
Hauv: - Koj tau txais cov ntaub ntawv no los ntawm qhov twg? Los yog koj puas mus rau qhov chaw khaws khoom no koj tus kheej?
EK: - Tsis yog, zoo, tau kawg, peb muaj qee yam kev ua haujlwm rov qab uas muaj qhov cuam tshuam rau peb cov kev txhawb nqa. Yog tias peb tsis paub tias qhov xwm txheej twg yog nyob rau hauv (piv txwv li, txog thaum lub sijhawm them nyiaj teb nrog lub sijhawm), peb tsis paub qhov tseem ceeb, uas yog, peb muab cov xwm txheej kawg nkaus xwb nrog kev ntseeg siab. Nyob rau hauv cov ntaub ntawv no, peb muab cov kev pauv mus rau ib tug tshwj xeeb xwm txheej rau phau ntawv ua. Thaum sawv ntxov, hnub tom qab, sai li sai tau thaum kev txhawb nqa tau txais cov ntaub ntawv hais tias xws li thiab xws li kev lag luam tseem nyob hauv kev them nyiaj, lawv manually ua lawv hauv qhov interface no.
Hauv: – Kuv muaj ob peb lo lus nug. Ib qho ntawm lawv yog qhov txuas ntxiv ntawm thaj chaw PCI DSS: koj yuav teev lawv li cas? Lo lus nug no yog vim tus tsim tawm tuaj yeem tso txhua yam hauv cov cav! Lo lus nug thib ob: koj yuav ua li cas thiaj li tshem tawm cov kev kho mob tshiab? Siv cov tes tuav hauv cov ntaub ntawv yog ib qho kev xaiv, tab sis tej zaum yuav muaj kev kho kub dawb - dab tsi yog txheej txheem muaj? Thiab lo lus nug thib peb yog tej zaum muaj feem xyuam nrog RTO, RPO. Koj qhov muaj nyob yog 99,97, yuav luag plaub cuaj, tab sis raws li kuv nkag siab nws, koj muaj qhov chaw thib ob, qhov chaw thib peb, thiab qhov chaw thib tsib ...EK: - Cia peb pib nrog thawj tus. Puas yog thawj lo lus nug txog cav? Thaum peb sau cov cav, peb muaj ib txheej uas npog tag nrho cov ntaub ntawv rhiab heev. Nws saib daim npog qhov ncauj thiab ntawm qhov chaw ntxiv. Raws li, peb cov ntaub ntawv tawm los nrog cov ntaub ntawv npog twb muaj lawm thiab PCI DSS Circuit Court. Qhov no yog ib qho ntawm cov haujlwm tsis tu ncua uas tau muab rau lub chaw kuaj mob. Lawv yuav tsum tau kuaj xyuas txhua txoj haujlwm, suav nrog cov ntawv teev lus uas lawv sau, thiab qhov no yog ib qho ntawm cov haujlwm tsis tu ncua thaum lub sijhawm tshuaj xyuas cov lej, txhawm rau tswj xyuas tias tus tsim tawm tsis sau qee yam. Cov kev txheeb xyuas tom qab ntawm qhov no tau ua tsis tu ncua los ntawm lub chaw saib xyuas kev ruaj ntseg hais txog ib zaug hauv ib lub lis piam: cov ntaub ntawv rau hnub kawg raug xaiv thiab lawv tau khiav los ntawm ib qho tshwj xeeb scanner-analyzer los ntawm cov servers kuaj xyuas txhua yam.
Hais txog kub-fixes. Qhov no suav nrog hauv peb txoj cai xa tawm. Peb muaj ib nqe lus cais hais txog kev kho mob. Peb ntseeg tias peb siv cov kev kho kub kub ib ncig ntawm lub moos thaum peb xav tau. Thaum lub version yog sib sau ua ke, sai li sai tau thaum nws khiav, sai li sai tau raws li peb muaj ib tug artifact, peb muaj ib tug system administrator nyob rau hauv lub luag hauj lwm ntawm ib tug hu los ntawm kev txhawb nqa, thiab nws deploy nws lub sij hawm thaum nws yog tsim nyog.Hais txog "plaub cuaj". Daim duab uas peb muaj tam sim no tau ua tiav tiag tiag, thiab peb tau mob siab rau nws hauv lwm qhov chaw khaws ntaub ntawv. Tam sim no peb muaj qhov chaw thib ob, thiab peb tab tom pib taug kev ntawm lawv, thiab qhov teeb meem ntawm qhov chaw hla cov ntaub ntawv rov ua dua yog cov lus nug tsis tseem ceeb. Peb sim daws nws ib zaug uas siv ntau txoj kev: peb sim siv tib "Tarantula" - nws tsis ua haujlwm rau peb, Kuv yuav qhia koj tam sim ntawd. Yog vim li cas peb thiaj li txiav txim siab "sens" manually. Qhov tseeb, txhua daim ntawv thov hauv peb qhov system khiav qhov tsim nyog "hloov - ua tiav" synchronization ntawm cov chaw zov me nyuam asynchronously.
Hauv: - Yog tias koj tau txais ib qho thib ob, vim li cas koj thiaj tsis tau txais qhov thib peb? Vim tsis muaj leej twg muaj lub hlwb tawg lawm...
EK: - Tab sis peb tsis muaj Split Brain. Vim tias txhua daim ntawv thov raug tsav los ntawm multimaster, nws tsis muaj teeb meem rau peb qhov twg qhov chaw thov tuaj. Peb npaj txhij rau qhov tseeb tias yog tias ib qho ntawm peb cov chaw khaws ntaub ntawv tsis ua haujlwm (peb cia siab rau qhov no) thiab nyob rau hauv nruab nrab ntawm cov neeg siv kev thov hloov mus rau qhov chaw thib ob cov ntaub ntawv, peb npaj txhij yuav plam tus neeg siv no, qhov tseeb; tab sis cov no yuav yog units, absolute units.
Hauv: - Nyob zoo hmo ntuj. Ua tsaug rau daim ntawv qhia. Koj tau tham txog koj qhov debugger, uas khiav qee qhov kev sim kev lag luam hauv kev tsim khoom. Tab sis qhia peb txog kev kuaj kev lag luam! Nws mus tob npaum li cas?
EK: - Nws mus dhau lub voj voog ntawm tag nrho cov khoom. Rau ib qho kev tivthaiv, tsis muaj qhov sib txawv ntawm qhov kev sim ua lag luam thiab kev lag luam ntau lawm. Tab sis los ntawm lub ntsiab lus ntawm kev xav, qhov no tsuas yog ib qho kev sib cais hauv qhov system, uas tsuas yog kev sim kev lag luam khiav.
Hauv: - Koj txiav nws qhov twg? Ntawm no Core xa ...
EK: - Peb nyob tom qab "Kor" nyob rau hauv cov ntaub ntawv no rau kev kuaj kev lag luam... Peb muaj xws li routing: "Kor" paub qhov kev them nyiaj xa mus rau - peb xa mus rau ib tug fake them nyiaj system, uas yooj yim muab ib tug http teeb liab thiab yog tag nrho.
Hauv: - Qhia rau kuv, thov, puas yog koj daim ntawv thov sau rau hauv ib lub monolith loj, lossis koj puas tau txiav nws rau qee qhov kev pabcuam lossis txawm tias microservices?
EK: - Peb tsis muaj monolith, tau kawg, peb muaj daim ntawv thov kev pabcuam. Peb tso dag tias peb cov kev pabcuam yog ua los ntawm monoliths - lawv yog cov loj heev. Nws nyuaj rau hu nws microservices, tab sis cov no yog cov kev pabcuam uas cov neeg ua haujlwm ntawm cov tshuab faib ua haujlwm.
Yog tias qhov kev pabcuam ntawm lub server raug cuam tshuam ...
Hauv: – Ces kuv muaj lus nug tom ntej. Txawm hais tias nws yog monolith, koj tseem hais tias koj muaj ntau ntawm cov servers tam sim no, lawv txhua tus txheej txheem cov ntaub ntawv, thiab cov lus nug yog: "Yog tias muaj kev cuam tshuam ntawm ib qho ntawm cov servers tam sim no lossis ib daim ntawv thov, ib tus neeg txuas. , lawv puas muaj qee yam kev tswj xyuas? Leej twg ntawm lawv tuaj yeem ua tau dab tsi? Kuv yuav tsum hu rau leej twg rau cov ntaub ntawv dab tsi?
EK: - Yog, twv yuav raug hu. Cov kev cai ruaj ntseg yog qhov hnyav heev. Ua ntej, peb muaj qhib cov ntaub ntawv txav, thiab cov chaw nres nkoj tsuas yog cov uas peb cia siab tias yuav muaj kev txav mus los ua ntej. Yog tias ib qho kev sib txuas lus nrog cov ntaub ntawv (hais, nrog Muskul) ntawm 5-4-3-2, tsuas yog 5-4-3-2 yuav qhib rau nws, thiab lwm qhov chaw nres nkoj thiab lwm cov lus qhia yuav tsis muaj. Tsis tas li ntawd, koj yuav tsum nkag siab tias hauv peb cov khoom tsim muaj txog 10 qhov sib txawv kev ruaj ntseg loops. Thiab txawm tias daim ntawv thov raug cuam tshuam los xij, Vajtswv txwv, tus neeg tawm tsam yuav tsis tuaj yeem nkag mus rau lub server tswj console, vim qhov no yog thaj chaw sib txawv network ruaj ntseg.Hauv: - Thiab nyob rau hauv cov ntsiab lus no, dab tsi yog qhov nthuav rau kuv yog tias koj muaj qee qhov kev cog lus nrog cov kev pabcuam - lawv tuaj yeem ua li cas, los ntawm "kev nqis tes ua" lawv tuaj yeem tiv tauj ib leeg ... Thiab hauv ib txwm ntws, qee qhov kev pabcuam tshwj xeeb thov qee qhov kab, ib daim ntawv teev "kev ua" ntawm lwm tus. Lawv zoo li tsis tig mus rau lwm tus hauv qhov xwm txheej ib txwm muaj, thiab lawv muaj lwm yam kev lav phib xaub. Yog tias ib qho ntawm lawv raug cuam tshuam, nws puas tuaj yeem cuam tshuam "kev ua" ntawm qhov kev pabcuam ntawd? ...
EK: - Kuv nkag siab. Yog hais tias nyob rau hauv ib qho xwm txheej nrog lwm tus neeg rau zaub mov kev sib txuas lus tau tso cai txhua, ces yog. Raws li daim ntawv cog lus SLA, peb tsis saib xyuas tias koj tsuas yog tso cai rau thawj 3 "kev ua", thiab koj tsis raug tso cai rau 4 "kev nqis tes ua". Qhov no yog qhov tsis txaus ntseeg rau peb, vim tias peb twb muaj 4-theem kev tiv thaiv, hauv txoj cai, rau circuits. Peb nyiam tiv thaiv peb tus kheej nrog cov contours, tsis yog nyob rau theem ntawm sab hauv.
Yuav ua li cas Visa, MasterCard thiab Sberbank ua haujlwm
Hauv: - Kuv xav qhia meej txog qhov hloov pauv tus neeg siv los ntawm ib lub chaw khaws ntaub ntawv mus rau lwm qhov. Raws li kuv paub, Visa thiab MasterCard ua haujlwm siv 8583 binary synchronous raws tu qauv, thiab muaj cov sib xyaw ua ke. Thiab kuv xav paub, tam sim no peb txhais tau tias hloov pauv - puas yog ncaj qha "Visa" thiab "MasterCard" lossis ua ntej them nyiaj, ua ntej ua haujlwm?
EK: - Qhov no yog ua ntej sib tov. Peb cov khoom sib xyaw yog nyob rau hauv tib lub chaw cov ntaub ntawv.
Hauv: - Hais lus ntxhib, koj puas muaj ib qho kev sib txuas?
EK: - "Visa" thiab "MasterCard" - yog. Tsuas yog vim Visa thiab MasterCard xav tau kev nqis peev loj heev hauv kev tsim kho vaj tse kom xaus cov ntawv cog lus sib cais kom tau txais ob khub sib xyaw, piv txwv. Lawv tau tshwj tseg hauv ib lub chaw khaws ntaub ntawv, tab sis yog tias, Vajtswv txwv, peb cov ntaub ntawv chaw, qhov twg muaj kev sib xyaw rau kev txuas rau Visa thiab MasterCard, tuag, ces peb yuav muaj kev sib txuas nrog Visa thiab MasterCard poob ...
Hauv: - Yuav ua li cas lawv thiaj li tseg tau? Kuv paub tias Visa tso cai rau ib qho kev sib txuas nkaus xwb!
EK: - Lawv muab cov khoom siv rau lawv tus kheej. Txawm li cas los xij, peb tau txais cov khoom siv uas tsis tshua muaj nyob hauv.
Hauv: - Yog li qhov sawv ntsug yog los ntawm lawv Cov Kev Sib Txuas Txiv Kab Ntxwv?..
EK: - Yog.
Hauv: - Tab sis li cas txog qhov no: yog tias koj lub chaw khaws ntaub ntawv ploj, koj tuaj yeem siv nws txuas ntxiv li cas? Los yog txoj kev tsheb nres xwb?
EK: - Tsis muaj. Nyob rau hauv rooj plaub no, peb tsuas yog hloov cov tsheb mus rau lwm qhov channel, uas, ib txwm, yuav kim dua rau peb thiab kim dua rau peb cov neeg siv khoom. Tab sis kev khiav tsheb yuav tsis dhau los ntawm peb qhov kev sib txuas ncaj qha rau Visa, MasterCard, tab sis dhau los ntawm cov xwm txheej Sberbank (hloov heev).
Kuv thov txim wildly yog tias kuv ua phem rau Sberbank cov neeg ua haujlwm. Tab sis raws li peb cov txheeb cais, ntawm cov tsev txhab nyiaj Lavxias, Sberbank feem ntau poob. Tsis yog ib lub hlis dhau mus yam tsis muaj ib yam dab tsi poob ntawm Sberbank.
Ib co ads 🙂
Ua tsaug uas koj tau nyob nrog peb. Koj puas nyiam peb cov ntawv? Xav pom cov ntsiab lus nthuav ntxiv? Txhawb nqa peb los ntawm kev tso ib qho kev txiav txim lossis qhia rau cov phooj ywg, , ib qho tshwj xeeb analogue ntawm nkag-theem servers, uas tau tsim los ntawm peb rau koj: (muaj nrog RAID1 thiab RAID10, mus txog 24 cores thiab mus txog 40GB DDR4).
Dell R730xd 2x pheej yig dua hauv Equinix Tier IV data center hauv Amsterdam? Tsuas yog nyob ntawm no hauv Netherlands! Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - los ntawm $ 99! Nyeem txog
Tau qhov twg los: www.hab.com