Nyob zoo! Kuv lub npe yog Vadim Madison, Kuv coj txoj kev loj hlob ntawm Avito System Platform. Nws tau hais ntau dua ib zaug yuav ua li cas peb hauv tuam txhab tau tsiv los ntawm monolithic architecture mus rau microservices ib qho. Nws yog lub sijhawm los qhia seb peb tau hloov pauv peb cov txheej txheem li cas kom tau txais txiaj ntsig zoo tshaj plaws ntawm microservices thiab tiv thaiv peb tus kheej kom tsis txhob poob rau hauv lawv. Yuav ua li cas PaaS pab peb ntawm no, yuav ua li cas peb simplified xa mus thiab txo cov creation ntawm ib tug microservice rau ib tug nias - nyeem rau. Tsis yog txhua yam kuv sau txog hauv qab no tau ua tiav hauv Avito, qee qhov nws yog qhov peb tsim peb lub platform.
(Thiab thaum kawg ntawm tsab xov xwm no, kuv mam li tham txog lub sijhawm los mus koom lub rooj sib tham peb-hnub los ntawm microservice architecture kws tshaj lij Chris Richardson).
Yuav ua li cas peb tuaj rau microservices
Avito yog ib lub vev xaib loj tshaj plaws hauv ntiaj teb; ntau tshaj 15 lab kev tshaj tawm tshiab tau tshaj tawm rau nws ib hnub. Peb lub backend txais ntau tshaj 20 txhiab thov ib ob. Tam sim no peb muaj ntau pua microservices.
Peb tau tsim lub microservice architecture rau ntau xyoo tam sim no. Yuav ua li cas raws nraim - peb cov npoj yaig hauv kev nthuav dav ntawm peb ntu ntawm RIT ++ 2017. Ntawm CodeFest 2017 (saib. ), Sergey Orlov thiab Mikhail Prokopchuk piav qhia meej vim li cas peb xav tau kev hloov pauv mus rau microservices thiab lub luag haujlwm dab tsi Kubernetes ua si ntawm no. Zoo, tam sim no peb tab tom ua txhua yam kom txo qis tus nqi scaling uas muaj nyob rau hauv xws li architecture.
Thaum pib, peb tsis tau tsim ib qho ecosystem uas yuav pab tau peb tsim thiab tsim microservices. Lawv tsuas sau cov kev daws teeb meem uas muaj txiaj ntsig zoo, qhib lawv hauv tsev thiab caw tus tsim tawm los daws lawv. Raws li qhov tshwm sim, nws tau mus rau kaum ob qhov chaw (dashboards, cov kev pabcuam sab hauv), tom qab ntawd nws tau muaj zog dua hauv nws txoj kev xav txiav cov lej qub, hauv monolith. Cov xim ntsuab hauv daim duab hauv qab no qhia tias tus tsim tawm ua li cas ib txoj hauv kev los yog lwm qhov nrog nws tus kheej tes, thiab xim daj qhia tias automation.
Tam sim no hauv PaaS CLI qhov hluav taws xob, qhov kev pabcuam tshiab tau tsim nrog ib qho kev hais kom ua, thiab cov ntaub ntawv tshiab tau ntxiv nrog ob ntxiv thiab xa mus rau theem.
Yuav ua li cas kov yeej lub sijhawm ntawm "microservice fragmentation"
Nrog lub monolithic architecture, rau lub hom phiaj ntawm qhov sib xws ntawm cov kev hloov pauv hauv cov khoom, cov neeg tsim khoom raug yuam kom txiav txim siab seb puas muaj dab tsi tshwm sim nrog lawv cov neeg nyob ze. Thaum ua haujlwm ntawm qhov tshiab architecture, cov ntsiab lus kev pabcuam tsis nyob ntawm ib leeg.
Tsis tas li ntawd, txhawm rau txhim kho microservice architecture kom ua tau zoo, ntau cov txheej txheem yuav tsum tau tsim, uas yog:
β’ sau npe;
β’ thov tracing (Jaeger);
β’ yuam kev aggregation (Sentry);
β’ cov xwm txheej, cov lus, cov xwm txheej los ntawm Kubernetes (Kev tshwm sim kwj ua haujlwm);
β’ haiv neeg txwv / Circuit Court breaker (koj tuaj yeem siv Hystrix);
β’ tswj kev pabcuam kev sib txuas (peb siv Netramesh);
β’ saib xyuas (Grafana);
β’ sib dhos (TeamCity);
β’ kev sib txuas lus thiab kev ceeb toom (Slack, email);
β’ nrhiav hauj lwm; (Jira)
β’ npaj cov ntaub ntawv.
Txhawm rau kom ntseeg tau tias lub kaw lus tsis poob nws txoj kev ncaj ncees thiab tseem ua haujlwm zoo thaum nws ntsuas, peb rov xav txog lub koom haum ntawm microservices hauv Avito.
Yuav ua li cas peb tswj microservices
Cov kev pab hauv qab no los ua kom muaj kev sib koom ua ke "txoj cai tog" ntawm ntau Avito microservices:
- faib infrastructure rau hauv txheej;
- Platform as a Service (PaaS) tswvyim;
- saib xyuas txhua yam uas tshwm sim nrog microservices.
Infrastructure abstraction txheej muaj peb txheej. Wb mus saum toj saud.
A. Sab saum toj - kev pabcuam mesh. Thaum xub thawj peb sim Istio, tab sis nws tau pom tias nws siv ntau cov peev txheej, uas yog kim heev rau peb cov ntim. Yog li ntawd, senior engineer nyob rau hauv lub architecture pab neeg Alexander Lukyanchenko tsim nws tus kheej kev daws teeb meem - (muaj nyob rau hauv Qhib Qhov Chaw), uas tam sim no peb siv hauv kev tsim khoom thiab siv ob peb zaug tsawg dua li Istio (tab sis tsis ua txhua yam uas Istio tuaj yeem khav theeb).
B. Nruab nrab - Kubernetes. Peb siv thiab ua haujlwm microservices rau nws.
C. Qab - liab qab hlau. Peb tsis siv huab lossis tej yam zoo li OpenStack, tab sis cia siab rau ntawm cov hlau liab qab.
Txhua txheej yog ua ke los ntawm PaaS. Thiab lub platform no, nyob rau hauv lem, muaj peb feem.
I. Generators, tswj los ntawm CLI kev siv hluav taws xob. Nws yog nws uas pab tus tsim tawm tsim microservice hauv txoj kev yog thiab nrog tsawg kawg ntawm kev siv zog.
II. Consolidated collector nrog kev tswj ntawm tag nrho cov cuab yeej los ntawm ib tug ntau dashboard.
III. Cia. Txuas nrog cov sijhawm teem sijhawm uas cia li teeb tsa ua rau cov haujlwm tseem ceeb. Ua tsaug rau cov txheej txheem zoo li no, tsis yog ib txoj haujlwm uas ploj lawm vim ib tus neeg tsis nco qab teeb tsa txoj haujlwm hauv Jira. Peb siv cov cuab yeej sab hauv hu ua Atlas rau qhov no.
Kev siv cov microservices hauv Avito kuj tau ua raws li ib lub tswv yim, uas yooj yim tswj lawv ntawm txhua theem ntawm kev txhim kho thiab tso tawm.
Cov txheej txheem microservice txoj kev loj hlob kav dej ua haujlwm li cas?
Feem ntau, microservice creation saw zoo li no:
CLI-push β Continuous Integration β Ci β Deploy β Artificial tests β Canary test β Squeeze Test β Production β Maintenance.
Cia peb mus dhau nws raws nraim hauv qhov kev txiav txim no.
CLI-pub
β’ Tsim microservice.
Peb tau tawm tsam ntev los qhia txhua tus tsim tawm yuav ua li cas microservices. Qhov no suav nrog sau cov lus qhia ntxaws ntxaws hauv Confluence. Tab sis cov schemes hloov thiab tau ntxiv. Qhov tshwm sim yog tias lub raj xa dej tau tshwm sim thaum pib ntawm kev taug kev: nws siv sijhawm ntau ntxiv los tsim cov microservices, thiab tseem muaj teeb meem feem ntau tshwm sim thaum lawv tsim.
Thaum kawg, peb tau tsim cov khoom siv CLI yooj yim uas ua rau cov kauj ruam yooj yim thaum tsim microservice. Qhov tseeb, nws hloov thawj git thawb. Nov yog qhov nws ua.
- Tsim cov kev pabcuam raws li tus qauv - ib kauj ruam dhau los, hauv "wizard" hom. Peb muaj cov qauv rau cov lus programming tseem ceeb hauv Avito backend: PHP, Golang thiab Python.
- Ib qho lus txib ntawm ib lub sij hawm siv ib puag ncig rau kev txhim kho hauv zos ntawm lub tshuab tshwj xeeb - Minikube tau tsim tawm, Helm kab kos tau txiav tawm thiab tso tawm hauv kubernetes hauv zos.
- Txuas cov ntaub ntawv xav tau. Tus tsim tawm tsis tas yuav tsum paub tus IP, nkag mus thiab lo lus zais kom nkag mus rau hauv cov ntaub ntawv nws xav tau - yog nws hauv zos, ntawm theem, lossis hauv kev tsim khoom. Ntxiv mus, cov ntaub ntawv khaws cia tau siv tam sim ntawd hauv kev teeb tsa tsis raug thiab nrog kev sib npaug.
- Nws ua haujlwm nyob sib dhos nws tus kheej. Cia peb hais tias tus tsim tawm kho qee yam hauv microservice los ntawm nws IDE. Lub tshuab hluav taws xob pom cov kev hloov pauv hauv cov ntaub ntawv kaw lus thiab, raws li lawv, rov tsim daim ntawv thov (rau Golang) thiab rov pib dua. Rau PHP, peb tsuas yog xa mus rau cov npe hauv lub voos xwmfab thiab muaj nyob-reload tau txais "automatically".
- Tsim autotests. Nyob rau hauv daim ntawv ntawm blanks, tab sis heev haum rau siv.
β’ Kev xa mus rau Microservice.
Kev siv microservice siv los ua haujlwm me ntsis rau peb. Cov hauv qab no yuav tsum tau:
I. Dockerfile.
II. Config.
III. Helm chart, uas nws tus kheej yog cumbersome thiab suav nrog:
- cov kab kos lawv tus kheej;
- cov qauv;
- cov txiaj ntsig tshwj xeeb coj mus rau hauv tus account txawv ib puag ncig.
Peb tau tshem qhov mob tawm ntawm kev rov ua haujlwm Kubernetes tshwm sim yog li tam sim no lawv tau tsim tawm. Tab sis qhov tseem ceeb tshaj, lawv simplified kev xa mus rau qhov txwv. Txij ntawm no mus peb muaj Dockerfile, thiab tus tsim tawm sau tag nrho cov teeb tsa hauv ib qho luv luv app.toml cov ntaub ntawv.
Yog, thiab hauv app.toml nws tus kheej tsis muaj dab tsi ua rau ib feeb. Peb qhia meej qhov twg thiab pes tsawg daim ntawv luam ntawm cov kev pabcuam los txhawb nqa (ntawm tus dev server, ntawm kev ua haujlwm, hauv kev tsim khoom), thiab qhia nws qhov kev vam khom. Pom cov kab loj = "me" hauv [ cav] thaiv. Qhov no yog qhov txwv uas yuav muab faib rau cov kev pabcuam ntawm Kubernetes.
Tom qab ntawd, raws li kev teeb tsa, txhua qhov tsim nyog Helm kab kos tau raug tsim tawm thiab kev sib txuas rau cov databases raug tsim.
β’ Basic validation. Xws li cov tshev kuj tseem siv tau.
Yuav tsum taug qab:
- puas muaj Dockerfile;
- puas muaj app.toml;
- Puas muaj cov ntaub ntawv?
- Puas yog kev vam khom hauv kev txiav txim?
- seb cov cai ceeb toom tau teeb tsa.
Mus rau qhov kawg taw tes: tus tswv ntawm qhov kev pabcuam nws tus kheej txiav txim siab seb cov khoom ntsuas twg los saib xyuas.
β’ Kev npaj cov ntaub ntawv.
Tseem yog thaj chaw teeb meem. Nws zoo nkaus li qhov pom tseeb tshaj plaws, tab sis tib lub sijhawm nws kuj yog cov ntaub ntawv "feem ntau tsis nco qab", thiab yog li ntawd qhov kev sib txuas tsis zoo hauv cov saw hlau.
Nws yog ib qho tsim nyog kom muaj cov ntaub ntawv rau txhua qhov kev pabcuam micro. Nws suav nrog cov blocks hauv qab no.
I. Cov lus piav qhia luv luv ntawm kev pabcuam. Muaj ob peb kab lus hais txog qhov nws ua thiab vim li cas nws thiaj li xav tau.
II. Architecture daim duab link. Nws yog ib qho tseem ceeb uas nrog ceev ceev ntawm nws yooj yim to taub, piv txwv li, seb koj puas siv Redis rau caching los yog raws li lub ntsiab cov ntaub ntawv khaws cia nyob rau hauv persistent hom. Hauv Avito tam sim no qhov no yog qhov txuas rau Confluence.
III. Phau ntawv. Cov lus qhia luv luv ntawm kev pib qhov kev pabcuam thiab qhov tsis txaus ntseeg ntawm kev tuav nws.
IV. FAQ, qhov twg nws yuav zoo rau kev cia siab txog cov teeb meem uas koj cov npoj yaig yuav ntsib thaum ua haujlwm nrog kev pabcuam.
V. Nqe lus piav qhia txog qhov kawg rau API. Yog tias mam li nco dheev koj tsis tau qhia meej qhov chaw, cov npoj yaig uas nws cov microservices cuam tshuam nrog koj li yuav luag them rau nws. Tam sim no peb siv Swagger thiab peb cov tshuaj hu ua luv luv rau qhov no.
VI. Daim ntawv lo. Los yog cov cim uas qhia tias yam khoom twg, kev ua haujlwm, lossis kev faib khoom ntawm lub tuam txhab cov kev pabcuam yog. Lawv pab koj nkag siab sai, piv txwv li, txawm tias koj tab tom txiav kev ua haujlwm uas koj cov npoj yaig tau nthuav tawm rau tib lub lag luam ib lub lis piam dhau los.
VII. Tus tswv lossis tus tswv ntawm qhov kev pabcuam. Feem ntau, nws - lossis lawv - tuaj yeem txiav txim siab siv PaaS, tab sis kom nyob rau sab nyab xeeb, peb xav kom tus tsim tawm qhia lawv tus kheej.
Thaum kawg, nws yog ib qho kev xyaum zoo los tshuaj xyuas cov ntaub ntawv, zoo ib yam li kev tshuaj xyuas cov cai.
Nruam Kev Nrhav
- Npaj repositories.
- Tsim ib lub raj xa dej hauv TeamCity.
- Kev teeb tsa txoj cai.
- Nrhiav rau cov tswv kev pabcuam. Muaj ib lub tswv yim hybrid ntawm no - phau ntawv cim thiab tsawg kawg automation los ntawm PaaS. Lub tswv yim tsis siv neeg tag nrho ua tsis tiav thaum cov kev pabcuam raug xa mus rau lwm pab pawg txhim kho lossis, piv txwv li, yog tus tsim tawm kev pabcuam tawm.
- Sau npe ib qho kev pabcuam hauv Atlas (saib saum toj). Nrog tag nrho nws cov tswv thiab kev vam meej.
- Tshawb xyuas kev tsiv teb tsaws. Peb xyuas seb lawv puas muaj peev xwm txaus ntshai. Piv txwv li, nyob rau hauv ib tug ntawm lawv ib tug hloov lub rooj los yog ib yam dab tsi lwm yam pops uas yuav ua txhaum lub compatibility ntawm cov ntaub ntawv schema ntawm txawv versions ntawm cov kev pab cuam. Tom qab ntawd qhov kev tsiv teb tsaws tsis tau ua, tab sis muab tso rau hauv kev tso npe - PaaS yuav tsum tau teeb tsa tus tswv kev pabcuam thaum nws muaj kev nyab xeeb siv nws.
Ci
Cov theem tom ntej yog ntim cov kev pabcuam ua ntej xa mus.
- Tsim daim ntawv thov. Raws li cov classics - hauv Docker duab.
- Generation of Helm kab kos rau kev pabcuam nws tus kheej thiab lwm yam kev pabcuam. Xws li rau databases thiab cache. Lawv raug tsim cia li ua raws li app.toml config uas tau tsim los ntawm CLI-push theem.
- Tsim daim pib rau admins qhib chaw nres nkoj (thaum xav tau).
- Khiav chav ntsuas kev ntsuas thiab suav cov lej them nyiaj. Yog tias cov cai tswjfwm qis dua qhov chaw pib, feem ntau qhov kev pabcuam yuav tsis mus ntxiv - mus rau kev xa tawm. Yog hais tias nws yog nyob rau hauv lub verge ntawm kev txais, ces cov kev pab cuam yuav muab ib tug "pessimizing" coefficient: ces, yog hais tias tsis muaj kev txhim kho nyob rau hauv lub sij hawm, tus tsim tawm yuav tau txais ib tug ceeb toom tias tsis muaj kev vam meej nyob rau hauv cov nqe lus ntawm kev xeem ( thiab ib yam dab tsi yuav tsum tau ua txog nws).
- Accounting rau nco thiab CPU txwv. Peb tsuas yog sau microservices hauv Golang thiab khiav lawv hauv Kubernetes. Li no ib subtlety cuam tshuam nrog peculiarity ntawm cov lus Golang: los ntawm lub neej ntawd, thaum pib, tag nrho cov cores ntawm lub tshuab yog siv, yog hais tias koj tsis meej meej teem lub GOMAXPROCS sib txawv, thiab thaum ob peb yam kev pab cuam yog launched rau tib lub tshuab, lawv pib. sib tw rau cov peev txheej, cuam tshuam ib leeg. Cov duab hauv qab no qhia tias lub sijhawm ua haujlwm hloov pauv li cas yog tias koj khiav daim ntawv thov yam tsis muaj kev sib cav thiab hauv kev sib tw rau hom kev pab. (Qhov chaw ntawm daim duab yog ).
Lub sijhawm ua haujlwm, tsawg dua yog qhov zoo dua. Qhov siab tshaj plaws: 643ms, yam tsawg kawg nkaus: 42ms. Daim duab yog clickable.
Lub sij hawm rau kev phais, tsawg dua yog zoo dua. Qhov siab tshaj plaws: 14091 ns, yam tsawg kawg nkaus: 151 ns. Daim duab yog clickable.
Nyob rau theem kev npaj sib dhos, koj tuaj yeem teeb tsa qhov sib txawv no meej lossis koj tuaj yeem siv lub tsev qiv ntawv los ntawm cov txiv neej los ntawm Uber.
Muab tso rau
β’ Tshawb xyuas cov lus pom zoo. Ua ntej koj pib xa cov kev pabcuam sibtham rau koj qhov chaw ib puag ncig, koj yuav tsum xyuas cov hauv qab no:
- API endpoints.
- Ua raws li API cov ntsiab lus teb cov lus teb nrog schema.
- Log format.
- Teem lub taub hau rau kev thov rau kev pabcuam (tam sim no qhov no yog ua los ntawm netramesh)
- Teem tus tswv token thaum xa cov lus mus rau qhov xwm txheej tsheb npav. Qhov no yog xav tau los taug qab kev sib txuas ntawm cov kev pabcuam thoob plaws lub npav. Koj tuaj yeem xa ob qho tib si idempotent cov ntaub ntawv mus rau lub tsheb npav, uas tsis nce kev sib txuas ntawm cov kev pabcuam (uas yog qhov zoo), thiab cov ntaub ntawv lag luam uas txhawb kev sib txuas ntawm cov kev pabcuam (uas yog qhov phem heev!). Thiab nyob rau lub sijhawm thaum qhov kev sib txuas no dhau los ua qhov teeb meem, kev nkag siab leej twg sau thiab nyeem lub npav pab cais cov kev pabcuam kom raug.
Tseem tsis tau muaj ntau lub rooj sib tham hauv Avito, tab sis lawv lub pas dej tau nthuav dav. Qhov ntau qhov kev pom zoo no muaj nyob rau hauv ib daim ntawv uas pab neeg tuaj yeem nkag siab thiab nkag siab, nws yooj yim dua kom tswj tau qhov sib xws ntawm microservices.
Synthetic xeem
β’ Kev ntsuas lub voj kaw. Rau qhov no peb tam sim no siv qhib qhov chaw . Ua ntej, nws sau cov load tiag tiag ntawm qhov kev pabcuam, tom qab ntawd - tsuas yog nyob rau hauv lub voj kaw - nws emulates nws.
β’ Kev Ntsuas Kev Nyuaj Siab. Peb sim nqa tag nrho cov kev pabcuam rau kev ua haujlwm zoo. Thiab tag nrho cov qauv ntawm txhua qhov kev pabcuam yuav tsum raug kuaj xyuas - qhov no peb tuaj yeem nkag siab txog qhov kev ua haujlwm tam sim no ntawm kev pabcuam thiab qhov sib txawv nrog cov qauv dhau los ntawm cov kev pabcuam tib yam. Yog tias, tom qab kev pabcuam hloov tshiab, nws qhov kev ua tau zoo poob los ntawm ib thiab ib nrab zaug, qhov no yog lub teeb liab meej rau nws cov tswv: koj yuav tsum khawb rau hauv cov cai thiab kho qhov xwm txheej.
Peb siv cov ntaub ntawv khaws tseg, piv txwv li, txhawm rau ua kom raug siv qhov ntsuas nws pib thiab, thaum kawg, feem ntau nkag siab tias qhov kev pabcuam scalable li cas.
Thaum kuaj kev thauj khoom, peb xyuas seb cov peev txheej siv tau raws li qhov kev txwv. Thiab peb tsom feem ntau ntawm qhov kawg.
a) Peb saib tag nrho cov load.
- me me dhau - feem ntau yuav muaj qee yam tsis ua haujlwm tag nrho yog tias lub load dheev poob ob peb zaug.
- Loj dhau - optimization yuav tsum tau.
b) Peb saib qhov kev txiav tawm raws li RPS.
Ntawm no peb saib qhov txawv ntawm qhov tam sim no version thiab yav dhau los ib qho thiab tag nrho kom muaj nuj nqis. Piv txwv li, yog tias ib qho kev pabcuam tsim tawm 100 rps, ces nws yog sau tsis zoo, lossis qhov no yog nws qhov tshwj xeeb, tab sis nyob rau hauv txhua rooj plaub, qhov no yog ib qho laj thawj los saib cov kev pabcuam zoo heev.
Yog tias, ntawm qhov tsis sib xws, muaj ntau RPS, ces tej zaum muaj qee yam kab laum thiab qee qhov kawg ntawm cov ntsiab lus tau tso tseg tsis ua haujlwm them nyiaj, thiab qee qhov lwm tus tsuas yog tshwm sim. return true;
Canary xeem
Tom qab peb dhau qhov kev ntsuam xyuas hluavtaws, peb sim microservice ntawm cov neeg siv tsawg. Peb pib ua tib zoo, nrog me me ntawm cov kev pabcuam cov neeg tuaj saib - tsawg dua 0,1%. Nyob rau theem no, nws yog ib qho tseem ceeb heev uas qhov tseeb technical thiab cov khoom ntsuas tau suav nrog hauv kev saib xyuas kom lawv qhia qhov teeb meem hauv kev pabcuam sai li sai tau. Lub sijhawm tsawg kawg nkaus rau kev xeem canary yog 5 feeb, qhov tseem ceeb yog 2 teev. Rau cov kev pabcuam nyuaj, peb teem sijhawm manually.
Cia peb tsom xam:
- cov lus qhia tshwj xeeb, tshwj xeeb, cov neeg ua haujlwm php-fpm;
- yuam kev hauv Sentry;
- teb cov xwm txheej;
- lub sij hawm teb, meej thiab nruab nrab;
- latency;
- kev zam, ua tiav thiab tsis siv;
- cov khoom ntsuas.
Squeeze Test
Squeeze Testing tseem hu ua "squeezing" testing. Lub npe ntawm cov txheej txheem tau nkag mus rau hauv Netflix. Nws cov ntsiab lus yog tias thawj zaug peb sau ib qho piv txwv nrog cov tsheb tiag tiag mus rau qhov tsis ua tiav thiab yog li teeb tsa nws qhov kev txwv. Tom qab ntawd peb ntxiv lwm qhov piv txwv thiab thauj cov khub no - dua rau qhov siab tshaj plaws; peb pom lawv qab nthab thiab delta nrog thawj "squeeze". Thiab yog li peb txuas ib qho piv txwv ntawm ib lub sijhawm thiab suav cov qauv ntawm kev hloov pauv.
Kuaj cov ntaub ntawv los ntawm "squeezing" kuj ntws mus rau hauv ib qho kev ntsuas cov ntaub ntawv, qhov twg peb tuaj yeem txhawb nqa cov khoom siv dag zog nrog lawv, lossis txawm hloov "synthetics" nrog lawv.
Ntau lawm
β’ Scaling. Thaum peb yob tawm ib qho kev pabcuam rau kev tsim khoom, peb saib xyuas seb nws ntsuas li cas. Hauv peb qhov kev paub dhau los, kev saib xyuas tsuas yog cov ntsuas CPU tsis muaj txiaj ntsig. Nws pib scaling nrog RPS benchmarking nyob rau hauv nws cov ntshiab daim ntawv ua hauj lwm, tab sis tsuas yog rau tej yam kev pab cuam, xws li streaming online. Yog li peb saib ua ntej ntawm daim ntawv thov tshwj xeeb cov khoom ntsuas.
Yog li ntawd, thaum scaling peb tsom xam:
- CPU thiab RAM ntsuas,
- tus naj npawb ntawm kev thov hauv kab,
- lub sijhawm teb,
- Kev kwv yees raws li cov ntaub ntawv keeb kwm sau tseg.
Thaum ntsuas qhov kev pabcuam, nws tseem ceeb heev uas yuav tsum tau saib xyuas nws qhov kev cia siab kom peb tsis txhob ntsuas qhov kev pabcuam thawj zaug hauv cov saw hlau, thiab cov uas nws nkag mus tau poob qis. Txhawm rau tsim kom muaj kev txaus siab rau tag nrho cov kev pabcuam, peb saib cov ntaub ntawv keeb kwm ntawm qhov "ze" kev pabcuam (raws li kev sib xyaw ntawm CPU thiab RAM ntsuas, ua ke nrog cov kev ntsuas tshwj xeeb) thiab sib piv nrog cov ntaub ntawv keeb kwm. ntawm qhov kev pabcuam pib, thiab lwm yam thoob plaws hauv "kev vam khom"", los ntawm sab saum toj mus rau hauv qab.
Kev pab
Tom qab lub microservice tau muab tso rau hauv kev ua haujlwm, peb tuaj yeem txuas cov txiaj ntsig rau nws.
Ntawm no yog cov xwm txheej uas tshwm sim.
- Pom muaj peev xwm tsiv teb tsaws chaw.
- Kev ruaj ntseg hloov tshiab tau raug tso tawm.
- Cov kev pabcuam nws tus kheej tsis tau hloov kho rau lub sijhawm ntev.
- Lub nra ntawm qhov kev pabcuam tau pom zoo txo ββqis lossis qee qhov ntawm nws cov khoom ntsuas tau nyob sab nraum qhov qub.
- Cov kev pabcuam tsis ua raws li lub platform tshiab.
Qee qhov kev cuam tshuam yog lub luag haujlwm rau kev ruaj ntseg ntawm kev ua haujlwm, qee qhov - raws li kev ua haujlwm ntawm kev saib xyuas lub cev - piv txwv li, qee qhov kev pabcuam tsis tau siv sijhawm ntev thiab nws cov duab hauv qab tau tso tseg tsis dhau kev kuaj xyuas kev ruaj ntseg.
Dashboard
Nyob rau hauv luv luv, lub dashboard yog cov tswj vaj huam sib luag ntawm peb tag nrho PaaS.
- Ib lub ntsiab lus ntawm cov ntaub ntawv hais txog kev pabcuam, nrog rau cov ntaub ntawv ntawm nws qhov kev xeem, tus lej ntawm nws cov duab, tus lej ntawm cov ntawv luam, cov qauv, thiab lwm yam.
- Ib lub cuab yeej rau kev lim cov ntaub ntawv los ntawm cov kev pabcuam thiab cov ntawv sau (cov cim ntawm cov khoom lag luam, kev ua haujlwm ntawm cov khoom, thiab lwm yam)
- Ib qho cuab yeej rau kev koom ua ke nrog cov cuab yeej cuab tam rau kev taug qab, txiav, thiab saib xyuas.
- Ib qho piv txwv ntawm cov ntaub ntawv kev pabcuam.
- Ib qho kev pom ntawm txhua qhov xwm txheej thoob plaws cov kev pabcuam.
Tag nrho
Ua ntej qhia PaaS, tus tsim tawm tshiab tuaj yeem siv sijhawm ntau lub lis piam nkag siab tag nrho cov cuab yeej tsim nyog los tsim cov kev pabcuam micro hauv kev tsim khoom: Kubernetes, Helm, peb sab hauv TeamCity nta, teeb tsa kev sib txuas rau databases thiab caches hauv qhov ua txhaum cai, thiab lwm yam. Tam sim no nws siv sij hawm ob peb teev los nyeem qhov pib sai thiab tsim cov kev pabcuam nws tus kheej.
Kuv tau muab ib daim ntawv qhia txog lub ncauj lus no rau HighLoad ++ 2018, koj tuaj yeem saib nws ΠΈ .
Bonus khiav rau cov neeg uas nyeem kom txog thaum kawg
Peb ntawm Avito tab tom npaj ib qho kev cob qhia sab hauv peb hnub rau cov neeg tsim khoom los ntawm , tus kws tshaj lij hauv microservice architecture. Peb xav muab lub sijhawm los koom nrog nws rau ib qho ntawm cov neeg nyeem ntawm no ncej. Txoj kev cob qhia tau muab tso tawm.
Kev cob qhia yuav muaj nyob rau lub Yim Hli 5 txog 7 hauv Moscow. Cov no yog cov hnub ua hauj lwm uas yuav tau nyob tag nrho. Noj su thiab kev cob qhia yuav nyob hauv peb lub chaw ua haujlwm, thiab cov neeg koom nrog xaiv yuav them rau kev mus ncig thiab kev pab nws tus kheej.
Koj tuaj yeem thov rau kev koom tes . Los ntawm koj - cov lus teb rau lo lus nug yog vim li cas koj thiaj li yuav tsum tau mus koom kev cob qhia thiab cov ntaub ntawv hais txog yuav tiv tauj koj li cas. Teb hauv lus Askiv, vim tias Chris yuav xaiv tus neeg koom nrog uas yuav tuaj koom kev cob qhia nws tus kheej.
Peb yuav tshaj tawm lub npe ntawm tus neeg koom nrog kev cob qhia hauv kev hloov tshiab rau cov ntawv tshaj tawm no thiab hauv kev tes hauj lwm Avito rau cov neeg tsim khoom (AvitoTech hauv , , ) tsis pub dhau lub Xya Hli 19.
Tau qhov twg los: www.hab.com