Sysadminka qhov kev sib tham tus thawj tswj hwm tau tshwm sim hauv Chelyabinsk, thiab thaum kawg kuv tau muab ib daim ntawv qhia txog peb cov kev daws teeb meem rau kev khiav haujlwm ntawm 1C-Bitrix hauv Kubernetes.
Bitrix, Kubernetes, Ceph - zoo sib xyaw?
Kuv yuav qhia koj li cas peb muab kev daws teeb meem ua haujlwm los ntawm txhua qhov no.
Cia peb mus!
Lub rooj sib tham tau tshwm sim rau lub Plaub Hlis 18 hauv Chelyabinsk. Koj tuaj yeem nyeem txog peb cov kev sib ntsib ntawm thiab saib .
Yog hais tias koj xav tuaj rau peb nrog ib daim ntawv qhia los yog raws li ib tug mloog - txais tos, sau rau [email tiv thaiv] thiab hauv Telegram t.me/vadimisakanov.
Kuv qhia
Kev daws "Bitrix hauv Kubernetes, version Southbridge 1.0"
Kuv yuav tham txog peb cov kev daws teeb meem hauv "rau dummies hauv Kubernetes" hom, raws li tau ua tiav ntawm lub rooj sib tham. Tab sis kuv xav tias koj paub cov lus Bitrix, Docker, Kubernetes, Ceph yam tsawg kawg ntawm cov ntawv hauv Wikipedia.
Dab tsi yog npaj-ua txog Bitrix hauv Kubernetes?
Muaj cov ntaub ntawv tsawg heev ntawm tag nrho Is Taws Nem txog kev ua haujlwm ntawm Bitrix daim ntawv thov hauv Kubernetes.
Kuv tsuas pom cov ntaub ntawv no:
Qhia los ntawm Alexander Serbul, 1C-Bitrix, thiab Anton Tuzlukov los ntawm Qsoft:
Kuv pom zoo kom mloog nws.
Tsim koj tus kheej kev daws teeb meem los ntawm tus neeg siv .
Pom ntau .
Aaand ... qhov tseeb, yog tag nrho.
Kuv ceeb toom rau koj, peb tsis tau kuaj xyuas qhov zoo ntawm cov kev daws teeb meem hauv cov ntawv txuas saum toj no :)
Los ntawm txoj kev, thaum npaj peb cov kev daws teeb meem, kuv tau tham nrog Alexander Serbul, ces nws daim ntawv qhia tseem tsis tau tshwm sim, yog li hauv kuv cov slides muaj ib yam khoom "Bitrix tsis siv Kubernetes."
Tab sis twb muaj ntau cov duab Docker npaj rau kev khiav Bitrix hauv Docker:
Puas yog qhov no txaus los tsim kev daws teeb meem rau Bitrix hauv Kubernetes?
Tsis muaj. Muaj ntau yam teeb meem uas yuav tsum tau daws.
Dab tsi yog teeb meem nrog Bitrix hauv Kubernetes?
Ua ntej, npaj cov duab los ntawm Dockerhub tsis haum rau Kubernetes
Yog hais tias peb xav tsim ib lub microservices architecture (thiab hauv Kubernetes peb feem ntau ua), peb yuav tsum cais peb cov ntawv thov Kubernetes rau hauv ntim thiab kom txhua lub thawv ua ib qho haujlwm me me (thiab ua nws zoo). Vim li cas ib leeg xwb? Hauv luv, qhov yooj yim dua qhov txhim khu kev qha.
Yuav kom paub meej ntxiv, saib kab lus no thiab video, thov:
Docker dluab hauv Dockerhub yog tsim los ntawm tag nrho-hauv-ib txoj cai, yog li peb tseem yuav tsum tau ua peb tus kheej lub tsheb kauj vab thiab txawm tsim cov duab los ntawm kos.
Thib ob - lub vev xaib code raug kho los ntawm admin vaj huam sib luag
Peb tsim ib ntu tshiab ntawm lub xaib - cov cai tau hloov kho (cov npe nrog lub npe ntawm ntu tshiab tau ntxiv).
Yog tias koj hloov cov khoom ntawm ib feem ntawm lub vaj huam sib luag admin, tus lej hloov.
Kubernetes "los ntawm lub neej ntawd" tsis tuaj yeem ua haujlwm nrog qhov no; cov thawv yuav tsum tsis muaj neeg nyob.
Yog vim li cas: Txhua lub thawv (pob tawb) hauv pawg txheej txheem tsuas yog ib feem ntawm kev khiav tsheb. Yog tias koj hloov cov cai hauv ib lub thawv nkaus xwb (pob ntseg), ces tus lej yuav txawv ntawm cov pods sib txawv, lub xaib yuav ua haujlwm sib txawv, thiab cov qauv sib txawv ntawm lub xaib yuav qhia rau cov neeg siv sib txawv. Koj ua tsis tau li ntawd.
Thib peb - koj yuav tsum daws qhov teeb meem nrog kev xa tawm
Yog hais tias peb muaj ib tug monolith thiab ib tug "classic" server, txhua yam yog heev yooj yim: peb deploy ib tug tshiab code puag, migrate lub database, hloov tsheb mus rau lub tshiab version ntawm code. Switching tshwm sim tam sim ntawd.
Yog tias peb muaj qhov chaw hauv Kubernetes, txiav rau hauv microservices, muaj ntau lub ntim nrog cov lej - huag. Koj yuav tsum tau sau cov thawv nrog cov qauv tshiab ntawm cov cai, yob lawv tawm ntawm cov qub, hloov cov ntaub ntawv kom raug, thiab ua qhov zoo tshaj plaws los ntawm cov neeg tuaj saib. Hmoov zoo, Kubernetes pab peb nrog qhov no, txhawb nqa ntau pawg ntawm ntau hom kev xa tawm.
Plaub - koj yuav tsum tau daws qhov teeb meem ntawm kev khaws cov statics
Yog tias koj lub vev xaib tsuas yog "tsuas yog" 10 gigabyte thiab koj siv nws tag nrho hauv cov thawv, koj yuav xaus nrog 10 gigabyte ntim uas yuav siv mus ib txhis.
Koj yuav tsum khaws cov "hnyav tshaj plaws" ntawm qhov chaw sab nraum lub thawv, thiab cov lus nug tshwm sim ntawm yuav ua li cas kom raug.
Dab tsi ploj ntawm peb qhov kev daws teeb meem?
Tag nrho cov cai Bitrix tsis muab faib ua microfunctions/microservices (yog li ntawd, kev sau npe yog cais, lub khw hauv online module yog cais, thiab lwm yam). Peb khaws tag nrho cov cai hauv paus hauv txhua lub thawv.
Peb kuj tsis khaws cov ntaub ntawv hauv Kubernetes (Kuv tseem siv cov kev daws teeb meem nrog cov ntaub ntawv hauv Kubernetes rau kev txhim kho ib puag ncig, tab sis tsis yog rau kev tsim khoom).
Nws tseem yuav pom tseeb rau cov thawj coj ntawm lub xaib uas lub vev xaib khiav ntawm Kubernetes. Lub "system check" muaj nuj nqi tsis ua hauj lwm kom raug; los kho qhov chaw code los ntawm lub admin vaj huam sib luag, koj yuav tsum xub nias lub khawm "Kuv xav hloov cov code".
Cov teeb meem tau raug txheeb xyuas, qhov xav tau los siv microservices tau txiav txim siab, lub hom phiaj yog qhov tseeb - kom tau txais cov txheej txheem ua haujlwm rau kev khiav haujlwm ntawm Bitrix hauv Kubernetes, khaws cia ob lub peev xwm ntawm Bitrix thiab qhov zoo ntawm Kubernetes. Cia peb pib siv.
architecture
Muaj ntau "ua haujlwm" pods nrog lub web server (cov neeg ua haujlwm).
Ib qho hauv qab nrog cron cov haujlwm (tsuas yog ib qho yuav tsum tau ua).
Ib qho kev hloov kho rau kev kho qhov chaw code los ntawm lub vaj huam sib luag admin (tseem tsuas yog yuav tsum muaj ib qho).
Peb daws cov lus nug:
- Yuav khaws cov kev sib tham nyob qhov twg?
- Yuav khaws lub cache qhov twg?
- Qhov twg khaws cov statics, tsis txhob tso gigabytes ntawm statics rau hauv ib pawg ntawm cov thawv?
- Lub database yuav ua haujlwm li cas?
Docker duab
Peb pib los ntawm kev tsim Docker duab.
Qhov kev xaiv zoo tshaj plaws yog tias peb muaj ib daim duab thoob ntiaj teb, ntawm nws lub hauv paus peb tau txais cov neeg ua haujlwm pods, pods nrog Crontasks, thiab hloov cov pods.
.
Nws suav nrog nginx, apache / php-fpm (tuaj yeem xaiv thaum tsim), msmtp rau xa ntawv, thiab cron.
Thaum sib sau ua ke cov duab, tag nrho cov cai hauv paus ntawm lub xaib tau theej rau /app directory (nrog rau qhov tshwj xeeb ntawm cov khoom uas peb yuav txav mus rau ib qho kev sib koom ua ke).
Microservices, kev pabcuam
cov neeg ua hauj lwm pods:
- Thawv nrog nginx + thawv apache/php-fpm + msmtp
- Nws tsis ua haujlwm kom txav msmtp mus rau hauv microservice cais, Bitrix tab tom pib npau taws tias nws tsis tuaj yeem xa ntawv ncaj qha
- Txhua lub thawv muaj qhov ua tiav codebase.
- Kev txwv tsis pub hloov cov cai hauv cov thawv.
cron hauv qab no:
- thawv nrog apache, php, cron
- ua tiav cov lej hauv paus suav nrog
- txwv tsis pub hloov code hauv cov thawv
upgrade raws li nram no:
- nginx thawv + apache/php-fpm thawv + msmtp
- Tsis muaj kev txwv tsis pub hloov code hauv ntim
kev kaw cia
Bitrix cache cia
Lwm qhov tseem ceeb: peb khaws cov passwords rau kev sib txuas rau txhua yam, los ntawm cov ntaub ntawv mus rau kev xa ntawv, hauv kubernetes secrets. Peb tau txais nyiaj ntxiv: cov passwords tsuas yog pom rau cov neeg uas peb muab kev nkag mus rau qhov tsis pub lwm tus paub, thiab tsis yog rau txhua tus neeg uas muaj kev nkag mus rau qhov project code base.
Cia rau statics
Koj tuaj yeem siv txhua yam: ceph, nfs (tab sis peb tsis pom zoo nfs rau kev tsim khoom), network cia los ntawm cov chaw muab huab, thiab lwm yam.
Qhov chaw cia yuav tsum tau txuas nrog rau hauv cov thawv rau / upload / directory ntawm lub xaib thiab lwm cov npe nrog cov ntsiab lus zoo li qub.
Database
Rau qhov yooj yim, peb pom zoo kom txav cov ntaub ntawv sab nraud ntawm Kubernetes. Lub hauv paus hauv Kubernetes yog ib txoj hauj lwm nyuaj; nws yuav ua rau lub tswv yim ntawm qhov kev txiav txim siab ntau dua.
Session cia
Peb siv memcached :)
Nws tuav kev sib kho cia zoo, yog pawg, thiab tau txais kev txhawb nqa "natively" raws li session.save_path hauv php. Xws li ib tug system tau raug sim ntau zaus nyob rau hauv classical monolithic architecture, thaum peb ua pawg nrog coob tus web servers. Rau kev xa tawm peb siv lub kaus mom hlau.
$ helm install stable/memcached --name sessionphp.ini - ntawm no cov duab muaj cov chaw rau khaws cov ntu hauv memcached
Peb siv Environment Variables kom dhau cov ntaub ntawv hais txog cov tswv nrog memcached .
Qhov no tso cai rau koj siv tib txoj cai nyob rau hauv dev, theem, xeem, prod ib puag ncig (lub memcached host lub npe nyob rau hauv lawv yuav txawv, yog li peb yuav tsum dhau ib lub npe tshwj xeeb rau kev sib ntsib rau txhua qhov chaw).
Bitrix cache cia
Peb xav tau ib qho chaw cia ua txhaum uas txhua lub pods tuaj yeem sau thiab nyeem los ntawm.
Peb kuj siv memcached.
Cov tshuaj no tau pom zoo los ntawm Bitrix nws tus kheej.
$ helm install stable/memcached --name cachebitrix/.settings_extra.php - ntawm no hauv Bitrix nws tau teev tseg qhov twg cov cache khaws cia
Peb kuj siv Environment Variables.
Krontaski
Muaj ntau txoj hauv kev los khiav Crontask hauv Kubernetes.
- cais kev xa tawm nrog lub pod rau khiav Crontasks
- cronjob rau kev ua haujlwm crontasks (yog tias qhov no yog lub vev xaib app - nrog wget , lossis kubectl exec hauv ib qho ntawm cov neeg ua haujlwm pods, thiab lwm yam.)
- thiab lwm yam
Koj tuaj yeem sib cav txog qhov tseeb tshaj plaws, tab sis qhov no peb xaiv qhov kev xaiv "sib cais xa tawm nrog cov pods rau Crontasks"
Nws ua li cas:
- ntxiv cron cov haujlwm ntawm ConfigMap lossis ntawm cov ntaub ntawv config/addcron
- Hauv ib qho piv txwv peb tso lub thawv zoo ib yam rau cov neeg ua haujlwm pod + tso cai rau kev ua tiav cov haujlwm hauv nws
- tib lub hauv paus yog siv, ua tsaug rau kev sib koom ua ke, lub thawv sib dhos yooj yim
Qhov zoo peb tau txais:
- peb tau ua haujlwm Crontasks hauv ib puag ncig zoo ib yam li cov neeg tsim khoom ib puag ncig (docker)
- Crontasks tsis tas yuav "rov sau dua" rau Kubernetes, lawv ua haujlwm hauv tib daim ntawv thiab hauv tib txoj cai raws li ua ntej
- cron cov dej num tuaj yeem ntxiv los ntawm txhua tus neeg koom nrog nrog kev cog lus rau cov ceg tsim khoom, tsis yog cov thawj coj xwb
Southbridge K8SDeploy module thiab code kho los ntawm admin vaj huam sib luag
Peb tab tom tham txog kev txhim kho hauv qab?
Yuav ua li cas coj tsheb mus rau qhov ntawd?
Hurray, peb tau sau ib qho module rau qhov no hauv PHP :) Qhov no yog ib qho me me classic module rau Bitrix. Nws tseem tsis tau muaj kev tshaj tawm rau pej xeem, tab sis peb npaj yuav qhib nws.
Lub module yog ntsia zoo li ib txwm module hauv Bitrix:
Thiab nws zoo li no:
Nws tso cai rau koj los teeb tsa lub ncuav qab zib uas txheeb xyuas tus thawj coj ntawm lub xaib thiab tso cai rau Kubernetes xa cov tsheb mus rau qhov hloov kho pod.
Thaum cov kev hloov pauv tiav, koj yuav tsum nyem git thawb, cov lej hloov pauv yuav raug xa mus rau git, tom qab ntawd lub kaw lus yuav tsim cov duab nrog tus lej tshiab thiab "yuav tawm" nws hla cov pawg, hloov cov qub pods. .
Yog lawm, nws yog me ntsis ntawm tus ntoo khaub lig, tab sis tib lub sijhawm peb tswj hwm microservice architecture thiab tsis txhob tshem tawm ntawm cov neeg siv Bitrix lawv nyiam lub sijhawm los kho cov cai los ntawm pawg tswj hwm. Thaum kawg, qhov no yog ib qho kev xaiv; koj tuaj yeem daws qhov teeb meem ntawm kev kho cov cai hauv ib txoj kev sib txawv.
Helm daim duab
Txhawm rau tsim cov ntawv thov ntawm Kubernetes, peb feem ntau siv Helm pob tus thawj tswj hwm.
Rau peb cov kev daws teeb meem Bitrix hauv Kubernetes, Sergey Bondarev, peb tus thawj coj tswj hwm, tau sau ib daim ntawv tshwj xeeb Helm.
Nws tsim cov neeg ua haujlwm, grade, cron pods, configures ingresses, kev pabcuam, thiab hloov pauv hloov pauv los ntawm Kubernetes secrets mus rau pods.
Peb khaws cov cai hauv Gitlab, thiab peb kuj khiav Helm tsim los ntawm Gitlab.
Hauv luv, nws zoo li no
$ helm upgrade --install project .helm --set image=registrygitlab.local/k8s/bitrix -f .helm/values.yaml --wait --timeout 300 --debug --tiller-namespace=productionHelm kuj tso cai rau koj ua qhov "seamless" rollback yog dheev ib yam dab tsi mus tsis ncaj ncees lawm thaum xa mus. Nws yog qhov zoo thaum koj tsis nyob hauv kev ntshai "kho cov cai ntawm ftp vim tias cov khoom poob," tab sis Kubernetes ua nws tau, thiab tsis muaj sijhawm poob.
Muab tso rau
Yog lawm, peb yog cov kiv cua ntawm Gitlab & Gitlab CI, peb siv nws :)
Thaum cog lus hauv Gitlab mus rau qhov project repository, Gitlab tso lub raj xa dej uas xa tawm ib puag ncig tshiab.
Cov theem:
- tsim (tsim Docker duab tshiab)
- xeem (test)
- huv si (tshem tawm qhov chaw sim)
- thawb (peb xa mus rau Docker npe)
- deploy (peb xa daim ntawv thov mus rau Kubernetes ntawm Helm).
Hurray, npaj tau, cia peb ua nws!
Zoo, los yog nug cov lus nug yog tias muaj.
Yog li ntawd peb tau ua dab tsi
Los ntawm ib tug technical point ntawm view:
- dockerized Bitrix;
- "txiav" Bitrix rau hauv ntim, txhua qhov ua haujlwm tsawg kawg nkaus;
- ua tiav lub xeev tsis muaj lub thawv ntim khoom;
- daws qhov teeb meem nrog kev hloov kho Bitrix hauv Kubernetes;
- tag nrho cov haujlwm Bitrix txuas ntxiv ua haujlwm (yuav luag tag nrho);
- Peb tau ua haujlwm ntawm kev xa mus rau Kubernetes thiab rollback ntawm versions.
Los ntawm qhov kev xav ntawm kev lag luam:
- ua txhaum cai;
- Kubernetes cov cuab yeej (kev koom ua ke yooj yim nrog Gitlab CI, seamless xa mus, thiab lwm yam);
- cov passwords zais cia (pom tsuas yog rau cov neeg uas tau txais kev tso cai ncaj qha rau cov passwords);
- Nws yooj yim los tsim ib puag ncig ntxiv (rau kev txhim kho, kev sim, thiab lwm yam) hauv ib qho kev tsim kho vaj tse.
Tau qhov twg los: www.hab.com