🥇 Keeb kwm ntawm kev tawm, uas cuam tshuam txhua yam

Cov yeeb ncuab ntawm Kev muaj tiag ua 12f2

Thaum kawg ntawm lub Plaub Hlis, thaum White Walkers tab tom tuav Winterfell, ib yam dab tsi nthuav ntxiv rau peb; peb tau ua qhov txawv txav. Hauv txoj ntsiab cai, peb tab tom dov tawm cov yam ntxwv tshiab rau hauv kev tsim khoom (zoo li txhua tus neeg). Tab sis qhov no txawv. Qhov ntsuas ntawm nws yog qhov ua yuam kev uas peb tuaj yeem ua tau yuav cuam tshuam tag nrho peb cov kev pabcuam thiab cov neeg siv. Yog li ntawd, peb dov tawm txhua yam raws li kev npaj, nyob rau hauv lub sij hawm npaj thiab tshaj tawm downtime, tsis muaj txim rau kev muag khoom. Kab lus yog hais txog yuav ua li cas peb ua tiav qhov no thiab yuav ua li cas leej twg tuaj yeem rov ua nws hauv tsev.

Tam sim no kuv yuav tsis piav qhia txog kev txiav txim siab ntawm vaj tse thiab kev ua haujlwm uas peb tau ua lossis qhia tias nws ua haujlwm li cas. Cov no yog cov ntawv sau nyob rau hauv cov npoo hais txog yuav ua li cas ib qho ntawm qhov nyuaj tshaj plaws rollouts coj qhov chaw, uas kuv pom thiab nyob rau hauv uas kuv tau ncaj qha mus koom. Kuv tsis lees paub qhov ua tiav lossis cov ntsiab lus technical; tej zaum lawv yuav tshwm sim hauv lwm tsab xov xwm.

Tom qab + dab tsi ua haujlwm yog qhov no?

Peb tab tom tsim huab platform Mail.ru Huab Solutions (MCS), qhov uas kuv ua hauj lwm ua tus thawj coj. Thiab tam sim no nws yog lub sijhawm los ntxiv IAM (Identity and Access Management) rau peb lub platform, uas muab kev tswj hwm kev sib koom ua ke ntawm txhua tus neeg siv nyiaj, cov neeg siv, passwords, lub luag haujlwm, kev pabcuam thiab ntau dua. Vim li cas nws thiaj li xav tau hauv huab yog ib lo lus nug pom tseeb: txhua tus neeg siv cov ntaub ntawv khaws cia hauv nws.

Feem ntau cov khoom zoo li no pib ua thaum pib ntawm txhua qhov haujlwm. Tab sis keeb kwm tej yam tau txawv me ntsis hauv MCS. MCS tau tsim hauv ob ntu:

Openstack nrog nws tus kheej Keystone tso cai module,
Hotbox (S3 cia) raws li Mail.ru Cloud project,

nyob ib ncig ntawm qhov kev pabcuam tshiab tshwm sim.

Qhov tseem ceeb, cov no yog ob hom kev tso cai sib txawv. Ntxiv rau, peb siv qee qhov sib cais Mail.ru kev txhim kho, piv txwv li, ib qho dav dav Mail.ru lo lus zais cia, nrog rau tus kheej sau openid connector, ua tsaug rau SSO (kev tso cai kawg-rau-kawg) tau muab rau hauv Horizon vaj huam sib luag. ntawm cov tshuab virtual (native OpenStack UI).

Ua IAM rau peb txhais tau tias txuas tag nrho rau hauv ib qho system, tag nrho peb tus kheej. Nyob rau tib lub sijhawm, peb yuav tsis poob ib qho kev ua haujlwm raws li txoj kev, tab sis yuav tsim lub hauv paus rau yav tom ntej uas yuav tso cai rau peb kom pob tshab refine nws yam tsis muaj refactoring thiab scale nws nyob rau hauv cov nqe lus ntawm functionality. Tsis tas li thaum pib, cov neeg siv tau muaj tus qauv rau kev nkag mus rau cov kev pabcuam (central RBAC, kev tswj hwm kev siv lub luag haujlwm) thiab lwm yam me me.

Txoj haujlwm tau hloov mus ua tsis tseem ceeb: python thiab perl, ob peb lub backends, cov kev pabcuam sau ntawv ntawm nws tus kheej, ntau pab pawg txhim kho thiab admins. Thiab qhov tseem ceeb tshaj plaws, muaj ntau txhiab tus neeg siv nyob ntawm kev sib ntaus sib tua. Tag nrho cov no yuav tsum tau sau thiab, qhov tseem ceeb tshaj plaws, dov tawm yam tsis muaj kev raug mob.

Dab tsi yog peb yuav dov tawm?

Txhawm rau muab nws roughly, li ntawm 4 lub hlis peb npaj cov hauv qab no:

Peb tau tsim ob peb lub daemons tshiab uas sib sau ua haujlwm uas yav dhau los ua haujlwm hauv ntau qhov chaw ntawm cov txheej txheem. Tus so ntawm cov kev pab cuam tau muab ib tug tshiab backend nyob rau hauv daim ntawv ntawm cov dab.
Peb tau sau peb tus kheej qhov chaw khaws cov passwords thiab cov yuam sij, muaj rau tag nrho peb cov kev pabcuam, uas tuaj yeem hloov kho dawb raws li peb xav tau.
Peb tau sau 4 backends tshiab rau Keystone los ntawm kos (cov neeg siv, tej yaam num, lub luag hauj lwm, lub luag hauj lwm assignments), uas, qhov tseeb, hloov nws database, thiab tam sim no ua raws li ib tug repository rau peb cov neeg siv passwords.
Peb tau qhia tag nrho peb cov kev pabcuam Openstack kom mus rau qhov kev pabcuam thib peb rau lawv cov cai es tsis txhob nyeem cov cai no hauv zos los ntawm txhua tus neeg rau zaub mov (yog, qhov ntawd yog li cas Openstack ua haujlwm los ntawm lub neej ntawd!)

Xws li ib tug loj rework yuav tsum loj, complex thiab, tseem ceeb tshaj, synchronous hloov nyob rau hauv ob peb lub tshuab sau los ntawm ntau yam kev loj hlob pab pawg. Thaum sib sau ua ke, tag nrho cov system yuav tsum ua haujlwm.

Yuav ua li cas dov tawm cov kev hloov no thiab tsis ntsia nws? Ua ntej peb txiav txim siab saib me ntsis rau yav tom ntej.

Rollout tswv yim

Nws yuav ua tau kom yob tawm cov khoom nyob rau hauv ob peb theem, tab sis qhov no yuav ua rau kom lub sij hawm loj hlob peb zaug. Tsis tas li ntawd, rau qee lub sij hawm peb yuav muaj kev ua tiav desynchronization ntawm cov ntaub ntawv hauv databases. Koj yuav tsum tau sau koj tus kheej cov cuab yeej synchronization thiab nyob nrog ntau lub khw muag khoom rau lub sijhawm ntev. Thiab qhov no ua rau muaj ntau yam txaus ntshai.
Txhua yam uas tuaj yeem npaj pob tshab rau tus neeg siv tau ua tiav ua ntej. Nws siv 2 lub hlis.
Peb tso cai rau peb tus kheej downtime rau ob peb teev - tsuas yog rau cov neeg siv kev ua haujlwm los tsim thiab hloov cov peev txheej.
Rau kev ua haujlwm ntawm tag nrho cov peev txheej uas twb tau tsim lawm, lub sijhawm poob tsis tuaj yeem lees txais. Peb tau npaj tias thaum lub sijhawm tso tawm, cov peev txheej yuav tsum ua haujlwm yam tsis muaj sijhawm poob thiab cuam tshuam rau cov neeg siv khoom.
Txhawm rau txo qhov cuam tshuam rau peb cov neeg siv khoom yog tias muaj qee yam tsis raug, peb tau txiav txim siab tawm rau hnub Sunday yav tsaus ntuj. Tsawg tus neeg siv khoom tswj hwm lub tshuab virtual thaum hmo ntuj.
Peb tau ceeb toom tag nrho peb cov neeg siv khoom tias thaum lub sijhawm xaiv rau kev tshaj tawm, kev tswj xyuas kev pabcuam yuav tsis muaj.

Digression: dab tsi yog qhov rollout?

<cua, philosophy>

Txhua tus kws tshaj lij IT tuaj yeem teb tau yooj yim tias qhov kev nthuav tawm yog dab tsi. Koj nruab CI / CD, thiab txhua yam yog xa mus rau lub khw. 🙂

Tau kawg qhov no muaj tseeb. Tab sis qhov nyuaj yog tias nrog cov cai niaj hnub xa khoom automation cov cuab yeej, kev nkag siab ntawm rollout nws tus kheej poob. Yuav ua li cas koj tsis nco qab txog lub epicness ntawm lub invention ntawm lub log thaum saib ntawm niaj hnub thauj. Txhua yam yog li ntawd automated tias qhov rollout feem ntau nqa tawm yam tsis nkag siab tag nrho cov duab.

Thiab tag nrho cov duab zoo li no. Rollout muaj plaub yam tseem ceeb:

Kev xa cov lej, suav nrog kev hloov kho cov ntaub ntawv. Piv txwv li, lawv tsiv teb tsaws chaw.
Code rollback yog lub peev xwm rov qab yog tias muaj qee yam yuam kev. Piv txwv li, los ntawm kev tsim cov thaub qab.
Lub sijhawm ua haujlwm ntawm txhua qhov rollout/rollback. Koj yuav tsum nkag siab lub sijhawm ntawm kev ua haujlwm ntawm thawj ob lub ntsiab lus.
Ua haujlwm cuam tshuam. Nws yog ib qho tsim nyog los ntsuas ob qho tib si qhov kev xav tau zoo thiab qhov tshwm sim tsis zoo.

Tag nrho cov no yuav tsum raug coj mus rau hauv tus account kom ua tiav qhov kev ua tiav. Feem ntau tsuas yog thawj zaug, lossis qhov zoo tshaj plaws thib ob, taw tes yog ntsuas, thiab tom qab ntawd qhov kev tshaj tawm tau suav tias ua tiav. Tab sis qhov thib peb thiab plaub tseem ceeb dua. Tus neeg siv yuav nyiam nws yog tias qhov kev tshaj tawm siv sijhawm 3 teev tsis yog ib feeb? Los yog yog tias qee yam tsis tsim nyog tau cuam tshuam thaum lub sijhawm tso tawm? Puas yog lub sijhawm poob ntawm ib qho kev pabcuam yuav ua rau muaj qhov tshwm sim tsis tau tshwm sim?

Txoj Cai 1..n, npaj rau kev tso tawm

Thaum xub thawj kuv xav txog luv luv piav txog peb cov rooj sib tham: tag nrho pab neeg, nws cov khoom, pawg ntawm kev sib tham ntawm kas fes cov ntsiab lus, kev sib cav, kev ntsuam xyuas, lub hlwb. Tom qab ntawd kuv xav tias nws yuav tsis tsim nyog. Plaub lub hlis ntawm txoj kev loj hlob ib txwm muaj qhov no, tshwj xeeb tshaj yog thaum koj tsis sau ib yam dab tsi uas tuaj yeem xa mus tas li, tab sis ib qho tseem ceeb rau lub neej nyob. Uas cuam tshuam rau tag nrho cov kev pabcuam, tab sis tsis muaj dab tsi yuav tsum hloov pauv rau cov neeg siv tsuas yog "ib pob hauv web interface."

Peb txoj kev nkag siab ntawm yuav ua li cas yob tawm tau hloov pauv los ntawm txhua lub rooj sib tham tshiab, thiab tseem ceeb heev. Piv txwv li, peb yuav hloov kho peb cov ntaub ntawv sau nqi tag nrho. Tab sis peb suav lub sijhawm thiab pom tau tias nws tsis tuaj yeem ua qhov no hauv lub sijhawm tsim nyog. Nws coj peb yuav luag ib lub lim tiam ntxiv rau shard thiab khaws cov ntaub ntawv sau nqi. Thiab thaum qhov kev cia siab rollout ceev tseem tsis txaus siab, peb tau xaj cov khoom siv ntxiv, muaj zog dua, qhov twg tag nrho lub hauv paus raug rub. Nws tsis yog tias peb tsis xav ua qhov no sai dua, tab sis tam sim no xav tau dov tawm tawm peb tsis muaj kev xaiv.

Thaum ib tus ntawm peb muaj kev tsis ntseeg tias qhov kev tshaj tawm tuaj yeem cuam tshuam rau peb lub tshuab virtual, peb siv sijhawm ib lub lim tiam ua kev sim, kev sim, kev txheeb xyuas cov lej thiab tau txais kev nkag siab meej tias qhov no yuav tsis tshwm sim hauv peb cov khoom, thiab txawm tias cov neeg tsis ntseeg siab tshaj plaws tau pom zoo. nrog qhov no.

Nyob rau hauv lub sijhawm no, cov txiv neej los ntawm kev txhawb nqa tau ua lawv tus kheej cov kev sim ywj pheej txhawm rau sau cov lus qhia rau cov neeg siv khoom ntawm txoj kev sib txuas, uas yuav tsum tau hloov pauv tom qab tso tawm. Lawv tau ua haujlwm ntawm tus neeg siv UX, npaj cov lus qhia thiab muab kev sab laj tus kheej.

Peb automated tag nrho cov kev khiav hauj lwm rollout uas ua tau. Txhua qhov haujlwm tau sau tseg, txawm tias qhov yooj yim tshaj plaws, thiab cov kev xeem tau ua haujlwm tas li. Lawv tau sib cav txog txoj hauv kev zoo tshaj plaws los tua cov kev pabcuam - tshem tawm cov daemon lossis thaiv kev nkag mus rau qhov kev pabcuam nrog firewall. Peb tsim ib daim ntawv teev npe ntawm pab pawg rau txhua theem ntawm kev nthuav tawm thiab hloov kho nws tas li. Peb kos thiab hloov kho tas li ib daim ntawv qhia Gantt rau txhua qhov kev ua haujlwm tawm, nrog rau lub sijhawm.

Thiab yog li ntawd…

Txoj cai kawg, ua ntej dov tawm

... nws yog lub sij hawm los yob tawm.

Raws li lawv hais, ib qho kev kos duab tsis tuaj yeem ua tiav, tsuas yog ua tiav ua tiav ntawm nws. Koj yuav tsum ua tib zoo xav, nkag siab tias koj yuav tsis pom txhua yam, tab sis ntseeg tias koj tau ua txhua yam kev xav uas tsim nyog, muab rau txhua qhov xwm txheej, kaw tag nrho cov kab mob tseem ceeb, thiab txhua tus neeg koom ua txhua yam lawv ua tau. Qhov ntau code koj dov tawm, qhov nyuaj dua nws yog kom ntseeg tau koj tus kheej ntawm qhov no (dua li, txhua tus nkag siab tias nws tsis tuaj yeem pom txhua yam).

Peb tau txiav txim siab tias peb tau npaj txhij los nthuav tawm thaum peb ntseeg tau tias peb tau ua txhua yam ua tau los npog tag nrho cov kev pheej hmoo rau peb cov neeg siv cuam tshuam nrog kev npaj txhij txog thiab kev poob qis. Ntawd yog, txhua yam tuaj yeem ua yuam kev tshwj tsis yog:

Cuam tshuam (dawb huv rau peb, muaj nuj nqis tshaj plaws) cov neeg siv khoom vaj tse,
Kev ua haujlwm: kev siv peb cov kev pabcuam tom qab kev tshaj tawm yuav tsum zoo ib yam li ua ntej.

Rolling tawm

Ob yob, 8 tsis txhob cuam tshuam

Peb muab downtime rau tag nrho cov kev thov los ntawm cov neeg siv 7 teev. Lub sijhawm no, peb muaj ob qho kev npaj ua tiav thiab kev npaj rov qab.

Lub rollout nws tus kheej yuav siv li 3 teev.
2 teev rau kev sim.
2 teev - tseg rau qhov ua tau rov qab hloov pauv.

Daim duab Gantt tau kos rau txhua qhov kev ua, nws yuav siv sijhawm ntev npaum li cas, yuav ua li cas ua ntu zus, ua dab tsi ua ke.

Ib daim ntawm ib daim ntawv qhia Gantt, ib qho ntawm cov qauv thaum ntxov (tsis muaj kev ua haujlwm sib luag). Qhov Zoo Tshaj Plaws Synchronization Tool

Txhua tus neeg koom tes muaj lawv lub luag haujlwm hauv kev txiav txim siab, lawv ua haujlwm li cas, thiab lawv lub luag haujlwm rau dab tsi. Peb sim coj txhua theem mus rau qhov tsis siv neeg, yob nws tawm, yob rov qab, sau cov lus tawm tswv yim thiab yob tawm dua.

Keeb kwm ntawm cov xwm txheej

Yog li, 15 tus neeg tuaj ua haujlwm hnub Sunday, Plaub Hlis 29, thaum 10 teev tsaus ntuj. Ntxiv rau cov neeg koom tseem ceeb, qee tus tuaj yeem los txhawb pab pawg, uas tshwj xeeb ua tsaug rau lawv.

Nws tseem tsim nyog hais tias peb tus neeg kuaj xyuas tseem ceeb yog nyob rau hnub so. Nws yog tsis yooj yim sua kom yob tawm yam tsis muaj kev sim, peb tab tom tshawb nrhiav kev xaiv. Ib tug npoj yaig pom zoo sim peb los ntawm kev so, uas nws tau txais txiaj ntsig zoo los ntawm tag nrho pab neeg.

00:00 ib. Nres
Peb nres cov neeg siv kev thov, dai ib qho kos npe hais tias kev ua haujlwm. Kev saib xyuas qw, tab sis txhua yam yog qhov qub. Peb xyuas tias tsis muaj dab tsi poob dua li qhov yuav tsum tau poob. Thiab peb pib ua haujlwm ntawm kev tsiv teb tsaws chaw.

Txhua leej txhua tus muaj daim phiaj xwm luam tawm los ntawm taw tes, txhua tus paub tias leej twg ua dab tsi thiab lub sijhawm twg. Tom qab txhua qhov kev nqis tes ua, peb xyuas lub sijhawm kom ntseeg tau tias peb tsis tshaj lawv, thiab txhua yam mus raws li kev npaj. Cov neeg uas tsis koom nrog kev tshaj tawm ncaj qha ntawm theem tam sim no tab tom npaj los ntawm kev tshaj tawm cov khoom ua si hauv online (Xonotic, hom 3 quacks) kom tsis txhob cuam tshuam rau lawv cov npoj yaig. 🙂

02:00 Nws. Rolled tawm
Ib qho kev xav tsis thoob - peb ua tiav qhov kev nthuav tawm ib teev ua ntej, vim yog kev ua kom zoo ntawm peb cov ntaub ntawv thiab cov ntawv tsiv teb tsaws chaw. Tus general quaj, "dov tawm!" Tag nrho cov haujlwm tshiab yog nyob rau hauv ntau lawm, tab sis kom deb li deb peb tsuas pom lawv nyob rau hauv lub interface. Txhua leej txhua tus mus rau hauv kev sim hom, txheeb lawv mus rau hauv pab pawg, thiab pib pom dab tsi tshwm sim thaum kawg.

Nws tsis tshwm sim zoo heev, peb paub qhov no tom qab 10 feeb, thaum tsis muaj dab tsi txuas lossis ua haujlwm hauv pab pawg cov neeg ua haujlwm. Quick sync, peb hais peb cov teeb meem, teeb qhov tseem ceeb, tawg rau hauv pab pawg thiab mus rau hauv kev debugging.

02:30 Nws. Ob qhov teeb meem loj vs plaub qhov muag
Peb pom ob qhov teeb meem loj. Peb pom tau hais tias cov neeg siv khoom yuav tsis pom qee qhov kev pabcuam txuas nrog, thiab cov teeb meem yuav tshwm sim nrog tus khub tus account. Ob leeg yog vim tsis zoo tag nrho cov ntawv tsiv teb tsaws chaw rau qee qhov rooj plaub ntug. Peb yuav tsum kho tam sim no.

Peb sau cov lus nug uas sau qhov no, nrog tsawg kawg 4 qhov muag. Peb sim lawv thaum ua ntej tsim khoom kom paub tseeb tias lawv ua haujlwm thiab tsis ua txhaum dab tsi. Koj tuaj yeem yob rau. Nyob rau tib lub sijhawm, peb khiav peb qhov kev sib koom ua ke tsis tu ncua, uas qhia ob peb yam teeb meem ntxiv. Lawv tag nrho cov me me, tab sis lawv kuj yuav tsum tau kho.

03:00 Nws. -2 teeb meem +2 teeb meem
Ob qhov teeb meem loj dhau los tau raug kho, thiab yuav luag tag nrho cov me thiab. Txhua tus neeg uas tsis nyob hauv kev kho yog nquag ua haujlwm hauv lawv cov nyiaj thiab tshaj tawm qhov lawv pom. Peb muab qhov tseem ceeb, faib cov pab pawg, thiab tso cov khoom tsis tseem ceeb rau sawv ntxov.

Peb khiav qhov kev sim dua, lawv pom ob qhov teeb meem loj tshiab. Tsis yog txhua txoj cai kev pabcuam tuaj txog raug, yog li qee cov neeg siv thov tsis dhau kev tso cai. Ntxiv rau qhov teeb meem tshiab nrog tus khub tus account. Wb maj mus saib.

03:20 ib. Kev kub ceev sync
Ib qhov teeb meem tshiab kho. Rau qhov thib ob, peb tab tom npaj ib qho xwm txheej ceev. Peb nkag siab tias muaj dab tsi tshwm sim: yav dhau los kho ib qho teeb meem, tab sis tsim lwm qhov. Peb so kom paub seb yuav ua li cas kom raug thiab tsis muaj qhov tshwm sim.

03:30 Nws. Rau qhov muag
Peb nkag siab tias lub xeev kawg ntawm lub hauv paus yuav tsum yog li cas kom txhua yam mus zoo rau txhua tus neeg koom tes. Peb sau daim ntawv thov nrog 6 lub qhov muag, yob nws tawm hauv kev tsim khoom ua ntej, sim nws, yob nws tawm rau kev tsim khoom.

04:00 Nws. Txhua yam ua haujlwm
Tag nrho cov kev xeem dhau, tsis muaj teeb meem tseem ceeb pom. Qee lub sij hawm, ib yam dab tsi hauv pab pawg tsis ua haujlwm rau ib tus neeg, peb teb tam sim ntawd. Feem ntau lub tswb yog cuav. Tab sis qee zaum qee yam tsis tuaj txog, lossis nplooj ntawv cais tsis ua haujlwm. Peb zaum, kho, kho, kho. Ib pab neeg sib cais tab tom pib ua qhov tseem ceeb kawg - kev them nqi.

04:30 Nws. Point of tsis rov
Lub ntsiab lus ntawm kev tsis rov qab los yog los txog, uas yog lub sijhawm thaum, yog tias peb pib rov qab, peb yuav tsis ua raws li lub sijhawm poob rau peb. Muaj teeb meem nrog kev them nqi, uas paub thiab sau txhua yam, tab sis tawv ncauj tsis kam sau nyiaj los ntawm cov neeg siv khoom. Muaj ntau yam kab mob ntawm tus kheej nplooj ntawv, kev ua, thiab xwm txheej. Lub luag haujlwm tseem ceeb ua haujlwm, txhua qhov kev xeem dhau los ua tiav. Peb txiav txim siab tias qhov rollout tau tshwm sim, peb yuav tsis thim rov qab.

06:00 Nws. Qhib rau txhua tus hauv UI
Cov kab kho. Qee qhov tsis txaus siab rau cov neeg siv raug tso tseg rau tom qab. Peb qhib lub interface rau txhua tus. Peb txuas ntxiv ua haujlwm ntawm kev them nqi, tos cov neeg siv lus tawm tswv yim thiab saib xyuas cov txiaj ntsig.

07:00 Nws. Teeb meem nrog API load
Nws pom tseeb tias peb me ntsis misplanned lub load ntawm peb API thiab sim no load, uas tsis tuaj yeem txheeb xyuas qhov teeb meem. Raws li qhov tshwm sim, ≈5% ntawm kev thov ua tsis tiav. Wb sib txhawb thiab nrhiav qhov laj thawj.

Kev them nqi yog tawv ncauj thiab tsis xav ua haujlwm ib yam nkaus. Peb txiav txim siab ncua nws mus txog thaum tom qab txhawm rau txhawm rau ua tiav cov kev hloov pauv hauv lub siab. Ntawd yog, tag nrho cov peev txheej tau sau rau hauv nws, tab sis sau tawm los ntawm cov neeg siv khoom tsis dhau mus. Tau kawg, qhov no yog ib qho teeb meem, tab sis piv rau qhov kev nthuav tawm dav dav nws zoo li tsis tseem ceeb.

08:00 Nws. Kho API
Peb dov tawm ib qho kev txhim kho rau lub load, qhov ua tsis tiav tau ploj mus. Peb pib mus tsev.

10:00 Nws. Tag nrho
Txhua yam yog kho. Nws nyob ntsiag to hauv kev saib xyuas thiab ntawm cov neeg siv khoom qhov chaw, pab neeg maj mam mus pw. Cov nqi them tseem tshuav, peb yuav rov ua dua tag kis.

Tom qab ntawd thaum nruab hnub muaj kev tshaj tawm uas kho cov cav, cov ntawv ceeb toom, rov qab cov lej thiab kev hloov kho rau qee qhov ntawm peb cov neeg siv khoom.

Yog li, qhov kev tshaj tawm tau ua tiav! Nws tuaj yeem ua tau zoo dua, tab sis peb tau txiav txim siab txog qhov tsis txaus rau peb kom ua tiav qhov zoo.

Tag nrho

Lub sijhawm 2 lub hlis ntawm kev npaj ua haujlwm rau kev tshaj tawm, 43 txoj haujlwm tau ua tiav, kav ntev li ob peb teev mus rau ob peb hnub.

Thaum lub sij hawm rollout:

tshiab thiab hloov dab - 5 daim, hloov 2 monoliths;
kev hloov pauv hauv cov ntaub ntawv - tag nrho 6 ntawm peb cov ntaub ntawv nrog cov neeg siv cov ntaub ntawv tau cuam tshuam, rub tawm tau los ntawm peb lub qub databases rau ib qho tshiab;
tag nrho redesigned frontend;
tus lej ntawm cov lej rub tawm - 33 txhiab kab ntawm cov lej tshiab, ≈ 3 txhiab kab ntawm cov lej hauv kev sim, ≈ 5 txhiab kab ntawm txoj cai tsiv teb tsaws chaw;
tag nrho cov ntaub ntawv tsis zoo, tsis yog ib tus neeg siv khoom lub tshuab virtual raug puas tsuaj. 🙂

Cov kev coj ua zoo rau kev nthuav tawm zoo

Lawv coj peb mus rau qhov teeb meem nyuaj no. Tab sis, feem ntau hais lus, nws yog ib qho tseem ceeb kom ua raws li lawv thaum lub sij hawm rollout. Tab sis qhov nyuaj tshaj qhov rollout, lub luag hauj lwm ntau dua lawv ua.

Thawj qhov koj yuav tsum tau ua yog nkag siab tias qhov kev nthuav tawm tuaj yeem ua li cas lossis yuav cuam tshuam rau cov neeg siv. Puas yuav muaj kev poob qis? Yog tias muaj, lub sijhawm poob yog dab tsi? Qhov no yuav cuam tshuam li cas rau cov neeg siv? Dab tsi yog qhov ua tau zoo tshaj plaws thiab qhov xwm txheej phem tshaj plaws? Thiab npog cov kev pheej hmoo.
Npaj txhua yam. Hauv txhua theem, koj yuav tsum nkag siab txhua yam ntawm kev nthuav tawm:
- code xa khoom;
- code rov qab;
- lub sijhawm ntawm txhua qhov haujlwm;
- cuam tshuam kev ua haujlwm.
Ua si los ntawm cov xwm txheej kom txog rau thaum txhua theem ntawm kev nthuav tawm, nrog rau cov kev pheej hmoo ntawm txhua tus ntawm lawv, ua kom pom tseeb. Yog tias koj muaj lus tsis txaus siab, koj tuaj yeem so thiab tshuaj xyuas cov lus nug ntawm theem sib cais.
Txhua theem tuaj yeem thiab yuav tsum tau txhim kho yog tias nws pab peb cov neeg siv. Piv txwv li, nws yuav txo qis qis lossis tshem tawm qee qhov kev pheej hmoo.
Kev xeem Rollback yog qhov tseem ceeb tshaj qhov kev xeem xa cov lej. Nws yog ib qho tsim nyog yuav tau xyuas tias vim li cas ntawm qhov rollback lub kaw lus yuav rov qab mus rau nws lub xeev qub, thiab paub meej tias qhov no nrog kev sim.
Txhua yam uas tuaj yeem ua tau automated yuav tsum tau automated. Txhua yam uas tsis tuaj yeem siv tau yuav tsum tau sau ua ntej ntawm daim ntawv khib nyiab.
Sau cov txheej txheem ua tiav. Dab tsi ua haujlwm yuav tsum muaj thiab thaum twg? Yog tias qhov no tsis tshwm sim, khiav ib txoj kev npaj rov qab.
Thiab qhov tseem ceeb tshaj plaws - cov neeg. Txhua tus neeg yuav tsum paub tias lawv ua dab tsi, vim li cas thiab dab tsi nyob ntawm lawv cov kev ua hauv cov txheej txheem rollout.

Thiab hauv ib kab lus, nrog kev npaj zoo thiab kev piav qhia koj tuaj yeem yob tawm txhua yam koj xav tau yam tsis muaj kev cuam tshuam rau kev muag khoom. Txawm tias ib yam dab tsi uas yuav cuam tshuam tag nrho koj cov kev pabcuam hauv kev tsim khoom.

Tau qhov twg los: www.hab.com

Ib zaj dab neeg nthuav tawm uas cuam tshuam txhua yam