Muaj tshwm sim ob peb hnub dhau los Hydra lub rooj sib tham. Cov hais mav los ntawm JUG.ru Pawg tau caw cov neeg hais lus npau suav (Leslie Lamport! Cliff Click! Martin Kleppmann!) thiab mob siab rau ob hnub rau kev faib khoom thiab kev suav. Kontur yog ib qho ntawm peb lub rooj sib tham sib koom tes. Peb tham ntawm qhov chaw muag khoom, tham txog peb qhov chaw khaws cia, ua si bingo, thiab daws teeb meem.

Nov yog tsab ntawv tshaj tawm nrog kev txheeb xyuas cov haujlwm ntawm Kontur sawv ntawm tus sau ntawm lawv cov ntawv nyeem. Rau cov neeg uas tau mus rau Hydra, qhov no yog koj lub caij nyoog kom nco qab cov kev paub qab ntxiag; rau cov uas tsis muaj, qhov no yog lub caij nyoog los ncab koj lub hlwb. loj O- sau ntawv.

Muaj txawm tias cov neeg koom nrog disassembled lub flipchart rau hauv slides los sau lawv cov kev daws teeb meem. Kuv tsis kidding - lawv xa cov ntawv no rau kev tshuaj xyuas:

Tag nrho muaj peb txoj haujlwm:

• hais txog xaiv replicas los ntawm qhov hnyav rau load ntsuas
• hais txog kev txheeb xyuas cov txiaj ntsig ntawm cov lus nug mus rau hauv lub cim xeeb database
• nyob rau hauv lub xeev hloov nyob rau hauv ib tug faib system nrog ib tug nplhaib topology

Task 1. ClusterClient

Nws yog ib qho tsim nyog los tawm tswv yim algorithm rau kev xaiv K tawm ntawm N hnyav replicas ntawm ib qho kev faib tawm:

Koj pab neeg tau ua hauj lwm los tsim ib lub tsev qiv ntawv cov neeg siv khoom rau ib pawg loj ntawm N nodes. Lub tsev qiv ntawv yuav taug qab ntau yam metadata cuam tshuam nrog cov nodes (piv txwv li, lawv cov latencies, 4xx / 5xx cov lus teb, thiab lwm yam) thiab muab cov taw tes taw hnyav W1..WN rau lawv. Txhawm rau txhawb nqa qhov kev ua tiav ib txhij, lub tsev qiv ntawv yuav tsum tuaj yeem xaiv K ntawm N nodes random-ib lub sijhawm ntawm kev xaiv yuav tsum yog qhov sib npaug ntawm qhov hnyav.

Tshaj tawm ib qho algorithm los xaiv cov nodes zoo. Kwv yees nws cov kev ua lej complex siv O cim loj.

Vim li cas txhua yam yog lus Askiv?

Vim qhov no yog li cas cov neeg tuaj koom lub rooj sib tham tau tawm tsam lawv thiab vim hais tias lus Askiv yog hom lus ntawm Hydra. Cov teeb meem zoo li no:

Nqa daim ntawv thiab xaum, xav, tsis txhob maj mus qhib spoilers tam sim ntawd :)

Analysis ntawm kev daws (Video)

Pib ntawm 5:53, tsuas yog 4 feeb:

Thiab ntawm no yog li cas cov tib neeg nrog lub flipchart pitched lawv daws:

Analysis of the solution (ntawv nyeem)

Cov kev daws teeb meem hauv qab no nyob ntawm qhov chaw: suav nrog qhov hnyav ntawm tag nrho cov replicas, tsim tus lej random ntawm 0 mus rau qhov sib npaug ntawm txhua qhov hnyav, tom qab ntawd xaiv ib qho i-replica xws li cov lej ntawm qhov hnyav ntawm cov replicas ntawm 0 mus rau ( i-1) th yog tsawg dua tus lej random, thiab qhov sib npaug ntawm qhov hnyav ntawm qhov hloov pauv ntawm 0 mus rau i-th - ntau dua li nws. Txoj kev no koj tuaj yeem xaiv ib qho replica, thiab xaiv tus tom ntej, koj yuav tsum rov ua dua tag nrho cov txheej txheem yam tsis xav txog qhov xaiv replica. Nrog xws li ib tug algorithm, qhov nyuaj ntawm kev xaiv ib tug replica yog O (N), qhov nyuaj ntawm xaiv K replicas yog O (N·K) ~ O (N2).

Quadratic complexity tsis zoo, tab sis nws tuaj yeem txhim kho. Rau qhov no peb yuav tsim ceg ntoo rau sums of weights. Qhov tshwm sim yog tsob ntoo ntawm qhov tob lg N, nplooj ntawm cov nplooj uas yuav muaj qhov hnyav ntawm cov replicas, thiab cov nodes seem yuav muaj ib feem sums, mus txog rau tag nrho cov luj ntawm lub hauv paus ntawm tsob ntoo. Tom ntej no, peb tsim ib tug random tooj los ntawm 0 mus rau lub sum ntawm tag nrho cov luj, nrhiav tus i-th replicas, tshem tawm ntawm tsob ntoo thiab rov ua cov txheej txheem mus nrhiav cov seem replicas. Nrog xws li cov algorithm, qhov nyuaj ntawm kev tsim ib tsob ntoo yog O (N), qhov nyuaj ntawm kev nrhiav cov i-th replica thiab tshem tawm ntawm tsob ntoo yog O (lg N), qhov nyuaj ntawm kev xaiv K replicas yog O (N + K lg N) ~ O(N lg N).

Linear-logarithmic complexity yog nicer dua quadratic complexity, tshwj xeeb tshaj yog rau loj K.

Nws yog qhov no algorithm siv nyob rau hauv code ClusterClient cov tsev qiv ntawv los ntawm qhov project "Sab Hnub Tuaj" (Muaj tsob ntoo ua hauv O (N lg N), tab sis qhov no tsis cuam tshuam rau qhov nyuaj kawg ntawm algorithm.)

Teeb meem 2. Zebra

Nws yog ib qho tsim nyog los tawm tswv yim algorithm rau kev txheeb xyuas cov ntaub ntawv hauv kev nco zoo los ntawm qhov chaw uas tsis yog indexed:

Koj pab neeg tau ua haujlwm los tsim cov ntaub ntawv khaws cia hauv lub cim xeeb. Ib qho kev ua haujlwm ntau yuav yog xaiv cov ntaub ntawv saum toj kawg nkaus N tau txheeb los ntawm qhov tsis txaus ntseeg (tsis suav nrog) cov lej los ntawm kev sau ntawm M (feem ntau N < 100 << M). Ib qho kev ua haujlwm tsawg me ntsis yuav yog xaiv sab saum toj N tom qab hla S cov ntaub ntawv saum toj kawg nkaus (S ~ N).

Tshaj tawm ib qho algorithm los ua cov lus nug zoo. Kwv yees nws cov kev ua lej tsis yooj yim siv O cim loj hauv qhov nruab nrab thiab qhov xwm txheej phem tshaj plaws.

Analysis ntawm kev daws (Video)

Pib ntawm 34:50, tsuas yog 6 feeb:

Analysis of the solution (ntawv nyeem)

Kev daws ntawm qhov chaw: txheeb tag nrho cov ntaub ntawv (piv txwv li, siv nrawm), tom qab ntawd nqa cov ntaub ntawv N + S. Hauv qhov no, qhov nyuaj ntawm kev txheeb xyuas qhov nruab nrab yog O (M lg M), qhov phem tshaj nws yog O (M2).

Pom tseeb, txheeb xyuas tag nrho cov ntaub ntawv M thiab tom qab ntawd tsuas yog ib feem me me ntawm lawv tsis muaj txiaj ntsig. Yuav kom tsis txhob txheeb tag nrho cov ntaub ntawv, ib qho algorithm yuav ua xaiv ceev, uas yuav xaiv N + S cov ntaub ntawv xav tau (lawv tuaj yeem txheeb los ntawm ib qho algorithm). Hauv qhov no, qhov nyuaj yuav raug txo mus rau O (M) qhov nruab nrab, thiab qhov phem tshaj plaws yuav nyob twj ywm tib yam.

Txawm li cas los xij, koj tuaj yeem ua nws ntau dua - siv cov algorithm binary heap streaming. Nyob rau hauv rooj plaub no, thawj N + S cov ntaub ntawv raug ntxiv rau min- lossis max-heap (nyob ntawm qhov kev taw qhia sorting), thiab tom qab ntawd txhua daim ntawv txuas ntxiv yog muab piv nrog cov hauv paus ntoo, uas muaj cov ntaub ntawv yam tsawg kawg nkaus lossis siab tshaj plaws tam sim no. , thiab, yog tias tsim nyog, ntxiv rau tsob ntoo. Hauv qhov no, qhov nyuaj ntawm qhov phem tshaj plaws, thaum koj yuav tsum rov tsim kho tsob ntoo tas li, yog O (M lg M), qhov nyuaj ntawm qhov nruab nrab yog O (M), zoo li thaum siv quickselect.

Txawm li cas los xij, heap streaming hloov tawm kom ua tau zoo dua vim qhov tseeb tias hauv kev xyaum feem ntau ntawm cov ntaub ntawv tuaj yeem muab pov tseg yam tsis tau rov tsim kho lub heap, tom qab ib qho kev sib piv nrog nws cov hauv paus ntsiab lus. Qhov kev txheeb xyuas no yog siv rau hauv Zebra hauv-nco cov ntaub ntawv database, tsim thiab siv hauv Kontur.

Teeb meem 3. Lub xeev swaps

Nws yog ib qho tsim nyog los tawm tswv yim zoo tshaj plaws algorithm rau kev hloov cov xeev:

Koj pab neeg tau ua haujlwm los tsim kom muaj lub xeev zoo nkauj pauv hloov pauv rau pawg faib ntawm N nodes. Lub xeev ntawm i-th yuav tsum raug xa mus rau (i + 1)-th node, N-th node lub xeev yuav tsum raug xa mus rau thawj qhov. Qhov kev txhawb nqa nkaus xwb yog lub xeev sib pauv thaum ob lub nodes pauv lawv lub xeev atomically. Nws paub tias kev sib pauv hauv xeev siv sijhawm M milliseconds. Txhua lub node tuaj yeem koom nrog hauv ib lub xeev sib pauv ntawm ib lub sijhawm.

Nws siv sijhawm ntev npaum li cas txhawm rau hloov cov xeev ntawm tag nrho cov nodes hauv pawg?

Analysis of the solution (ntawv nyeem)

Kev daws ntawm qhov chaw: pauv cov xeev ntawm thawj thiab thib ob, tom qab ntawd thawj thiab thib peb, tom qab ntawd thawj thiab plaub, thiab lwm yam. Tom qab txhua qhov kev sib pauv, lub xeev ntawm ib lub caij yuav nyob rau hauv txoj haujlwm xav tau. Koj yuav tau ua O(N) permutations thiab siv O(N·M) sij hawm.

Lub sij hawm linear ntev, yog li koj tuaj yeem pauv cov xeev ntawm cov ntsiab lus hauv cov khub: thawj zaug nrog thib ob, thib peb nrog plaub, thiab lwm yam. Tom qab txhua qhov kev sib pauv ntawm lub xeev, txhua lub ntsiab lus thib ob yuav nyob hauv txoj haujlwm xav tau. Koj yuav tau ua O (lg N) permutations thiab siv O (M lg N) sij hawm.

Txawm li cas los xij, koj tuaj yeem ua qhov kev hloov pauv ntau dua - tsis yog hauv linear, tab sis nyob rau lub sijhawm tas li. Ua li no, thawj kauj ruam koj yuav tsum tau pauv lub xeev ntawm thawj lub caij nrog lub xeem, lub thib ob nrog lub penultimate, thiab hais txog. Lub xeev ntawm lub xeem lub caij yuav nyob rau hauv qhov xav tau txoj hauj lwm. Thiab tam sim no peb yuav tsum tau pauv lub xeev ntawm lub caij thib ob nrog qhov kawg, qhov thib peb nrog rau qhov kawg, thiab lwm yam. Tom qab qhov kev sib pauv no, cov xeev ntawm txhua lub ntsiab lus yuav nyob hauv txoj haujlwm xav tau. Nyob rau hauv tag nrho, O (2M) ~ O (1) permutations yuav ua.

Cov kev daws teeb meem zoo li no yuav tsis xav tsis thoob rau tus lej ntawm txhua tus, uas tseem nco ntsoov tias kev sib hloov yog ib qho kev sib xyaw ntawm ob lub axial symmetries. Los ntawm txoj kev, nws yog qhov tsis tseem ceeb ntawm kev hloov pauv tsis yog los ntawm ib qho, tab sis los ntawm K <N txoj haujlwm. (Sau hauv cov lus raws nraim li cas.)

Koj puas nyiam cov puzzles? Koj puas paub lwm txoj kev daws teeb meem? Qhia tawm hauv cov lus.

Nov yog qee qhov kawg siv tau txuas:

• paub ntau ntxiv txog kev txhim kho infrastructure hauv Kontur
• saib cov ntaub ntawv sab hauv flyer nrog cov ntawv ceeb toom hais txog kev faib tshuab
• saib cov series ntawm cov lus qhuab qhia "Cov algorithms txawv txawv rau tib neeg»
• subscribe rau peb Telegram channel

Tau qhov twg los: www.hab.com