TL;DR: Plaub xyoos dhau los kuv tawm hauv Google nrog ib lub tswv yim rau cov cuab yeej saib xyuas neeg rau zaub mov tshiab. Lub tswv yim yog los ua ke feem ntau cais cov haujlwm rau hauv ib qho kev pabcuam thiab kev txheeb xyuas log, kev sau cov metrics, thiab dashboards. Ib lub hauv paus ntsiab lus yog qhov kev pabcuam yuav tsum yog tiag tiag ceev, muab devops nrog ib qho yooj yim, sib tham sib, kev lom zem. Qhov no yuav tsum tau ua ntau-gigabyte cov ntaub ntawv teev nyob rau hauv feem ntawm ib tug thib ob thaum nyob rau hauv pob nyiaj siv. Cov cuab yeej tswj cov cav uas twb muaj lawm feem ntau qeeb thiab clunky, yog li peb tau ntsib nrog qhov kev sib tw zoo: smartly tsim cov cuab yeej los muab cov neeg siv kev paub tshiab.
Kab lus no piav qhia txog yuav ua li cas peb ntawm Scalyr daws qhov teeb meem no los ntawm kev siv cov txheej txheem hauv tsev kawm qub, kev siv dag zog yuam kev, tshem tawm cov txheej txheem tsis tsim nyog thiab zam cov ntaub ntawv nyuaj. Koj tuaj yeem siv cov kev qhia no rau koj tus kheej cov teeb meem engineering.
Old School Power
Kev txheeb xyuas lub cav feem ntau pib nrog kev tshawb nrhiav: nrhiav txhua cov lus uas phim ib qho qauv. Hauv Scalyr, cov no yog kaum lossis pua pua ntawm gigabytes ntawm cov cav los ntawm ntau lub servers. Cov txheej txheem niaj hnub no, raws li txoj cai, koom nrog kev tsim kho ntawm qee cov ntaub ntawv cov qauv tsim kom zoo rau kev tshawb nrhiav. Kuv yeej pom qhov no hauv Google, qhov twg lawv zoo nkauj heev ntawm qhov no. Tab sis peb tau txiav txim siab rau ntau txoj hauv kev: linear scanning ntawm cav. Thiab nws ua haujlwm - peb muab qhov kev tshawb nrhiav tau uas yog qhov kev txiav txim siab nrawm dua li peb cov neeg sib tw (saib cov animation kawg).
Qhov kev pom tseem ceeb yog tias cov txheej txheem niaj hnub no yog qhov ceev heev ntawm kev ua haujlwm yooj yim, ncaj nraim. Qhov no yog ib qho yooj yim kom nco nyob rau hauv complex, multi-layer systems uas cia siab rau I / O ceev thiab network kev khiav hauj lwm, thiab xws li systems yog heev niaj hnub no. Yog li ntawd, peb tau tsim ib qho kev tsim uas txo qis cov txheej txheem thiab cov khib nyiab ntau dhau. Nrog ntau tus processors thiab servers nyob rau hauv tib lub sijhawm, kev tshawb nrhiav ceev nce mus txog 1 TB ib ob.
Cov ntsiab lus tseem ceeb ntawm kab lus no:
- Kev tshawb nrhiav Brute-force yog ib txoj hauv kev siv tau los daws cov teeb meem loj hauv ntiaj teb.
- Brute force yog txheej txheem tsim, tsis yog kev daws teeb meem ua haujlwm. Zoo li txhua yam txheej txheem, nws yog qhov zoo dua rau qee qhov teeb meem dua li lwm tus, thiab tuaj yeem siv tsis zoo lossis zoo.
- Brute quab yuam tshwj xeeb tshaj yog zoo rau kev ua tiav ruaj khov kev tsim khoom.
- Kev siv lub zog brute kom zoo yuav tsum tau ua kom zoo code thiab siv cov peev txheej txaus thaum lub sijhawm. Nws yog qhov tsim nyog yog tias koj cov servers nyob rau hauv qhov hnyav tsis siv neeg siv thiab cov neeg siv kev ua haujlwm tseem ceeb.
- Kev ua tau zoo nyob ntawm tus tsim ntawm tag nrho cov system, tsis yog tsuas yog lub voj algorithm.
(Cov kab lus no piav qhia txog kev tshawb nrhiav cov ntaub ntawv hauv lub cim xeeb. Feem ntau, thaum tus neeg siv ua qhov kev tshawb nrhiav cav, Scalyr servers twb tau cached nws. Cov kab lus tom ntej no yuav sib tham txog kev tshawb nrhiav cov cav tsis raug. nrog cov khoom siv suav loj loj).
Brute force method
Kev lig kev cai, cov ntaub ntawv loj yog tshawb nrhiav siv cov ntsiab lus tseem ceeb. Thaum ua ntawv thov rau server cav, qhov no txhais tau tias nrhiav txhua lo lus tshwj xeeb hauv lub cav. Rau txhua lo lus, koj yuav tsum tau ua ib daim ntawv teev tag nrho cov inclusions. Qhov no ua rau nws yooj yim mus nrhiav tau tag nrho cov lus nrog lo lus no, piv txwv li 'yuam kev', 'firefox' lossis "transaction_16851951" - tsuas yog saib hauv qhov ntsuas.
Kuv siv txoj hauv kev no hauv Google thiab nws ua haujlwm zoo. Tab sis nyob rau hauv Scalyr peb tshawb cov cav byte byte.
Vim li cas? Los ntawm qhov pom tsis meej algorithmic ntawm qhov pom, cov ntsiab lus tseem ceeb indexes muaj txiaj ntsig zoo dua li kev tshawb nrhiav brute force. Txawm li cas los xij, peb tsis muag algorithms, peb muag kev ua haujlwm. Thiab kev ua tau zoo tsis yog tsuas yog hais txog algorithms, tab sis kuj hais txog cov tshuab engineering. Peb yuav tsum xav txog txhua yam: ntim ntawm cov ntaub ntawv, hom kev tshawb fawb, cov khoom siv kho vajtse thiab cov ntsiab lus ntawm software. Peb txiav txim siab tias rau peb qhov teeb meem tshwj xeeb, ib yam dab tsi zoo li 'grep' yog qhov zoo dua li qhov ntsuas.
Index zoo heev, tab sis lawv muaj kev txwv. Ib lo lus yooj yim nrhiav tau. Tab sis kev tshawb nrhiav cov lus nrog ntau lo lus, xws li 'googlebot' thiab '404', yog qhov nyuaj dua. Kev tshawb nrhiav cov kab lus zoo li 'kev zam tsis raug cai' yuav tsum muaj qhov ntsuas ntau dua uas sau tsis yog tag nrho cov lus nrog lo lus, tab sis kuj yog qhov chaw tshwj xeeb ntawm lo lus.
Qhov nyuaj tiag tiag los thaum koj tsis nrhiav cov lus. Cia peb hais tias koj xav pom ntau npaum li cas tsheb khiav los ntawm bots. Thawj qhov kev xav yog tshawb xyuas cov cav rau lo lus 'bot'. Nov yog qhov koj yuav pom qee cov bots: Googlebot, Bingbot thiab ntau lwm tus. Tab sis ntawm no 'bot' tsis yog ib lo lus, tab sis ib feem ntawm nws. Yog tias peb tshawb nrhiav 'bot' hauv qhov ntsuas, peb yuav tsis pom cov ntawv tshaj tawm nrog lo lus 'Googlebot'. Yog tias koj tshawb xyuas txhua lo lus hauv qhov ntsuas thiab tom qab ntawd luam theej duab qhov ntsuas rau cov ntsiab lus pom, kev tshawb nrhiav yuav qeeb heev. Yog li ntawd, qee qhov kev pabcuam log tsis tso cai rau kev tshawb nrhiav ib feem lossis (qhov zoo tshaj) tso cai tshwj xeeb syntax nrog kev ua haujlwm qis dua. Peb xav zam qhov no.
Lwm qhov teeb meem yog sau ntawv. Koj puas xav nrhiav txhua qhov kev thov los ntawm 50.168.29.7? Yuav ua li cas txog debugging cav muaj [error]? Cov ntawv sau feem ntau hla cov ntawv sau.
Thaum kawg, engineers nyiam cov cuab yeej muaj zog, thiab qee zaum qhov teeb meem tsuas yog daws tau nrog kev qhia tsis tu ncua. Cov ntsiab lus tseem ceeb index tsis haum rau qhov no.
Tsis tas li ntawd, cov indices complex. Txhua cov lus yuav tsum tau ntxiv rau ntau lub npe lo lus tseem ceeb. Cov npe no yuav tsum tau khaws cia rau hauv ib qho kev tshawb nrhiav tau yooj yim txhua lub sijhawm. Cov lus nug nrog cov kab lus, cov lus tawg, lossis cov kab lus tsis tu ncua yuav tsum tau muab txhais ua ntau daim ntawv teev npe, thiab cov txiaj ntsig tau soj ntsuam thiab ua ke los tsim cov txiaj ntsig. Nyob rau hauv cov ntsiab lus ntawm ib tug loj-scale, ntau tus neeg xauj tsev, qhov complexity tsim teeb meem kev ua tau zoo uas tsis pom thaum soj ntsuam cov algorithms.
Cov ntsiab lus tseem ceeb kuj siv ntau qhov chaw, thiab kev cia khoom yog tus nqi tseem ceeb hauv kev tswj xyuas lub cav.
Ntawm qhov tod tes, txhua qhov kev tshawb nrhiav tuaj yeem siv ntau lub tshuab xam zauv. Peb cov neeg siv txaus siab rau kev tshawb nrhiav ceev rau cov lus nug tshwj xeeb, tab sis cov lus nug no tau ua tsis tshua muaj. Rau cov lus nug tshawb fawb, piv txwv li, rau lub dashboard, peb siv cov txheej txheem tshwj xeeb (peb yuav piav qhia rau lawv hauv tsab xov xwm tom ntej). Lwm qhov kev thov tsis tshua muaj txaus uas koj tsis tshua tau ua ntau dua ib zaug. Tab sis qhov no tsis tau txhais hais tias peb cov servers tsis tibneeg hu tauj coob: lawv tsis khoom nrog kev ua haujlwm ntawm kev txais, tshuaj xyuas thiab compressing cov lus tshiab, ntsuas kev ceeb toom, compressing cov ntaub ntawv qub, thiab lwm yam. Yog li, peb muaj cov khoom siv tseem ceeb ntawm cov txheej txheem uas tuaj yeem siv los ua cov lus nug.
Brute force ua haujlwm yog tias koj muaj teeb meem brute (thiab ntau lub zog)
Brute quab yuam ua haujlwm zoo tshaj plaws ntawm cov teeb meem yooj yim nrog cov voj voog me me. Feem ntau koj tuaj yeem txhim kho lub voj voog sab hauv kom khiav ntawm qhov nrawm heev. Yog hais tias cov cai yog complex, nws yog ntau nyuaj rau optimize nws.
Peb qhov kev tshawb nrhiav cov cai Ameslikas muaj lub voj voog loj loj. Peb khaws cov lus ntawm nplooj ntawv ntawm 4K; txhua nplooj ntawv muaj qee cov lus (hauv UTF-8) thiab metadata rau txhua cov lus. Metadata yog cov qauv uas encodes qhov ntev ntawm tus nqi, cov lus hauv ID, thiab lwm yam. Lub voj voog tshawb nrhiav zoo li no:
Qhov no yog ib tug yooj yim version ntawm lub tiag code. Tab sis txawm nyob ntawm no, ntau qhov chaw tso khoom, cov ntaub ntawv luam, thiab kev hu xov tooj tuaj yeem pom. JVM zoo nkauj heev ntawm optimizing muaj nuj nqi hu thiab faib cov khoom ephemeral, yog li cov cai no ua haujlwm zoo dua li peb tsim nyog. Thaum kuaj, cov neeg siv khoom siv nws zoo heev. Tab sis thaum kawg peb coj nws mus rau qib tom ntej.
(Koj tuaj yeem nug yog vim li cas peb khaws cov lus hauv hom ntawv no nrog 4K nplooj ntawv, ntawv nyeem thiab metadata, tsis yog ua haujlwm nrog cov cav ncaj qha. Muaj ntau yam laj thawj, uas ua rau lub fact tias nyob rau hauv lub Scalyr cav yog zoo li ib tug faib database tshaj ib tug. Cov ntaub ntawv tshawb fawb feem ntau ua ke nrog DBMS-style cov ntxaij lim dej hauv cov npoo tom qab kev txheeb xyuas cov ntaub ntawv. Peb tuaj yeem tshawb nrhiav ntau txhiab lub cav ib zaug, thiab cov ntaub ntawv yooj yim tsis haum rau peb cov kev hloov pauv, rov ua dua, faib cov ntaub ntawv tswj hwm).
Thaum xub thawj, nws zoo li cov cai no tsis haum rau brute force optimization. "Kev ua haujlwm tiag tiag" hauv String.indexOf() tsis txawm dominate CPU profile. Ntawd yog, optimizing txoj kev no ib leeg yuav tsis coj cov txiaj ntsig tseem ceeb.
Nws thiaj li tshwm sim tias peb khaws cov metadata thaum pib ntawm txhua nplooj ntawv, thiab cov ntawv ntawm txhua cov lus hauv UTF-8 tau ntim rau lwm qhov kawg. Ua kom zoo dua ntawm qhov no, peb rov sau lub voj mus tshawb tag nrho nplooj ntawv ib zaug:
Qhov no version ua haujlwm ncaj qha rau ntawm qhov pom raw byte[] thiab tshawb xyuas tag nrho cov lus ib zaug hla tag nrho 4K nplooj ntawv.
Qhov no yooj yim dua rau optimize rau txoj kev brute quab yuam. Lub voj tshawb nrhiav sab hauv yog hu ua ib txhij rau tag nrho 4K nplooj ntawv, tsis yog cais ntawm txhua tus ncej. Tsis muaj kev theej cov ntaub ntawv, tsis muaj kev faib khoom. Thiab ntau txoj haujlwm metadata raug hu tsuas yog thaum qhov tshwm sim zoo, thiab tsis yog ntawm txhua cov lus. Txoj kev no peb tau tshem tawm ib tuj ntawm cov nyiaj siv ua haujlwm, thiab cov khoom seem ntawm cov khoom siv yog concentrated nyob rau hauv ib qho kev tshawb nrhiav me me, uas yog qhov zoo rau kev txhim kho ntxiv.
Peb qhov kev tshawb nrhiav algorithm tiag tiag yog nyob ntawm . Nws zoo ib yam li Boyer-Moore algorithm, hla kwv yees li qhov ntev ntawm txoj hlua tshawb nrhiav ntawm txhua kauj ruam. Qhov sib txawv tseem ceeb yog tias nws tshawb xyuas ob bytes ntawm ib lub sijhawm kom txo qis qhov sib tw tsis tseeb.
Peb qhov kev siv yuav tsum tsim kom muaj 64K nrhiav lub rooj rau txhua qhov kev tshawb nrhiav, tab sis qhov ntawd tsis muaj dab tsi piv rau gigabytes ntawm cov ntaub ntawv uas peb tab tom nrhiav. Lub voj sab hauv txheej txheem ntau gigabytes ib ob ntawm ib qho tseem ceeb. Hauv kev xyaum, kev ua haujlwm ruaj khov yog nyob ib puag ncig 1,25 GB ib ob ntawm txhua tus tub ntxhais, thiab muaj chaw rau kev txhim kho. Nws muaj peev xwm tshem tawm qee qhov nyiaj siv ua haujlwm sab nraud ntawm lub voj sab hauv, thiab peb npaj yuav sim nrog lub voj sab hauv C tsis yog Java.
Peb siv zog
Peb tau tham txog tias kev tshawb nrhiav cav tuaj yeem siv tau "kwv yees", tab sis peb muaj "lub zog" ntau npaum li cas? Ntau heev.
1 cev: Thaum siv kom raug, ib qho tseem ceeb ntawm cov txheej txheem niaj hnub no muaj zog heev hauv nws txoj cai.
8 cev: Peb tab tom khiav ntawm Amazon hi1.4xlarge thiab i2.4xlarge SSD servers, txhua tus muaj 8 cores (16 xov). Raws li tau hais los saum no, cov cores no feem ntau tsis khoom nrog kev ua haujlwm tom qab. Thaum tus neeg siv ua qhov kev tshawb fawb, kev ua haujlwm tom qab raug ncua, tso tawm tag nrho 8 cores rau kev tshawb nrhiav. Kev tshawb nrhiav feem ntau ua tiav hauv qhov sib cais thib ob, tom qab ntawd cov haujlwm tom qab rov pib dua (qhov kev pabcuam cuam tshuam kom ntseeg tau tias cov lus nug tshawb nrhiav tsis cuam tshuam nrog kev ua haujlwm tseem ceeb hauv keeb kwm yav dhau).
16 cev: rau kev ntseeg siab, peb npaj cov servers rau hauv pawg tswv / qhev. Txhua tus tswv muaj ib SSD thiab ib EBS server raws li nws cov lus txib. Yog tias tus neeg rau zaub mov tseem ceeb poob, SSD server tam sim ntawd siv nws qhov chaw. Yuav luag txhua lub sijhawm, tus tswv thiab qhev ua haujlwm zoo, yog li txhua cov ntaub ntawv thaiv tuaj yeem tshawb nrhiav ntawm ob lub servers sib txawv (EBS qhev server muaj cov txheej txheem tsis muaj zog, yog li peb tsis xav txog nws). Peb faib cov hauj lwm ntawm lawv, kom peb muaj tag nrho ntawm 16 cores muaj.
Ntau cores: Nyob rau yav tom ntej, peb yuav faib cov ntaub ntawv hla cov servers hauv txoj hauv kev uas lawv txhua tus koom nrog ua txhua qhov kev thov tsis tseem ceeb. Txhua lub hauv paus yuav ua haujlwm. [Nco tseg: peb tau ua raws li txoj kev npaj thiab nce qhov kev tshawb nrhiav nrawm mus rau 1 TB / s, saib daim ntawv kawg ntawm tsab xov xwm].
Simplicity kom ntseeg tau
Lwm qhov zoo dua ntawm txoj kev brute force yog nws qhov kev ua tau zoo sib xws. Feem ntau, kev tshawb nrhiav tsis tshua nkag siab rau cov ntsiab lus ntawm qhov teeb meem thiab cov ntaub ntawv teeb tsa (Kuv twv tias yog vim li cas nws hu ua "coarse").
Cov ntsiab lus tseem ceeb qee zaum ua rau cov txiaj ntsig tau nrawm heev, thiab lwm zaus nws tsis ua. Wb hais tias koj muaj 50 GB ntawm cov cav uas lo lus 'customer_5987235982' tshwm raws nraim peb zaug. Kev tshawb nrhiav rau lo lus no suav peb qhov chaw ncaj qha los ntawm qhov ntsuas thiab yuav ua tiav tam sim ntawd. Tab sis txoj kev tshawb nrhiav tsiaj qus tuaj yeem tshawb xyuas ntau txhiab lo lus tseem ceeb thiab siv sijhawm ntev.
Ntawm qhov tod tes, brute force searches ua tau ntawm ntau dua lossis tsawg dua qhov nrawm rau ib qho lus nug. Kev tshawb nrhiav cov lus ntev yog qhov zoo dua, tab sis txawm tias kev tshawb nrhiav ib tus cwj pwm yog nrawm heev.
Qhov yooj yim ntawm txoj kev brute quab yuam txhais tau hais tias nws qhov kev ua tau zoo nyob ze rau nws qhov siab tshaj plaws theoretical. Muaj tsawg txoj kev xaiv rau kev npaj txhij txog disk overloads, xauv qhov kev sib cav, taw tes caum, thiab ntau txhiab qhov laj thawj rau kev ua tsis tiav. Kuv tsuas yog saib cov lus thov los ntawm cov neeg siv Scalyr lub lim tiam dhau los ntawm peb cov neeg siv khoom loj tshaj plaws. Muaj 14 thov. Raws nraim yim ntawm lawv tau siv ntau tshaj ib thib ob; 000% ua tiav hauv 99 milliseconds (yog tias koj tsis tau siv cov cuab yeej ntsuas ntsuas, ntseeg kuv: nws ceev ceev).
ruaj khov, txhim khu kev qha kev ua tau zoo yog qhov tseem ceeb rau kev yooj yim ntawm kev siv cov kev pabcuam. Yog tias nws lags ib ntus, cov neeg siv yuav pom tias nws tsis ntseeg tau thiab tsis kam siv nws.
Nkag mus tshawb hauv kev nqis tes ua
Ntawm no yog cov animation luv luv uas qhia tau tias Scalyr nrhiav hauv kev nqis tes ua. Peb muaj tus account demo qhov twg peb import txhua qhov xwm txheej hauv txhua qhov chaw khaws cia Github pej xeem. Hauv qhov demo no, kuv tshuaj xyuas ib lub lim tiam tus nqi ntawm cov ntaub ntawv: kwv yees li 600 MB ntawm cov ntaub ntawv nyoos.
Cov vis dis aus tau kaw nyob, tsis muaj kev npaj tshwj xeeb, ntawm kuv lub desktop (kwv yees li 5000 mais ntawm lub server). Qhov kev ua tau zoo koj yuav pom yog vim muaj ntau , nrog rau qhov ceev thiab txhim khu kev qha backend. Thaum twg muaj kev ncua tsis muaj qhov ntsuas 'loading', nws yog kuv nres kom koj tuaj yeem nyeem qhov kuv tab tom yuav nias.
Nyob rau hauv xaus
Thaum ua tiav cov ntaub ntawv ntau, nws yog ib qho tseem ceeb uas yuav tau xaiv ib qho algorithm zoo, tab sis "zoo" tsis txhais hais tias "zoo". Xav txog seb koj cov cai yuav ua haujlwm li cas hauv kev xyaum. Kev tsom xam theoretical ntawm algorithms tawm ntawm qee yam uas tuaj yeem yog qhov tseem ceeb hauv lub ntiaj teb tiag. Cov algorithms yooj yim yog yooj yim rau optimize thiab ruaj khov nyob rau hauv cov xwm txheej ntug.
Kuj xav txog cov ntsiab lus uas cov cai yuav raug tua. Hauv peb qhov xwm txheej, peb xav tau cov servers muaj zog txaus los tswj cov haujlwm tom qab. Cov neeg siv pib tshawb nrhiav tsis tshua muaj, yog li peb tuaj yeem qiv tag nrho pawg servers rau lub sijhawm luv luv uas xav tau los ua kom tiav txhua qhov kev tshawb nrhiav.
Siv txoj kev quab yuam brute, peb tau siv qhov kev tshawb nrhiav nrawm, txhim khu kev qha, hloov pauv thoob plaws ib txheej ntawm cov cav. Peb vam tias cov tswv yim no muaj txiaj ntsig zoo rau koj qhov project.
Kho kom raug: Lub npe thiab cov ntawv tau hloov pauv los ntawm "Nrhiav ntawm 20 GB ib ob" mus rau "Nrhiav ntawm 1 TB ib ob" los cuam tshuam qhov kev ua tau zoo nce ntxiv hauv ob peb xyoos dhau los. Qhov kev nce hauv qhov nrawm no feem ntau yog vim muaj kev hloov pauv ntawm hom thiab tus naj npawb ntawm EC2 servers peb tab tom tso hnub no los pab peb cov neeg siv khoom nce ntxiv. Muaj kev hloov pauv sai sai uas yuav ua rau muaj kev txhawb nqa ntxiv hauv kev ua haujlwm tau zoo, thiab peb tos tsis tau los qhia lawv.
Tau qhov twg los: www.hab.com