Zabbix yog ib qho kev saib xyuas. Ib yam li lwm qhov system, nws ntsib peb qhov teeb meem tseem ceeb ntawm txhua qhov kev saib xyuas: sau thiab ua cov ntaub ntawv, khaws keeb kwm, thiab ntxuav nws.
Cov theem ntawm kev txais, ua thiab sau cov ntaub ntawv yuav siv sij hawm. Tsis ntau, tab sis rau lub kaw lus loj qhov no tuaj yeem ua rau qeeb qeeb. Qhov teeb meem khaws cia yog qhov teeb meem nkag mus rau cov ntaub ntawv. Lawv yog siv rau kev tshaj tawm, kev kuaj xyuas thiab ua rau muaj kev cuam tshuam. Latencies hauv kev nkag tau cov ntaub ntawv tseem cuam tshuam kev ua haujlwm. Thaum databases loj tuaj, cov ntaub ntawv tsis cuam tshuam yuav tsum tau muab tshem tawm. Kev tshem tawm yog ib qho kev ua haujlwm nyuaj uas tseem noj tau qee qhov peev txheej.
Cov teeb meem ntawm kev ncua sij hawm thaum sau thiab khaws cia hauv Zabbix yog daws los ntawm caching: ntau hom caches, caching hauv database. Txhawm rau daws qhov teeb meem thib peb, caching tsis haum, yog li Zabbix siv TimescaleDB. Nws yuav qhia koj txog nws Andrey Gushchin - technical support engineer Zabbix SIA. Andrey tau txhawb nqa Zabbix rau ntau tshaj 6 xyoo thiab muaj kev paub ncaj qha nrog kev ua haujlwm.
TimescaleDB ua haujlwm li cas, nws tuaj yeem ua haujlwm zoo li cas piv rau PostgreSQL li niaj zaus? Zabbix ua lub luag haujlwm dab tsi rau TimescaleDB database? Yuav pib li cas los ntawm kos thiab yuav ua li cas los ntawm PostgreSQL thiab qhov kev teeb tsa twg muaj kev ua tau zoo dua? Txog tag nrho cov no hauv qab txiav.
Productivity Challenges
Txhua qhov kev soj ntsuam xyuas ntsib cov teeb meem kev ua haujlwm tshwj xeeb. Kuv yuav tham txog peb ntawm lawv: kev sau cov ntaub ntawv thiab kev ua haujlwm, kev khaws cia, thiab keeb kwm clearing.
Cov ntaub ntawv ceev ceev khaws thiab ua tiav. Kev saib xyuas zoo yuav tsum tau txais tag nrho cov ntaub ntawv sai sai thiab ua raws li cov lus qhia - raws li nws cov qauv. Tom qab ua tiav, lub kaw lus yuav tsum ceev nrooj khaws cov ntaub ntawv no hauv cov ntaub ntawv rau kev siv tom qab.
Clearing keeb kwm. Qee zaum muaj ib hnub thaum koj tsis tas yuav khaws cov ntsuas. Vim li cas koj thiaj li xav tau cov ntaub ntawv uas tau sau tseg 5 xyoo dhau los, ib hlis lossis ob lub hlis: qee cov nodes tau raug tshem tawm, qee tus tswv lossis kev ntsuas tsis xav tau ntxiv lawm vim tias lawv dhau los thiab tsis tau sau ntxiv lawm. Kev saib xyuas zoo yuav tsum khaws cov ntaub ntawv keeb kwm thiab tshem tawm ib ntus kom cov ntaub ntawv tsis loj hlob.
Kev tu cov ntaub ntawv stale yog qhov teeb meem tseem ceeb uas cuam tshuam rau kev ua haujlwm ntawm database.
Caching hauv Zabbix
Hauv Zabbix, thawj thiab thib ob hu tau daws siv caching. RAM yog siv los sau thiab ua cov ntaub ntawv. Rau kev khaws cia - keeb kwm hauv kev cuam tshuam, graphs thiab suav cov ntsiab lus. Nyob rau sab database muaj qee qhov caching rau kev xaiv yooj yim, piv txwv li, graphs.
Caching ntawm sab ntawm Zabbix server nws tus kheej yog:
ConfigurationCache;
ValueCache;
Keeb kwmCache;
TrendsCache.
Cia peb xav txog lawv hauv kev nthuav dav ntxiv.
ConfigurationCache
Qhov no yog lub hauv paus tseem ceeb uas peb khaws cov ntsuas ntsuas, cov tswv tsev, cov khoom siv cov ntaub ntawv, ua rau - txhua yam peb xav tau rau PreProcessing thiab rau kev sau cov ntaub ntawv.
Tag nrho cov no yog khaws cia hauv ConfigurationCache kom tsis txhob tsim cov lus nug tsis tsim nyog hauv cov ntaub ntawv. Tom qab lub server pib, peb hloov kho qhov cache no, tsim thiab hloov kho cov teeb tsa ib ntus.
Kev sau cov ntaub ntawv
Daim duab loj heev, tab sis qhov tseem ceeb hauv nws yog cov neeg sau. Cov no yog ntau yam "pollers" - cov txheej txheem sib dhos. Lawv yog lub luag haujlwm rau ntau hom kev sib dhos: lawv sau cov ntaub ntawv ntawm SNMP, IPMI, thiab xa tag nrho mus rau PreProcessing.
Cov neeg sau tau piav qhia hauv txiv kab ntxwv.
Zabbix tau suav cov khoom sib sau ua ke uas xav tau los sau cov tshev. Yog tias peb muaj lawv, peb nqa cov ntaub ntawv rau lawv ncaj qha los ntawm ValueCache.
PreProcessing HistoryCache
Txhua tus neeg sau khoom siv ConfigurationCache kom tau txais haujlwm. Tom qab ntawd lawv hloov mus rau PreProcessing.
PreProcessing siv ConfigurationCache kom tau txais cov kauj ruam ua ntej. Nws ua cov ntaub ntawv no ntau yam.
Tom qab ua cov ntaub ntawv siv PreProcessing, peb khaws nws hauv HistoryCache rau kev ua haujlwm. Qhov no xaus kev sau cov ntaub ntawv thiab peb txav mus rau cov txheej txheem tseem ceeb hauv Zabbix - keeb kwm syncerVim nws yog ib tug monolithic architecture.
Nco tseg: PreProcessing yog ib qho nyuaj heev. Nrog v 4.2 nws tau raug tsiv mus rau npe. Yog tias koj muaj Zabbix loj heev nrog ntau cov ntaub ntawv thiab kev sau ntau zaus, qhov no ua rau kev ua haujlwm yooj yim dua.
Keeb kwm syncer tsim ib qho kev tshwm sim, nce ntxiv los tsim kev ceeb toom yog tias xav tau los ntawm kev teeb tsa, thiab cov ntaub ntawv. Yog tias muaj qhov tshwm sim rau kev ua haujlwm tom ntej, ces nws khaws cov nqi no hauv ValueCache kom tsis txhob nkag mus rau cov lus keeb kwm. Qhov no yog li cas ValueCache tau sau nrog cov ntaub ntawv uas tsim nyog los laij cov txiaj ntsig thiab suav cov ntsiab lus.
Keeb kwm syncer sau tag nrho cov ntaub ntawv mus rau lub database, thiab nws sau rau disk. Cov txheej txheem ua tiav ntawm no.
Caching nyob rau hauv lub database
Nyob rau sab database muaj ntau yam caches thaum koj xav saib cov duab lossis qhia txog cov xwm txheej:
Innodb_buffer_pool ntawm MySQL sab;
shared_buffers nyob rau sab PostgreSQL;
effective_cache_size nyob rau sab Oracle;
shared_pool ntawm DB2 sab.
Muaj ntau ntau lwm yam caches, tab sis cov no yog cov tseem ceeb rau tag nrho cov databases. Lawv tso cai rau koj khaws cov ntaub ntawv hauv RAM uas feem ntau xav tau rau cov lus nug. Lawv muaj lawv tus kheej technologies rau qhov no.
Database kev ua tau zoo yog qhov tseem ceeb
Lub Zabbix server tas li khaws cov ntaub ntawv thiab sau nws. Thaum rov pib dua, nws kuj nyeem los ntawm keeb kwm kom sau ValueCache. Siv cov ntawv sau thiab cov ntawv ceeb toom Zabbix API, uas yog tsim los ntawm Web interface. Zabbix API nkag mus rau hauv cov ntaub ntawv thiab khaws cov ntaub ntawv tsim nyog rau daim duab, cov ntawv ceeb toom, cov npe tshwm sim thiab cov teeb meem tshiab.
Rau visualization - ua grafana. Qhov no yog ib qho kev daws teeb meem nrov ntawm peb cov neeg siv. Nws tuaj yeem xa ncaj qha thov los ntawm Zabbix API thiab mus rau cov ntaub ntawv, thiab tsim kom muaj kev sib tw rau kev txais cov ntaub ntawv. Yog li ntawd, kev kho kom zoo dua thiab zoo dua ntawm cov ntaub ntawv yuav tsum tau ua kom haum rau kev xa khoom sai ntawm cov txiaj ntsig thiab kev sim.
neeg tu vajtse
Qhov kev sib tw thib peb hauv Zabbix yog keeb kwm tshem tawm siv Tus Saib Xyuas Tsev. Nws ua raws li tag nrho cov chaw - cov ntaub ntawv hais qhia tias ntev npaum li cas los khaws cov kev hloov pauv (kev hloov pauv) hauv hnub.
Peb xam TrendsCache ntawm ya. Thaum cov ntaub ntawv tuaj txog, peb sib sau ua ke rau ib teev thiab sau nws rau hauv cov ntxhuav rau cov kev hloov pauv hloov pauv.
Tus neeg saib xyuas tsev pib thiab tshem tawm cov ntaub ntawv los ntawm cov ntaub ntawv siv cov "xaiv". Qhov no tsis yog ib txwm ua tau zoo, raws li tuaj yeem pom los ntawm cov duab ua haujlwm ntawm cov txheej txheem sab hauv.
Daim duab liab qhia tau hais tias Keeb Kwm syncer tsis tu ncua. Daim duab txiv kab ntxwv nyob rau sab saum toj yog Housekeeper, uas ua haujlwm tas li. Nws tos rau lub database rho tawm tag nrho cov kab uas nws tau teev tseg.
Thaum twg koj yuav tsum lov tes taw Tus Saib Xyuas Tsev? Piv txwv li, muaj "Ib Yam Khoom" thiab koj yuav tsum tshem tawm 5 txhiab kab kawg hauv ib lub sijhawm. Tau kawg, qhov no tshwm sim los ntawm index. Tab sis feem ntau cov dataset loj heev, thiab cov ntaub ntawv tseem nyeem los ntawm disk thiab muab tso rau hauv lub cache. Qhov no ib txwm ua haujlwm kim heev rau cov ntaub ntawv thiab, nyob ntawm qhov loj ntawm cov ntaub ntawv, tuaj yeem ua rau muaj teeb meem kev ua haujlwm.
Tus neeg saib xyuas tsev yooj yim rau lov tes taw. Hauv Web interface muaj qhov teeb tsa hauv "Administration general" rau Tus Neeg Saib Xyuas Tsev. Peb lov tes taw Internal Housekeeping rau internal trend keeb kwm thiab nws tsis tswj nws lawm.
Tus neeg saib xyuas tsev raug kaw, cov duab qhia tau zoo - cov teeb meem dab tsi tuaj yeem muaj nyob rau hauv rooj plaub no thiab dab tsi tuaj yeem pab daws qhov kev sib tw thib peb?
Cov txiaj ntsig tuaj yeem hloov pauv yog tias "kev teeb tsa" loj heev. Yog tias ib qho me me "setup" yog mus txog 5 nvps (tus nqi tshiab ib ob), qhov nruab nrab yog los ntawm 000 txog 5, ces qhov loj yog siab dua 000 nvps. Cov no yog cov loj thiab loj heev installation uas yuav tsum tau ceev faj configuration ntawm database.
Hauv kev teeb tsa loj heev, lub sijhawm ntawm ib hnub yuav tsis zoo. Kuv tau pom MySQL partitions ntawm 40 GB lossis ntau dua ib hnub. Qhov no yog cov ntaub ntawv loj heev uas tuaj yeem ua rau muaj teeb meem thiab yuav tsum tau txo qis.
Partitioning muab dab tsi?
Kev faib cov rooj. Feem ntau cov no yog cov ntaub ntawv cais ntawm disk. Lub phiaj xwm nug xaiv ib qho kev faib ua kom zoo dua. Feem ntau muab faib yog siv los ntawm ntau yam - qhov no kuj muaj tseeb rau Zabbix. Peb siv "timestamp" nyob rau ntawd - lub sij hawm txij thaum pib ntawm lub era. Cov no yog cov lej qub rau peb. Koj teem caij pib thiab xaus ntawm hnub - qhov no yog kev faib.
TimescaleDB muaj lub tswv yim hypertable (hypertable) uas koj tsim. Nws muaj tej kab - muab faib. Cov chunks tau txais kev tswj hwm cov khoom tawg uas tsis cuam tshuam rau lwm cov khoom tawg. Txhua chunks muaj nws tus kheej lub sijhawm.
TimescaleDB vs PostgreSQL
TimescaleDB ua haujlwm tau zoo. Extensions manufacturers thov tias lawv siv cov lus nug kom raug ntau dua ua cov txheej txheem, tshwj xeeb hauv inserts . Raws li dataset ntxig loj hlob tuaj, lub algorithm tswj kev ua haujlwm tas li.
Tom qab 200 lab kab, PostgreSQL feem ntau pib sag ho thiab poob kev ua tau zoo rau 0. TimescaleDB tso cai rau koj kom zoo ntxig "inserts" rau txhua tus nqi ntawm cov ntaub ntawv.
chaw
Txhim kho TimescaleDB yog qhov yooj yim haum rau txhua pob. IN cov ntaub ntawv txhua yam tau piav qhia meej - nws nyob ntawm cov pob khoom ntawm PostgreSQL. TimescaleDB kuj tseem tuaj yeem tsim thiab muab sau ua ke.
Kuv siv ob lub servers. Ua ntej - VMware tshuab. Nws yog me me: 20 Intel® Xeon® CPU E5-2630 v 4 @ 2.20GHz processors, 16 GB ntawm RAM thiab 200 GB SSD.
Kuv ntsia PostgreSQL 10.8 rau nws nrog Debian 10.8-1.pgdg90+1 OS thiab xfs cov ntaub ntawv kaw lus. Kuv teeb tsa txhua yam tsawg kawg los siv cov ntaub ntawv tshwj xeeb no, rho tawm dab tsi Zabbix nws tus kheej yuav siv.
Nyob rau tib lub tshuab muaj Zabbix server, PostgreSQL thiab load tus neeg sawv cev. Kuv muaj 50 tus neeg ua haujlwm nquag uas tau siv LoadableModulekom sai heev tsim cov txiaj ntsig sib txawv: cov lej, cov hlua. Kuv sau cov ntaub ntawv nrog ntau cov ntaub ntawv.
Thaum xub thawj lub configuration muaj 5 yam data per host. Yuav luag txhua lub hauv paus muaj qhov ua kom zoo ib yam li kev teeb tsa tiag tiag. Qee lub sij hawm muaj ntau tshaj ib qho kev tshwm sim. Rau ib lub network node muaj 3-000 tawm.
Cov ntaub ntawv hloov tshiab ncua sij hawm - 4-7 vib nas this. Kuv tswj lub load nws tus kheej los ntawm kev siv tsis tau tsuas yog 50 tus neeg sawv cev, tab sis ntxiv ntxiv. Tsis tas li ntawd, siv cov ntaub ntawv ntsiab lus, kuv hloov kho lub load thiab txo qhov hloov tshiab rau 4 s.
PostgreSQL. 35 NPV
Kuv thawj zaug khiav ntawm cov khoom siv no yog nyob rau ntawm cov ntshiab PostgreSQL - 35 txhiab tus nqi ib ob. Raws li koj tuaj yeem pom, ntxig cov ntaub ntawv yuav siv cov feem ntawm ib thib ob - txhua yam zoo thiab nrawm. Qhov tsuas yog qhov uas 200 GB SSD disk puv sai sai.
Qhov no yog tus qauv Zabbix server kev ua haujlwm dashboard.
Thawj daim duab xiav yog tus naj npawb ntawm qhov tseem ceeb hauv ib ob. Daim duab thib ob ntawm sab xis yog kev thauj khoom ntawm cov txheej txheem tsim. Qhov thib peb yog thauj cov txheej txheem tsim sab hauv: keeb kwm syncers thiab Housekeeper, uas tau khiav ntawm no rau qee lub sijhawm.
Daim duab plaub qhia HistoryCache siv. Qhov no yog hom tsis ua ntej tso rau hauv cov ntaub ntawv. Daim duab thib tsib ntsuab qhia txog kev siv ValueCache, uas yog, pes tsawg ValueCache hits rau qhov tshwm sim - qhov no yog ob peb txhiab tus nqi ib ob.
PostgreSQL. 50 NPV
Tom qab ntawd kuv tau nce lub load rau 50 txhiab tus nqi ib ob ntawm tib lub kho vajtse.
Thaum thauj khoom los ntawm Tus Saib Xyuas Tsev, ntxig 10 txhiab tus nqi siv 2-3 vib nas this.
Tus neeg saib xyuas tsev twb pib cuam tshuam kev ua haujlwm.
Qhov thib peb daim duab qhia tau hais tias, nyob rau hauv dav dav, lub load ntawm trappers thiab keeb kwm syncers tseem nyob ntawm 60%. Hauv daim duab plaub, HistoryCache twb tau pib ua kom muaj zog heev thaum lub sijhawm ua haujlwm hauv tsev. Nws yog 20% tag nrho, uas yog hais txog 0,5 GB.
PostgreSQL. 80 NPV
Tom qab ntawd kuv tau nce lub load rau 80 txhiab tus nqi ib ob. Qhov no yog kwv yees li 400 txhiab cov ntaub ntawv thiab 280 txhiab tus ua rau.
Tus nqi thauj khoom ntawm peb caug keeb kwm syncers twb tau siab heev.
Kuv kuj tau nce ntau yam tsis muaj: keeb kwm syncers, caches.
Ntawm kuv cov khoom kho vajtse, qhov chaw thau khoom ntawm keeb kwm syncers tau nce mus rau qhov siab tshaj plaws. HistoryCache tau sau sai sai nrog cov ntaub ntawv - cov ntaub ntawv rau kev ua tiav tau khaws cia hauv qhov tsis muaj.
Tag nrho lub sij hawm no kuv tau soj ntsuam yuav ua li cas lub processor, RAM thiab lwm yam system tsis tau siv, thiab pom tias disk siv yog nyob rau hauv nws siab tshaj plaws.
Kuv tau ua tiav kev siv siab tshaj disk muaj peev xwm ntawm no hardware thiab ntawm lub tshuab virtual no. Nrog rau qhov kev siv zog li no, PostgreSQL pib yaug cov ntaub ntawv heev, thiab lub disk tsis muaj sijhawm los sau thiab nyeem.
Thib ob server
Kuv coj lwm tus neeg rau zaub mov, uas twb muaj 48 processors thiab 128 GB ntawm RAM. Kuv tuned - muab tso rau 60 keeb kwm syncer, thiab ua tiav qhov kev ua tau zoo.
Qhov tseeb, qhov no yog qhov txwv ntawm kev tsim khoom uas ib yam dab tsi yuav tsum tau ua.
TimescaleDB. 80 NP
Kuv lub luag haujlwm tseem ceeb yog kuaj lub peev xwm ntawm TimescaleDB tiv thaiv Zabbix load. 80 txhiab tus nqi ib ob yog ntau heev, qhov zaus ntawm kev sau cov metrics (tshwj tsis yog Yandex, tau kawg) thiab qhov loj loj "tsim".
Muaj qhov poob rau hauv txhua daim duab - qhov no yog qhov tseeb ntawm cov ntaub ntawv tsiv teb tsaws. Tom qab ua tsis tiav hauv Zabbix server, qhov chaw thau khoom profile ntawm keeb kwm syncer hloov pauv ntau - nws poob peb zaug.
TimescaleDB tso cai rau koj ntxig cov ntaub ntawv yuav luag 3 zaug sai dua thiab siv HistoryCache tsawg dua.
Tom qab ntawd kuv tau nce tus naj npawb ntawm cov ntaub ntawv mus rau 500 txhiab. Lub luag haujlwm tseem ceeb yog los ntsuas lub peev xwm ntawm TimescaleDB - Kuv tau txais tus nqi suav ntawm 125 txhiab tus nqi ib ob.
Qhov no yog kev ua haujlwm "kev teeb tsa" uas tuaj yeem ua haujlwm ntev. Tab sis txij li kuv lub disk tsuas yog 1,5 TB, kuv tau sau nws hauv ob peb hnub.
Qhov tseem ceeb tshaj plaws yog tias tib lub sijhawm tshiab TimescaleDB partitions tau tsim.
Qhov no yog unnoticeable kiag li rau kev ua tau zoo. Thaum cov partitions raug tsim hauv MySQL, piv txwv li, txhua yam txawv. Qhov no feem ntau tshwm sim thaum hmo ntuj vim tias nws thaiv kev nkag dav dav, ua haujlwm nrog cov ntxhuav thiab tuaj yeem tsim kev pabcuam degradation. Qhov no tsis yog qhov xwm txheej nrog TimescaleDB.
Ua piv txwv, kuv yuav qhia ib daim duab los ntawm ntau tus neeg hauv zej zog. Nyob rau hauv daim duab, TimescaleDB yog enabled, ua tsaug uas lub load rau siv io.weight ntawm lub processor tau poob. Kev siv cov txheej txheem sab hauv kuj txo qis. Ntxiv mus, qhov no yog lub tshuab virtual zoo tib yam ntawm cov pancake disks, tsis yog SSD.
tshawb pom
TimescaleDB yog ib qho kev daws teeb meem zoo rau "kev teeb tsa" me me, uas cuam tshuam rau kev ua haujlwm disk. Nws yuav tso cai rau koj mus txuas ntxiv ua haujlwm zoo kom txog thaum cov ntaub ntawv tau tsiv mus rau kho vajtse sai li sai tau.
TimescaleDB yooj yim rau kev teeb tsa, muab kev ua tau zoo, ua haujlwm zoo nrog Zabbix thiab muaj qhov zoo dua PostgreSQL.
Yog tias koj siv PostgreSQL thiab tsis npaj yuav hloov nws, kuv pom zoo siv PostgreSQL nrog TimescaleDB txuas ntxiv ua ke nrog Zabbix. Qhov kev daws teeb meem no ua haujlwm zoo mus txog qhov nruab nrab "kev teeb tsa".
Thaum peb hais tias "kev ua haujlwm siab" peb txhais tau tias HighLoad ++. Koj yuav tsis muaj sijhawm ntev los kawm txog cov thev naus laus zis thiab kev coj ua uas ua rau muaj kev pabcuam rau ntau lab tus neeg siv. Sau npe qhia rau lub Kaum Ib Hlis 7 thiab 8 peb twb tau sau, tab sis ntawm no kev sib ntsib tuaj yeem hais ntxiv.
Subscribe rau peb tsab ntawv xov xwm и telegram, nyob rau hauv uas peb nthuav tawm cov yam ntxwv ntawm lub rooj sib tham yav tom ntej, thiab nrhiav kom tau qhov zoo tshaj plaws ntawm nws.