{"id":55734,"date":"2020-01-27T00:00:00","date_gmt":"2020-01-26T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/highload-andrej-gushhin-zabbix-vysokaya-proizvoditelnost-i-nativnoe-partitsionirovanie"},"modified":"2020-02-18T14:03:52","modified_gmt":"2020-02-18T11:03:52","slug":"highload-andrej-gushhin-zabbix-vysokaya-proizvoditelnost-i-nativnoe-partitsionirovanie","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/highload-andrej-gushhin-zabbix-vysokaya-proizvoditelnost-i-nativnoe-partitsionirovanie","title":{"rendered":"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>Ne do t\u00eb shqyrtojm\u00eb funksionimin e Zabbix me baz\u00ebn e t\u00eb dh\u00ebnave TimescaleDB si backend. Do t\u00eb tregojm\u00eb si t\u00eb nisni nga zero dhe si t\u00eb migroni nga PostgreSQL. Do t\u00eb ofrojm\u00eb gjithashtu teste t\u00eb krahasimit t\u00eb performanc\u00ebs t\u00eb dy konfigurimeve.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/fb4f7ea4585b6dcdafec9d0d1e3a71e4.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nHighLoad++ Siberia 2019. Salla \"Tomsk\". 24 qershor, 16:00. Tezat dhe <noindex><a rel=\"nofollow\" href=\"https:\/\/www.highload.ru\/siberia\/2019\/abstracts\/5390\">prezantimi<\/a><\/noindex>. Konferenca e ardhshme HighLoad++ do t\u00eb mbahet m\u00eb 6 dhe 7 prill 2020 n\u00eb Sh\u00ebn Petersburg. Detajet dhe biletat <noindex><a rel=\"nofollow\" href=\"http:\/\/bit.ly\/2sSxgBx\">n\u00eb lidhje<\/a><\/noindex>.<\/p>\n<p><b>Andrei Gushchin (m\u00eb pas \u2013 AG):<\/b> \u2013 Un\u00eb jam inxhinier i mb\u00ebshtetjes teknike t\u00eb ZABBIX (m\u00eb pas \u2013 \"Zabbix\"), trainer. Kam punuar m\u00eb shum\u00eb se 6 vjet n\u00eb mb\u00ebshtetje teknike dhe kam pasur eksperienc\u00eb t\u00eb drejtp\u00ebrdrejt\u00eb me performanc\u00ebn. Sot do t\u00eb flas p\u00ebr performanc\u00ebn q\u00eb mund t\u00eb ofroj\u00eb TimescaleDB, n\u00eb krahasim me PostgreSQL 10. Gjithashtu, disa informacione hyr\u00ebse \u2013 se si funksionon n\u00eb t\u00ebr\u00ebsi.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h3>Sfida kryesore e performanc\u00ebs: nga grumbullimi deri te pastrimi i t\u00eb dh\u00ebnave<\/h3>\n<p>\nLe t\u00eb fillojm\u00eb me faktin se ka sfida t\u00eb caktuara t\u00eb performanc\u00ebs me t\u00eb cilat p\u00ebrballet \u00e7do sistem monitorimi. Sfida e par\u00eb e performanc\u00ebs \u00ebsht\u00eb grumbullimi dhe p\u00ebrpunimi i shpejt\u00eb i t\u00eb dh\u00ebnave.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/a2cb78b549a55c3b59d7fcdb8b8865b3.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNj\u00eb sistem monitorimi i mir\u00eb duhet t\u00eb marr\u00eb dhe p\u00ebrpunoj\u00eb t\u00eb gjitha t\u00eb dh\u00ebnat n\u00eb koh\u00eb, sipas shprehjeve t\u00eb ndjeshm\u00ebris\u00eb, dometh\u00ebn\u00eb, t\u00eb p\u00ebrpunoj\u00eb sipas disa kritereve (n\u00eb sisteme t\u00eb ndryshme, ky proces ndodhet ndryshe) dhe t\u00eb ruaj\u00eb n\u00eb baz\u00ebn e t\u00eb dh\u00ebnave, n\u00eb m\u00ebnyr\u00eb q\u00eb m\u00eb von\u00eb t\u00eb p\u00ebrdoren k\u00ebto t\u00eb dh\u00ebna.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/a3ce55c3aec3eb93948150684838a5b3.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSfida e dyt\u00eb e performanc\u00ebs \u00ebsht\u00eb ruajtja e historis\u00eb. Ruajtja e t\u00eb dh\u00ebnave n\u00eb nj\u00eb baz\u00eb t\u00eb dh\u00ebnash \u00ebsht\u00eb shpesh e nevojshme dhe duhet t\u00eb ofrohet nj\u00eb akses i shpejt\u00eb dhe i leht\u00eb n\u00eb k\u00ebto metrika q\u00eb jan\u00eb mbledhur p\u00ebr nj\u00eb periudh\u00eb t\u00eb caktuar kohe. M\u00eb e r\u00ebnd\u00ebsishmja, q\u00eb k\u00ebto t\u00eb dh\u00ebna t\u00eb jen\u00eb t\u00eb lehta p\u00ebr t'u marr\u00eb, t'i p\u00ebrdor\u00ebsh ato n\u00eb raporte, grafike, ndjeshm\u00ebri, n\u00eb disa vlera pragore, p\u00ebr njoftime etj.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/325363d79f0270d620f956eeb8ab3402.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSfida e tret\u00eb e performanc\u00ebs \u00ebsht\u00eb pastrimi i historis\u00eb, q\u00eb do t\u00eb thot\u00eb kur vjen nj\u00eb dit\u00eb kur nuk duhet t\u00eb ruash disa metrika t\u00eb detajuara q\u00eb jan\u00eb mbledhur p\u00ebr 5 vjet (edhe p\u00ebr muaj ose dy muaj). Disa nyje t\u00eb rrjetit jan\u00eb fshir\u00eb, ose disa hoste, metrika tashm\u00eb nuk jan\u00eb t\u00eb nevojshme sepse ato jan\u00eb b\u00ebr\u00eb t\u00eb vjetra dhe nuk po mbledhen m\u00eb. Po, t\u00eb gjitha k\u00ebto duhet t\u00eb pastrohen, n\u00eb m\u00ebnyr\u00eb q\u00eb baza e t\u00eb dh\u00ebnave t\u00eb mos rritet n\u00eb nj\u00eb madh\u00ebsi t\u00eb madhe. Dhe n\u00eb p\u00ebrgjith\u00ebsi, pastrimi i historis\u00eb shpesh \u00ebsht\u00eb nj\u00eb prov\u00eb serioze p\u00ebr depot \u2013 zakonisht ndikon shum\u00eb n\u00eb performanc\u00eb.<\/p>\n<h3>Si t\u00eb zgjidhni problemet e cache-it?<\/h3>\n<p>\nDua t\u00eb flas konkretisht p\u00ebr \"Zabbix\". N\u00eb \"Zabbix\", sfidat e para dhe t\u00eb dyta zgjidhen p\u00ebrmes cache-it.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/d5510b31553ee8e63a0c240789d51622.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nGrumbullimi dhe p\u00ebrpunimi i t\u00eb dh\u00ebnave \u2013 ne p\u00ebrdorim kujtes\u00ebn operuese p\u00ebr t\u00eb ruajtur t\u00eb gjitha k\u00ebto t\u00eb dh\u00ebna. Tani do t\u00eb flitet m\u00eb shum\u00eb p\u00ebr k\u00ebto t\u00eb dh\u00ebna.<\/p>\n<p>Po ashtu, nga ana e baz\u00ebs s\u00eb t\u00eb dh\u00ebnave ka nj\u00eb cache t\u00eb caktuar p\u00ebr p\u00ebrzgjedhjet kryesore \u2013 p\u00ebr grafike, dhe gj\u00ebra t\u00eb tjera.<\/p>\n<p>Cache-i n\u00eb an\u00ebn e vet\u00eb serverit Zabbix: ne kemi ConfigurationCache, ValueCache, HistoryCache, TrendsCache. \u00c7far\u00eb jan\u00eb k\u00ebto?<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/241a131145eccd280a7411c43b2cdce4.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nConfigurationCache \u00ebsht\u00eb cache-i kryesor, n\u00eb t\u00eb cilin ne ruajm\u00eb metrikat, hostet, element\u00ebt e t\u00eb dh\u00ebnave, ndjeshm\u00ebrit\u00eb; gjith\u00e7ka q\u00eb nevojitet p\u00ebr p\u00ebrpunimin e parapro\u00e7esit, grumbullimin e t\u00eb dh\u00ebnave, nga cilat hoste t\u00eb grumbullojm\u00eb, me cil\u00ebn frekuenc\u00eb. T\u00eb gjitha k\u00ebto ruhen n\u00eb ConfigurationCache, p\u00ebr t\u00eb mos e kaluar n\u00eb baz\u00ebn e t\u00eb dh\u00ebnave, p\u00ebr t\u00eb mos krijuar k\u00ebrkesa t\u00eb tep\u00ebrta. Pas nisjes s\u00eb serverit, ne p\u00ebrdit\u00ebsojm\u00eb k\u00ebt\u00eb cache (krijojm\u00eb) dhe e rifreskojm\u00eb her\u00eb pas here (n\u00eb var\u00ebsi t\u00eb cil\u00ebsimeve t\u00eb konfigurimit).<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/474ac0db2e46aa945a0da62197f72987.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Cache-i n\u00eb Zabbix. Grumbullimi i t\u00eb dh\u00ebnave<\/h3>\n<p>\nK\u00ebtu skema \u00ebsht\u00eb mjaft e madhe:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/a6ad9dbd5deebac5440433a47c693fcf.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nElementet kryesore n\u00eb skem\u00eb \u2013 jan\u00eb k\u00ebta grumbullues:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/feb932586302e2284bd44595caf9ee1b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nK\u00ebto jan\u00eb proceset e grumbullimit, \"pollers\", q\u00eb p\u00ebrgjigjen p\u00ebr lloje t\u00eb ndryshme grumbullimi. Ata mbledhin t\u00eb dh\u00ebna p\u00ebr icmp, ipmi, p\u00ebr protokolle t\u00eb ndryshme dhe e d\u00ebrgojn\u00eb gjith\u00e7ka n\u00eb parapro\u00e7esim.<\/p>\n<h3>HistoriCache e PreProcessing<\/h3>\n<p>\nPo ashtu, n\u00ebse kemi element\u00eb t\u00eb dh\u00ebnash t\u00eb llogaritur (kush \u00ebsht\u00eb i njohur me \"Zabbix\" e di), jan\u00eb element\u00eb t\u00eb llogaritur, aggregative, ne i marrim ato direkt nga ValueCache. Do t\u00eb flas m\u00eb von\u00eb se si ai mbushet. T\u00eb gjith\u00eb k\u00ebta grumbullues p\u00ebrdorin ConfigurationCache p\u00ebr t\u00eb marr\u00eb detyrat e tyre dhe m\u00eb pas i d\u00ebrgojn\u00eb p\u00ebr parapro\u00e7esim.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/3948ee1167c52856949c25e2043be974.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nParapro\u00e7esimi gjithashtu p\u00ebrdor ConfigurationCache p\u00ebr t\u00eb marr\u00eb hapat e parapro\u00e7esimit, p\u00ebrpunon k\u00ebto t\u00eb dh\u00ebna n\u00eb m\u00ebnyra t\u00eb ndryshme. Duke filluar nga versioni 4.2, ai \u00ebsht\u00eb vendosur te proxy. Kjo \u00ebsht\u00eb shum\u00eb e p\u00ebrshtatshme, sepse vet\u00eb parapro\u00e7esimi \u00ebsht\u00eb nj\u00eb operacion mjaft i r\u00ebnd\u00eb. Dhe n\u00ebse keni nj\u00eb \"Zabbix\" shum\u00eb t\u00eb madh, me nj\u00eb num\u00ebr t\u00eb madh element\u00ebsh t\u00eb t\u00eb dh\u00ebnave dhe nj\u00eb frekuenc\u00eb t\u00eb lart\u00eb grumbullimi, kjo leht\u00ebsisht leht\u00ebson pun\u00ebn.<\/p>\n<p>P\u00ebrkat\u00ebsisht, pasi p\u00ebrpunojm\u00eb k\u00ebto t\u00eb dh\u00ebna n\u00eb ndonj\u00eb m\u00ebnyr\u00eb duke p\u00ebrdorur parapro\u00e7esimin, i ruajm\u00eb ato n\u00eb HistoryCache p\u00ebr t'i p\u00ebrpunuar m\u00eb tej. K\u00ebtu p\u00ebrfundon grumbullimi i t\u00eb dh\u00ebnave. Kalojm\u00eb n\u00eb procesin kryesor.<\/p>\n<h3>Puna e History syncer<\/h3>\n<p>\n<img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/66539e4184d041ed84f797080226ba24.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nProcesi kryesor n\u00eb \u00abZabbix\u00bb (p\u00ebr shkak se kjo \u00ebsht\u00eb nj\u00eb arkitektur\u00eb monolitike) \u2013 History syncer. Ky \u00ebsht\u00eb procesi kryesor q\u00eb merret me p\u00ebrpunimin atomar t\u00eb \u00e7do elementi t\u00eb dh\u00ebnash, dometh\u00ebn\u00eb \u00e7do vler\u00eb:<\/p>\n<ul>\n<li>vjen nj\u00eb vler\u00eb (e merr nga HistoryCache);<\/li>\n<li>kontrollon n\u00eb Configuration syncer: a ka ndonj\u00eb trigger p\u00ebr llogaritje \u2013 i llogarit ato;<br \/>\nn\u00ebse ka \u2013 krijon ngjarje, krijon eskalimin p\u00ebr t\u00eb krijuar nj\u00eb njoftim, n\u00ebse \u00ebsht\u00eb e nevojshme sipas konfigurimit;<\/li>\n<li>regjistron triggerat p\u00ebr p\u00ebrpunim t\u00eb m\u00ebvonsh\u00ebm, agregim; n\u00ebse jeni duke agreguar p\u00ebr or\u00ebn e fundit dhe k\u00ebshtu me radh\u00eb, kjo vler\u00eb e ruan n\u00eb ValueCache, p\u00ebr ta shmangur aksesin n\u00eb tabel\u00ebn e historis\u00eb; k\u00ebshtu, ValueCache mbushet me t\u00eb dh\u00ebnat e nevojshme p\u00ebr llogaritjen e triggerave, elementeve t\u00eb llogaritura etj.;<\/li>\n<li>pastaj History syncer regjistron t\u00eb gjitha t\u00eb dh\u00ebnat n\u00eb baz\u00ebn e t\u00eb dh\u00ebnave;<\/li>\n<li>baza e t\u00eb dh\u00ebnave i regjistron ato n\u00eb disk \u2013 n\u00eb k\u00ebt\u00eb pik\u00eb procesi i p\u00ebrpunimit p\u00ebrfundohet.<\/li>\n<\/ul>\n<p><\/p>\n<h3>Baza t\u00eb dh\u00ebnash. Keshimi<\/h3>\n<p>\nN\u00eb an\u00ebn e DB-s\u00eb, kur d\u00ebshironi t\u00eb shihni grafiko ose ndonj\u00eb raport p\u00ebr ngjarjet, ka disa keshime t\u00eb ndryshme. Por n\u00eb kuad\u00ebr t\u00eb k\u00ebtij raporti nuk do flas p\u00ebr to.<\/p>\n<p>P\u00ebr MySQL ka Innodb_buffer_pool, dhe shum\u00eb keshime t\u00eb tjera t\u00eb ndryshme q\u00eb gjithashtu mund t\u00eb konfigurohen.<br \/>\nPor k\u00ebto jan\u00eb kryesor\u00ebt:<\/p>\n<ul>\n<li>shared_buffers;<\/li>\n<li>effective_cache_size;<\/li>\n<li>shared_pool.<\/li>\n<\/ul>\n<p>\n<img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/925e7abb21cfaa2516ff2b33ab161dc0.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nUn\u00eb kam paraqitur p\u00ebr t\u00eb gjitha bazat e t\u00eb dh\u00ebnave q\u00eb ka disa keshime, q\u00eb lejojn\u00eb mbajtjen n\u00eb memorje p\u00ebr ato t\u00eb dh\u00ebna q\u00eb jan\u00eb shpesh t\u00eb nevojshme p\u00ebr k\u00ebrkesat. Aty kan\u00eb teknologjit\u00eb e tyre p\u00ebr k\u00ebt\u00eb.<\/p>\n<h3>P\u00ebr performanc\u00ebn e baz\u00ebs s\u00eb t\u00eb dh\u00ebnave<\/h3>\n<p>\nSipas k\u00ebsaj, ka nj\u00eb ambient konkurrues, dometh\u00ebn\u00eb serveri \u00abZabbix\u00bb mbledh t\u00eb dh\u00ebnat dhe i regjistron ato. Gjat\u00eb rimarrjes ai gjithashtu lexon nga historia p\u00ebr t\u00eb mbushur ValueCache dhe k\u00ebshtu me radh\u00eb. Po ashtu, mund t\u00eb keni skriptet dhe raportet q\u00eb p\u00ebrdorin \u00abZabbix\u00bb-API, i cili \u00ebsht\u00eb nd\u00ebrtuar mbi nd\u00ebrfaqen web. \u00abZabbix\u00bb-API hyn n\u00eb DB dhe merr t\u00eb dh\u00ebnat e nevojshme p\u00ebr t\u00eb marr\u00eb grafiko, raporte ose ndonj\u00eb list\u00eb ngjarjesh, problemet e fundit.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/10a55cb1af660aace3172a572bea6692.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNj\u00eb zgjidhje shum\u00eb e njohur p\u00ebr vizualizimin \u00ebsht\u00eb Grafana, q\u00eb e p\u00ebrdorin p\u00ebrdoruesit tan\u00eb. Ajo di t\u00eb hyj\u00eb direkt p\u00ebrmes \u00abZabbix\u00bb-API ose p\u00ebrmes DB. Ajo gjithashtu krijon nj\u00eb gar\u00eb t\u00eb caktuar p\u00ebr t\u00eb marr\u00eb t\u00eb dh\u00ebnat: k\u00ebrkohet nj\u00eb konfigurim m\u00eb i holl\u00eb dhe i mir\u00eb i DB-s\u00eb p\u00ebr t\u00eb p\u00ebrmbushur shp\u00ebrndarjen e shpejt\u00eb t\u00eb rezultateve dhe testimin.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/8a008e55dbee5635d386cec2fc1c40f5.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Pastrimi i historis\u00eb. N\u00eb Zabbix ka nj\u00eb Housekeeper<\/h3>\n<p>\nThirrja e tret\u00eb, q\u00eb p\u00ebrdoret n\u00eb \u00abZabbix\u00bb \u2013 \u00ebsht\u00eb pastrimi i historis\u00eb me an\u00eb t\u00eb Housekeeper. \u00abHousekeeper\u00bb respekton t\u00eb gjitha konfigurimet, dometh\u00ebn\u00eb ne n\u00eb element\u00ebt e dh\u00ebnash kemi p\u00ebrcaktuar se sa t\u00eb mbajm\u00eb (n\u00eb dit\u00eb), sa t\u00eb mbajm\u00eb trendet, dinamika e ndryshimeve.<\/p>\n<p>Nuk kam folur p\u00ebr TrendCache, t\u00eb cilat i llogarisim n\u00eb flutur: t\u00eb dh\u00ebnat vijn\u00eb, ne i agregojm\u00eb p\u00ebr nj\u00eb or\u00eb (m\u00eb s\u00eb shumti jan\u00eb numrat e or\u00ebs s\u00eb fundit), mesatarja \/ minimale dhe regjistrojm\u00eb k\u00ebt\u00eb nj\u00eb her\u00eb n\u00eb or\u00eb n\u00eb tabel\u00ebn e dinamikave t\u00eb ndryshimeve (\u00abTrends\u00bb). \u00abHousekeeper\u00bb niset dhe fshin t\u00eb dh\u00ebnat nga DB me seleksione normale, q\u00eb nuk \u00ebsht\u00eb gjithmon\u00eb efikase.<\/p>\n<p>Si ta kuptoni q\u00eb kjo nuk \u00ebsht\u00eb efikase? Mund t\u00eb shihni nj\u00eb pamje t\u00eb till\u00eb n\u00eb grafiket e performanc\u00ebs s\u00eb proceseve t\u00eb brendshme:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/f413ec989491d4186a05c779978e2f6d.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nJu keni History syncer t\u00eb angazhuar vazhdimisht (grafiku i kuq). Dhe grafiku \"portokalli\", q\u00eb kalon n\u00eb kresht k\u00ebt\u00eb. Kjo \u00ebsht\u00eb \u00abHousekeeper\u00bb, i cili niset dhe pret nga DB q\u00eb t\u00eb fshij\u00eb t\u00eb gjitha rreshtat q\u00eb ai cakton.<\/p>\n<p>Merrni ndonj\u00eb ID Artikulli: duhet t\u00eb fshini 5 mij\u00eb t\u00eb fundit; sigurisht, sipas indekseve. Por zakonisht dataset \u00ebsht\u00eb mjaft i madh \u2013 baza e t\u00eb dh\u00ebnave gjithsesi e lexon at\u00eb nga disku dhe e ngre at\u00eb n\u00eb cache, dhe kjo \u00ebsht\u00eb nj\u00eb operacion shum\u00eb i kushtuesh\u00ebm p\u00ebr DB. N\u00eb var\u00ebsi t\u00eb madh\u00ebsive t\u00eb saj, kjo mund t\u00eb \u00e7oj\u00eb n\u00eb disa probleme performanc\u00ebs.<\/p>\n<p>P\u00ebr t\u00eb \u00e7aktivizuar \u00abHousekeeper\u00bb mund t\u00eb b\u00ebni nj\u00eb m\u00ebnyr\u00eb t\u00eb thjesht\u00eb \u2013 ne kemi nd\u00ebrfaqen web t\u00eb njohur p\u00ebr t\u00eb gjith\u00eb. Konfigurimi n\u00eb Administration general (konfigurimet p\u00ebr \u00abHousekeeper\u00bb) ne \u00e7aktivizojm\u00eb housekeeping-un e brendsh\u00ebm p\u00ebr historin\u00eb dhe trendet e brendshme. Si rezultat, \u00abHousekeeper\u00bb nuk menaxhon m\u00eb k\u00ebt\u00eb:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/c9dc3ca86e7fd0c1229f6b9bc5d31270.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n\u00c7far\u00eb mund t\u00eb b\u00ebni m\u00eb tej? E keni \u00e7aktivizuar, grafiket tuaja jan\u00eb rregulluar... Cilat mund t\u00eb jen\u00eb problemet n\u00eb k\u00ebt\u00eb rast? \u00c7far\u00eb mund t\u00eb ndihmoj\u00eb?<\/p>\n<h3>Particionimi (sekcionimi)<\/h3>\n<p>\nKjo zakonisht konfigurohet n\u00eb \u00e7do baz\u00eb t\u00eb dh\u00ebnash relacionale, t\u00eb cilat i p\u00ebrmenda, n\u00eb m\u00ebnyra t\u00eb ndryshme. N\u00eb MySQL ka teknologjin\u00eb e saj. Por n\u00eb p\u00ebrgjith\u00ebsi ato jan\u00eb shum\u00eb t\u00eb ngjashme, n\u00ebse flasim p\u00ebr PostgreSQL 10 dhe MySQL. Sigurisht, ka shum\u00eb ndryshime t\u00eb brendshme, si\u00e7 \u00ebsht\u00eb realizuar gjith\u00e7ka dhe si ndikon t\u00eb gjitha k\u00ebto n\u00eb performanc\u00eb. Por n\u00eb p\u00ebrgjith\u00ebsi, krijimi i nj\u00eb partie t\u00eb re shpesh \u00e7on gjithashtu n\u00eb disa probleme t\u00eb caktuara.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/729114d61794314e19383f09ee9b5d61.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb var\u00ebsi t\u00eb konfigurimit tuaj (sa shum\u00eb t\u00eb dh\u00ebna krijohen n\u00eb nj\u00eb dit\u00eb), zakonisht p\u00ebrjashtoni minimumin \u2013 kjo \u00ebsht\u00eb 1 dit\u00eb\/particion, nd\u00ebrsa p\u00ebr \"trendet\", dinamik\u00ebn e ndryshimeve \u2013 1 muaj \/ nj\u00eb partition t\u00eb ri. Kjo mund t\u00eb ndryshoj\u00eb n\u00ebse keni nj\u00eb konfigurim shum\u00eb t\u00eb madh.<\/p>\n<p>Dua t\u00eb them menj\u00ebher\u00eb p\u00ebr p\u00ebrmasat e konfigurimit: deri n\u00eb 5 mij\u00eb vlera t\u00eb reja \u00e7do sekond\u00eb (nvps, si\u00e7 quhet) \u2013 kjo do t\u00eb konsiderohet nj\u00eb \"set up\" i vog\u00ebl. Mesatarja \u00ebsht\u00eb nga 5 deri n\u00eb 25 mij\u00eb vlera \u00e7do sekond\u00eb. \u00c7do gj\u00eb m\u00eb shum\u00eb se kaq \u2013 \u00ebsht\u00eb instalime t\u00eb m\u00ebdha dhe shum\u00eb t\u00eb m\u00ebdha, t\u00eb cilat k\u00ebrkojn\u00eb konfigurim shum\u00eb t\u00eb kujdessh\u00ebm t\u00eb baz\u00ebs s\u00eb t\u00eb dh\u00ebnave.<\/p>\n<p>N\u00eb instalime shum\u00eb t\u00eb m\u00ebdha 1 dit\u00eb \u2013 kjo mund t\u00eb mos jet\u00eb optimale. Un\u00eb vet\u00eb kam par\u00eb n\u00eb MySQL partitione prej 40 gigabajt n\u00eb dit\u00eb (dhe m\u00eb shum\u00eb mund t\u00eb jen\u00eb). Ky \u00ebsht\u00eb nj\u00eb volum shum\u00eb i madh t\u00eb dh\u00ebnash, i cili mund t\u00eb shkaktoj\u00eb disa probleme. Duhet ta zvog\u00ebloni.<\/p>\n<h3>P\u00ebrse \u00ebsht\u00eb e nevojshme partitizimi?<\/h3>\n<p>\n\u00c7far\u00eb p\u00ebrfitimi ofron Partitioning, mendoj se t\u00eb gjith\u00eb e din\u00eb \u2013 kjo \u00ebsht\u00eb ndarja e tabelave. Zakonisht, k\u00ebto jan\u00eb skedar\u00eb t\u00eb ve\u00e7ant\u00eb n\u00eb disk dhe k\u00ebrkesa t\u00eb spanuara. Ai zgjedh m\u00eb optimal nj\u00eb partition, n\u00ebse kjo p\u00ebrfshihet n\u00eb ndarjen normale.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/4920fb3415733799f0a90b1bc7af0114.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nP\u00ebr \"Zabbix\", n\u00eb ve\u00e7anti, p\u00ebrdoret sipas rangut, sipas intervalit, pra ne p\u00ebrdorim timestamp (numri normal, koha q\u00eb nga fillimi i epok\u00ebs). Ju p\u00ebrcaktoni fillimin e dit\u00ebs \/ mbarimin e dit\u00ebs, dhe kjo \u00ebsht\u00eb nj\u00eb partition. Sipas k\u00ebsaj, n\u00ebse ju referoheni t\u00eb dh\u00ebnave dy dit\u00eb m\u00eb par\u00eb, gjith\u00e7ka do t\u00eb zgjedh\u00eb nga baza e t\u00eb dh\u00ebnave m\u00eb shpejt, sepse duhet t\u00eb ngarkoni vet\u00ebm nj\u00eb skedar n\u00eb cache dhe ta jepni (e jo nj\u00eb tabel\u00eb t\u00eb madhe).<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/f5ee4e0b471eb5ebe701615aa617d66d.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nShum\u00eb DB gjithashtu p\u00ebrshpejtojn\u00eb insertimin (shtimi n\u00eb nj\u00eb child-table). Deri tani po flas n\u00eb m\u00ebnyr\u00eb abstrakte, por kjo \u00ebsht\u00eb gjithashtu e mundur. Partitizimi shpesh ndihmon.<\/p>\n<h3>Elasticsearch p\u00ebr NoSQL<\/h3>\n<p>\nS\u00eb fundmi, n\u00eb 3.4, ne kemi futur nj\u00eb zgjidhje p\u00ebr NoSQL. Shtuam mund\u00ebsin\u00eb p\u00ebr t\u00eb shkruajtur n\u00eb Elasticsearch. Ju mund t\u00eb shkruani disa lloje t\u00eb ve\u00e7anta: zgjidhni \u2013 ose shkruani numra, ose ndonj\u00eb shenj\u00eb; ne kemi tekst-string, mund t\u00eb shkruani log n\u00eb Elasticsearch\u2026 P\u00ebrkat\u00ebsisht, nd\u00ebrfaqja web do t\u00eb drejtohet gjithashtu drejt Elasticsearch. Kjo funksionon mjaft mir\u00eb n\u00eb disa raste, por p\u00ebr momentin mund t\u00eb p\u00ebrdoret.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/24fe7d19c9e42474cd786ba59846c18c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>TimescaleDB. Hipertable<\/h3>\n<p>\nP\u00ebr 4.4.2 v\u00ebm\u00eb re nj\u00eb gj\u00eb, si\u00e7 \u00ebsht\u00eb TimescaleDB. \u00c7far\u00eb \u00ebsht\u00eb kjo? Kjo \u00ebsht\u00eb nj\u00eb zgjerim p\u00ebr \"Postgres\", pra ka nj\u00eb nd\u00ebrfaqe natyrore p\u00ebr PostgreSQL. P\u00ebrve\u00e7 k\u00ebsaj, ky zgjerim lejon t\u00eb punosh shum\u00eb m\u00eb efektivisht me t\u00eb dh\u00ebnat e koh\u00ebs dhe t\u00eb kesh partitizim automatik. Si duket:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/de921f06453476215b6240ab9da6fae1.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKy \u00ebsht\u00eb hipertable \u2013 ka nj\u00eb koncept t\u00eb till\u00eb n\u00eb Timescale. Kjo \u00ebsht\u00eb hipertabela q\u00eb ju krijoni, dhe n\u00eb t\u00eb ka chunks (chunk). Chunks jan\u00eb partitione, k\u00ebto jan\u00eb child-tables, n\u00ebse nuk gaboj. Kjo \u00ebsht\u00eb v\u00ebrtet efikase.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/719eb3f5d8c9f57544dfc4010850610c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>TimescaleDB dhe PostgreSQL<\/h3>\n<p>\nSi\u00e7 garantohen prodhuesit e TimescaleDB, ata p\u00ebrdorin nj\u00eb algorit\u00ebm m\u00eb t\u00eb sakt\u00eb p\u00ebr p\u00ebrpunimin e k\u00ebrkesave, sidomos insert\u2019ave, i cili lejon t\u00eb kesh nj\u00eb performanc\u00eb pothuajse konstante me rritjen e madh\u00ebsis\u00eb s\u00eb dataset-insertit. Pra, pas 200 milion rreshtash \"Postgres\" normal fillon t\u00eb bjer\u00eb shum\u00eb dhe humbet performanc\u00ebn deri n\u00eb zero, nd\u00ebrsa \"Timescale\" lejon q\u00eb insertet t\u00eb realizohen sa m\u00eb efikasht t\u00eb jet\u00eb e mundur me \u00e7do sasi t\u00eb dh\u00ebnash.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/a613d505a9dfe6008551ce981da7b24e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Si t\u00eb instaloni TimescaleDB? \u00cbsht\u00eb shum\u00eb e thjesht\u00eb!<\/h3>\n<p>\nKa dokumentacion p\u00ebr t\u00eb, e p\u00ebrshkruar \u2013 mund t\u00eb instaloni nga paketat p\u00ebr \u00e7do... Ky varion nga paketat zyrtare t\u00eb \"Postgres\". Mund t\u00eb kompiloni manualisht. M\u00eb ndodhi t\u00eb kompiloj p\u00ebr DB.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/6767927fa92260103d318f9aa70ef5b3.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb \"Zabbix\" ne thjesht aktivizojm\u00eb Extention. Mendoj se ata q\u00eb kan\u00eb p\u00ebrdorur n\u00eb \"Postgres\" Extention\u2026 Thjesht aktivizoni Extention, krijoni at\u00eb p\u00ebr DB \"Zabbix\" q\u00eb po p\u00ebrdorni.<\/p>\n<p>Dhe hapi i fundit...<\/p>\n<h3>TimescaleDB. Migrimi i tabelave t\u00eb historis\u00eb<\/h3>\n<p>\nJu duhet t\u00eb krijoni hipertable. P\u00ebr k\u00ebt\u00eb ka nj\u00eb funksion t\u00eb ve\u00e7ant\u00eb \u2013 Create hypertable. N\u00eb t\u00eb, parametri i par\u00eb tregon tabel\u00ebn q\u00eb nevojitet n\u00eb k\u00ebt\u00eb DB (p\u00ebr t\u00eb cil\u00ebn duhet t\u00eb krijoni hipertabel\u00ebn).<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/47350f79fd9829c12685c1df8f0ff9a6.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nFusha, mbi t\u00eb cil\u00ebn duhen krijuar, dhe chunk_time_interval (kjo \u00ebsht\u00eb intervali i chunks (particioneve q\u00eb duhet t\u00eb p\u00ebrdorin). 86 400 \u2013 kjo \u00ebsht\u00eb nj\u00eb dit\u00eb. <\/p>\n<p>Parametri migrate_data: n\u00ebse e vendosni n\u00eb true, at\u00ebher\u00eb ky transferon t\u00eb gjitha t\u00eb dh\u00ebnat aktuale n\u00eb chunks t\u00eb krijuara paraprakisht.<\/p>\n<p>Un\u00eb vet\u00eb e kam p\u00ebrdorur migrate_data \u2013 kjo merr nj\u00eb koh\u00eb t\u00eb konsiderueshme, n\u00eb var\u00ebsi t\u00eb p\u00ebrmasave t\u00eb DB-s\u00eb tuaj. Un\u00eb kisha mbi nj\u00eb terabajt \u2013 krijimi zgjati m\u00eb shum\u00eb se nj\u00eb or\u00eb. N\u00eb disa raste gjat\u00eb testimit, kam hequr t\u00eb dh\u00ebnat historike p\u00ebr tekst (history_text) dhe string (history_str), q\u00eb t\u00eb mos i transferoja \u2013 ato n\u00eb t\u00eb v\u00ebrtet\u00eb nuk m\u00eb interesonin.<\/p>\n<p>Dhe p\u00ebrdit\u00ebsimi i fundit q\u00eb po b\u00ebjm\u00eb n\u00eb db_extention ton\u00eb: ne instalojm\u00eb timescaledb, q\u00eb edhe baza e t\u00eb dh\u00ebnave dhe, n\u00eb ve\u00e7anti, \u2018Zabbix\u2019 t\u00eb kuptoj\u00eb se ekziston db_extention. Ai e aktivizon at\u00eb dhe p\u00ebrdor sintaks\u00ebn dhe k\u00ebrkesat e duhura p\u00ebr baz\u00ebn e t\u00eb dh\u00ebnave, duke shfryt\u00ebzuar \u201ctiparet\u201d q\u00eb k\u00ebrkohen p\u00ebr TimescaleDB.<\/p>\n<h3>Konfigurimi i serverit<\/h3>\n<p>\nKam p\u00ebrdorur dy server\u00eb. Serveri i par\u00eb \u2013 \u00ebsht\u00eb nj\u00eb makin\u00eb virtuale mjaft e vog\u00ebl, me 20 procese, 16 gigabajt RAM. E kam konfiguruar me PostgreSQL 10.8:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/990805374a2380a5c645c578404489bd.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSistemi operativ ishte Debian, sistemi i skedar\u00ebve \u2013 xfs. Kam b\u00ebr\u00eb konfigurime minimale q\u00eb t\u00eb p\u00ebrdor sakt\u00ebsisht k\u00ebt\u00eb baz\u00eb t\u00eb dh\u00ebnash, p\u00ebrve\u00e7 asaj q\u00eb do t\u00eb p\u00ebrdorte vet\u00eb \u2018Zabbix\u2019. N\u00eb t\u00eb nj\u00ebjt\u00ebn makin\u00eb kishte serverin \u2018Zabbix\u2019, PostgreSQL dhe agjent\u00ebt e ngarkes\u00ebs.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/a6efa9ab9cd58f0061d04f13fe31018c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKam p\u00ebrdorur 50 agjent\u00eb aktiv\u00eb, q\u00eb p\u00ebrdorin LoadableModule, p\u00ebr t\u00eb gjeneruar shpejt rezultate t\u00eb ndryshme. Ata gjeneruan rreshta, numra dhe k\u00ebshtu me radh\u00eb. E mbusha baz\u00ebn e t\u00eb dh\u00ebnave me nj\u00eb sasi t\u00eb madhe t\u00eb dh\u00ebnash. Fillimisht, konfigurimi p\u00ebrmbante 5 mij\u00eb element\u00eb t\u00eb dh\u00ebnash p\u00ebr \u00e7do host, dhe \u00e7do element t\u00eb dh\u00ebnash kishte nj\u00eb trigger \u2013 n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb ishte nj\u00eb konfigurim real. Ndonj\u00ebher\u00eb duhen m\u00eb shum\u00eb se nj\u00eb trigger p\u00ebr p\u00ebrdorim.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/d5a3c306402e08bf2532eb6203a4aab4.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nIntervali i p\u00ebrdit\u00ebsimit, ngarkesa e vet\u00eb, e regulloja duke p\u00ebrdorur jo vet\u00ebm 50 agjent\u00eb (shtoja edhe t\u00eb tjer\u00eb), por gjithashtu me elemente dinamike t\u00eb dh\u00ebnash dhe e ulja intervalin e p\u00ebrdit\u00ebsimit n\u00eb 4 sekonda.<\/p>\n<h3>Testi i performanc\u00ebs. PostgreSQL: 36 mij\u00eb NVPs<\/h3>\n<p>\nP\u00ebr her\u00eb t\u00eb par\u00eb, konfigurimi im ishte me PostgreSQL 10 t\u00eb past\u00ebr n\u00eb k\u00ebt\u00eb hardware (35 mij\u00eb vlera n\u00eb sekond\u00eb). Si\u00e7 duket n\u00eb ekran, futja e t\u00eb dh\u00ebnave zgjat fraksione sekondash \u2013 gjith\u00e7ka \u00ebsht\u00eb mir\u00eb dhe e shpejt\u00eb, SSD-t\u00eb (200 gigabajt). E vetmja gj\u00eb \u00ebsht\u00eb se 20 GB plotsish jan\u00eb plot\u00ebsisht t\u00eb shpejta.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/6803a64fedd26093b9117a036d2993ae.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDo t\u00eb ket\u00eb ende shum\u00eb grafi t\u00eb tilla. Kjo \u00ebsht\u00eb dashboards standarde e performanc\u00ebs s\u00eb serverit \u2018Zabbix\u2019.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/081cc11c6bf42db3f8f16b6cc4e6ee0c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nGrafiku i par\u00eb \u2013 numri i vlerave n\u00eb sekond\u00eb (blu, sip\u00ebr majtas), 35 mij\u00eb vlera n\u00eb k\u00ebt\u00eb rast. Ky (sip\u00ebr n\u00eb qend\u00ebr) \u00ebsht\u00eb ngarkesa e proceseve t\u00eb mbledhjes, nd\u00ebrsa ky (sip\u00ebr n\u00eb t\u00eb djatht\u00eb) \u00ebsht\u00eb ngarkesa e proceseve t\u00eb brendshme: history syncers dhe housekeeper, t\u00eb cil\u00ebt k\u00ebtu (n\u00eb fund n\u00eb qend\u00ebr) u ekzekutuan p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb.<\/p>\n<p>Ky grafik (n\u00eb fund n\u00eb qend\u00ebr) tregon p\u00ebrdorimin e ValueCache \u2013 sa hit-e t\u00eb ValueCache p\u00ebr triggerat (disa mij\u00ebra vlera n\u00eb sekond\u00eb). Nj\u00eb grafik tjet\u00ebr i r\u00ebnd\u00ebsish\u00ebm \u2013 grafiku i kat\u00ebrt (n\u00eb fund majtas) q\u00eb tregon p\u00ebrdorimin e HistoryCache, p\u00ebr t\u00eb cilin kam folur, i cili \u00ebsht\u00eb nj\u00eb buffer para futjes n\u00eb baz\u00ebn e t\u00eb dh\u00ebnave.<\/p>\n<h3>Testi i performanc\u00ebs. PostgreSQL: 50 mij\u00eb NVPs<\/h3>\n<p>\nM\u00eb pas e rrita ngarkes\u00ebn n\u00eb 50 mij\u00eb vlera n\u00eb sekond\u00eb n\u00eb k\u00ebt\u00eb hardware. Gjat\u00eb ngarkes\u00ebs nga \u2018Housekeeper\u2019, 10 mij\u00eb vlera u regjistruan brenda 2-3 sekondave me llogaritje. \u00c7far\u00eb, n\u00eb fakt, \u00ebsht\u00eb treguar n\u00eb screenshot-in e ardhsh\u00ebm:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/80824da986dbee2f8d5ebe0ead4fbcfd.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n\u2018Housekeeper\u2019 tashm\u00eb po fillon t\u00eb pengoj\u00eb funksionimin, por p\u00ebrgjith\u00ebsisht ngarkesa e triggerave t\u00eb history-syncers akoma \u00ebsht\u00eb n\u00eb nivelin 60 % (grafiku i tret\u00eb, sip\u00ebr n\u00eb t\u00eb djatht\u00eb). HistoryCache gjat\u00eb pun\u00ebs s\u00eb \u2018Housekeeper\u2019 fillon t\u00eb mbushet aktivisht (n\u00eb fund majtas). Ai ishte rreth gjysm\u00eb gigabait, duke u mbushur n\u00eb 20%.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/9e1bcd4f3b9f3e8a1c9a36b2083ec61b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Testi i performanc\u00ebs. PostgreSQL: 80 mij\u00eb NVPs<\/h3>\n<p>\nM\u00eb pas e rita n\u00eb 80 mij\u00eb vlera n\u00eb sekond\u00eb:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/13ae1589d6b9d35b4dda9c6c7d989cc2.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nIshte rreth 400 mij\u00eb element\u00eb t\u00eb dh\u00ebnash, 280 mij\u00eb triggera. Futja, si\u00e7 e shihni, me ngarkes\u00ebn e history-syncers (t\u00eb cil\u00ebt ishin 30) ishte mjaft e lart\u00eb. M\u00eb pas e rita parametra t\u00eb ndrysh\u00ebm: history-syncers, cache\u2026 N\u00eb k\u00ebt\u00eb hardware ngarkesa e history-syncers filloi t\u00eb rritej n\u00eb maksimum, pothuajse \u2018n\u00eb raft\u2019 \u2013 p\u00ebrkat\u00ebsisht, HistoryCache shkoi n\u00eb nj\u00eb ngarkes\u00eb shum\u00eb t\u00eb lart\u00eb:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/0ec528256c8fa7b16e1c7c95c8119535.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nGjith\u00eb k\u00ebt\u00eb koh\u00eb kam mbajtur n\u00ebn v\u00ebshtrim t\u00eb gjith\u00eb parametrat e sistemit (si p\u00ebrdoret procesori, memoria RAM) dhe kam zbuluar se shfryt\u00ebzimi i disqeve ishte maksimal \u2013 arrita kapacitetin maksimal t\u00eb k\u00ebtij disku n\u00eb k\u00ebt\u00eb hardware, n\u00eb k\u00ebt\u00eb makin\u00eb virtuale. \u2018PostgreSQL\u2019 filloi, me nj\u00eb intensitet t\u00eb till\u00eb, t\u00eb hiqte t\u00eb dh\u00ebna mjaft aktivisht, dhe disku nuk arrinte m\u00eb p\u00ebr t\u00eb shkruar, lexuar\u2026<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/0809dbd7638e7ee003ea24c611984a0c.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nMora nj\u00eb server tjet\u00ebr, i cili tashm\u00eb kishte 48 procese dhe 128 gigabajt RAM:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/53410b502a4574aac5d40f1a2a6d3f42.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPo ashtu e kam \u2018optimizuar\u2019 \u2013 vendosa history syncer (60 cop\u00eb) dhe arrita nj\u00eb performanc\u00eb t\u00eb pranueshme. N\u00eb fakt, ne nuk jemi \u2018n\u00eb raft\u2019, por k\u00ebt\u00eb \u00ebsht\u00eb, mbase, kufiri i performanc\u00ebs, ku tashm\u00eb duhet t\u00eb nd\u00ebrmarrim disa veprime.<\/p>\n<h3>Testi i performanc\u00ebs. TimescaleDB: 80 mij\u00eb NVPs<\/h3>\n<p>\nIshte nj\u00eb nga detyrat e mia kryesore \u2013 t\u00eb p\u00ebrdor TimescaleDB. N\u00eb \u00e7do grafik shihet r\u00ebnia:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/b0064068895a34b93b5aa6771aba05cb.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nK\u00ebto d\u00ebshtime jan\u00eb pik\u00ebrisht migrimi i t\u00eb dh\u00ebnave. Pas k\u00ebsaj, n\u00eb serverin \u00abZabbix\u00bb, profili i ngarkes\u00ebs p\u00ebr historin\u00eb e sinkronizuesve ka ndryshuar ndjesh\u00ebm. Ai lejon futjen e t\u00eb dh\u00ebnave pothuajse 3 her\u00eb m\u00eb shpejt dhe p\u00ebrdor m\u00eb pak HistoryCache \u2013 p\u00ebr pasoj\u00eb, t\u00eb dh\u00ebnat do t\u00eb d\u00ebrgohen n\u00eb koh\u00eb. P\u00ebrs\u00ebri, 80 mij\u00eb vlera n\u00eb sekond\u00eb \u2013 \u00ebsht\u00eb nj\u00eb nivel mjaft i lart\u00eb (sigurisht, jo p\u00ebr \u00abYandex\u00bb). N\u00eb p\u00ebrgjith\u00ebsi, kjo \u00ebsht\u00eb nj\u00eb konfigurim mjaft i madh, me nj\u00eb server.<\/p>\n<h3>Testi i performanc\u00ebs PostgreSQL: 120 mij\u00eb NVPs<\/h3>\n<p>\nP\u00ebr m\u00eb tep\u00ebr, e rrisja vler\u00ebn e numrit t\u00eb elementeve t\u00eb dh\u00ebnave n\u00eb gjysm\u00eb milioni dhe mora nj\u00eb vler\u00ebsim prej 125 mij\u00eb n\u00eb sekond\u00eb:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/7d0cf2ef7c6691c1bbf4b90afd34e4bb.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDhe mora k\u00ebto grafika:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/9c09333a50e016921a8a5b70e00397a1.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb parim, kjo \u00ebsht\u00eb nj\u00eb konfigurim funksional, ai mund t\u00eb punoj\u00eb p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb. Por pasi kisha nj\u00eb disk prej vet\u00ebm 1.5 terabajt\u00ebsh, e konsumova at\u00eb brenda disa dit\u00ebsh. E r\u00ebnd\u00ebsishmja \u00ebsht\u00eb se n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb po krijoheshin partit\u00eb t\u00eb reja n\u00eb TimescaleDB, dhe kjo p\u00ebr performanc\u00ebn ndodhte krejt\u00ebsisht pa u v\u00ebn\u00eb re, ndryshe nga MySQL.<\/p>\n<p>Zakonisht, partit\u00eb krijohen nat\u00ebn, sepse ato bllokojn\u00eb plot\u00ebsisht futjen dhe pun\u00ebn me tabelat, duke mund\u00ebsuar degradimin e sh\u00ebrbimit. N\u00eb k\u00ebt\u00eb rast, nuk ka asnj\u00eb \u00e7\u00ebshtje! Q\u00ebllimi kryesor ishte t\u00eb kontrollonim kapacitetet e TimescaleDB. Rezultati ishte kjo shif\u00ebr: 120 mij\u00eb vlera n\u00eb sekond\u00eb.<\/p>\n<p>Ka gjithashtu shembuj n\u00eb \u00abkomunitet\u00bb:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/66aa6b1d4c559d12082e4d91b6a1ca10.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nNj\u00eb person gjithashtu aktivizoi TimescaleDB dhe ngarkesa p\u00ebr p\u00ebrdorimin e io.weight ra n\u00eb procesor; dhe p\u00ebrdorimi i elementeve t\u00eb proceseve t\u00eb brendshme gjithashtu ra fal\u00eb aktivizimit t\u00eb TimescaleDB. Dhe kjo ndodhi edhe me disqet e zakonshme, pra nj\u00eb virtualizim normal n\u00eb disqe t\u00eb zakonshme (jo SSD)!<\/p>\n<p>P\u00ebr disa konfigurime t\u00eb vogla, t\u00eb cilat kufizohen nga performanca e disqit, TimescaleDB, mendoj se \u00ebsht\u00eb nj\u00eb zgjidhje shum\u00eb e mir\u00eb. Ai do t\u00eb lejoj\u00eb q\u00eb t\u00eb vazhdoj\u00eb t\u00eb punoj\u00eb deri sa t\u00eb migrosh n\u00eb nj\u00eb harduer m\u00eb t\u00eb shpejt\u00eb p\u00ebr baz\u00ebn e t\u00eb dh\u00ebnave.<\/p>\n<p>Ju ftoj t\u00eb gjith\u00eb n\u00eb ngjarjet tona: Konferenca \u2013 n\u00eb Mosk\u00eb, Samiti \u2013 n\u00eb Rig\u00eb. P\u00ebrdorni kanalet tona \u2013 \u00abTelegram\u00bb, forum, IRC. N\u00ebse keni ndonj\u00eb pyetje \u2013 ejani tek stenda jon\u00eb, mund t\u00eb flasim p\u00ebr \u00e7do gj\u00eb.<\/p>\n<h3>Pyetje nga publiku<\/h3>\n<p>\nPyetje nga publiku (m\u00eb tej \u2013 A): \u2013 N\u00ebse TimescaleDB \u00ebsht\u00eb kaq e leht\u00eb p\u00ebr tu konfiguruar dhe ofron nj\u00eb rritje t\u00eb till\u00eb t\u00eb performanc\u00ebs, ndoshta do t\u00eb ishte m\u00eb mir\u00eb ta p\u00ebrdorim si nj\u00eb praktik\u00eb t\u00eb mir\u00eb t\u00eb konfigurimit t\u00eb \u00abZabbix\u00bb me \u00abPostgreSQL\u00bb? Dhe a ka ndonj\u00eb penges\u00eb ose disavantazh t\u00eb k\u00ebsaj zgjidhjeje, apo n\u00ebse vendosa t\u00eb b\u00ebj \u00abZabbix\u00bb, mund t\u00eb marr \u00abPostgreSQL\u00bb, ta vendos aty \u00abTimescale\u00bb menj\u00ebher\u00eb, ta p\u00ebrdor dhe t\u00eb mos mendoj p\u00ebr ndonj\u00eb problem?<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/314845a019724806283a957b15d857cc.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<b>AG:<\/b> \u2013 Po, do t\u00eb thoja se kjo \u00ebsht\u00eb nj\u00eb rekomandim i mir\u00eb: t\u00eb p\u00ebrdor\u00ebsh \u00abPostgreSQL\u00bb menj\u00ebher\u00eb me zgjerimin TimescaleDB. Si\u00e7 kam th\u00ebn\u00eb, ka shum\u00eb komente pozitive, megjith\u00ebse ky \u00abfeature\u00bb \u00ebsht\u00eb eksperimental. Por n\u00eb t\u00eb v\u00ebrtet\u00eb, testet tregojn\u00eb se \u00ebsht\u00eb nj\u00eb zgjidhje e shk\u00eblqyer (me TimescaleDB), dhe mendoj se do t\u00eb zhvillohet! Ne e ndjekim si zhvillohet ky zgjerim dhe do t\u00eb rregullojm\u00eb \u00e7kaf un\u00eb nevojitet.<\/p>\n<p>Madje, gjat\u00eb zhvillimit u mb\u00ebshtet\u00ebm te nj\u00eb nga karakteristikat e njohura t\u00eb tij: aty mund t\u00eb punoje me chunks pak ndryshe. Por m\u00eb pas ata e hoq\u00ebn at\u00eb n\u00eb versionin e ardhsh\u00ebm, dhe ne u detyruam t\u00eb mos mb\u00ebshteteshim m\u00eb n\u00eb k\u00ebt\u00eb kod. Do t'i rekomandoja k\u00ebsaj zgjidhjeje n\u00eb shum\u00eb konfigurime. N\u00ebse po p\u00ebrdorni MySQL... P\u00ebr konfigurimet mesatare, \u00e7do zgjidhje funksionon mjaft mir\u00eb.<\/p>\n<p><b>A:<\/b> \u2013 N\u00eb grafikat e fundit, q\u00eb jan\u00eb nga community, kishte nj\u00eb grafik me \u00abHousekeeperin\u00bb:<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/d01549b6b9b97d8bdf5372efe05d9039.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAi vazhdoi t\u00eb punoj\u00eb. \u00c7far\u00eb b\u00ebn \u00abHousekeeper\u00bb n\u00eb rastin e TimescaleDB?<\/p>\n<p><b>AG:<\/b> \u2013 Tani nuk mund t\u00eb them sakt\u00ebsisht \u2013 do ta shoh kodin dhe do t\u00eb them m\u00eb n\u00eb detaje. Ai p\u00ebrdor k\u00ebrkesat e TimescaleDB jo p\u00ebr t\u00eb fshir\u00eb chunks, por si\u00e7 duket, i agregon. Deri tani nuk jam i gatsh\u00ebm t\u00eb p\u00ebrgjigjem p\u00ebr k\u00ebt\u00eb pyetje teknike. N\u00eb stand sot ose nes\u00ebr do ta sqarohem.<\/p>\n<p><b>A:<\/b> \u2013 Kam nj\u00eb pyetje t\u00eb ngjashme \u2013 p\u00ebr performanc\u00ebn e operacionit t\u00eb fshirjes n\u00eb \u00abTimescale\u00bb.<br \/>\nA (p\u00ebrgjigje nga publiku): \u2013 Kur fshini t\u00eb dh\u00ebna nga tabela, n\u00ebse e b\u00ebni k\u00ebt\u00eb p\u00ebrmes fshirjes, duhet t\u00eb kaloni p\u00ebrmes tabel\u00ebs \u2013 t\u00eb fshini, t\u00eb pastroni, t\u00eb gjitha t\u00eb sh\u00ebnohen p\u00ebr vakuumin n\u00eb t\u00eb ardhmen. N\u00eb \u00abTimescale\u00bb, p\u00ebr shkak se keni chunks, mund t\u00eb hiqni direkt. Thjesht i thoni skedarit, i cili ndodhet n\u00eb big data: \u00abFshij!\u00bb<\/p>\n<p>\u00abTimescale\u00bb thjesht e kupton se ai chunk nuk ekziston m\u00eb. Dhe pasi ai integrohet n\u00eb planifikuesin e k\u00ebrkesave, ai kap kushtet tuaja n\u00eb select ose n\u00eb operacione t\u00eb tjera dhe menj\u00ebher\u00eb kupton se ai chunk nuk ekziston m\u00eb \u2013 \u00abNuk do t\u00eb shkoj m\u00eb atje!\u00bb (t\u00eb dh\u00ebnat mungojn\u00eb). K\u00ebshtu, skanimi i tabel\u00ebs z\u00ebvend\u00ebsohet me fshirjen e skedarit binar, prandaj kjo \u00ebsht\u00eb e shpejt\u00eb.<\/p>\n<p><b>A:<\/b> \u2013 Kemi trajtuar tem\u00ebn e jo-SQL. Sa kam kuptuar, \u00abZabbix\u00bb s'ka nevoj\u00eb t\u00eb modifikoj\u00eb t\u00eb dh\u00ebnat shum\u00eb, dhe gjith\u00e7ka \u00ebsht\u00eb nj\u00eb lloj logu. A \u00ebsht\u00eb e mundur t\u00eb p\u00ebrdoren DB-specializuara, t\u00eb cilat nuk mund t\u00eb ndryshojn\u00eb t\u00eb dh\u00ebnat e tyre, por n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb t\u00eb ruajn\u00eb, akumullojn\u00eb dhe ofrojn\u00eb shum\u00eb m\u00eb shpejt \u2013 Clickhouse, p\u00ebr shembull, di\u00e7ka n\u00eb m\u00ebnyr\u00eb kafkiane?.. Kafka gjithashtu \u00ebsht\u00eb nj\u00eb log! A \u00ebsht\u00eb e mundur t\u00eb integrohen ndonj\u00ebher\u00eb?<\/p>\n<p><b>AG:<\/b> \u2013 Ekstraktimi mund t\u00eb b\u00ebhet. Ne kemi nj\u00eb \u00abfeature\u00bb t\u00eb caktuar q\u00eb nga versioni 3.4: ju mund t\u00eb shkruani n\u00eb skedar\u00eb t\u00eb gjith\u00eb skedar\u00ebt historik\u00eb, ngjarjet, gjith\u00e7ka tjet\u00ebr; dhe m\u00eb pas me ndihm\u00ebn e ndonj\u00eb procesori d\u00ebrgoni n\u00eb \u00e7do DB tjet\u00ebr. N\u00eb t\u00eb v\u00ebrtet\u00eb shum\u00eb njer\u00ebz e ristrukturojn\u00eb dhe shkruajn\u00eb drejtp\u00ebrdrejt n\u00eb DB. Historik\u00eb-synkronizuesit e shkruajn\u00eb gjith\u00e7ka n\u00eb skedar\u00eb, rotullojn\u00eb k\u00ebta skedar\u00eb dhe k\u00ebshtu me radh\u00eb, dhe k\u00ebt\u00eb mund ta kaloni n\u00eb \u00abClickhouse\u00bb. Nuk mund t\u00eb them asgj\u00eb p\u00ebr planet, por ndoshta mb\u00ebshtetje e m\u00ebtejshme p\u00ebr zgjidhjet NoSQL (t\u00eb tilla si \u00abClickhouse\u00bb) do t\u00eb vazhdoj\u00eb.<\/p>\n<p><b>A:<\/b> \u2013 Pra, a \u00ebsht\u00eb e mundur t\u00eb shkurtohet plot\u00ebsisht PostgreSQL?<\/p>\n<p><b>AG:<\/b> \u2013 Sigurisht, pjesa m\u00eb e v\u00ebshtir\u00eb n\u00eb \u00abZabbix\u00bb jan\u00eb tabelat historike, t\u00eb cilat krijojn\u00eb m\u00eb shum\u00eb probleme, dhe ngjarjet. N\u00eb k\u00ebt\u00eb rast, n\u00ebse nuk do t\u00eb ruani ngjarjet p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb dhe do t\u00eb mbani historin\u00eb me trendet n\u00eb ndonj\u00eb depo m\u00eb t\u00eb shpejt\u00eb, at\u00ebher\u00eb n\u00eb p\u00ebrgjith\u00ebsi nuk do t\u00eb ket\u00eb probleme, mendoj.<\/p>\n<p><b>A:<\/b> \u2013 A mund t\u00eb vler\u00ebsoni, sa m\u00eb shpejt do t\u00eb funksionoj\u00eb gjith\u00e7ka, n\u00ebse kalojm\u00eb n\u00eb \u00abClickhouse\u00bb, p\u00ebr shembull?<\/p>\n<p><b>AG:<\/b> \u2013 Nuk e kam testuar. Mendoj se t\u00eb pakt\u00ebn numrat e nj\u00ebjt\u00eb mund t\u00eb arrihen mjaft leht\u00eb, duke marr\u00eb parasysh q\u00eb \u00abClickhouse\u00bb ka nd\u00ebrfaqen e tij, por nuk mund t\u00eb them me siguri. M\u00eb mir\u00eb t\u00eb testoni. Gjith\u00e7ka varet nga konfigurimi: sa host-e keni dhe k\u00ebshtu me radh\u00eb. Futja \u00ebsht\u00eb nj\u00eb gj\u00eb, por duhet gjithashtu t\u00eb merrni k\u00ebto t\u00eb dh\u00ebna \u2013 me Grafana ose di\u00e7ka tjet\u00ebr.<\/p>\n<p><b>A:<\/b> \u2013 Pra, b\u00ebhet fjal\u00eb p\u00ebr nj\u00eb luft\u00eb t\u00eb barabart\u00eb, jo nj\u00eb avantazh t\u00eb madh t\u00eb k\u00ebtyre DB-ve t\u00eb shpejta?<\/p>\n<p><b>AG:<\/b> \u2013 Mendoj se kur t\u00eb integrojm\u00eb, do t\u00eb kemi teste m\u00eb t\u00eb sakta.<\/p>\n<p><b>A:<\/b> \u2013 Ku e la i vjetri RRD? \u00c7far\u00eb e detyroi kalimin n\u00eb bazat e t\u00eb dh\u00ebnave SQL? Fillimisht, t\u00eb gjitha metrikat jan\u00eb mbledhur n\u00eb RRD.<\/p>\n<p><b>AG:<\/b> \u2013 N\u00eb \u00abZabbix\u00bb RRD, ndoshta ka qen\u00eb n\u00eb nj\u00eb version shum\u00eb t\u00eb lasht\u00eb. Kan\u00eb ekzistuar gjithmon\u00eb bazat e t\u00eb dh\u00ebnave SQL \u2013 qasja klasike. Qasja klasike \u00ebsht\u00eb MySQL, PostgreSQL (jan\u00eb ekzistuar prej koh\u00ebsh). Ne kemi nj\u00eb nd\u00ebrfaqe t\u00eb p\u00ebrbashk\u00ebt p\u00ebr bazat e t\u00eb dh\u00ebnave SQL dhe RRD, q\u00eb pothuajse kurr\u00eb nuk e kemi p\u00ebrdorur.<\/p>\n<p><img decoding=\"async\" alt=\"HighLoad++, Andi Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe ndarje natyrale\" src=\"\/wp-content\/uploads\/2020\/01\/ac5f02494c63983601cc09c0b22e722b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<center><div class=\"youtube-placeholder\" data-id=\"umRk94j5M8o\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/umRk94j5M8o\/hqdefault.jpg\" alt=\"Luaj videon\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><\/p>\n<h3>Pak reklam\u00eb \ud83d\ude42<\/h3>\n<p>\nFaleminderit q\u00eb q\u00ebndroni me ne. Ju p\u00eblqejn\u00eb artikujt tan\u00eb? D\u00ebshironi t\u00eb shihni m\u00eb shum\u00eb materiale interesante? Na mb\u00ebshtetni duke b\u00ebr\u00eb nj\u00eb porosi ose duke na rekomanduar njohurive tuaj, <noindex><a rel=\"nofollow\" href=\"https:\/\/ua-hosting.company\/cloudvps\/nl\">VPS cloud p\u00ebr zhvillues nga $4.99<\/a><\/noindex>, <b>nj\u00eb analog unik i server\u00ebve entry-level, i nd\u00ebrtuar p\u00ebr ju:<\/b> <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/company\/ua-hosting\/blog\/347386\/\">E gjith\u00eb e v\u00ebrteta rreth VPS (KVM) E5-2697 v3 (6 Nuclea) 10GB DDR4 480GB SSD 1Gbps nga $19 ose si ta ndajm\u00eb sakt\u00ebsisht serverin?<\/a><\/noindex> (disponohen variante me RAID1 dhe RAID10, deri n\u00eb 24 b\u00ebrthama dhe deri n\u00eb 40GB DDR4).<\/p>\n<p><b>Dell R730xd dyfish m\u00eb i lir\u00eb n\u00eb qendr\u00ebn e t\u00eb dh\u00ebnave Equinix Tier IV n\u00eb Amsterdam?<\/b> Vet\u00ebm k\u00ebtu <b><noindex><a rel=\"nofollow\" href=\"https:\/\/ua-hosting.company\/serversnl\">2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TB nga $199<\/a><\/noindex> n\u00eb Holand\u00eb! <b>Dell R420 \u2014 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB \u2014 nga $99!<\/b><\/b> Lexoni rreth <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/company\/ua-hosting\/blog\/329618\/\">Si t\u00eb nd\u00ebrtosh nj\u00eb infrastruktur\u00eb t\u00eb klas\u00ebs korporative me p\u00ebrdorimin e server\u00ebve Dell R730xd E5-2650 v4 me \u00e7mim prej 9000 euro p\u00ebr pak para?<\/a><\/noindex><br \/>\n<br \/>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/ua-hosting\/blog\/485470\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041c\u044b \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0440\u0430\u0431\u043e\u0442\u0443 Zabbix \u0441 \u0431\u0430\u0437\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 TimescaleDB \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 backend. \u041f\u043e\u043a\u0430\u0436\u0435\u043c, \u043a\u0430\u043a \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0441 \u043d\u0443\u043b\u044f \u0438 \u043a\u0430\u043a \u043c\u0438\u0433\u0440\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u0441 PostgreSQL. \u0422\u0430\u043a\u0436\u0435 \u043f\u0440\u0438\u0432\u0435\u0434\u0435\u043c \u0441\u0440\u0430\u0432\u043d\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0435 \u0442\u0435\u0441\u0442\u044b \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u0434\u0432\u0443\u0445 \u043a\u043e\u043d\u0444\u0438\u0433\u0443\u0440\u0430\u0446\u0438\u0439. HighLoad++ Siberia 2019. \u0417\u0430\u043b \u00ab\u0422\u043e\u043c\u0441\u043a\u00bb. 24 \u0438\u044e\u043d\u044f, 16:00. \u0422\u0435\u0437\u0438\u0441\u044b \u0438 \u043f\u0440\u0435\u0437\u0435\u043d\u0442\u0430\u0446\u0438\u044f. \u0421\u043b\u0435\u0434\u0443\u044e\u0449\u0430\u044f \u043a\u043e\u043d\u0444\u0435\u0440\u0435\u043d\u0446\u0438\u044f HighLoad++ \u043f\u0440\u043e\u0439\u0434\u0435\u0442 6 \u0438 7 \u0430\u043f\u0440\u0435\u043b\u044f 2020 \u0433\u043e\u0434\u0430 \u0432 \u0421\u0430\u043d\u043a\u0442-\u041f\u0435\u0442\u0435\u0440\u0431\u0443\u0440\u0433\u0435. \u041f\u043e\u0434\u0440\u043e\u0431\u043d\u043e\u0441\u0442\u0438 \u0438 \u0431\u0438\u043b\u0435\u0442\u044b \u043f\u043e [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-55734","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.0.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041c\u044b \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0440\u0430\u0431\u043e\u0442\u0443 Zabbix \u0441 \u0431\u0430\u0437\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 TimescaleDB \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 backend. \u041f\u043e\u043a\u0430\u0436\u0435\u043c, \u043a\u0430\u043a \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0441 \u043d\u0443\u043b\u044f \u0438 \u043a\u0430\u043a \u043c\u0438\u0433\u0440\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u0441 PostgreSQL.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/highload-andrej-gushhin-zabbix-vysokaya-proizvoditelnost-i-nativnoe-partitsionirovanie\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.0.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47HighLoad++, \u0410\u043d\u0434\u0440\u0435\u0439 \u0413\u0443\u0449\u0438\u043d (Zabbix): \u0432\u044b\u0441\u043e\u043a\u0430\u044f \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u0438 \u043d\u0430\u0442\u0438\u0432\u043d\u043e\u0435 \u043f\u0430\u0440\u0442\u0438\u0446\u0438\u043e\u043d\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041c\u044b \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0440\u0430\u0431\u043e\u0442\u0443 Zabbix \u0441 \u0431\u0430\u0437\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 TimescaleDB \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 backend. \u041f\u043e\u043a\u0430\u0436\u0435\u043c, \u043a\u0430\u043a \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0441 \u043d\u0443\u043b\u044f \u0438 \u043a\u0430\u043a \u043c\u0438\u0433\u0440\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u0441 PostgreSQL.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/highload-andrej-gushhin-zabbix-vysokaya-proizvoditelnost-i-nativnoe-partitsionirovanie\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-01-26T21:00:00+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-18T11:03:52+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47HighLoad++, Andrey Gushchin (Zabbix): performanc\u00eb e lart\u00eb dhe particionim natyror | ProHoster","description":"Ne do t\u00eb shqyrtojm\u00eb pun\u00ebn e Zabbix me baz\u00ebn e t\u00eb dh\u00ebnave TimescaleDB si backend. Do t\u00eb tregojm\u00eb si ta nisni nga e para dhe si t\u00eb migroni nga PostgreSQL.","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/highload-andrej-gushhin-zabbix-vysokaya-proizvoditelnost-i-nativnoe-partitsionirovanie","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47HighLoad++, \u0410\u043d\u0434\u0440\u0435\u0439 \u0413\u0443\u0449\u0438\u043d (Zabbix): \u0432\u044b\u0441\u043e\u043a\u0430\u044f \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u0438 \u043d\u0430\u0442\u0438\u0432\u043d\u043e\u0435 \u043f\u0430\u0440\u0442\u0438\u0446\u0438\u043e\u043d\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 | ProHoster","og:description":"\u041c\u044b \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u0440\u0430\u0431\u043e\u0442\u0443 Zabbix \u0441 \u0431\u0430\u0437\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 TimescaleDB \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 backend. \u041f\u043e\u043a\u0430\u0436\u0435\u043c, \u043a\u0430\u043a \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0441 \u043d\u0443\u043b\u044f \u0438 \u043a\u0430\u043a \u043c\u0438\u0433\u0440\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u0441 PostgreSQL.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/highload-andrej-gushhin-zabbix-vysokaya-proizvoditelnost-i-nativnoe-partitsionirovanie","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-01-26T21:00:00+00:00","article:modified_time":"2020-02-18T11:03:52+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"55734","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 19:37:39","updated":"2022-09-28 01:51:35","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/55734","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=55734"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/55734\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=55734"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=55734"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=55734"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}