{"id":94153,"date":"2020-09-13T19:42:30","date_gmt":"2020-09-13T17:42:30","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/victoriametrics-i-monitoring-privatnyh-oblakov-pavel-kolobaev"},"modified":"2020-09-13T19:42:30","modified_gmt":"2020-09-13T17:42:30","slug":"victoriametrics-i-monitoring-privatnyh-oblakov-pavel-kolobaev","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/victoriametrics-i-monitoring-privatnyh-oblakov-pavel-kolobaev","title":{"rendered":"VictoriaMetrics dhe monitorimi i \u043e\u0431\u043b\u0430qeve \u043f\u0440\u0438\u0432\u0430\u0442\u043d\u044b\u0435. Pavel Kolobaev","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"VictoriaMetrics<\/p>\n

<\/p>\n

VictoriaMetrics \u2014 nj\u00eb DBMS e shpejt\u00eb dhe e shkall\u00ebzuar p\u00ebr ruajtjen dhe p\u00ebrpunimin e t\u00eb dh\u00ebnave n\u00eb form\u00ebn e serive temporale (nj\u00eb regjistrim krijon koh\u00ebn dhe nj\u00eb grup vlerash p\u00ebrkat\u00ebse p\u00ebr at\u00eb koh\u00eb, p\u00ebr shembull, t\u00eb marra n\u00ebp\u00ebrmjet pyetjeve periodike t\u00eb gjendjeve t\u00eb sensor\u00ebve ose t\u00eb grumbullimit t\u00eb metrikave).<\/p>\n

<\/a><\/noindex>
\n

\r\n \"Luaj\r\n
<\/div>\r\n <\/div><\/center><\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

M\u00eb quajn\u00eb Kolobajev Pavel. DevOps, SRE, LeroyMerlin, gjith\u00e7ka si kod \u2013 kjo \u00ebsht\u00eb gjith\u00e7ka p\u00ebr ne: p\u00ebr mua dhe p\u00ebr koleg\u00ebt e tjer\u00eb t\u00eb LeroyMerlin.<\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

https:\/\/bit.ly\/3jf1fIK<\/a><\/noindex><\/p>\n

<\/p>\n

Ka nj\u00eb cloud t\u00eb bazuar n\u00eb OpenStack. Atje ka nj\u00eb lidhje t\u00eb vog\u00ebl p\u00ebr radarin teknik. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

E nd\u00ebrtuar mbi baz\u00ebn e harduerit Kubernetes, si dhe mbi t\u00eb gjitha sh\u00ebrbimet p\u00ebrkat\u00ebse t\u00eb OpenStack dhe regjistrimit. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Skema ishte k\u00ebshtu gjat\u00eb zhvillimit. Kur e zhvilluam, kishim nj\u00eb operator Prometheus, i cili ruante t\u00eb dh\u00ebnat brenda vet\u00eb klas\u00ebs K8s. Ai automatiksht gjen at\u00eb q\u00eb duhet t\u00eb shkrapohet dhe e sistemon, th\u00ebn\u00eb ndryshe. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

T\u00eb gjitha t\u00eb dh\u00ebnat duhet t'i transferojm\u00eb jasht\u00eb klas\u00ebs Kubernetes, sepse n\u00ebse ndodh di\u00e7ka, ne duhet t\u00eb kuptojm\u00eb se \u00e7far\u00eb dhe ku. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Zgjidhja e par\u00eb \u2013 ne p\u00ebrdorim federation, kur kemi nj\u00eb Prometheus t\u00eb jasht\u00ebm, kur qarkullojm\u00eb n\u00eb klas\u00ebn Kubernetes p\u00ebrmes mekanizmit federation.<\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Por k\u00ebtu dalin disa probleme. N\u00eb rastin ton\u00eb problemi filloi kur pat\u00ebm 250,000 metrika, dhe kur arrit\u00ebm n\u00eb 400,000 metrika, kuptuam se nuk mund t\u00eb punojm\u00eb k\u00ebshtu. Ne rrit\u00ebm scrape_timeout n\u00eb 25 sekonda.<\/p>\n

<\/p>\n

Pse na u desh t\u00eb b\u00ebnim k\u00ebt\u00eb? Prometheus fillon t\u00eb num\u00ebroj\u00eb koh\u00ebn e skadimit q\u00eb nga momenti i marrjes s\u00eb t\u00eb dh\u00ebnave. Dhe nuk ka r\u00ebnd\u00ebsi se t\u00eb dh\u00ebnat ende po l\u00ebvizin. N\u00ebse brenda k\u00ebtij intervali t\u00eb caktuar kohor t\u00eb dh\u00ebnat nuk jan\u00eb shkarkuar dhe sesioni nuk do t\u00eb mbyllet p\u00ebrmes http, at\u00ebher\u00eb sesioni konsiderohet d\u00ebshtim dhe t\u00eb dh\u00ebnat nuk arrijn\u00eb n\u00eb Prometheus.<\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Grafik\u00ebt q\u00eb t\u00eb gjith\u00eb i njohim, t\u00eb cilat marrim kur mungojn\u00eb disa t\u00eb dh\u00ebna. Grafik\u00ebt jan\u00eb t\u00eb shqyer dhe kjo nuk na k\u00ebnaq. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Opcioni tjet\u00ebr \u2013 \u00ebsht\u00eb sharding bazuar n\u00eb dy Prometheus t\u00eb ndrysh\u00ebm p\u00ebrmes t\u00eb nj\u00ebjtit mekaniz\u00ebm federation.<\/p>\n

<\/p>\n

P\u00ebr shembull, thjesht t\u00eb merret dhe t\u00eb shkarkohet sipas emrit. Kjo mund t\u00eb p\u00ebrdoret gjithashtu, por ne vendos\u00ebm t\u00eb l\u00ebvizim p\u00ebrpara. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Tani do na duhet t\u00eb p\u00ebrpunojm\u00eb k\u00ebto sharde n\u00eb nj\u00ebfar\u00eb m\u00ebnyre. Mund t\u00eb p\u00ebrdorni promxy, i cili shkon n\u00eb zon\u00ebn e shardit, dhe b\u00ebn shum\u00ebfishimin e t\u00eb dh\u00ebnave. Ai punon me dy sharde si me nj\u00eb pik\u00eb t\u00eb vetme hyrje. Kjo mund t\u00eb realizohet p\u00ebrmes promxy, por \u00ebsht\u00eb ende shum\u00eb e komplikuar. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Opcioni i par\u00eb \u2013 ne duam t\u00eb heqim dor\u00eb nga mekanizmi federation, sepse \u00ebsht\u00eb shum\u00eb i ngadalsh\u00ebm. <\/p>\n

<\/p>\n

Zhvilluesit e Prometheus thon\u00eb qart\u00eb: \u00abDjem, p\u00ebrdorni databaza t\u00eb tjera si TimescaleDB, sepse ne nuk do t\u00eb mb\u00ebshtesim ruajtjen afatgjat\u00eb t\u00eb metrikave\u00bb. Kjo nuk \u00ebsht\u00eb detyra e tyre. \"VictoriaMetrics<\/p>\n

<\/p>\n

Ne e shkruajm\u00eb n\u00eb let\u00ebr se na duhet nj\u00eb eksport jasht\u00eb, p\u00ebr t\u00eb mos ruajtur gjith\u00e7ka n\u00eb nj\u00eb vend. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Disavantazhi i dyt\u00eb \u2013 \u00ebsht\u00eb konsumimi i memories. Po, e kuptoj q\u00eb shumica do t\u00eb thon\u00eb se n\u00eb vitin 2020 disa gigabajt memorie jan\u00eb nj\u00eb \u00e7mim i vog\u00ebl, por megjithat\u00eb. <\/p>\n

<\/p>\n

Tani kemi nj\u00eb ambient dev dhe prodhimi. N\u00eb dev, ajo ka rreth 9 gigabajt p\u00ebr 350,000 metrika. N\u00eb prodhim \u2013 rreth 14 gigabajt p\u00ebr 780,000 metrika. N\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, koha e ruajtjes \u00ebsht\u00eb vet\u00ebm 30 minuta. Kjo \u00ebsht\u00eb keq. Dhe tani do ta shpjegoj pse.<\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Ne b\u00ebjm\u00eb nj\u00eb llogari, pra, n\u00eb 1.5 milion metrika, dhe ne jemi tashm\u00eb af\u00ebr, n\u00eb faz\u00ebn e projektimit, marrim 35-37 gigabajt memorie. Por tashm\u00eb p\u00ebr 4 milion metrika k\u00ebrkohet rreth 90 gigabajt memorie. Pra, kjo \u00ebsht\u00eb llogaritur sipas formul\u00ebs q\u00eb ofrojn\u00eb zhvilluesit e Prometheus. Ne kemi par\u00eb korelacionin dhe kuptuam se nuk duam t\u00eb paguajm\u00eb disa miliona p\u00ebr nj\u00eb server thjesht p\u00ebr monitorim. <\/p>\n

<\/p>\n

Numri i makinerive nuk do t\u00eb rritet vet\u00ebm, ne gjithashtu po monitorojm\u00eb vet\u00eb makinat virtuale. Prandaj, sa m\u00eb shum\u00eb makina virtuale, aq m\u00eb shum\u00eb metrika t\u00eb ndryshme etj. Do t\u00eb kemi nj\u00eb rritje t\u00eb ve\u00e7ant\u00eb t\u00eb klas\u00ebs son\u00eb n\u00eb lidhje me metrikat.<\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Me hap\u00ebsir\u00ebn e diskut k\u00ebtu nuk \u00ebsht\u00eb aq keq, por do t\u00eb deshiim t\u00eb p\u00ebrmir\u00ebsojm\u00eb. Ne kemi marr\u00eb p\u00ebr 15 dit\u00eb gjithsej 120 gigabajt, nga t\u00eb cilat 100 jan\u00eb t\u00eb dh\u00ebna t\u00eb kompresuara, 20 jan\u00eb t\u00eb dh\u00ebna t\u00eb pakompresuara, por gjithmon\u00eb d\u00ebshirojm\u00eb m\u00eb pak. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

P\u00ebr rrjedhoj\u00eb, ne shkruajm\u00eb nj\u00eb pik\u00eb tjet\u00ebr \u2013 \u00ebsht\u00eb konsumimi i madh i burimeve, t\u00eb cilat d\u00ebshirojm\u00eb t'i kursejm\u00eb, sepse nuk duam q\u00eb klasa jon\u00eb e monitorimit t\u00eb konsumoj\u00eb m\u00eb shum\u00eb burime sesa klasa jon\u00eb q\u00eb merret me menaxhimin e OpenStack.<\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Ka nj\u00eb tjet\u00ebr disavantazh t\u00eb Prometheus, q\u00eb ne e kemi zbuluar, \u00ebsht\u00eb ndonj\u00eb kufizim n\u00eb memorie. Me Prometheus k\u00ebtu \u00ebsht\u00eb duke u b\u00ebr\u00eb shum\u00eb m\u00eb keq, sepse ai sakt\u00ebsisht nuk ka k\u00ebto kufizime. P\u00ebrdorimi i kufizimit n\u00eb docker gjithashtu nuk \u00ebsht\u00eb nj\u00eb zgjidhje. N\u00ebse ndonj\u00ebher\u00eb RAFT bie dhe atje jan\u00eb 20-30 gigabajt, at\u00ebher\u00eb do t\u00eb rifilloj\u00eb shum\u00eb ngadal\u00eb. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Kjo \u00ebsht\u00eb nj\u00eb arsye tjet\u00ebr pse Prometheus nuk na p\u00ebrshtatet, pra, nuk mund t\u00eb kufizohet konsumimi i memories. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Mund t\u00eb krijohet nj\u00eb skem\u00eb t\u00eb till\u00eb. Kjo skem\u00eb na nevojitet p\u00ebr t\u00eb organizuar nj\u00eb cluster HA. Ne duam q\u00eb metrike tona t\u00eb jen\u00eb t\u00eb aksesueshme gjithmon\u00eb dhe kudo, edhe kur serveri q\u00eb ruan k\u00ebto metrika bie. K\u00ebshtu q\u00eb do t\u00eb duhet t\u00eb nd\u00ebrtojm\u00eb nj\u00eb skem\u00eb t\u00eb till\u00eb. <\/p>\n

<\/p>\n

Kjo skem\u00eb thot\u00eb se do t\u00eb kemi dyfishim t\u00eb shard-\u00ebve, dhe gjithashtu dyfishim t\u00eb shpenzimeve t\u00eb resurseve t\u00eb p\u00ebrdorura. Mund t\u00eb shkall\u00ebzohet pothuajse horizontalisht, por megjithat\u00eb, konsumi i burimeve do t\u00eb jet\u00eb i lart\u00eb. <\/p>\n

<\/p>\n

\"VictoriaMetrics<\/p>\n

<\/p>\n

Mang\u00ebsit\u00eb sipas rendit si\u00e7 i kemi shkruar:<\/p>\n

<\/p>\n