Tiwona momwe Zabbix imagwirira ntchito ndi database ya TimescaleDB ngati backend. Tikuwonetsani momwe mungayambitsire kuchokera pachiyambi komanso momwe mungasamukire kuchokera ku PostgreSQL. Tidzaperekanso zoyeserera zofananira za masinthidwe awiriwo.
HighLoad ++ Siberia 2019. Tomsk Hall. June 24, 16:00. Izi ndi . Msonkhano wotsatira wa HighLoad ++ udzachitika pa Epulo 6 ndi 7, 2020 ku St. Tsatanetsatane ndi matikiti .
Andrey Gushchin (pano - AG): - Ndine injiniya wothandizira zaukadaulo wa ZABBIX (omwe amatchedwa "Zabbix"), mphunzitsi. Ndakhala ndikugwira ntchito yothandizira ukadaulo kwa zaka zopitilira 6 ndipo ndakhala ndikuchita bwino kwambiri. Lero ndilankhula za machitidwe omwe TimescaleDB angapereke poyerekeza ndi PostgreSQL 10 yanthawi zonse.
Zovuta zapantchito zapamwamba: kuyambira kusonkhanitsa deta mpaka kuyeretsa deta
Poyambira, pali zovuta zina zomwe kachitidwe kowunikira kalikonse kamayang'anizana nazo. Vuto loyamba la zokolola ndikusonkhanitsa ndikukonza deta mwachangu.
Dongosolo labwino loyang'anira liyenera kulandira mwachangu, munthawi yake, ndikulikonza molingana ndi mawu oyambitsa, ndiye kuti, lizikonza motsatira njira zina (izi ndizosiyana m'machitidwe osiyanasiyana) ndikuzisunga mu nkhokwe kuti mugwiritse ntchito izi mu m'tsogolo.
Vuto lachiwiri la magwiridwe antchito ndikusunga mbiri. Sungani mu nkhokwe nthawi zambiri ndipo mukhale ndi mwayi wofikira mwachangu komanso mosavuta ku ma metrics awa omwe adasonkhanitsidwa pakapita nthawi. Chofunika kwambiri ndi chakuti izi ndizosavuta kuzipeza, zigwiritseni ntchito mu malipoti, ma graph, zoyambitsa, pazigawo zina, zochenjeza, ndi zina zotero.
Vuto lachitatu la magwiridwe antchito ndikuchotsa mbiri yakale, ndiye kuti, mukafika pomwe simukufunika kusunga ma metric omwe asonkhanitsidwa pazaka 5 (ngakhale miyezi kapena miyezi iwiri). Ma netiweki ena adachotsedwa, kapena ena, ma metrics sakufunikanso chifukwa ndi akale ndipo sasonkhanitsidwa. Zonsezi ziyenera kuyeretsedwa kuti database yanu isakule kwambiri. Kawirikawiri, kuchotsa mbiri yakale nthawi zambiri kumakhala kuyesa kwakukulu kwa kusungirako - nthawi zambiri kumakhala ndi zotsatira zamphamvu kwambiri pa ntchito.
Momwe mungathetsere mavuto a caching?
Tsopano ndilankhula mwachindunji za Zabbix. Ku Zabbix, kuyimba koyamba ndi kwachiwiri kumathetsedwa pogwiritsa ntchito caching.
Kusonkhanitsa ndi kukonza deta - Timagwiritsa ntchito RAM kusunga deta yonseyi. Deta iyi tsopano idzakambidwa mwatsatanetsatane.
Komanso kumbali ya database pali zosungirako zosankha zazikulu - ma graph ndi zinthu zina.
Kusunga kumbali ya seva ya Zabbix palokha: tili ndi ConfigurationCache, ValueCache, HistoryCache, TrendsCache. Ndi chiyani?
ConfigurationCache ndiye cache yayikulu momwe timasungira ma metric, makamu, zinthu za data, zoyambitsa; Chilichonse chomwe mungafune kuti mukonzeretu, sonkhanitsani deta, kuchokera komwe makamu azitolera, pafupipafupi. Zonsezi zimasungidwa mu ConfigurationCache kuti musapite ku database ndikupanga mafunso osafunikira. Seva ikayamba, timasintha cache iyi (kupanga) ndikuyisintha nthawi ndi nthawi (malingana ndi makonzedwe a kasinthidwe).
Kusungidwa mu Zabbix. Kusonkhanitsa deta
Apa chithunzicho ndi chachikulu kwambiri:
Zomwe zili mu ndondomekoyi ndi osonkhanitsa awa:
Izi ndi njira zochitira misonkhano yokha, "oponya voti" osiyanasiyana omwe ali ndi udindo wamitundu yosiyanasiyana yamisonkhano. Amasonkhanitsa zidziwitso kudzera pa icmp, ipmi, ndi ma protocol osiyanasiyana ndikusamutsa zonse ku preprocessing.
PreProcessing HistoryCache
Komanso, ngati tawerengera zinthu za data (omwe amadziwika bwino ndi Zabbix amadziwa), ndiko kuti, kuwerengera, kusonkhanitsa deta, timazitenga mwachindunji kuchokera ku ValueCache. Ine ndikuuzani inu momwe izo zidzadzazidwa mtsogolo. Osonkhanitsa onsewa amagwiritsa ntchito ConfigurationCache kuti alandire ntchito zawo ndikuzipereka ku preprocessing.
Preprocessing imagwiritsanso ntchito ConfigurationCache kuti ipeze njira zosinthira ndikusintha detayi m'njira zosiyanasiyana. Kuyambira ku mtundu 4.2, tachisuntha kupita ku proxy. Izi ndizothandiza kwambiri, chifukwa preprocessing yokha ndi ntchito yovuta. Ndipo ngati muli ndi Zabbix yayikulu kwambiri, yokhala ndi zinthu zambiri za data komanso kusonkhanitsa pafupipafupi, ndiye kuti izi zimathandizira kwambiri ntchitoyi.
Chifukwa chake, titakonza izi mwanjira ina pogwiritsa ntchito preprocessing, timazisunga mu HistoryCache kuti tikonzenso. Izi zimamaliza kusonkhanitsa deta. Timapita ku njira yayikulu.
ntchito syncer mbiri
Njira yayikulu mu Zabbix (popeza ndi zomangamanga za monolithic) ndi syncer ya Mbiri. Iyi ndiye njira yayikulu yomwe imachita makamaka ndi kukonza kwa ma atomiki pa data iliyonse, ndiye kuti, mtengo uliwonse:
- mtengo umabwera (zimatengera kuchokera ku HistoryCache);
- imayang'ana mu Configuration syncer: ngati pali zoyambitsa kuwerengera - kuziwerengera;
ngati pali - imapanga zochitika, imapanga kukwera kuti apange chenjezo, ngati kuli kofunikira malinga ndi kasinthidwe; - zolemba zomwe zimayambitsa kukonzedwa kotsatira, kuphatikizika; ngati muphatikiza pa ola lomaliza ndi zina zotero, mtengo uwu umakumbukiridwa ndi ValueCache kuti musapite ku mbiri yakale; Choncho, ValueCache imadzazidwa ndi deta yofunikira yomwe ili yofunikira kuwerengera zoyambitsa, zinthu zowerengedwa, ndi zina zotero;
- ndiye syncer ya Mbiri imalemba zonse ku database;
- database imawalembera ku diski - apa ndipamene ntchito yokonza imathera.
Nawonsomba. Kusunga
Kumbali ya database, mukafuna kuwona ma graph kapena malipoti ena pazochitika, pali ma cache osiyanasiyana. Koma mu lipoti ili sindiyankhula za iwo.
Kwa MySQL pali Innodb_buffer_pool, ndi ma cache osiyanasiyana omwe angathenso kukhazikitsidwa.
Koma zazikulu ndi izi:
- share_buffers;
- ogwira_cache_size;
- adagawana_dziwe.
Pama database onse, ndinanena kuti pali zosungira zina zomwe zimakulolani kusunga mu RAM zomwe zimafunikira nthawi zambiri pamafunso. Iwo ali ndi matekinoloje awo pa izi.
About Database Performance
Chifukwa chake, pali malo opikisana, ndiko kuti, seva ya Zabbix imasonkhanitsa deta ndikulemba. Mukayambiranso, imawerengeranso mbiri yakale kuti mudzaze ValueCache ndi zina zotero. Pano mukhoza kukhala ndi zolemba ndi malipoti omwe amagwiritsa ntchito Zabbix API, yomwe imamangidwa pa intaneti. Zabbix API imalowa m'dawunilodi ndikulandira deta yofunikira kuti ipeze ma grafu, malipoti, kapena mtundu wina wa mndandanda wa zochitika, mavuto aposachedwa.
Komanso njira yodziwika bwino yowonera ndi Grafana, yomwe ogwiritsa ntchito athu amagwiritsa ntchito. Kutha kulowa mwachindunji kudzera mu Zabbix API komanso kudzera mu database. Zimapanganso mpikisano wina wopezera deta: kukonzanso bwino, koyenera kwa nkhokwe kumafunika kuti zigwirizane ndi kuperekedwa mofulumira kwa zotsatira ndi kuyesa.
Kuchotsa mbiri. Zabbix ali ndi Wosunga Nyumba
Kuitana kwachitatu komwe kumagwiritsidwa ntchito ku Zabbix ndikuchotsa mbiri yakale pogwiritsa ntchito Wosunga Nyumba. Woyang'anira nyumba amatsata zoikamo zonse, ndiye kuti, zinthu zathu za data zimawonetsa nthawi yayitali yosungira (m'masiku), nthawi yosungiramo zinthu, komanso kusintha kwakusintha.
Sindinalankhule za TrendCache, yomwe timawerengera pa ntchentche: deta ifika, timayiphatikiza kwa ola limodzi (makamaka izi ndi manambala a ola lomaliza), kuchuluka kwake ndi pafupifupi / kuchepera ndipo timalemba kamodzi pa ola limodzi. Table of the Dynamics of Changes ("Trends") . "Woyang'anira nyumba" akuyamba ndikuchotsa deta kuchokera ku database pogwiritsa ntchito zosankha zokhazikika, zomwe sizigwira ntchito nthawi zonse.
Kodi mungamvetse bwanji kuti sizothandiza? Mutha kuwona chithunzi chotsatira pazithunzi zamachitidwe amkati:
Syncher Yanu Yambiri imakhala yotanganidwa nthawi zonse (chithunzi chofiyira). Ndipo graph "yofiira" yomwe imapita pamwamba. Uyu ndi "Woyang'anira Nyumba" yemwe amayamba ndikudikirira kuti database ichotse mizere yonse yomwe yafotokoza.
Tiyeni titenge ID Yachinthu: muyenera kuchotsa 5 zikwi zapitazo; ndithudi, ndi indexes. Koma nthawi zambiri deta imakhala yayikulu kwambiri - nkhokwe imawerengabe kuchokera ku disk ndikuyiyika mu cache, ndipo iyi ndi ntchito yodula kwambiri pa database. Kutengera ndi kukula kwake, izi zimatha kuyambitsa zovuta zina.
Mutha kuletsa Wosunga Nyumba m'njira yosavuta - tili ndi mawonekedwe odziwika bwino apaintaneti. Zochunira mu Administration general (zokonda za "Housekeeper") timayimitsa kusunga m'nyumba kwa mbiri yamkati ndi zomwe zikuchitika. Chifukwa chake, Woyang'anira Nyumba sakuwongoleranso izi:
Kodi mungatani kenako? Munazimitsa, ma graph anu afika ... Ndi mavuto ena ati omwe angabwere pankhaniyi? Nβchiyani chingathandize?
Kugawa (gawo)
Nthawi zambiri izi zimakonzedwa mwanjira ina pa database iliyonse yaubale yomwe ndalemba. MySQL ili ndi ukadaulo wake. Koma zonse ndizofanana kwambiri zikafika ku PostgreSQL 10 ndi MySQL. Inde, pali kusiyana kwakukulu kwamkati momwe zonse zimagwiritsidwira ntchito komanso momwe zimakhudzira ntchito. Koma kawirikawiri, kupanga gawo latsopano nthawi zambiri kumabweretsa mavuto ena.
Kutengera kukhazikitsidwa kwanu (kuchuluka kwa deta komwe mumapanga tsiku limodzi), nthawi zambiri amakhazikitsa zochepera - izi ndi tsiku limodzi / batch, ndi "mayendedwe", kusintha kwakusintha - mwezi umodzi / batch yatsopano. Izi zitha kusintha ngati muli ndi khwekhwe lalikulu kwambiri.
Tinene nthawi yomweyo kukula kwa khwekhwe: mpaka 5 zikwi zatsopano pa sekondi imodzi (otchedwa nvps) - izi zidzaonedwa ngati "kukhazikitsa" kakang'ono. Avereji - kuchokera pa 5 mpaka 25 zikwi zikwi pamphindikati. Zonse zomwe zili pamwambapa ndizoyika kale zazikulu komanso zazikulu kwambiri zomwe zimafunikira kusanja koyenera kwa database.
Pakuyika kwakukulu, tsiku limodzi silingakhale labwino. Ine ndekha ndawonapo magawo pa MySQL a 1 gigabytes patsiku (ndipo pakhoza kukhala zambiri). Izi ndizochuluka kwambiri za deta, zomwe zingayambitse mavuto ena. Iyenera kuchepetsedwa.
Chifukwa chiyani muyenera kugawa?
Zomwe Partitioning imapereka, ndikuganiza kuti aliyense amadziwa, ndikugawa tebulo. Nthawi zambiri awa ndi mafayilo osiyana pa disk ndi zopempha za span. Imasankha gawo limodzi bwino kwambiri ngati lili gawo la magawo abwinobwino.
Kwa Zabbix, makamaka, imagwiritsidwa ntchito ndi osiyanasiyana, ndi osiyanasiyana, ndiko kuti, timagwiritsa ntchito timestamp (nambala yokhazikika, nthawi kuyambira chiyambi cha nthawi). Mumatchula chiyambi cha tsiku / kutha kwa tsiku, ndipo uku ndi kugawa. Chifukwa chake, ngati mukupempha deta yomwe ili ndi masiku awiri, zonse zimachotsedwa ku database mwachangu, chifukwa mumangofunika kutsitsa fayilo imodzi mu cache ndikuyibweza (osati tebulo lalikulu).
Ma database ambiri amafulumizitsanso kuyika (kuyika patebulo lamwana limodzi). Ndikulankhula mwachisawawa pakadali pano, koma izi ndizothekanso. Kugawa nthawi zambiri kumathandiza.
Elasticsearch ya NoSQL
Posachedwa, mu 3.4, tidagwiritsa ntchito yankho la NoSQL. Anawonjezera luso lolemba mu Elasticsearch. Mukhoza kulemba mitundu ina: mumasankha - kulemba manambala kapena zizindikiro zina; tili ndi zingwe zolemba, mutha kulemba zipika ku Elasticsearch... Mogwirizana ndi izi, mawonekedwe a intaneti apezanso Elasticsearch. Izi zimagwira ntchito bwino nthawi zina, koma pakadali pano zitha kugwiritsidwa ntchito.
NthawiDB. Hypertables
Kwa 4.4.2 tidasamalira chinthu chimodzi monga TimescaleDB. Ndi chiyani? Uku ndikuwonjezera kwa PostgreSQL, ndiko kuti, ili ndi mawonekedwe amtundu wa PostgreSQL. Kuphatikiza apo, kukulitsa uku kumakupatsani mwayi wogwira ntchito bwino kwambiri ndi data ya timeseries ndikugawana zokha. Momwe zimawonekera:
Izi ndizowopsa - pali lingaliro lotere mu Timescale. Ichi ndi hypertable kuti mumapanga, ndipo muli chunks. Chunks ndi magawo, awa ndi matebulo a ana, ngati sindikulakwitsa. Ndizothandiza kwenikweni.
TimescaleDB ndi PostgreSQL
Monga opanga TimescaleDB akutsimikizira, amagwiritsa ntchito algorithm yolondola kwambiri poyankha mafunso, makamaka zoyika, zomwe zimawalola kuti azikhala ndi magwiridwe antchito pafupifupi nthawi zonse ndi kukula kochulukira kwa dataset. Ndiye kuti, pambuyo pa mizere ya 200 miliyoni ya Postgres, yomwe imayamba kutsika kwambiri ndikutaya magwiridwe antchito mpaka zero, pomwe Timescale imakulolani kuti muyike zoyikamo moyenera momwe mungathere ndi kuchuluka kwa data.
Momwe mungayikitsire TimescaleDB? Ndi zophweka!
Zili m'zolemba, zafotokozedwa - mukhoza kuziyika kuchokera ku phukusi lililonse ... Zimatengera mapepala ovomerezeka a Postgres. Itha kupangidwa pamanja. Zinachitika kuti ndiyenera kusonkhanitsa kwa database.
Pa Zabbix timangoyambitsa Extention. Ndikuganiza kuti omwe adagwiritsa ntchito Extention mu Postgres ... Mukungoyambitsa Extention, kupanga izo ku Zabbix database yomwe mukugwiritsa ntchito.
Ndipo sitepe yomaliza...
NthawiDB. Kusamuka kwa ma tebulo a mbiriyakale
Muyenera kupanga hypertable. Pali ntchito yapadera ya izi - Pangani hypertable. M'menemo, gawo loyamba ndilo tebulo lomwe likufunika mu database iyi (yomwe muyenera kupanga hypertable).
Gawo lopangira, ndi chunk_time_interval (iyi ndi nthawi ya chunks (magawo omwe akuyenera kugwiritsidwa ntchito). 86 ndi tsiku limodzi.
Migrate_data parameter: Ngati muyika kuzoona, ndiye kuti izi zisuntha zonse zomwe zilipo kuzinthu zomwe zidapangidwa kale.
Ndagwiritsapo ntchito migrate_data ndekha - zimatengera nthawi yokwanira, kutengera kukula kwa database yanu. Ndinali ndi terabyte yopitilira - zidatenga ola limodzi kuti ndipange. Nthawi zina, poyesa, ndidachotsa mbiri yakale yamawu (history_text) ndi chingwe (history_str) kuti ndisamutsatire - sizinali zondisangalatsa kwenikweni.
Ndipo timapanga zosintha zomaliza mu db_extention yathu: timayika timescaledb kuti databaseyo ndipo, makamaka, Zabbix yathu imvetsetse kuti pali db_extention. Amayiyambitsa ndikugwiritsa ntchito mawu olondola ndi mafunso ku database, pogwiritsa ntchito "mawonekedwe" omwe ali ofunikira ku TimescaleDB.
Kusintha kwa seva
Ndinagwiritsa ntchito ma seva awiri. Seva yoyamba ndi makina ang'onoang'ono, mapurosesa 20, gigabytes 16 ya RAM. Ndinakonza Postgres 10.8 pa izo:
Makina ogwiritsira ntchito anali Debian, mawonekedwe a fayilo anali xfs. Ndidapanga zoikamo zochepa kuti ndigwiritse ntchito nkhokwe iyi, kusiya zomwe Zabbix izigwiritsa ntchito. Pa makina omwewo panali seva ya Zabbix, PostgreSQL ndi othandizira katundu.
Ndagwiritsa ntchito othandizira 50 omwe amagwiritsa ntchito LoadableModule kuti apange zotsatira zosiyanasiyana. Ndiwo omwe adapanga zingwe, manambala, ndi zina zotero. Ndinadzaza nkhokwe ndi deta yambiri. Poyambirira, kasinthidwe kameneka kanali ndi zinthu zokwana 5 pa wolandira, ndipo pafupifupi chinthu chilichonse cha data chinali ndi choyambitsa - kuti izi zikhale zokonzekera zenizeni. Nthawi zina mumafunikira choyambitsa chimodzi kuti mugwiritse ntchito.
Ndinayendetsa nthawi yosinthira ndi katundu wokha osagwiritsa ntchito othandizira 50 (kuwonjezera zambiri), komanso kugwiritsa ntchito zinthu zamtundu wa data ndikuchepetsa nthawi yosinthira kukhala masekondi 4.
Mayeso a magwiridwe antchito. PostgreSQL: 36 NVPs
Kukhazikitsa koyamba, kukhazikitsidwa koyamba komwe ndinali nako kunali pa PostreSQL 10 yoyera pa hardware iyi (35 zikwi zikwi pamphindi). Kawirikawiri, monga momwe mukuonera pazenera, kuyika deta kumatenga magawo a sekondi - chirichonse chiri chabwino komanso chachangu, ma drive a SSD (200 gigabytes). Chokhacho ndikuti 20 GB imadzaza mwachangu.
Padzakhala ma graph ochuluka kwambiri m'tsogolomu. Iyi ndi dashboard yokhazikika ya seva ya Zabbix.
Grafu yoyamba ndi kuchuluka kwamtengo pamphindikati (buluu, pamwamba kumanzere), 35 zikwizikwi pankhaniyi. Izi (pakatikati) ndikutsitsa njira zomangira, ndipo izi (pamwamba kumanja) ndikutsitsa njira zamkati: zolumikizira mbiri yakale ndi wosamalira nyumba, zomwe pano (pakatikati) zakhala zikuyenda kwa nthawi yayitali.
Grafu iyi (pakatikati) ikuwonetsa kugwiritsa ntchito kwa ValueCache - ndi zingati za ValueCache zomwe zimagunda zoyambitsa (masauzande angapo pamphindikati). Girafu ina yofunika kwambiri ndi yachinayi (pansi kumanzere), yomwe ikuwonetsa kugwiritsa ntchito HistoryCache, yomwe ndidayankhula, yomwe ndi buffer isanalowe mu database.
Mayeso a magwiridwe antchito. PostgreSQL: 50 NVPs
Kenako, ndidawonjezera katunduyo kukhala 50 zikwizikwi pamphindikati pazida zomwezo. Ikapakidwa ndi Wosunga Nyumba, zinthu 10 zikwizikwi zidalembedwa mumasekondi 2-3 ndikuwerengera. Zomwe, kwenikweni, zikuwonetsedwa pazithunzi zotsatirazi:
"Woyang'anira nyumba" wayamba kale kusokoneza ntchito, koma kawirikawiri, katundu wa oyendetsa mbiri yakale akadali pamlingo wa 60% (graph yachitatu, pamwamba kumanja). HistoryCache iyamba kale kudzaza mwachangu pamene Wosunga Nyumba akuthamanga (pansi kumanzere). Zinali pafupifupi theka la gigabyte, 20% yodzaza.
Mayeso a magwiridwe antchito. PostgreSQL: 80 NVPs
Kenako ndidakulitsa mpaka 80 zikwizikwi pamphindikati:
Zinali pafupifupi 400 zikwi deta zinthu, 280 zikwi zoyambitsa. Kuyikapo, monga mukuwonera, potengera kuchuluka kwa zozama za mbiri yakale (anali 30 mwa izo) zinali kale kwambiri. Kenako ndinawonjezera magawo osiyanasiyana: zozama za mbiriyakale, posungira ... Pazida izi, katundu pazida za mbiri yakale adayamba kuchulukirachulukira, pafupifupi "pa alumali" - motero, HistoryCache idalowa mumtolo waukulu kwambiri:
Nthawi yonseyi ndimayang'anira magawo onse adongosolo (momwe purosesa imagwiritsidwira ntchito, RAM) ndikupeza kuti kugwiritsa ntchito disk kunali kokwanira - ndidapeza kuchuluka kwa disk iyi pazida izi, pamakina awa. "Postgres" idayamba kutaya deta mwachangu kwambiri, ndipo disk inalibenso nthawi yolemba, kuwerenga ...
Ndinatenga seva ina yomwe inali kale ndi 48 processors ndi 128 gigabytes ya RAM:
"Ndinayikonza" - ndinayika syncer ya Mbiri (zidutswa 60) ndikuchita bwino. Ndipotu, sitili "pa alumali," koma izi mwina ndi malire a zokolola, kumene kuli kofunikira kale kuchitapo kanthu.
Mayeso a magwiridwe antchito. TimescaleDB: 80 NVPs
Ntchito yanga yayikulu inali kugwiritsa ntchito TimescaleDB. Grafu iliyonse ikuwonetsa kuviika:
Zolephera izi ndikusamuka kwa data. Pambuyo pake, mu seva ya Zabbix, mbiri yodzaza mbiri yakale, monga momwe mukuonera, inasintha kwambiri. Zimakulolani kuti muyike deta pafupifupi maulendo atatu mofulumira ndikugwiritsa ntchito HistoryCache yochepa - motero, mudzakhala ndi deta yoperekedwa panthawi yake. Apanso, 3 zikwi zamtengo wapatali pa sekondi imodzi ndizokwera kwambiri (zowona, osati za Yandex). Ponseponse uku ndikukhazikitsa kwakukulu, komwe kumakhala ndi seva imodzi.
Mayeso a PostgreSQL: 120 ma NVP zikwizikwi
Kenako, ndinawonjezera mtengo wazinthu za data kufika theka la milioni ndikulandira mtengo wowerengeka wa 125 pamphindikati:
Ndipo ndili ndi ma graph awa:
M'malo mwake, iyi ndi njira yogwirira ntchito, imatha kugwira ntchito kwa nthawi yayitali. Koma popeza ndinali ndi disk ya 1,5 terabyte yokha, ndinaigwiritsa ntchito m'masiku angapo. Chofunika kwambiri ndi chakuti panthawi imodzimodziyo magawo atsopano adapangidwa pa TimescaleDB, ndipo izi sizinali zodziwika bwino pa ntchito, zomwe sizinganene za MySQL.
Nthawi zambiri, magawo amapangidwa usiku, chifukwa izi nthawi zambiri zimalepheretsa kuyika ndikugwira ntchito ndi matebulo ndipo zimatha kuyambitsa kuwonongeka kwa ntchitoyo. Pamenepa sizili choncho! Ntchito yayikulu inali kuyesa kuthekera kwa TimescaleDB. Zotsatira zake zinali chiwerengero chotsatira: 120 zikwi zikwi pamphindikati.
Palinso zitsanzo m'deralo:
Munthuyo adatsegulanso TimescaleDB ndipo katundu wogwiritsa ntchito io.weight adagwera pa purosesa; ndi kugwiritsa ntchito zinthu zamkati zamkati zatsikanso chifukwa cha kuphatikizidwa kwa TimescaleDB. Kuphatikiza apo, awa ndi ma disks wamba, ndiko kuti, makina wamba pa disk wamba (osati ma SSD)!
Kwa makonzedwe ang'onoang'ono omwe amachepetsedwa ndi disk performance, TimescaleDB, mwa lingaliro langa, ndi yankho labwino kwambiri. Ikuthandizani kuti mupitirize kugwira ntchito musanasamuke kupita ku hardware yofulumira ya database.
Ndikukuitanani nonse ku zochitika zathu: Msonkhano ku Moscow, Summit ku Riga. Gwiritsani ntchito mayendedwe athu - Telegraph, forum, IRC. Ngati muli ndi mafunso, bwerani ku desiki yathu, titha kukambirana chilichonse.
Mafunso Omvera
Funso lochokera kwa omvera (pambuyo pake - A): - Ngati TimescaleDB ndiyosavuta kuyisintha, ndipo imapereka kulimbikitsa kotereku, ndiye kuti mwina izi ziyenera kugwiritsidwa ntchito ngati njira yabwino yokonzekera Zabbix ndi Postgres? Ndipo kodi pali zovuta ndi zovuta za yankho ili, kapena pambuyo pake, ngati ndinaganiza zodzipangira Zabbix ndekha, nditha kutenga Postgres mosavuta, kuika Timescale pomwepo, gwiritsani ntchito osaganizira mavuto aliwonse ?
AG: - Inde, ndinganene kuti uwu ndiupangiri wabwino: gwiritsani ntchito Postgres nthawi yomweyo ndikuwonjezera kwa TimescaleDB. Monga ndanenera kale, ndemanga zambiri zabwino, ngakhale kuti "chinthu" ichi ndi choyesera. Koma mayeso akuwonetsa kuti ili ndi yankho labwino (ndi TimescaleDB) ndipo ndikuganiza kuti lisintha! Tikuyang'anira momwe chiwonjezerochi chikukulira ndipo tisintha ngati pakufunika.
Ngakhale panthawi ya chitukuko, tinkadalira chimodzi mwa "mawonekedwe" awo odziwika bwino: zinali zotheka kugwira ntchito ndi chunks mosiyana. Koma kenako adazidula m'chiwonetsero chotsatira, ndipo tidayenera kusiya kudalira code iyi. Ndikupangira kugwiritsa ntchito yankho ili pamakhazikitsidwe ambiri. Ngati mugwiritsa ntchito MySQL ... Pakukhazikitsa kwapakati, yankho lililonse limagwira ntchito bwino.
A: - Pazithunzi zomaliza za anthu ammudzi, panali graph yokhala ndi "Woyang'anira Nyumba":
Anapitiriza kugwira ntchito. Kodi Housekeeper amachita chiyani ndi TimescaleDB?
AG: - Tsopano sindingathe kunena motsimikiza - ndiyang'ana pa code ndikukuuzani mwatsatanetsatane. Imagwiritsa ntchito mafunso a TimescaleDB kuti musachotse chunks, koma kuti muwaphatikize. Sindinakonzekere kuyankha funso ili laukadaulo panobe. Tipeza zambiri pa stand lero kapena mawa.
A: - Ndili ndi funso lofananalo - lokhudza magwiridwe antchito ochotsa mu Timescale.
A (yankho kuchokera kwa omvera): - Mukachotsa deta patebulo, ngati muyichotsa, ndiye kuti muyenera kudutsa patebulo - chotsani, chotsani, chotsani chilichonse kuti chisachotsedwe mtsogolo. Mu Timescale, popeza muli ndi chunks, mutha kusiya. Mwachidule, mumangouza fayilo yomwe ili mu data yayikulu: "Chotsani!"
Timescale imangomvetsetsa kuti chunk yotereyi kulibenso. Ndipo popeza imaphatikizidwa muzokonzekera zamafunso, imagwiritsa ntchito mbedza kuti igwire zomwe mwasankha kapena ntchito zina ndipo nthawi yomweyo imamvetsetsa kuti chunk iyi kulibenso - "Sindipitanso kumeneko!" (zambiri palibe). Ndizomwezo! Ndiko kuti, kujambula kwa tebulo kumasinthidwa ndi kuchotsedwa kwa fayilo ya binary, kotero kumathamanga.
A: - Takhudza kale mutu wa non-SQL. Momwe ndikumvera, Zabbix safunikira kwenikweni kusintha deta, ndipo zonsezi ndizofanana ndi chipika. Kodi ndizotheka kugwiritsa ntchito ma database apadera omwe sangathe kusintha deta yawo, koma nthawi yomweyo sungani, sonkhanitsani, ndikugawirani mofulumira kwambiri - Clickhouse, mwachitsanzo, chinachake chonga Kafka?.. Kafka ndi chipika! Kodi ndizotheka kuwaphatikiza mwanjira ina?
AG: - Kutsitsa kutha kuchitika. Tili ndi "chinthu" china kuyambira mtundu 3.4: mutha kulemba mafayilo onse a mbiri yakale, zochitika, china chilichonse ku mafayilo; ndiyeno tumizani ku database ina iliyonse pogwiritsa ntchito chowongolera. M'malo mwake, anthu ambiri amagwiranso ntchito ndikulemba mwachindunji ku database. Pa ntchentche, ozama mbiri amalemba zonsezi m'mafayilo, tembenuzani mafayilowa, ndi zina zotero, ndipo mukhoza kusamutsa izi ku Clickhouse. Sindinganene za mapulani, koma mwina thandizo lina la mayankho a NoSQL (monga Clickhouse) lipitilira.
A: - Nthawi zambiri, zimakhala kuti mutha kuchotsa ma postgres?
AG: - Zachidziwikire, gawo lovuta kwambiri mu Zabbix ndi matebulo am'mbiri, omwe amabweretsa zovuta komanso zochitika. Pankhaniyi, ngati simukusunga zochitika kwa nthawi yayitali ndikusunga mbiri yakale ndi zochitika m'malo ena osungira mwachangu, ndiye kuti ambiri, ndikuganiza, sipadzakhala mavuto.
A: - Kodi mungayerekeze kuti chilichonse chidzagwira ntchito mwachangu bwanji ngati mutasinthira ku Clickhouse, mwachitsanzo?
AG: - Sindinayese. Ndikuganiza kuti ziwerengero zomwezo zitha kupezeka mosavuta, chifukwa Clickhouse ili ndi mawonekedwe ake, koma sindinganene motsimikiza. Ndi bwino kuyesa. Zonse zimadalira kasinthidwe: ndi makamu angati omwe muli nawo, ndi zina zotero. Kuyika ndi chinthu chimodzi, koma muyenera kupezanso deta iyi - Grafana kapena china.
A: - Ndiye tikukamba za nkhondo yofanana, osati za ubwino waukulu wa nkhokwe zachanguzi?
AG: - Ndikuganiza kuti tikaphatikiza, padzakhala mayesero olondola kwambiri.
A: - Kodi RRD yabwino yakale idapita kuti? Ndi chiyani chinakupangitsani kuti musinthe ma database a SQL? Poyambirira, ma metric onse adasonkhanitsidwa pa RRD.
AG: - Zabbix anali ndi RRD, mwina mu mtundu wakale kwambiri. Pakhala pali nkhokwe za SQL - njira yachikale. Njira yachikale ndi MySQL, PostgreSQL (iwo akhalapo kwa nthawi yayitali kwambiri). Sitinagwiritsepo ntchito mawonekedwe wamba pamasamba a SQL ndi RRD.
Zotsatsa zina π
Zikomo chifukwa chokhala nafe. Kodi mumakonda zolemba zathu? Mukufuna kuwona zambiri zosangalatsa? Tithandizeni potipatsa oda kapena kulimbikitsa anzathu, , ma analogi apadera a ma seva olowera, omwe adakupangirani inu: (ikupezeka ndi RAID1 ndi RAID10, mpaka 24 cores mpaka 40GB DDR4).
Dell R730xd 2x yotsika mtengo ku Equinix Tier IV data center ku Amsterdam? Pokhapokha ku Netherlands! Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - kuchokera $99! Werengani za
Source: www.habr.com