{"id":53944,"date":"2019-12-13T00:00:00","date_gmt":"2019-12-12T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/clickhouse-graphite-kak-znachitelno-umenshit-potreblyaemoe-mesto-na-diskah"},"modified":"2020-02-18T14:01:53","modified_gmt":"2020-02-18T11:01:53","slug":"clickhouse-graphite-kak-znachitelno-umenshit-potreblyaemoe-mesto-na-diskah","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/clickhouse-graphite-kak-znachitelno-umenshit-potreblyaemoe-mesto-na-diskah","title":{"rendered":"ClickHouse + Graphite: kuidas m\u00e4rkimisv\u00e4\u00e4rselt v\u00e4hendada kettaruumi tarbimist","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"ClickHouse + Graphite: kuidas m\u00e4rkimisv\u00e4\u00e4rselt v\u00e4hendada kettaruumi tarbimist\" src=\"\/wp-content\/uploads\/2019\/12\/5a2d19e6fea2a28d8d4aa9220653aefc.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Tere, habr.<\/p>\n<p><\/p>\n<p>Kui keegi kasutab s\u00fcsteemi <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/graphite-project\/graphite-web\">graphite-web<\/a><\/noindex> ja on j\u00f5udnud salvestusruumi j\u00f5udlusprobleemide taha <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/graphite-project\/whisper\">whisper<\/a><\/noindex> (IO, kasutatud ketas ruum), siis t\u00f5en\u00e4osus, et ClickHouse'le kui asendajale on t\u00e4helepanu p\u00f6\u00f6ratud, peaks l\u00e4henema \u00fchele. See v\u00e4ide eeldab, et metrikate demonina on juba kasutusel kolmanda osapoole rakendus, n\u00e4iteks <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/grobian\/carbonwriter\">carbonwriter<\/a><\/noindex> v\u00f5i <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/go-graphite\/go-carbon\">go-carbon<\/a><\/noindex>.<\/p>\n<p><\/p>\n<p>ClickHouse lahendab kirjeldatud probleemid h\u00e4sti. N\u00e4iteks, p\u00e4rast 2TiB andmete kolimist whisperist, mahtusid nad 300GiB. Ma ei hakka detailidesse laskuma, sellel teemal on piisavalt artikleid. Lisaks, hiljuti polnud meie ClickHouse salvestus mitte k\u00f5ik h\u00e4sti.<\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h3 id=\"problemy-s-potreblyaemym-mestom\">Kohustusliku ruumi probleemid<\/h3>\n<p><\/p>\n<p>Esmapilgul peaks k\u00f5ik h\u00e4sti t\u00f6\u00f6tama. J\u00e4rgides <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.yandex\/docs\/ru\/operations\/table_engines\/graphitemergetree\/#primer-konfiguratsii\">dokumentatsioon<\/a><\/noindex>, loome metrikate salvestamise skeemi konfiguratsiooni (edasi <code>retention<\/code>), seej\u00e4rel loome tabeli vastavalt valitud tagasiside soovitusele graphite-web jaoks: <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/lomik\/carbon-clickhouse\">carbon-clickhouse<\/a><\/noindex>+<noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/lomik\/graphite-clickhouse\">graphite-clickhouse<\/a><\/noindex> v\u00f5i <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/ClickHouse\/graphouse\">graphouse<\/a><\/noindex>, s\u00f5ltuvalt sellest, millist teeki kasutatakse. Ja\u2026 k\u00e4ivitub aeglane pomm.<\/p>\n<p><\/p>\n<p>Selleks, et m\u00f5ista, milline see on, tuleb teada, kuidas toimivad sisestused ja andmete edasine elu tabelites mootort\u00fc\u00fcpide *<em>MergeTree<\/em> ClickHouse (diagrammid on p\u00e4rit <noindex><a rel=\"nofollow\" href=\"https:\/\/youtu.be\/PLMSA_gDdyM\">esitlusest.<\/a><\/noindex> Aleksei Zatelepinilt):<\/p>\n<p><\/p>\n<ul>\n<li>Sisestatakse <code>blok<\/code> andmeid. Meie puhul on need metrikad.<br \/>\n<img decoding=\"async\" alt=\"ClickHouse + Graphite: kuidas m\u00e4rkimisv\u00e4\u00e4rselt v\u00e4hendada kettaruumi tarbimist\" src=\"\/wp-content\/uploads\/2019\/12\/b60d482acca3c8d187d394f58a2d8a16.png\" style=\"display:block;margin: 0 auto;\" \/><\/li>\n<li>Iga selline block enne ketast kirjutamist sorteeritakse vastavalt v\u00f5tmele <code>CLR kasutaja m\u00e4\u00e4ratletud agregaatfunktsioon<\/code>, mis on m\u00e4\u00e4ratud tabeli loomisel.<\/li>\n<li>P\u00e4rast sorteerimist, <code>t\u00fckk<\/code> (<code>part<\/code>) andmeid kirjutatakse kettale.<br \/>\n<img decoding=\"async\" alt=\"ClickHouse + Graphite: kuidas m\u00e4rkimisv\u00e4\u00e4rselt v\u00e4hendada kettaruumi tarbimist\" src=\"\/wp-content\/uploads\/2019\/12\/a54903b8efa0866d24da303fa7a68134.png\" style=\"display:block;margin: 0 auto;\" \/><\/li>\n<li>Server j\u00e4lgib taustal, et selliseid t\u00fckkide arvu ei oleks liiga palju ja k\u00e4ivitab tausta <code>\u00fclendused<\/code> (<code>merge<\/code>, edasi mergenid).<br \/>\n<img decoding=\"async\" alt=\"ClickHouse + Graphite: kuidas m\u00e4rkimisv\u00e4\u00e4rselt v\u00e4hendada kettaruumi tarbimist\" src=\"\/wp-content\/uploads\/2019\/12\/151c18c7b73ebc4097493b13a2e26765.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<img decoding=\"async\" alt=\"ClickHouse + Graphite: kuidas m\u00e4rkimisv\u00e4\u00e4rselt v\u00e4hendada kettaruumi tarbimist\" src=\"\/wp-content\/uploads\/2019\/12\/1921908554981eca0847541f77a15790.png\" style=\"display:block;margin: 0 auto;\" \/><\/li>\n<li>Server l\u00f5petab mergenide automaatse k\u00e4ivitamise, kui andmed enam aktiivselt ei sisene <code>partitsioon<\/code> (<code>partition<\/code>), kuid protsessi saab k\u00e4sitsi k\u00e4ivitada k\u00e4suga <code>OPTIMIZE<\/code>.<\/li>\n<li>Kui partitsioonis on j\u00e4\u00e4nud vaid \u00fcks t\u00fckk, siis ei saa tavalise k\u00e4suga merge'd k\u00e4ivitada, tuleb kasutada <code>OPTIMIZE ... FINAL<\/code><\/li>\n<\/ul>\n<p><\/p>\n<p>Ja nii hakkavad esimesed metrikad saabuma. Ning nad h\u00f5ivavad mingisuguse ruumi. Edasised s\u00fcndmused v\u00f5ivad varieeruda s\u00f5ltuvalt paljusid teguritest:<\/p>\n<p><\/p>\n<ul>\n<li>Partitsioneerimise v\u00f5ti v\u00f5ib olla kas v\u00e4ga v\u00e4ike (p\u00e4ev) v\u00f5i v\u00e4ga suur (m\u00f5ned kuud).<\/li>\n<li>Retention \u043a\u043e\u043d\u0444\u0438\u0433\u0443\u0440\u0430\u0446\u0438\u044f \u043c\u043e\u0436\u0435\u0442 \u0432\u043a\u043b\u044e\u0447\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u043f\u043e\u0440\u043e\u0433\u043e\u0432 \u0430\u0433\u0440\u0435\u0433\u0430\u0446\u0438\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u0432\u043d\u0443\u0442\u0440\u0438 \u0430\u043a\u0442\u0438\u0432\u043d\u043e\u0439 \u043f\u0430\u0440\u0442\u0438\u0446\u0438\u0438 (\u043a\u0443\u0434\u0430 \u0437\u0430\u043f\u0438\u0441\u044b\u0432\u0430\u044e\u0442\u0441\u044f \u043c\u0435\u0442\u0440\u0438\u043a\u0438), \u0430 \u043c\u043e\u0436\u0435\u0442 \u0438 \u043d\u0435\u0442.<\/li>\n<li>\u0415\u0441\u043b\u0438 \u0434\u0430\u043d\u043d\u044b\u0445 \u043e\u0447\u0435\u043d\u044c \u043c\u043d\u043e\u0433\u043e, \u0442\u043e \u0441\u0430\u043c\u044b\u0435 \u0440\u0430\u043d\u043d\u0438\u0435 \u043a\u0443\u0441\u043a\u0438, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0438\u0437-\u0437\u0430 \u0444\u043e\u043d\u043e\u0432\u044b\u0445 \u043c\u0435\u0440\u0436\u0435\u0439 \u043c\u043e\u0433\u0443\u0442 \u0443\u0436\u0435 \u0441\u0442\u0430\u0442\u044c \u043e\u0433\u0440\u043e\u043c\u043d\u044b\u043c\u0438 (\u043f\u0440\u0438 \u0432\u044b\u0431\u043e\u0440\u0435 \u043d\u0435\u043e\u043f\u0442\u0438\u043c\u0430\u043b\u044c\u043d\u043e\u0433\u043e \u043a\u043b\u044e\u0447\u0430 \u043f\u0430\u0440\u0442\u0438\u0446\u0438\u043e\u043d\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f), \u043d\u0435 \u0431\u0443\u0434\u0443\u0442 \u043c\u0435\u0440\u0436\u0438\u0442\u044c\u0441\u044f \u0441\u0430\u043c\u0438 \u0441 \u043d\u043e\u0432\u044b\u043c\u0438 \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u043c\u0438 \u043a\u0443\u0441\u043a\u0430\u043c\u0438.<\/li>\n<\/ul>\n<p><\/p>\n<p>\u0418 \u0432\u0441\u0451 \u0437\u0430\u043a\u0430\u043d\u0447\u0438\u0432\u0430\u0435\u0442\u0441\u044f \u0432\u0441\u0435\u0433\u0434\u0430 \u043e\u0434\u0438\u043d\u0430\u043a\u043e\u0432\u043e. \u041c\u0435\u0441\u0442\u043e, \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u043c\u043e\u0435 \u043c\u0435\u0442\u0440\u0438\u043a\u0430\u043c\u0438 \u0432 ClickHouse, \u0442\u043e\u043b\u044c\u043a\u043e \u0440\u0430\u0441\u0442\u0451\u0442, \u0435\u0441\u043b\u0438:<\/p>\n<p><\/p>\n<ul>\n<li>\u043d\u0435 \u043f\u0440\u0438\u043c\u0435\u043d\u044f\u0442\u044c <code>OPTIMIZE ... FINAL<\/code> \u0432\u0440\u0443\u0447\u043d\u0443\u044e \u0438\u043b\u0438<\/li>\n<li>\u043d\u0435 \u0432\u0441\u0442\u0430\u0432\u043b\u044f\u0442\u044c \u0434\u0430\u043d\u043d\u044b\u0435 \u0432\u043e \u0432\u0441\u0435 \u043f\u0430\u0440\u0442\u0438\u0446\u0438\u0438 \u043d\u0430 \u043f\u043e\u0441\u0442\u043e\u044f\u043d\u043d\u043e\u0439 \u043e\u0441\u043d\u043e\u0432\u0435, \u0447\u0442\u043e\u0431\u044b \u0440\u0430\u043d\u043e \u0438\u043b\u0438 \u043f\u043e\u0437\u0434\u043d\u043e \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0444\u043e\u043d\u043e\u0432\u044b\u0439 \u043c\u0435\u0440\u0436.<\/li>\n<\/ul>\n<p><\/p>\n<p>\u0412\u0442\u043e\u0440\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431 \u043a\u0430\u0436\u0435\u0442\u0441\u044f \u043d\u0430\u0438\u0431\u043e\u043b\u0435\u0435 \u043f\u0440\u043e\u0441\u0442\u044b\u043c \u0432 \u0440\u0435\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u0438 \u0438, \u0441\u043b\u0435\u0434\u043e\u0432\u0430\u0442\u0435\u043b\u044c\u043d\u043e, \u043e\u043d \u043d\u0435\u043f\u0440\u0430\u0432\u0438\u043b\u044c\u043d\u044b\u0439 \u0438 \u0431\u044b\u043b \u043f\u0440\u043e\u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d \u0432 \u043f\u0435\u0440\u0432\u0443\u044e \u043e\u0447\u0435\u0440\u0435\u0434\u044c.<br \/>\n\u042f \u043d\u0430\u043f\u0438\u0441\u0430\u043b \u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e \u043f\u0440\u043e\u0441\u0442\u043e\u0439 \u0441\u043a\u0440\u0438\u043f\u0442 \u043d\u0430 python, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u043e\u0442\u043f\u0440\u0430\u0432\u043b\u044f\u043b \u0444\u0438\u043a\u0442\u0438\u0432\u043d\u044b\u0435 \u043c\u0435\u0442\u0440\u0438\u043a\u0438 \u0434\u043b\u044f \u043a\u0430\u0436\u0434\u043e\u0433\u043e \u0434\u043d\u044f \u0437\u0430 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u0435 4 \u0433\u043e\u0434\u0430 \u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0430\u043b\u0441\u044f \u043a\u0430\u0436\u0434\u044b\u0439 \u0447\u0430\u0441 \u043a\u0440\u043e\u043d\u043e\u043c.<br \/>\n\u041f\u043e\u0441\u043a\u043e\u043b\u044c\u043a\u0443 \u0432\u0441\u044f \u0440\u0430\u0431\u043e\u0442\u0430 ClickHouse DBMS \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u0430 \u043d\u0430 \u0442\u043e\u043c, \u0447\u0442\u043e \u044d\u0442\u0430 \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0440\u0430\u043d\u043e \u0438\u043b\u0438 \u043f\u043e\u0437\u0434\u043d\u043e \u0432\u044b\u043f\u043e\u043b\u043d\u0438\u0442 \u0432\u0441\u044e \u0444\u043e\u043d\u043e\u0432\u0443\u044e \u0440\u0430\u0431\u043e\u0442\u0443, \u043d\u043e \u043d\u0435\u0438\u0437\u0432\u0435\u0441\u0442\u043d\u043e \u043a\u043e\u0433\u0434\u0430, \u043c\u043d\u0435 \u043d\u0435 \u0443\u0434\u0430\u043b\u043e\u0441\u044c \u0434\u043e\u0436\u0434\u0430\u0442\u044c\u0441\u044f \u043c\u043e\u043c\u0435\u043d\u0442\u0430, \u043a\u043e\u0433\u0434\u0430 \u0441\u0442\u0430\u0440\u044b\u0435 \u043e\u0433\u0440\u043e\u043c\u043d\u044b\u0435 \u043a\u0443\u0441\u043a\u0438 \u043d\u0430\u0447\u043d\u0443\u0442 \u043c\u0435\u0440\u0436 \u0441 \u043d\u043e\u0432\u044b\u043c\u0438 \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u043c\u0438. \u0421\u0442\u0430\u043b\u043e \u044f\u0441\u043d\u043e, \u0447\u0442\u043e \u043d\u0443\u0436\u043d\u043e \u0438\u0441\u043a\u0430\u0442\u044c \u0441\u043f\u043e\u0441\u043e\u0431 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043f\u0440\u0438\u043d\u0443\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0435 \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u0438.<\/p>\n<p>\n<img decoding=\"async\" alt=\"ClickHouse + Graphite: kuidas m\u00e4rkimisv\u00e4\u00e4rselt v\u00e4hendada kettaruumi tarbimist\" src=\"\/wp-content\/uploads\/2019\/12\/60b0354cb65dcba6e8402b67261041ff.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<h3 id=\"informaciya-v-sistemnyh-tablicah-clickhouse\">\u0418\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044f \u0432 \u0441\u0438\u0441\u0442\u0435\u043c\u043d\u044b\u0445 \u0442\u0430\u0431\u043b\u0438\u0446\u0430\u0445 ClickHouse<\/h3>\n<p><\/p>\n<p>\u0414\u0430\u0432\u0430\u0439\u0442\u0435 \u043f\u043e\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u043d\u0430 \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0443 \u0442\u0430\u0431\u043b\u0438\u0446\u044b <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.yandex\/docs\/ru\/operations\/system_tables\/#system_tables-parts\">system.parts<\/a><\/noindex>. \u042d\u0442\u043e \u0438\u0441\u0447\u0435\u0440\u043f\u044b\u0432\u0430\u044e\u0449\u0430\u044f \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044f \u043e \u043a\u0430\u0436\u0434\u043e\u043c \u043a\u0443\u0441\u043a\u0435 \u0432\u0441\u0435\u0445 \u0442\u0430\u0431\u043b\u0438\u0446 \u043d\u0430 \u0441\u0435\u0440\u0432\u0435\u0440\u0435 ClickHouse. \u0421\u043e\u0434\u0435\u0440\u0436\u0438\u0442, \u0432 \u0442\u043e\u043c \u0447\u0438\u0441\u043b\u0435, \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0438\u0435 \u0441\u0442\u043e\u043b\u0431\u0446\u044b:<\/p>\n<p><\/p>\n<ul>\n<li>\u0438\u043c\u044f \u0431\u0430\u0437\u044b \u0434\u0430\u043d\u043d\u044b\u0445 (<code>database<\/code>);<\/li>\n<li>\u0438\u043c\u044f \u0442\u0430\u0431\u043b\u0438\u0446\u044b (<code>table<\/code>);<\/li>\n<li>\u0438\u043c\u044f \u0438 ID \u043f\u0430\u0440\u0442\u0438\u0446\u0438\u0438 (<code>partition<\/code> &amp; <code>partition_id<\/code>);<\/li>\n<li>\u0432\u0440\u0435\u043c\u044f \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043a\u0443\u0441\u043a\u0430 (<code>modification_time<\/code>);<\/li>\n<li>\u043c\u0438\u043d\u0438\u043c\u0430\u043b\u044c\u043d\u0430\u044f \u0438 \u043c\u0430\u043a\u0441\u0438\u043c\u0430\u043b\u044c\u043d\u0430\u044f \u0434\u0430\u0442\u0430 \u0432 \u043a\u0443\u0441\u043a\u0435 (\u043f\u0430\u0440\u0442\u0438\u0446\u0438\u043e\u043d\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u043f\u0440\u043e\u0438\u0441\u0445\u043e\u0434\u0438\u0442 \u043f\u043e \u0434\u043d\u044f\u043c) (<code>min_date<\/code> &amp; <code>max_date<\/code>);<\/li>\n<\/ul>\n<p><\/p>\n<p>\u0422\u0430\u043a\u0436\u0435 \u0435\u0441\u0442\u044c \u0442\u0430\u0431\u043b\u0438\u0446\u0430 <noindex><a rel=\"nofollow\" href=\"https:\/\/clickhouse.yandex\/docs\/ru\/operations\/system_tables\/#system-graphite-retentions\">system.graphite_retentions<\/a><\/noindex>, \u0441\u043e \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0438\u043c\u0438 \u0438\u043d\u0442\u0435\u0440\u0435\u0441\u043d\u044b\u043c\u0438 \u043f\u043e\u043b\u044f\u043c\u0438:<\/p>\n<p><\/p>\n<ul>\n<li>\u0438\u043c\u044f \u0431\u0430\u0437\u044b \u0434\u0430\u043d\u043d\u044b\u0445 (<code>Tables.database<\/code>);<\/li>\n<li>\u0438\u043c\u044f \u0442\u0430\u0431\u043b\u0438\u0446\u044b (<code>Tables.table<\/code>);<\/li>\n<li>\u0432\u043e\u0437\u0440\u0430\u0441\u0442 \u043c\u0435\u0442\u0440\u0438\u043a\u0438, \u043a\u043e\u0433\u0434\u0430 \u0434\u043e\u043b\u0436\u043d\u0430 \u0431\u044b\u0442\u044c \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0430 \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0430\u044f \u0430\u0433\u0440\u0435\u0433\u0430\u0446\u0438\u044f (<code>age<\/code>);<\/li>\n<\/ul>\n<p><\/p>\n<p>Nii:<\/p>\n<p><\/p>\n<ol>\n<li>\u0423 \u043d\u0430\u0441 \u0435\u0441\u0442\u044c \u0442\u0430\u0431\u043b\u0438\u0446\u0430 \u043a\u0443\u0441\u043a\u043e\u0432 \u0438 \u0442\u0430\u0431\u043b\u0438\u0446\u0430 \u043f\u0440\u0430\u0432\u0438\u043b \u0430\u0433\u0440\u0435\u0433\u0430\u0446\u0438\u0438.<\/li>\n<li>\u041e\u0431\u044a\u0435\u0434\u0438\u043d\u044f\u0435\u043c \u0438\u0445 \u043f\u0435\u0440\u0435\u0441\u0435\u0447\u0435\u043d\u0438\u0435 \u0438 \u043f\u043e\u043b\u0443\u0447\u0430\u0435\u043c \u0432\u0441\u0435 \u0442\u0430\u0431\u043b\u0438\u0446\u044b *GraphiteMergeTree.<\/li>\n<li>\u0418\u0449\u0435\u043c \u0432\u0441\u0435 \u043f\u0430\u0440\u0442\u0438\u0446\u0438\u0438, \u0432 \u043a\u043e\u0442\u043e\u0440\u044b\u0445:\n<ul>\n<li>\u0431\u043e\u043b\u044c\u0448\u0435 \u043e\u0434\u043d\u043e\u0433\u043e \u043a\u0443\u0441\u043a\u0430<\/li>\n<li>\u0438\u043b\u0438 \u043d\u0430\u0441\u0442\u0430\u043b\u0430 \u043f\u043e\u0440\u0430 \u043f\u0440\u0438\u043c\u0435\u043d\u0438\u0442\u044c \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0435\u0435 \u043f\u0440\u0430\u0432\u0438\u043b\u043e \u0430\u0433\u0440\u0435\u0433\u0430\u0446\u0438\u0438, \u0438 <code>modification_time<\/code> \u0441\u0442\u0430\u0440\u0448\u0435 \u044d\u0442\u043e\u0433\u043e \u043c\u043e\u043c\u0435\u043d\u0442\u0430.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p><\/p>\n<h3 id=\"realizaciya\">Rakendamine<\/h3>\n<p>\n<b class=\"spoiler_title\">\u0414\u0430\u043d\u043d\u044b\u0439 \u0437\u0430\u043f\u0440\u043e\u0441<\/b><\/p>\n<pre><code class=\"sql\">VALI\n    concat(p.database, '.', p.table) KUI table,\n    p.partition_id KUI partition_id,\n    p.partition KUI partition,\n    -- K\u00f5ige \"vanem\" reegel, mida saab rakendada\n    -- partitsioonile, kuid mitte tulevikus, vt (*)\n    max(g.age) KUI age,\n    -- Osade arv partitsioonis\n    countDistinct(p.name) KUI parts,\n    -- K\u00f5ige vanemaks m\u00f5\u00f5dikuks partitsioonis peetakse j\u00e4rgmise p\u00e4eva 00:00:00\n    toDateTime(max(p.max_date + 1)) KUI max_time,\n    -- Millal partitsioon tuleks optimeerida\n    max_time + age KUI rollup_time,\n    -- Millal k\u00f5ige vanem t\u00fckk partitsioonis viimati uuendati\n    min(p.modification_time) KUI modified_at\nFROM system.parts KUI p\nINNER JOIN\n(\n    -- K\u00f5ik reeglid k\u00f5ikide tabelite *GraphiteMergeTree jaoks\n    VALI\n        Tables.database KUI database,\n        Tables.table KUI table,\n        age\n    FROM system.graphite_retentions\n    ARRAY JOIN Tables\n    GROUP BY\n        database,\n        table,\n        age\n) KUI g ON\n    (p.table = g.table)\n    JA (p.database = g.database)\nKUS\n    -- Ainult aktiivsed t\u00fckid\n    p.active\n    -- (*) Ja ainult need read, kus aggregeerimise reeglid peaksid juba olema rakendatud\n    JA ((toDateTime(p.max_date + 1) + g.age) &lt; now())\nGROUP BY\n    table,\n    partition\nHAVING\n    -- Ainult partitsioonid, mis on nooremad optimeerimise hetkest\n    (modified_at  1)\nKORDA\n    table ASC,\n    partition ASC,\n    age ASC<\/code><\/pre>\n<p><\/p>\n<p>tagastab iga *GraphiteMergeTree tabelite partitsiooni, mille \u00fchinemine peaks vabastama kettaruumi. J\u00e4\u00e4b vaid t\u00fckkide mitmekordsus l\u00e4bi k\u00e4ia k\u00e4suga <code>OPTIMIZE ... FINAL<\/code>. L\u00f5plikus teostuses on samuti arvestatud, et aktiivse kirjutamisega partitsioone pole vaja puudutada.<\/p>\n<p><\/p>\n<p>Just seda teeb projekt <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/innogames\/graphite-ch-optimizer\">graphite-ch-optimizer<\/a><\/noindex>. Endised kolleegid Yandex.Market'ist katsetasid seda tootmises, t\u00f6\u00f6 tulemusi saab n\u00e4ha allpool. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"ClickHouse + Graphite: kuidas m\u00e4rkimisv\u00e4\u00e4rselt v\u00e4hendada kettaruumi tarbimist\" src=\"\/wp-content\/uploads\/2019\/12\/858dcbf538135d8d57964a71b2b6ca23.png\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Kui k\u00e4ivitada programm ClickHouse serveris, siis see lihtsalt t\u00f6\u00f6tab deemonire\u017eiimis. Iga tunni j\u00e4rel k\u00e4ivitatakse p\u00e4ring, kontrollides, kas on tekkinud uusi partitsioone, mis on vanemad kui kolm p\u00e4eva, mida v\u00f5iks optimeerida.<\/p>\n<p><\/p>\n<p>L\u00e4hitulevikus on plaanis pakkuda v\u00e4hemalt deb-pakette ja v\u00f5imalusel ka rpm.<\/p>\n<p><\/p>\n<h3 id=\"vmesto-zaklyucheniya\">L\u00f5petuseks<\/h3>\n<p><\/p>\n<p>Viimase 9 kuu jooksul olen oma ettev\u00f5ttes <noindex><a rel=\"nofollow\" href=\"https:\/\/www.innogames.com\/ru\/\">InnoGames<\/a><\/noindex> veetnud palju aega, olles ClickHouse ja graphite-web vahel. See oli hea kogemus, mille tulemuseks on peaaegu v\u00f5imalik \u00fcleminek whisperilt ClickHouse'ile m\u00f5\u00f5tmete salvestamiseks. Loodan, et see artikkel on midagi, mis sarnaneb ts\u00fckli algusele selle kohta, milliseid t\u00e4iustusi oleme teinud nende tehnoloogiate erinevates osades ja mida tulevikus veel tehakse.<\/p>\n<p><\/p>\n<p>Projekti taotluse arendamisele kulus mitu liitrit \u00f5lut ja administratiivp\u00e4evi koos <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/users\/v0devil\/\" class=\"user_link\">v0devil<\/a><\/noindex>, mille eest tahan talle oma t\u00e4nu avaldada. Samuti selle artikli \u00fclevaatamise eest.<\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/innogames\/graphite-ch-optimizer\">Projektileht githubis<\/a><\/noindex><\/p>\n<p>Allikas: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/479564\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u044e, habr. \u0415\u0441\u043b\u0438 \u043a\u0442\u043e-\u0442\u043e \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0438\u0440\u0443\u0435\u0442 \u0441\u0438\u0441\u0442\u0435\u043c\u0443 graphite-web \u0438 \u0441\u0442\u043e\u043b\u043a\u043d\u0443\u043b\u0441\u044f \u0441 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u043e\u0439 \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 whisper (IO, \u043f\u043e\u0442\u0440\u0435\u0431\u043b\u044f\u0435\u043c\u043e\u0435 \u0434\u0438\u0441\u043a\u043e\u0432\u043e\u0435 \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u043e), \u0442\u043e \u0448\u0430\u043d\u0441 \u0442\u043e\u0433\u043e, \u0447\u0442\u043e \u0431\u044b\u043b \u0431\u0440\u043e\u0448\u0435\u043d \u0432\u0437\u0433\u043b\u044f\u0434 \u043d\u0430 ClickHouse \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 \u0437\u0430\u043c\u0435\u043d\u044b, \u0434\u043e\u043b\u0436\u0435\u043d \u0441\u0442\u0440\u0435\u043c\u0438\u0442\u044c\u0441\u044f \u043a \u0435\u0434\u0438\u043d\u0438\u0446\u0435. \u0414\u0430\u043d\u043d\u043e\u0435 \u0443\u0442\u0432\u0435\u0440\u0436\u0434\u0435\u043d\u0438\u0435 \u043f\u043e\u0434\u0440\u0430\u0437\u0443\u043c\u0435\u0432\u0430\u0435\u0442 \u0442\u043e, \u0447\u0442\u043e \u0432 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0435 \u043f\u0440\u0438\u043d\u0438\u043c\u0430\u044e\u0449\u0435\u0433\u043e \u043c\u0435\u0442\u0440\u0438\u043a\u0438 \u0434\u0435\u043c\u043e\u043d\u0430 \u0443\u0436\u0435 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0441\u044f \u0441\u0442\u043e\u0440\u043e\u043d\u043d\u044f\u044f \u0440\u0435\u0430\u043b\u0438\u0437\u0430\u0446\u0438\u044f, \u043d\u0430\u043f\u0440\u0438\u043c\u0435\u0440 carbonwriter \u0438\u043b\u0438 go-carbon. ClickHouse \u0445\u043e\u0440\u043e\u0448\u043e \u0440\u0435\u0448\u0430\u0435\u0442 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-53944","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.1.1 - aioseo.com -->\n\t<meta name=\"description\" content=\".\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/clickhouse-graphite-kak-znachitelno-umenshit-potreblyaemoe-mesto-na-diskah\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.1.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"et_EE\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47ClickHouse + Graphite: \u043a\u0430\u043a \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0443\u043c\u0435\u043d\u044c\u0448\u0438\u0442\u044c \u043f\u043e\u0442\u0440\u0435\u0431\u043b\u044f\u0435\u043c\u043e\u0435 \u043c\u0435\u0441\u0442\u043e \u043d\u0430 \u0434\u0438\u0441\u043a\u0430\u0445 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\".\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/clickhouse-graphite-kak-znachitelno-umenshit-potreblyaemoe-mesto-na-diskah\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-12-12T21:00:00+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-18T11:01:53+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47ClickHouse + Graphite: kuidas oluliselt v\u00e4hendada kettaruumi kasutust | ProHoster","description":".","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/clickhouse-graphite-kak-znachitelno-umenshit-potreblyaemoe-mesto-na-diskah","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47ClickHouse + Graphite: \u043a\u0430\u043a \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u0443\u043c\u0435\u043d\u044c\u0448\u0438\u0442\u044c \u043f\u043e\u0442\u0440\u0435\u0431\u043b\u044f\u0435\u043c\u043e\u0435 \u043c\u0435\u0441\u0442\u043e \u043d\u0430 \u0434\u0438\u0441\u043a\u0430\u0445 | ProHoster","og:description":".","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/clickhouse-graphite-kak-znachitelno-umenshit-potreblyaemoe-mesto-na-diskah","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-12-12T21:00:00+00:00","article:modified_time":"2020-02-18T11:01:53+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"53944","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 09:26:21","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 20:15:54","updated":"2026-01-24 09:26:21","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/53944","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=53944"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/53944\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=53944"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=53944"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=53944"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}