ClickHouse + Graphite: yadda ake rage yawan amfani da sararin samaniya

Salam, habr.

Idan wani yayi amfani da tsarin graphite-web kuma sun ci karo da batun aikin ajiya waswasi (IO, sararin faifai da aka cinye), to, damar da aka jefa ClickHouse a matsayin maye ya kamata ya kasance ɗaya. Wannan bayanin yana nuna cewa an riga an yi amfani da aiwatar da wani ɓangare na uku azaman daemon mai karɓar awo, misali carbon marubuci ko ga carbon.

ClickHouse yana magance matsalolin da aka bayyana da kyau. Misali, bayan canja wurin 2TiB na bayanai daga raɗaɗi, sun dace da 300GiB. Ba zan tsaya kan kwatancen daki-daki ba; akwai labarai da yawa akan wannan batu. Bugu da kari, har zuwa kwanan nan, ba komai yayi daidai ba tare da ma'ajiyar ClickHouse.

Matsaloli tare da cinye sarari

A kallon farko, komai ya kamata yayi aiki da kyau. Masu bi takardun, ƙirƙiri saiti don tsarin ma'aunin ma'auni (ƙari retention), sannan ƙirƙirar tebur bisa ga shawarar da aka zaɓa na baya don graphite-web: carbon-clickhouse+graphite-clickhouse ko graphhouse, dangane da wace tari aka yi amfani da shi. Kuma... lokacin bam ya tashi.

Don fahimtar wanene, kuna buƙatar sanin yadda abubuwan da ake sakawa ke aiki da ƙarin hanyar rayuwa na bayanai a cikin tebur na injunan dangin *MergeTree ClickHouse (shafukan da aka ɗauka daga gabatarwa Alexey Zatelepin:

• Saka блок data. A cikin yanayinmu, ma'auni ne ya zo.
  
• Kowane irin wannan toshe ana jerawa bisa maɓalli kafin a rubuta shi zuwa faifai. ORDER BYƙayyade lokacin ƙirƙirar tebur.
• Bayan an jera, кусок (part) an rubuta bayanai zuwa faifai.
  
• Sabar tana sa ido a bayan fage ta yadda ba za a sami irin waɗannan guntu ba, kuma ta ƙaddamar da bango слияния (merge, Lahira hade).
  
  
• Sabar tana daina aiki da haɗin kai da kanta da zaran bayanai sun daina shiga cikin rayayye партицию (partition), amma zaka iya fara aikin da hannu tare da umarnin OPTIMIZE.
• Idan yanki ɗaya ne kawai ya rage a cikin ɓangaren, to ba za ku iya gudanar da haɗakar ta amfani da umarnin da aka saba ba; dole ne ku yi amfani da shi. OPTIMIZE ... FINAL

Don haka, ma'auni na farko sun zo. Kuma sun dauki wani sarari. Abubuwan da ke biyo baya na iya bambanta da ɗan dangane da abubuwa da yawa:

• Maɓallin rarraba na iya zama ko dai ƙanƙanta (rana ɗaya) ko babba (watanni da yawa).
• Saitin riƙon yana iya dacewa da mahimmin madaidaitan madaidaitan bayanan tattara bayanai a cikin ɓangaren aiki (inda ake yin rikodin awo), ko wataƙila a'a.
• Idan akwai mai yawa bayanai, sa'an nan na farko chunks, wanda saboda baya hadawa iya riga ya zama babba (idan ka zabi wani mara kyau partitioning key), ba zai ci kansu da sabo ne kananan chunks.

Kuma yana ƙarewa koyaushe. Wurin da awo ke mamaye a ClickHouse yana ƙaruwa kawai idan:

• kar a nema OPTIMIZE ... FINAL da hannu ko
• kar a saka bayanai a cikin duk ɓangarori a kan ci gaba, ta yadda ba dade ko ba dade za a fara haɗa bayanan baya

Hanya na biyu da alama ita ce mafi sauƙi don aiwatarwa kuma, saboda haka, ba daidai ba ne kuma an fara gwadawa.
Na rubuta rubutun python mai sauƙi wanda ya aiko da ma'auni na yau da kullun don shekaru 4 da suka gabata kuma yana gudana cron kowace awa.
Tun da dukan aikin ClickHouse DBMS ya dogara ne akan gaskiyar cewa wannan tsarin zai yi duk aikin baya ko ba da jimawa ba, amma ba a san lokacin da ba, na kasa jira har sai tsofaffin manyan guda deign don fara haɗuwa tare da sababbin ƙananan. . Ya bayyana a fili cewa muna buƙatar nemo hanyar da za a iya sarrafa ingantawar tilastawa.

Bayani a cikin tsarin tsarin ClickHouse

Bari mu dubi tsarin tebur tsarin. sassa. Wannan cikakken bayani ne game da kowane yanki na duk tebur akan sabar ClickHouse. Ya ƙunshi, a tsakanin wasu abubuwa, ginshiƙai masu zuwa:

• db suna (database);
• sunan tebur (table);
• sunan partition da ID (partition & partition_id);
• lokacin da aka kafa guntu (modification_time);
• mafi ƙanƙanta da matsakaicin kwanan wata a cikin yanki (ana yin rabuwa da rana) (min_date & max_date);

Akwai kuma tebur tsarin.graphite_retentions, tare da fagage masu ban sha'awa:

• db suna (Tables.database);
• sunan tebur (Tables.table);
• shekarun awo lokacin da yakamata a yi amfani da tara na gaba (age);

Saboda haka:

1. Muna da tebur na chunks da tebur na ƙa'idodin tarawa.
2. Muna haɗa hanyar haɗin su kuma muna samun duk tebur * GraphiteMergeTree.
3. Muna neman duk sassan da:
   • fiye da guda ɗaya
   • ko kuma lokaci ya yi da za a yi amfani da ƙa'idar tarawa ta gaba, kuma modification_time sun girmi wannan lokacin.

Aiwatarwa

Wannan bukata

SELECT
    concat(p.database, '.', p.table) AS table,
    p.partition_id AS partition_id,
    p.partition AS partition,
    -- Самое "старое" правило, которое может быть применено для
    -- партиции, но не в будущем, см (*)
    max(g.age) AS age,
    -- Количество кусков в партиции
    countDistinct(p.name) AS parts,
    -- За самую старшую метрику в партиции принимается 00:00:00 следующего дня
    toDateTime(max(p.max_date + 1)) AS max_time,
    -- Когда партиция должна быть оптимизированна
    max_time + age AS rollup_time,
    -- Когда самый старый кусок в партиции был обновлён
    min(p.modification_time) AS modified_at
FROM system.parts AS p
INNER JOIN
(
    -- Все правила для всех таблиц *GraphiteMergeTree
    SELECT
        Tables.database AS database,
        Tables.table AS table,
        age
    FROM system.graphite_retentions
    ARRAY JOIN Tables
    GROUP BY
        database,
        table,
        age
) AS g ON
    (p.table = g.table)
    AND (p.database = g.database)
WHERE
    -- Только активные куски
    p.active
    -- (*) И только строки, где правила аггрегации уже должны быть применены
    AND ((toDateTime(p.max_date + 1) + g.age) < now())
GROUP BY
    table,
    partition
HAVING
    -- Только партиции, которые младше момента оптимизации
    (modified_at < rollup_time)
    -- Или с несколькими кусками
    OR (parts > 1)
ORDER BY
    table ASC,
    partition ASC,
    age ASC

yana dawo da kowane bangare na *GraphiteMergeTree tebur wanda hadawarsu yakamata ya 'yantar da sarari. Abinda ya rage shine a bi su duka tare da buƙata OPTIMIZE ... FINAL. Har ila yau, aiwatarwa na ƙarshe yana la'akari da gaskiyar cewa babu buƙatar taɓa sassan tare da rikodi mai aiki.

Wannan shi ne ainihin abin da aikin yake yi graphite-ch-optimizer. Tsoffin abokan aiki daga Yandex.Market sun gwada shi a cikin samarwa, ana iya ganin sakamakon aikin a ƙasa.

Idan kun gudanar da shirin akan uwar garken tare da ClickHouse, kawai zai fara aiki a yanayin daemon. Sau ɗaya a sa'a guda za a aiwatar da buƙatar, duba ko sabbin ɓangarori waɗanda suka girmi kwanaki uku sun bayyana waɗanda za a iya inganta su.

Shirye-shiryen mu na gaggawa shine samar da aƙalla fakitin bashi, kuma idan zai yiwu, kuma rpm.

Maimakon a ƙarshe

A cikin watanni 9+ da suka gabata ina cikin kamfani na Wasannin Inno ɓata lokaci mai yawa don yin tinkering a mahadar ClickHouse da graphite-web. Kwarewa ce mai kyau, wanda ya haifar da saurin sauyawa daga raɗaɗi zuwa ClickHouse azaman ma'aunin ma'auni. Ina fatan wannan labarin wani abu ne na farkon jerin abubuwan da muka inganta a sassa daban-daban na wannan tarin, da abin da za a yi a nan gaba.

An kashe lita da yawa na giya da kwanakin admin don haɓaka buƙatar, tare da v0 shaidan, wanda nake son in nuna masa godiya ta. Kuma don sake duba wannan labarin.

Shafin aikin akan github

source: www.habr.com