{"id":33803,"date":"2019-10-31T21:54:45","date_gmt":"2019-10-31T18:54:45","guid":{"rendered":"https:\/\/prohoster.info\/blog\/on-vam-ne-drook\/"},"modified":"2019-10-31T21:54:45","modified_gmt":"2019-10-31T18:54:45","slug":"on-vam-ne-drook","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/on-vam-ne-drook","title":{"rendered":"Ta ei ole teile dRook","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Seoses Rooki suurenemisega soovin r\u00e4\u00e4kida selle varjatud probleemidest ja takistustest, millega te v\u00f5ib-olla kokku puutute.<\/p>\n

Minust: Ceph haldamise kogemus alates versioonist hammer, kogukonna asutaja. t.me\/ceph_ru<\/a><\/noindex> Telegramis.<\/p>\n

Et mitte olla s\u00f5nam\u00e4nge, viitan ma Habr's aktsepteeritud (j\u00e4rgides reitingut) postitustele Ceph probleemidest. Enamikku neist probleemidest olen ka ise kogenud. Kasutatud materjalide lingid on postituse l\u00f5pus.<\/p>\n

Rooki postituses ei mainita Cephi lihtsalt niisama \u2014 Rook on p\u00f5him\u00f5tteliselt Ceph, m\u00e4hitud Kubernetesesse, seega p\u00e4rib see k\u00f5ik selle probleemid. Alustame Ceph probleemidest.
\n<\/a><\/noindex><\/p>\n

Klastri haldamise lihtsustamine<\/h3>\n

\n\u00dcks Rooki eelistest on Cephi mugav haldamine l\u00e4bi Kubernetes.<\/p>\n

Siiski sisaldab Ceph \u00fcle 1000 seadistuse parameetri, samas kui Rooki kaudu saame muuta vaid v\u00e4iksema osa neist.<\/p>\n

N\u00e4ide Luminous'e pealt
\n> ceph daemon mon.a config show | wc -l
\n1401<\/p><\/blockquote>\n

Rooki turundatakse kui mugavat viisi Cephi paigaldamiseks ja v\u00e4rskendamiseks.<\/b>
\nCephi paigaldamisel ilma Rookita pole probleeme \u2014 Ansible'i playbooki saab kirjutada 30 minutiga, kuid v\u00e4rskendamisega on palju probleeme. <\/p>\n

Tsitaat Krok'i postitusest<\/i><\/p>\n

N\u00e4ide: crush seadistused ei t\u00f6\u00f6ta p\u00e4rast v\u00e4rskendust hummersist jewel'ile<\/p>\n

> ceph osd crush show-tunables
\n{
\n\u2026
\n \u201estraw_calc_version\u201d: 1,
\n \u201eallowed_bucket_algs\u201d: 22,
\n \u201eprofile\u201d: \u201eunknown\u201d,
\n \u201eoptimal_tunables\u201d: 0,
\n\u2026
\n}<\/p><\/blockquote>\n

Kuid isegi v\u00e4ikeste versioonide puhul on probleeme.<\/p>\n

N\u00e4ide: Versiooni 12.2.6 v\u00e4rskendamine viib klastrite seisundisse health err ja tinglikult vigase PG.
\nceph.com\/releases\/v12-2-8-released<\/a><\/noindex><\/p>\n

\u00c4ra v\u00e4rskenda, oota ja testige? Aga me kasutame Rooki just mugava v\u00e4rskendamise t\u00f5ttu.<\/p>\n

Rooki klastrite h\u00e4daolukordade taastamise keerukus <\/h3>\n

\nN\u00e4ide: OSD kukub, visates endale vigu. Kahtlustate, et probleem on m\u00f5nes seadistuse parameetris, soovite muuta konfi konkreetse demoni jaoks, kuid ei saa, kuna teil on Kubernetes ja DaemonSet.<\/p>\n

Alternatiivi pole. ceph tell osd.Num injectargs ei t\u00f6\u00f6ta \u2014 OSD on ju maas.<\/p>\n

Silumine keerukus<\/h3>\n

\nM\u00f5ne seadistuse ja j\u00f5udluse testimise jaoks on vajalik otsene \u00fchendus OSD demoni soketiga. Rooki puhul tuleb k\u00f5igepealt leida \u00f5ige konteiner, seej\u00e4rel siseneda sellesse, avastada, et debug t\u00f6\u00f6riistad puuduvad ja v\u00e4ga pettuda.<\/p>\n

OSD-de j\u00e4rjepidev t\u00f5usmise keerukus<\/h3>\n

\nN\u00e4ide: OSD kukub OOM-i t\u00f5ttu, algab \u00fcmberjaotamine ja seej\u00e4rel kukuvad j\u00e4rgmised.<\/p>\n

Lahendus: T\u00f5stke OSD \u00fche kaupa, oodake, kuni see on t\u00e4ielikult klastrisse \u00fchendatud, ja t\u00f5stke j\u00e4rgmised. (Rohkem teavet leiate raportist Ceph. Katastroofide anatoomia).<\/p>\n

Baremetal installatsiooni korral tehakse see lihtsalt k\u00e4sitsi, Rooki ja \u00fche OSD puhul node'is ei ole erilisi probleeme, probleemid j\u00e4rjestikuste t\u00f5usudega tekivad juhul, kui OSD > 1 node'is. <\/p>\n

Muidugi, need on lahendatavad, kuid me ju toome Rooki lihtsustamiseks, aga saame keerukuse.<\/p>\n

Ceph deemonite limiitide valimise keerukus.<\/h3>\n

\nBaremetal Ceph installatsiooni korral on vajalikud ressursid klastrisse piisavalt lihtne arvutada \u2014 valemid ja uuringud on olemas. N\u00f5rkade CPU-de kasutamisel peate siiski l\u00e4bima rea j\u00f5udluskatsed, \u00f5ppima, mis on Numa, kuid see on ikkagi lihtsam kui Rookis.<\/p>\n

Rooki puhul, v\u00e4lja arvatud m\u00e4lu piirid, mille saab arvutada, tekib k\u00fcsimus CPU piire seadmise kohta.<\/p>\n

Ja siin peate tegelema j\u00f5udluskatsed. Kui piirid on liiga madalad, saate aeglase klastriga, kui m\u00e4\u00e4rate unlimi, siis saate aktiivse CPU kasutuse rebalance'i ajal, mis avaldab halba m\u00f5ju teie rakendustele Kuberneteses.<\/p>\n

Probleemid v\u00f5rgus kommunikatsiooniga v1.<\/h3>\n

\nCeph jaoks soovitatakse kasutada 2x10Gb v\u00f5rku. \u00dcks kliendiliikluseks, teine Cephi teenuste vajaduste (rebalance) jaoks. Kui elate Cephiga baremetal keskkonnas, on see jaotus lihtsalt seadistatav, kui aga elate Rookiga, p\u00f5hjustab v\u00f5rgu jagamine teile probleeme, kuna kaugel pole iga klastrikonfiguratsioon, mis lubaks pod-ile kaks erinevat v\u00f5rku.<\/p>\n

Probleemid v\u00f5rgus kommunikatsiooniga v2.<\/h3>\n

\nKui loobute v\u00f5rgu jagamisest, siis rebalance'i ajal blokeerib Cephi liiklus kogu kanali, ja teie rakendused Kuberneteses hakkavad t\u00f5rkuma v\u00f5i kukuvad kokku. Ceph rebalance'i kiirus on v\u00f5imalik v\u00e4hendada, kuid siis pika rebalance'i t\u00f5ttu suureneb risk, et teine node kukub klastrist v\u00e4lja diskide v\u00f5i OOM-i t\u00f5ttu, ja seal on garantii, et klaster muutub read-only.<\/p>\n

Pikk rebalance \u2014 pikad rakenduste t\u00f5rked.<\/h3>\n

\nTsitaat postitusest Ceph. Katastroofide anatoomia.<\/i><\/p>\n

Testklastri j\u00f5udlus:<\/p>\n

4 KB suurune kirjutamisoperatsioon v\u00f5tab 1 ms aega, j\u00f5udlus 1000 operatsiooni\/sekundis 1 voos.<\/p>\n

4 MB operatsioon (objekti suurus) v\u00f5tab 22 ms aega, j\u00f5udlus 45 operatsiooni\/sekundis.<\/p>\n

Seega, kui \u00fcks kolmest domeenist eba\u00f5nnestub, on klaster m\u00f5nda aega degradeerunud seisundis, ja pool kuumadest objektidest jaotub erinevatesse versioonidesse, siis pool kirjutamisoperatsioonidest algab sundrestaureerimisega.<\/p>\n

Sundrestaureerimise aega arvutame umbkaudu \u2014 kirjutamisoperatsioonid degradatsiooniga objektil.<\/p>\n

Esiteks loeme 4 MB 22 ms, kirjutame 22 ms, ja siis kirjutame 1 ms jooksul 4 KB tegelikke andmeid. Kokku 45 ms \u00fche kirjutamisoperatsiooni jaoks degradeerunud objektil SSD-l, kui meie tavaline j\u00f5udlus oli 1 ms \u2014 j\u00f5udluse langus 45 korda.<\/p>\n

Mida suurem on degradeerunud objektide protsent, seda hullemaks k\u00f5ik muutub.<\/p><\/blockquote>\n

Seega on rebalanseerimise kiirus kriitilise t\u00e4htsusega klastrite korrektseks toimimiseks.<\/p>\n

Kohandatud serveri seaded ceph'i jaoks<\/h3>\n

\nCeph vajab sageli spetsiifilist hosti h\u00e4\u00e4lestust. <\/p>\n

N\u00e4ide: sysctl s\u00e4tted ja samad JumboFrame'id, m\u00f5ned neist seadetest v\u00f5ivad teie koormusele negatiivselt m\u00f5juda.<\/p>\n

Rook'i tegelik vajadus on kahtluse all<\/h3>\n

\nKui olete pilves ja teil on salvestus teie pilveteenuse pakkujalt, on see oluliselt mugavam. <\/p>\n

Kui olete oma serverites, siis ceph'i haldamine on mugavam ilma kubernetest. <\/p>\n

Kas rendite servereid m\u00f5nest odavast hostimisest? Siis ootab teid ees palju l\u00f5busat v\u00f5rgu, selle latentsuse ja l\u00e4bilaskev\u00f5imega, mis kindlasti m\u00f5jutab ceph'i negatiivselt.<\/p>\n

Kokkuv\u00f5tteks:<\/b> Kuberentes'e juurutamine ja salvestuse juurutamine on erinevad \u00fclesanded erinevate sisendi ja lahenduste variatsioonidega \u2014 nende segamine t\u00e4hendab v\u00f5imaliku ohtliku kompromissi tegemist \u00fche v\u00f5i teise kasuks. Nende lahenduste \u00fchitamine on projekteerimise etapis v\u00e4ga keeruline, r\u00e4\u00e4kimata ekspluateerimise ajast.<\/p>\n

Kasutatud kirjanduse nimekiri:<\/p>\n

Postitus #1<\/a><\/noindex> Aga teie r\u00e4\u00e4gite Ceph'ist\u2026 kui hea see on tegelikult?
\nPostitus #2<\/a><\/noindex> Ceph. Katastroofi anatoomia
\n
Allikas: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0412 \u0441\u0432\u044f\u0437\u0438 \u0441 \u043d\u0430\u0431\u0438\u0440\u0430\u044e\u0449\u0435\u0439 \u043f\u043e\u043f\u0443\u043b\u044f\u0440\u043d\u043e\u0441\u0442\u044c\u044e Rook \u0445\u043e\u0447\u0435\u0442\u0441\u044f \u043f\u043e\u0433\u043e\u0432\u043e\u0440\u0438\u0442\u044c \u043e \u0435\u0433\u043e \u043f\u043e\u0434\u0432\u043e\u0434\u043d\u044b\u0445 \u043a\u0430\u043c\u043d\u044f\u0445 \u0438 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u0430\u0445, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0436\u0434\u0443\u0442 \u0432\u0430\u0441 \u043d\u0430 \u043f\u0443\u0442\u0438. \u041e \u0441\u0435\u0431\u0435: \u041e\u043f\u044b\u0442 \u0430\u0434\u043c\u0438\u043d\u0438\u0441\u0442\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f ceph \u0441 \u0432\u0435\u0440\u0441\u0438\u0438 hammer, \u043e\u0441\u043d\u043e\u0432\u0430\u0442\u0435\u043b\u044c \u043a\u043e\u043c\u044c\u044e\u043d\u0438\u0442\u0438 t.me\/ceph_ru \u0432 \u0442\u0435\u043b\u0435\u0433\u0440\u0430\u043c. \u0414\u0430\u0431\u044b \u043d\u0435 \u0431\u044b\u0442\u044c \u0433\u043e\u043b\u043e\u0441\u043b\u043e\u0432\u043d\u044b\u043c \u044f \u0431\u0443\u0434\u0443 \u0441\u0441\u044b\u043b\u0430\u0442\u044c\u0441\u044f \u043d\u0430 \u043f\u0440\u0438\u043d\u044f\u0442\u044b\u0435 \u0445\u0430\u0431\u0440\u043e\u043c (\u0441\u0443\u0434\u044f \u043f\u043e \u0440\u0435\u0439\u0442\u0438\u043d\u0433\u0443) \u043f\u043e\u0441\u0442\u044b \u043e \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u0430\u0445 \u0441 ceph. \u0421 \u0431\u041e\u043b\u044c\u0448\u0435\u0439 \u0447\u0430\u0441\u0442\u044c\u044e \u043f\u0440\u043e\u0431\u043b\u0435\u043c \u0432 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-33803","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Ta ei ole teile dRook | ProHoster","description":"Seoses Rook'i kasvava populaarsusega soovin r\u00e4\u00e4kida selle varjatud takistustest ja probleemidest, mis teid teel ootavad. Minust: Ceph'i haldamise kogemus alates versioonist hammer, kogukonna asutaja","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/on-vam-ne-drook","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041e\u043d \u0432\u0430\u043c \u043d\u0435 \u0434Rook | ProHoster","og:description":"\u0412 \u0441\u0432\u044f\u0437\u0438 \u0441 \u043d\u0430\u0431\u0438\u0440\u0430\u044e\u0449\u0435\u0439 \u043f\u043e\u043f\u0443\u043b\u044f\u0440\u043d\u043e\u0441\u0442\u044c\u044e Rook \u0445\u043e\u0447\u0435\u0442\u0441\u044f \u043f\u043e\u0433\u043e\u0432\u043e\u0440\u0438\u0442\u044c \u043e \u0435\u0433\u043e \u043f\u043e\u0434\u0432\u043e\u0434\u043d\u044b\u0445 \u043a\u0430\u043c\u043d\u044f\u0445 \u0438 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u0430\u0445, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0436\u0434\u0443\u0442 \u0432\u0430\u0441 \u043d\u0430 \u043f\u0443\u0442\u0438. \u041e \u0441\u0435\u0431\u0435: \u041e\u043f\u044b\u0442 \u0430\u0434\u043c\u0438\u043d\u0438\u0441\u0442\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f ceph \u0441 \u0432\u0435\u0440\u0441\u0438\u0438 hammer, \u043e\u0441\u043d\u043e\u0432\u0430\u0442\u0435\u043b\u044c \u043a\u043e\u043c\u044c\u044e\u043d\u0438\u0442\u0438","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/on-vam-ne-drook","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:54:45+00:00","article:modified_time":"2019-10-31T18:54:45+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"33803","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 16:46:22","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 02:33:24","updated":"2026-01-21 16:46:22","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/33803","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=33803"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/33803\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=33803"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=33803"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=33803"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}