{"id":113018,"date":"2024-01-20T23:28:04","date_gmt":"2024-01-20T21:28:06","guid":{"rendered":"https:\/\/prohoster.info\/blog\/novosti-interneta\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu"},"modified":"2024-01-20T23:28:04","modified_gmt":"2024-01-20T21:28:06","slug":"opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","status":"publish","type":"post","link":"https:\/\/prohoster.info\/pt\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","title":{"rendered":"Experi\u00eancia na cria\u00e7\u00e3o de um armazenamento Ceph com capacidade de tebibits por segundo","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Um engenheiro da Clyso resumiu a experi\u00eancia de criar um cluster de armazenamento baseado em um sistema distribu\u00eddo de alta disponibilidade Ceph, com uma capacidade de mais de um tebibyte por segundo. \u00c9 not\u00e1vel que este \u00e9 o primeiro cluster baseado em Ceph a atingir tal desempenho, mas antes de alcan\u00e7ar o resultado apresentado, os engenheiros tiveram que superar uma s\u00e9rie de desafios dif\u00edceis de identificar. <\/p>\n

Por exemplo, para aumentar o desempenho em 10-20%, foi suficiente ativar nos servidores, nas configura\u00e7\u00f5es de economia de energia da BIOS, apenas o modo de desempenho m\u00e1ximo e desativar o c-state (o c-state altera as configura\u00e7\u00f5es de economia de energia dependendo da carga, o que afeta o Ceph). Tamb\u00e9m foi descoberto que ao usar unidades NVMe, o n\u00facleo do Linux gastava um tempo consider\u00e1vel processando bloqueios spin durante a atualiza\u00e7\u00e3o do mapeamento IOMMU. Desativar o IOMMU no n\u00facleo resultou em um aumento significativo no desempenho nos testes de leitura e escrita de blocos de 4MB. <\/p>\n

Desligar o IOMMU n\u00e3o resolveu o problema de queda de desempenho durante a grava\u00e7\u00e3o aleat\u00f3ria de blocos de 4KB. Ao investigar, os engenheiros descobriram corre\u00e7\u00f5es nos scripts de compila\u00e7\u00e3o do Ceph nos projetos Gentoo e Ubuntu, que inclu\u00edam a constru\u00e7\u00e3o com a op\u00e7\u00e3o RelWithDebInfo, pois, com isso, o GCC aplicava o modo de otimiza\u00e7\u00e3o '-O2', durante o qual o desempenho do Ceph aumentava significativamente. A compila\u00e7\u00e3o com a biblioteca TCMalloc tamb\u00e9m levou \u00e0 redu\u00e7\u00e3o do desempenho. A mudan\u00e7a nas op\u00e7\u00f5es de compila\u00e7\u00e3o e a interrup\u00e7\u00e3o do uso do TCMalloc resultou em uma diminui\u00e7\u00e3o do tempo de compacta\u00e7\u00e3o em tr\u00eas vezes e aumentou a performance de grava\u00e7\u00f5es aleat\u00f3rias em blocos de 4K em duas vezes. No final, tamb\u00e9m foi realizada uma otimiza\u00e7\u00e3o das configura\u00e7\u00f5es do Reef RocksDB e dos grupos de coloca\u00e7\u00e3o (PG). <\/p>\n

O cluster \u00e9 formado por 68 n\u00f3s baseados em servidores<\/a> Dell PowerEdge R6615 com CPU AMD EPYC 9454P 48C\/96T. Cada n\u00f3 cont\u00e9m 10 unidades NVMe Dell de 15,36TB, dois adaptadores Ethernet 100GbE Mellanox ConnectX-6 e 192GB de RAM. O software \u00e9 baseado no Ubuntu 20.04.6 e Ceph 17.2.7. No cluster, h\u00e1 630 OSD (Object Storage Daemon, processo em segundo plano que gerencia o armazenamento de dados no armazenamento local, um OSD por unidade NVMe), tr\u00eas processos MON (monitor, que rastreia o estado do cluster) e um processo MGR (Manager, servi\u00e7o de gerenciamento). O tamanho do armazenamento \u00e9 de 8,2 PB. <\/p>\n

A largura de banda durante opera\u00e7\u00f5es sequenciais de leitura de blocos de 4M foi de 1025 GiB\/s, e para grava\u00e7\u00e3o \u2014 270 GiB\/s. Durante a leitura aleat\u00f3ria de blocos de 4KB, o desempenho foi de 25,5 milh\u00f5es de opera\u00e7\u00f5es de leitura por segundo e 4,9 milh\u00f5es durante a grava\u00e7\u00e3o. A ativa\u00e7\u00e3o da criptografia reduziu a largura de banda de leitura para aproximadamente 750 GiB\/s. Com a utiliza\u00e7\u00e3o de c\u00f3digos de corre\u00e7\u00e3o de erros EC62, a largura de banda foi de 547 GiB\/s durante a leitura e 387 GiB\/s durante a grava\u00e7\u00e3o (a velocidade de grava\u00e7\u00e3o foi superior \u00e0quela sem c\u00f3digos de corre\u00e7\u00e3o), enquanto no acesso aleat\u00f3rio obteve-se 3,4M IOPS na leitura e 936K IOPS na grava\u00e7\u00e3o.

\"Experi\u00eancia<\/center>
\"Experi\u00eancia<\/center> <\/p>\n

Al\u00e9m disso, \u00e9 importante destacar que em setembro um marco semelhante de largura de banda em tebibytes por segundo foi alcan\u00e7ado no cluster de armazenamento de exabytes do CERN, implementado com o sistema de armazenamento distribu\u00eddo aberto EOS, baseado no protocolo XRootD.
\n
Fonte: opennet.ru<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0418\u043d\u0436\u0435\u043d\u0435\u0440 \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Clyso \u043e\u0431\u043e\u0431\u0449\u0438\u043b \u043e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0439 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0451\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u044b Ceph \u0441 \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e, \u043f\u0440\u0435\u0432\u044b\u0448\u0430\u044e\u0449\u0435\u0439 \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443. \u041e\u0442\u043c\u0435\u0447\u0430\u0435\u0442\u0441\u044f, \u0447\u0442\u043e \u044d\u0442\u043e \u043f\u0435\u0440\u0432\u044b\u0439 \u043a\u043b\u0430\u0441\u0442\u0435\u0440 \u043d\u0430 \u0431\u0430\u0437\u0435 Ceph, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0441\u043c\u043e\u0433 \u0434\u043e\u0441\u0442\u0438\u0433\u043d\u0443\u0442\u044c \u043f\u043e\u0434\u043e\u0431\u043d\u043e\u0433\u043e \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044f, \u043d\u043e \u043f\u0435\u0440\u0435\u0434 \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u0438\u0435\u043c \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u043d\u043e\u0433\u043e \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0430 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u0430\u043c \u043f\u043e\u0442\u0440\u0435\u0431\u043e\u0432\u0430\u043b\u043e\u0441\u044c \u043f\u0440\u0435\u043e\u0434\u043e\u043b\u0435\u0442\u044c \u0441\u0435\u0440\u0438\u044e \u043d\u0435 \u043e\u0447\u0435\u0432\u0438\u0434\u043d\u044b\u0445 \u043f\u043e\u0434\u0432\u043e\u0434\u043d\u044b\u0445 \u043a\u0430\u043c\u043d\u0435\u0439. \u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u0434\u043b\u044f \u043f\u043e\u0432\u044b\u0448\u0435\u043d\u0438\u044f \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u043d\u0430 10-20% \u0431\u044b\u043b\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":113019,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-113018","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Experi\u00eancia na cria\u00e7\u00e3o de armazenamento Ceph com largura de banda de um tebibyte por segundo | ProHoster","description":"Um engenheiro da Clyso resumiu a experi\u00eancia de cria\u00e7\u00e3o de um cluster de armazenamento baseado em um sistema distribu\u00eddo tolerante a falhas Ceph, com largura de banda superior a um tebibyte por segundo.","canonical_url":"https:\/\/prohoster.info\/pt\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"pt_PT","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 Ceph c \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443 | ProHoster","og:description":"\u0418\u043d\u0436\u0435\u043d\u0435\u0440 \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Clyso \u043e\u0431\u043e\u0431\u0449\u0438\u043b \u043e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0439 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0451\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u044b Ceph \u0441 \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e, \u043f\u0440\u0435\u0432\u044b\u0448\u0430\u044e\u0449\u0435\u0439 \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443.","og:url":"https:\/\/prohoster.info\/pt\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2024-01-20T21:28:06+00:00","article:modified_time":"2024-01-20T21:28:06+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"113018","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2026-02-09 16:50:41","updated":"2026-02-09 16:50:41","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/posts\/113018","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/comments?post=113018"}],"version-history":[{"count":1,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/posts\/113018\/revisions"}],"predecessor-version":[{"id":158784,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/posts\/113018\/revisions\/158784"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/media\/113019"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/media?parent=113018"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/categories?post=113018"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/pt\/wp-json\/wp\/v2\/tags?post=113018"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}