{"id":113018,"date":"2024-01-20T23:28:04","date_gmt":"2024-01-20T21:28:06","guid":{"rendered":"https:\/\/prohoster.info\/blog\/novosti-interneta\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu"},"modified":"2024-01-20T23:28:04","modified_gmt":"2024-01-20T21:28:06","slug":"opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","title":{"rendered":"Experien\u021b\u0103 \u00een crearea unui depozit Ceph cu l\u0103\u021bimi de band\u0103 de tebibait pe secund\u0103.","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Un inginer de la compania Clyso a sintetizat experien\u021ba cre\u0103rii unui cluster de stocare bazat pe un sistem distribuit rezistent la defec\u021biuni Ceph, cu o l\u0103\u021bime de band\u0103 care dep\u0103\u0219e\u0219te un tebibait pe secund\u0103. Se observ\u0103 c\u0103 acesta este primul cluster bazat pe Ceph care a reu\u0219it s\u0103 ating\u0103 o astfel de valoare, dar \u00eenainte de a ob\u021bine rezultatul prezentat, inginerii au trebuit s\u0103 dep\u0103\u0219easc\u0103 o serie de capcane evidente. <\/p>\n

De exemplu, pentru a \u00eembun\u0103t\u0103\u021bi performan\u021ba cu 10-20%, a fost suficient s\u0103 activam pe servere, \u00een set\u0103rile BIOS de economisire a energiei, func\u021bionarea doar \u00een modul de performan\u021b\u0103 maxim\u0103 \u0219i s\u0103 dezactiv\u0103m c-state (c-state modific\u0103 parametrii de economisire a energiei \u00een func\u021bie de sarcin\u0103, ceea ce afecteaz\u0103 Ceph). De asemenea, s-a dovedit c\u0103, atunci c\u00e2nd se utilizeaz\u0103 unit\u0103\u021bi NVMe, nucleul Linux petrece o perioad\u0103 semnificativ\u0103 de timp proces\u00e2nd blocajele spin \u00een timpul actualiz\u0103rii map\u0103rii IOMMU. Dezactivarea IOMMU \u00een nucleu a dus la o cre\u0219tere notabil\u0103 a performan\u021bei \u00een testele de scriere \u0219i citire a blocurilor de 4MB. <\/p>\n

Dezactivarea IOMMU nu a solu\u021bionat problema sc\u0103derii performan\u021bei \u00een cazul scrierii aleatorii a blocurilor de 4KB. \u00cen \u00eencercarea de a identifica cauza, inginerii au descoperit corec\u021bii \u00een scripturile de compilare Ceph din proiectele Gentoo \u0219i Ubuntu, care includeau compilarea cu op\u021biunea RelWithDebInfo, deoarece aceasta activa modul de optimizare '-O2' \u00een GCC, ceea ce \u00eembun\u0103t\u0103\u021bea semnificativ performan\u021ba Ceph. De asemenea, sc\u0103derea performan\u021bei era cauzat\u0103 de compilarea cu biblioteca TCMalloc. Modificarea flagurilor de compilare \u0219i oprirea utiliz\u0103rii TCMalloc a dus la o reducere de trei ori a timpului de compactare \u0219i o dublare a performan\u021bei opera\u021biunilor de scriere aleatoare cu blocuri de 4K. \u00cen final, au fost realizate optimiz\u0103ri suplimentare ale set\u0103rilor Reef RocksDB \u0219i grupurilor de plasare (PG). <\/p>\n

Clusterul este format din 68 de noduri bazate pe servere<\/a> Dell PowerEdge R6615 cu CPU AMD EPYC 9454P 48C\/96T. Fiecare nod con\u021bine 10 unit\u0103\u021bi NVMe Dell de 15.36TB, dou\u0103 adaptoare Ethernet 100GbE Mellanox ConnectX-6 \u0219i 192 GB RAM. Software-ul este bazat pe Ubuntu 20.04.6 \u0219i Ceph 17.2.7. \u00cen clusterul format din 63 de noduri sunt configurate 630 OSD (Object Storage Daemon, proces de fundal care gestioneaz\u0103 stocarea datelor \u00een stocarea local\u0103, c\u00e2te unul OSD pe unitate NVMe), trei procese MON (monitor, care urm\u0103re\u0219te starea clusterului) \u0219i un proces MGR (Manager, serviciul de administrare). Dimensiunea stoc\u0103rii este de 8.2 PB. <\/p>\n

L\u0103\u021bimea de band\u0103 pentru opera\u021biunile secven\u021biale de citire a blocurilor de 4M a fost de 1025 GiB\/s, iar pentru scriere de 270 GiB\/s. \u00cen cazul citirii aleatorii a blocurilor de 4KB, performan\u021ba a fost de 25,5 milioane de opera\u021biuni de citire pe secund\u0103 \u0219i 4,9 milioane pentru scriere. Activarea cript\u0103rii a redus l\u0103\u021bimea de band\u0103 la citire la aproximativ 750 GiB\/s. Atunci c\u00e2nd s-au utilizat coduri de corectare a erorilor EC62, l\u0103\u021bimea de band\u0103 a fost de 547 GiB\/s pentru citire \u0219i 387 GiB\/s pentru scriere (viteza de scriere a fost mai mare dec\u00e2t f\u0103r\u0103 codurile de corectare), iar \u00een cazul accesului aleatoriu s-au ob\u021binut 3,4M IOPS la citire \u0219i 936K IOPS la scriere.

\"Experien\u021b\u0103<\/center>
\"Experien\u021b\u0103<\/center> <\/p>\n

\u00cen plus, se poate observa c\u0103 \u00een septembrie un prag similar de l\u0103\u021bime de band\u0103 \u00een tebibai pe secund\u0103 a fost atins \u00een clusterul de stocare Exabyte de la CERN, realizat pe baza stoc\u0103rii distribuite deschise EOS, construit pe protocolul XRootD.
\n
Sursa: opennet.ro<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0418\u043d\u0436\u0435\u043d\u0435\u0440 \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Clyso \u043e\u0431\u043e\u0431\u0449\u0438\u043b \u043e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0439 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0451\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u044b Ceph \u0441 \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e, \u043f\u0440\u0435\u0432\u044b\u0448\u0430\u044e\u0449\u0435\u0439 \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443. \u041e\u0442\u043c\u0435\u0447\u0430\u0435\u0442\u0441\u044f, \u0447\u0442\u043e \u044d\u0442\u043e \u043f\u0435\u0440\u0432\u044b\u0439 \u043a\u043b\u0430\u0441\u0442\u0435\u0440 \u043d\u0430 \u0431\u0430\u0437\u0435 Ceph, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0441\u043c\u043e\u0433 \u0434\u043e\u0441\u0442\u0438\u0433\u043d\u0443\u0442\u044c \u043f\u043e\u0434\u043e\u0431\u043d\u043e\u0433\u043e \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044f, \u043d\u043e \u043f\u0435\u0440\u0435\u0434 \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u0438\u0435\u043c \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u043d\u043e\u0433\u043e \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0430 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u0430\u043c \u043f\u043e\u0442\u0440\u0435\u0431\u043e\u0432\u0430\u043b\u043e\u0441\u044c \u043f\u0440\u0435\u043e\u0434\u043e\u043b\u0435\u0442\u044c \u0441\u0435\u0440\u0438\u044e \u043d\u0435 \u043e\u0447\u0435\u0432\u0438\u0434\u043d\u044b\u0445 \u043f\u043e\u0434\u0432\u043e\u0434\u043d\u044b\u0445 \u043a\u0430\u043c\u043d\u0435\u0439. \u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u0434\u043b\u044f \u043f\u043e\u0432\u044b\u0448\u0435\u043d\u0438\u044f \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u043d\u0430 10-20% \u0431\u044b\u043b\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":113019,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-113018","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Experien\u021ba de creare a stoc\u0103rii Ceph cu l\u0103\u021bime de band\u0103 de tebibai pe secund\u0103 | ProHoster","description":"Un inginer de la Clyso a rezumat experien\u021ba de creare a unui cluster de stocare bazat pe un sistem distribuit rezistent la defectiuni Ceph cu o l\u0103\u021bime de band\u0103 ce dep\u0103\u0219e\u0219te tebibai pe secund\u0103.","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 Ceph c \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443 | ProHoster","og:description":"\u0418\u043d\u0436\u0435\u043d\u0435\u0440 \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Clyso \u043e\u0431\u043e\u0431\u0449\u0438\u043b \u043e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0439 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0451\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u044b Ceph \u0441 \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e, \u043f\u0440\u0435\u0432\u044b\u0448\u0430\u044e\u0449\u0435\u0439 \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443.","og:url":"https:\/\/prohoster.info\/ro\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2024-01-20T21:28:06+00:00","article:modified_time":"2024-01-20T21:28:06+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"113018","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2026-02-09 16:50:41","updated":"2026-02-09 16:50:41","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/113018","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=113018"}],"version-history":[{"count":1,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/113018\/revisions"}],"predecessor-version":[{"id":158784,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/113018\/revisions\/158784"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media\/113019"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=113018"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=113018"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=113018"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}