{"id":113018,"date":"2024-01-20T23:28:04","date_gmt":"2024-01-20T21:28:06","guid":{"rendered":"https:\/\/prohoster.info\/blog\/novosti-interneta\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu"},"modified":"2024-01-20T23:28:04","modified_gmt":"2024-01-20T21:28:06","slug":"opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sv\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","title":{"rendered":"Erfarenhet av att skapa en Ceph-lagring med en genomstr\u00f6mning p\u00e5 tebibyte per sekund","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

En ingenj\u00f6r fr\u00e5n Clyso sammanfattade erfarenheterna av att skapa ett lagringskluster baserat p\u00e5 det fel toleranta distribuerade systemet Ceph med en bandbredd som \u00f6verstiger tebibyte per sekund. Det noteras att detta \u00e4r det f\u00f6rsta Ceph-klustret som har uppn\u00e5tt en s\u00e5dan siffra, men innan ingenj\u00f6rerna kunde uppn\u00e5 det presenterade resultatet beh\u00f6vde de \u00f6vervinna en rad icke uppenbara fallgropar. <\/p>\n

Till exempel var det tillr\u00e4ckligt att aktivera maximala prestandainst\u00e4llningar och st\u00e4nga av C-states (C-states \u00e4ndrar energibesparingsinst\u00e4llningarna beroende p\u00e5 belastningen, vilket p\u00e5verkar Ceph) p\u00e5 servrarna f\u00f6r att \u00f6ka prestandan med 10-20%. Det visade sig ocks\u00e5 att n\u00e4r man anv\u00e4nde NVMe-enheter spenderade Linux-k\u00e4rnan en betydande tid p\u00e5 att bearbeta spinl\u00e5ses under uppdateringen av IOMMU-mapping. Avst\u00e4ngning av IOMMU i k\u00e4rnan ledde till en m\u00e4rkbar \u00f6kning av prestandan i testerna f\u00f6r skriv- och l\u00e4sblock p\u00e5 4 MB. <\/p>\n

Trots detta l\u00f6ste avst\u00e4ngningen av IOMMU inte problemet med prestandaf\u00f6rlust vid slumpm\u00e4ssig skrivning av block om 4KB. N\u00e4r ingenj\u00f6rerna granskade problemet, st\u00f6tte de p\u00e5 korrigeringar i Ceph:s byggscenarier fr\u00e5n projekten Gentoo och Ubuntu, vilka inkluderade byggen med alternativet RelWithDebInfo, eftersom med det anv\u00e4ndes optimeringsl\u00e4ge «-O2» i GCC, vilket avsev\u00e4rt \u00f6kade Ceph:s prestanda. Prestandaf\u00f6rlust orsakades ocks\u00e5 av kompilering med biblioteket TCMalloc. \u00c4ndring av kompilationsflaggorna och avskaffandet av TCMalloc ledde till en trefaldig minskning av komprimeringstiden och f\u00f6rdubblade prestandan f\u00f6r slumpm\u00e4ssiga 4K-skrivoperationer. Slutligen gjordes \u00e4ven en optimering av inst\u00e4llningarna f\u00f6r Reef RocksDB och placering av grupper (PG). <\/p>\n

Kluster \u00e4r bildat av 68 noder baserat p\u00e5 servrar<\/a> Dell PowerEdge R6615 med CPU AMD EPYC 9454P 48C\/96T. Varje nod inneh\u00e5ller 10 NVMe-enheter Dell 15.36TB, tv\u00e5 Ethernet-adaptrar 100GbE Mellanox ConnectX-6 och 192 GB RAM. Mjukvaran baseras p\u00e5 Ubuntu 20.04.6 och Ceph 17.2.7. I klustret med 63 noder har 630 OSD (Object Storage Daemon, en bakgrundsprocess som hanterar datalagring i lokalt lager, en OSD per NVMe-enhet) startats, tre MON-processer (monitor, \u00f6vervakar klustrets tillst\u00e5nd) och en MGR-process (Manager, hanterar tj\u00e4nsten). Lagringsstorleken \u00e4r 8.2 PB. <\/p>\n

Bandbredden vid sekventiella l\u00e4soperationer av block p\u00e5 4M var 1025 GiB\/s, skrivningar \u2014 270 GiB\/s. Vid slumpm\u00e4ssig l\u00e4sning av block p\u00e5 4KB var prestandan 25,5 miljoner l\u00e4soperationer per sekund och 4,9 miljoner vid skrivning. Aktivering av kryptering minskade bandbredden vid l\u00e4sning till cirka 750 GiB\/s. N\u00e4r felkorrektionkoderna EC62 anv\u00e4ndes var bandbredden 547 GiB\/s vid l\u00e4sning och 387 GiB\/s vid skrivning (skrivhastigheten var h\u00f6gre \u00e4n utan felkorrektion), och vid slumpm\u00e4ssig \u00e5tkomst var det 3,4M IOPS vid l\u00e4sning och 936K IOPS vid skrivning.

\"Erfarenhet<\/center>
\"Erfarenhet<\/center> <\/p>\n

Det kan ocks\u00e5 noteras att en liknande genomsnittsfl\u00f6desgr\u00e4ns i tebibyte per sekund n\u00e5ddes i det exabyteskala lagringsklustret vid CERN i september, som byggdes med det \u00f6ppna distribuerade lagringssystemet EOS, baserat p\u00e5 protokollet XRootD.
\n
K\u00e4lla: opennet.ru<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0418\u043d\u0436\u0435\u043d\u0435\u0440 \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Clyso \u043e\u0431\u043e\u0431\u0449\u0438\u043b \u043e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0439 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0451\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u044b Ceph \u0441 \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e, \u043f\u0440\u0435\u0432\u044b\u0448\u0430\u044e\u0449\u0435\u0439 \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443. \u041e\u0442\u043c\u0435\u0447\u0430\u0435\u0442\u0441\u044f, \u0447\u0442\u043e \u044d\u0442\u043e \u043f\u0435\u0440\u0432\u044b\u0439 \u043a\u043b\u0430\u0441\u0442\u0435\u0440 \u043d\u0430 \u0431\u0430\u0437\u0435 Ceph, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0441\u043c\u043e\u0433 \u0434\u043e\u0441\u0442\u0438\u0433\u043d\u0443\u0442\u044c \u043f\u043e\u0434\u043e\u0431\u043d\u043e\u0433\u043e \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044f, \u043d\u043e \u043f\u0435\u0440\u0435\u0434 \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u0438\u0435\u043c \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u043d\u043e\u0433\u043e \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0430 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u0430\u043c \u043f\u043e\u0442\u0440\u0435\u0431\u043e\u0432\u0430\u043b\u043e\u0441\u044c \u043f\u0440\u0435\u043e\u0434\u043e\u043b\u0435\u0442\u044c \u0441\u0435\u0440\u0438\u044e \u043d\u0435 \u043e\u0447\u0435\u0432\u0438\u0434\u043d\u044b\u0445 \u043f\u043e\u0434\u0432\u043e\u0434\u043d\u044b\u0445 \u043a\u0430\u043c\u043d\u0435\u0439. \u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u0434\u043b\u044f \u043f\u043e\u0432\u044b\u0448\u0435\u043d\u0438\u044f \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u043d\u0430 10-20% \u0431\u044b\u043b\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":113019,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-113018","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Erfarenhet av att skapa Ceph-lagring med en genomsnittsfl\u00f6de p\u00e5 tebibyte per sekund | ProHoster","description":"En ingenj\u00f6r fr\u00e5n Clyso sammanfattade erfarenheten av att skapa ett lagringskluster baserat p\u00e5 den felresistenta distribuerade systemet Ceph, med en genomsnittsfl\u00f6de som \u00f6verstiger tebibyte per sekund.","canonical_url":"https:\/\/prohoster.info\/sv\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sv_SE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 Ceph c \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443 | ProHoster","og:description":"\u0418\u043d\u0436\u0435\u043d\u0435\u0440 \u0438\u0437 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Clyso \u043e\u0431\u043e\u0431\u0449\u0438\u043b \u043e\u043f\u044b\u0442 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u0442\u043a\u0430\u0437\u043e\u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u043e\u0439 \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0451\u043d\u043d\u043e\u0439 \u0441\u0438\u0441\u0442\u0435\u043c\u044b Ceph \u0441 \u043f\u0440\u043e\u043f\u0443\u0441\u043a\u043d\u043e\u0439 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c\u044e, \u043f\u0440\u0435\u0432\u044b\u0448\u0430\u044e\u0449\u0435\u0439 \u0442\u0435\u0431\u0438\u0431\u0430\u0439\u0442 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443.","og:url":"https:\/\/prohoster.info\/sv\/blog\/news\/opyt-sozdaniya-hranilishha-ceph-c-propusknoj-sposobnostyu-tebibajt-v-sekundu","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2024-01-20T21:28:06+00:00","article:modified_time":"2024-01-20T21:28:06+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"113018","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2026-02-09 16:50:41","updated":"2026-02-09 16:50:41","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/posts\/113018","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/comments?post=113018"}],"version-history":[{"count":1,"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/posts\/113018\/revisions"}],"predecessor-version":[{"id":158784,"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/posts\/113018\/revisions\/158784"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/media\/113019"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/media?parent=113018"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/categories?post=113018"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sv\/wp-json\/wp\/v2\/tags?post=113018"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}