{"id":42134,"date":"2019-03-18T00:00:00","date_gmt":"2019-03-17T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas"},"modified":"2020-02-18T13:44:56","modified_gmt":"2020-02-18T10:44:56","slug":"uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","title":{"rendered":"Accelerarea input\/output-ului de fi\u0219iere C\/C++, f\u0103r\u0103 a te stresa prea mult","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Accelerarea<\/p>\n

Prefa\u021b\u0103<\/h4>\n

\nExist\u0103 \u00een lume o unealt\u0103 at\u00e2t de simpl\u0103 \u0219i foarte util\u0103 \u2014 BDelta<\/a><\/noindex>, \u0219i astfel s-a \u00eent\u00e2mplat c\u0103 s-a integrat foarte bine \u00een procesul nostru de produc\u021bie (de\u0219i nu am reu\u0219it s\u0103-i instalez versiunea, dar cu siguran\u021b\u0103 nu era ultima disponibil\u0103). O folosim conform destina\u021biei sale \u2014 construirea patch-urilor binare. Dac\u0103 arunc\u0103m o privire asupra ceea ce este \u00een depozit \u2014 devine pu\u021bin trist: \u00een esen\u021b\u0103, a fost abandonat demult \u0219i multe acolo sunt deja \u00eenvechite (c\u00e2ndva, un fost coleg de-al meu a adus c\u00e2teva modific\u0103ri, dar a fost acum mult timp). \u00cen general, am decis s\u0103 revigorez acest proiect: m-am forkuit, am eliminat ce nu inten\u021bionez s\u0103 folosesc, am trecut proiectul pe cmake<\/a><\/noindex>, am inlinat microfunc\u021biile \u201ecalde\u201d, am eliminat din stiv\u0103 matrici mari (\u0219i matrici cu lungime variabil\u0103, care \u00een mod deschis m\u0103 \u201ebombardeaz\u0103\u201d), am rulat din nou profilerul \u2014 \u0219i am aflat c\u0103 aproximativ 40% din timp este consumat pe fwrite<\/a><\/noindex>\u2026
\n<\/a><\/noindex><\/p>\n

Ce este cu fwrite?<\/h4>\n

\n\u00cen acest cod, fwrite (\u00een cazul meu specific: construirea patch-ului \u00eentre fi\u0219ierele apropiate de 300 MB, datele de intrare fiind complet \u00een memorie) este apelat de milioane de ori cu un buffer de dimensiuni mici. Este evident c\u0103 acest lucru va \u00eencetini, a\u0219a c\u0103 a\u0219 dori s\u0103 influen\u021bez acest haos f\u0103r\u0103 a implementa diverse tipuri de surse de date sau I\/O asincron; voiam s\u0103 g\u0103sesc o solu\u021bie mai simpl\u0103. Primul lucru care mi-a venit \u00een minte a fost \u2014 s\u0103 cresc dimensiunea buffer-ului<\/p>\n

setvbuf(file, nullptr, _IOFBF, 64* 1024)<\/code><\/pre>\n
\ndar nu am ob\u021binut o \u00eembun\u0103t\u0103\u021bire semnificativ\u0103 a rezultatului (acum fwrite ocupa aproximativ 37% din timp) \u2014 a\u0219a c\u0103 problema nu este, de fapt, \u00een scrierea frecvent\u0103 a datelor pe disc. Dac\u0103 ne uit\u0103m \u201esub capot\u0103\u201d la fwrite putem observa c\u0103 \u00een interior se \u00eent\u00e2mpl\u0103 un lock\/unlock al structurii FILE cam a\u0219a (pseudocod, tot analiza s-a realizat \u00een Visual Studio 2017):<\/p>\n
\nsize_t fwrite (const void *buffer, size_t size, size_t count, FILE *stream)\n{\n   size_t retval = 0;\n   _lock_str(stream);   \n   __try\n   {\n      retval = _fwrite_nolock(buffer, size, count, stream);\n   }\n   __finally \n   {\n       _unlock_str(stream);   \n   }\n   return retval;\n}\n<\/code><\/pre>\n
\nDac\u0103 credem profilerului, pe _fwrite_nolock cade doar 6% din timp, restul este overhead. \u00cen cazul meu specific, securitatea fluxului este un exces evident, a\u0219a c\u0103 sacrific asta, \u00eenlocuind apelul fwrite cu _fwrite_nolock<\/a><\/noindex> \u2014 nici cu argumentele nu trebuie s\u0103 te complici. A\u0219adar: aceast\u0103 manipulare simpl\u0103 a redus semnificativ costurile pentru salvarea rezultatelor, care \u00een varianta ini\u021bial\u0103 reprezentau aproape jum\u0103tate din timpul necesar. Apropo, \u00een lumea POSIX exist\u0103 o func\u021bie similar\u0103 \u2014 fwrite_unlocked<\/a><\/noindex>. \u00cen general vorbind, acela\u0219i lucru se aplic\u0103 \u0219i pentru fread. Astfel, cu ajutorul a c\u00e2torva #define putem ob\u021bine o solu\u021bie complet multiplatform\u0103 f\u0103r\u0103 blocaje suplimentare, dac\u0103 acestea nu sunt necesare (ceea ce se \u00eent\u00e2mpl\u0103 destul de des).<\/p>\n
fwrite, _fwrite_nolock, setvbuf<\/h4>\n
\nS\u0103 ne abstragem de la proiectul original \u0219i s\u0103 ne ocup\u0103m de testarea unui caz concret: salvarea unui fi\u0219ier mare (512 MB) \u00een por\u021bii extrem de mici \u2014 de 1 byte. Sistemul de testare: AMD Ryzen 7 1700, 16 GB RAM, HDD 7200 rpm 64 MB cache, Windows 10 1809, binarul construit pe 32 de bi\u021bi, optimiz\u0103rile incluse, biblioteca fiind legat\u0103 static.<\/p>\n
Exemplu pentru desf\u0103\u0219urarea experimentului:<\/p>\n
\n#include <chrono>\n#include <cstdio>\n#include <inttypes.h>\n#include <memory>\n\n#ifdef _MSC_VER\n#define fwrite_unlocked _fwrite_nolock\n#endif\n\nusing namespace std::chrono;\n\nint main()\n{\n    std::unique_ptr<FILE, int(*)(FILE*)> file(fopen(\"test.bin\", \"wb\"), fclose);\n    if (!file)\n        return 1;\n\n    constexpr size_t TEST_BUFFER_SIZE = 256 * 1024;\n    if (setvbuf(file.get(), nullptr, _IOFBF, TEST_BUFFER_SIZE) != 0)\n        return 2;\n\n    auto start = steady_clock::now();\n    const uint8_t b = 77;\n    constexpr size_t TEST_FILE_SIZE = 512 * 1024 * 1024;\n    for (size_t i = 0; i < TEST_FILE_SIZE; ++i)\n        fwrite_unlocked(&b, 1, sizeof(b), file.get());\n\n    auto end = steady_clock::now();\n    auto interval = duration_cast<microseconds>(end - start);\n    printf(\"Time: %lldn\", interval.count());\n\n    return 0;\n}\n<\/code><\/pre>\n
\nCa variabile vor fi folosite TEST_BUFFER_SIZE, iar pentru c\u00e2teva cazuri vom \u00eenlocui fwrite_unlocked cu fwrite. S\u0103 \u00eencepem cu cazul fwrite f\u0103r\u0103 a seta explicit dimensiunea buffer-ului (vom comenta setvbuf \u0219i codul asociat): timpul 27048906 \u03bcs, viteza de scriere \u2014 18.93 MB\/s. Acum vom seta dimensiunea buffer-ului la 64 KB: timpul \u2014 25037111 \u03bcs, viteza \u2014 20.44 MB\/s. Acum s\u0103 test\u0103m func\u021bionarea _fwrite_nolock f\u0103r\u0103 a apela setvbuf: 7262221 \u03bcs, viteza \u2014 70.5 MB\/s! <\/p>\n
Mai departe, vom experimenta cu dimensiunea buffer-ului (setvbuf):<\/p>\n
\"Accelerarea
\n
\nDatele au fost ob\u021binute prin medierea a 5 experimente, nu am avut r\u0103bdare s\u0103 contabilizez erorile. Dup\u0103 p\u0103rerea mea, 93 MB\/s la scrierea de 1 byte pe un HDD obi\u0219nuit este un rezultat foarte bun, trebuie doar s\u0103 alegi dimensiunea optim\u0103 a buffer-ului (\u00een cazul meu 256 KB \u2014 perfect) \u0219i s\u0103 \u00eenlocuie\u0219ti fwrite cu _fwrite_nolock\/fwrite_unlocked (\u00een cazul \u00een care nu este necesar\u0103 siguran\u021ba de fir, desigur). 
\nSimilar cu fread \u00een astfel de condi\u021bii. Deoarece nu am la dispozi\u021bie o ma\u0219in\u0103 \"hardware\" cu linux (pl\u0103cile unice nu se pun la socoteal\u0103), am decis s\u0103 desf\u0103\u0219or un experiment limitat pe o ma\u0219in\u0103 virtual\u0103 (Hyper-V, OpenSUSE 15, GCC 8.3.1) \u2014 tendin\u021ba este, \u00een principiu, aceea\u0219i: fwrite \"goal\u0103\" 20 MB\/s, fwrite + buffer de 256 KB a dat 23 MB\/s, fwrite_unlocked cu acela\u0219i buffer \u2014 35 MB\/s (binar 64 de bi\u021bi, construit cu g++ -o2 -s -static-libgcc -static-libstdc++ fwrite_test.cpp -o fwrite_test).<\/p>\n
Cuv\u00e2nt \u00eenainte<\/h4>\n
\nScopul acestui articol a fost descrierea unei metode simple \u0219i eficiente, care poate fi aplicat\u0103 \u00een multe cazuri (func\u021biile _fwrite_nolock\/fwrite_unlocked nu le am \u00eent\u00e2lnit anterior, nu sunt foarte populare \u2013 \u0219i pe bun\u0103 dreptate). Nu pretind noutatea materialului, dar sper c\u0103 articolul va fi de folos comunit\u0103\u021bii.<\/p>\n
Sursa: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c \u043d\u0430 \u0441\u0432\u0435\u0442\u0435 \u0442\u0430\u043a\u0430\u044f \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0438 \u043e\u0447\u0435\u043d\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0430\u044f \u0443\u0442\u0438\u043b\u0438\u0442\u0430 \u2014 BDelta, \u0438 \u0442\u0430\u043a \u0432\u044b\u0448\u043b\u043e, \u0447\u0442\u043e \u043e\u043d\u0430 \u043e\u0447\u0435\u043d\u044c \u0434\u0430\u0432\u043d\u043e \u0443\u043a\u043e\u0440\u0435\u043d\u0438\u043b\u0430\u0441\u044c \u0432 \u043d\u0430\u0448\u0435\u043c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u043e\u043c \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0435 (\u043f\u0440\u0430\u0432\u0434\u0430 \u0435\u0451 \u0432\u0435\u0440\u0441\u0438\u044e \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c \u043d\u0435 \u0443\u0434\u0430\u043b\u043e\u0441\u044c, \u043d\u043e \u043e\u043d\u0430 \u0442\u043e\u0447\u043d\u043e \u0431\u044b\u043b\u0430 \u043d\u0435 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0439 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0439). \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0435\u0451 \u043f\u043e \u043f\u0440\u044f\u043c\u043e\u043c\u0443 \u043d\u0430\u0437\u043d\u0430\u0447\u0435\u043d\u0438\u044e \u2014 \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u0438\u0435 \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u043f\u0430\u0442\u0447\u0435\u0439. \u0415\u0441\u043b\u0438 \u0432\u0437\u0433\u043b\u044f\u043d\u0443\u0442\u044c, \u0447\u0442\u043e \u0442\u0430\u043c \u0432 \u0440\u0435\u043f\u043e\u0437\u0438\u0442\u043e\u0440\u0438\u0438, \u2014 \u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u0441\u044f \u0441\u043b\u0435\u0433\u043a\u0430 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-42134","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47 Accelerarea I\/O-ului de fi\u0219iere C\/C++, f\u0103r\u0103 prea mult efort | ProHoster","description":"Prefa\u021b\u0103 Exist\u0103.","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0423\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u0435 \u0444\u0430\u0439\u043b\u043e\u0432\u043e\u0433\u043e \u0432\u0432\u043e\u0434\u0430-\u0432\u044b\u0432\u043e\u0434\u0430 C\/C++, \u043d\u0435 \u043e\u0441\u043e\u0431\u043e \u043d\u0430\u043f\u0440\u044f\u0433\u0430\u044f\u0441\u044c | ProHoster","og:description":"\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c.","og:url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-03-17T21:00:00+00:00","article:modified_time":"2020-02-18T10:44:56+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"42134","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 07:22:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:05:22","updated":"2026-01-22 07:22:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/42134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=42134"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/42134\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=42134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=42134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=42134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}