{"id":42134,"date":"2019-03-18T00:00:00","date_gmt":"2019-03-17T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas"},"modified":"2020-02-18T13:44:56","modified_gmt":"2020-02-18T10:44:56","slug":"uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","title":{"rendered":"Failide sisendi- ja v\u00e4ljalaske kiirus C\/C++-ga, ilma liialt pingutamata","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Failide<\/p>\n

Eess\u00f5na<\/h4>\n

\nOn maailmas \u00fcks lihtne ja v\u00e4ga kasulik utiliit \u2014 BDelta<\/a><\/noindex>, ja nii juhtuski, et see on juba ammu juurdunud meie tootmisprotsessi (t\u00f5si, selle versiooni installimine ei \u00f5nnestunud, kuid see ei olnud kindlasti viimase saadaval versioon). Kasutame seda oma algses otstarbes - binaarsete patchide loomisel. Kui vaadata, mis seal repos on, siis muutub pisut kurvaks: p\u00f5him\u00f5tteliselt on see juba ammu maha j\u00e4etud ja palju seal on tugevalt vananenud (kunagi tegi sinna mitmeid parandusi mu endine kolleeg, kuid see juhtus kaua aega tagasi). \u00dches\u00f5naga, otsustasin selle asja elustada: forkisin, viskasin v\u00e4lja selle, mida ei kavatse kasutada, ning t\u00f5in projekti \u00fcle cmake<\/a><\/noindex>, inlinesin \"kuumad\" mikrofunktsioonid, eemaldasin virnast suured massiivid (ja muutuva pikkusega massiivid, millest mul t\u00f5eliselt \"p\u00f6\u00f6rdub\"), l\u00e4bisin uuesti profiilerit - ja sain teada, et umbes 40% ajast kulub fwrite<\/a><\/noindex>\u2026
\n<\/a><\/noindex><\/p>\n

Mis seal fwrite\u2019iga siis toimub?<\/h4>\n

\nAntud koodis kutsutakse fwrite (minu konkreetses testjuhtumis: patchi loomine sarnaste 300 MB failide vahel, sisendandmed on t\u00e4ielikult m\u00e4lus) miljoneid kordi v\u00e4lja v\u00e4ikese suurusega puhvriga. On ilmselge, et see asi hakkab k\u00f5vasti segama, ja seet\u00f5ttu tahaks kuidagi selle asja \u00fcle m\u00f5jutada. Erinevate andmeallikate, as\u00fcnkroonse I\/O rakendamine ei ole hetkel soovitav, tahaks leida lihtsamat lahendust. Esimene asi, mis p\u00e4he tuli \u2014 suurendada puhversuurust<\/p>\n

setvbuf(file, nullptr, _IOFBF, 64* 1024)<\/code><\/pre>\n
\naga tulemus ei paranenud oluliselt (n\u00fc\u00fcd kulus fwrite\u2019ile umbes 37% ajast) - t\u00e4hendas, et asi ei ole siiski sagedases andmete salvestamises kettale. Vaatamata fwrite \u201ckapoti alla\u201d v\u00f5ib n\u00e4ha, et seal toimub FILE struktuuri lukustamine\/avalukustamine umbes nii (pseudokood, kogu anal\u00fc\u00fcs viidi l\u00e4bi Visual Studio 2017 all):<\/p>\n
\nsize_t fwrite (const void *buffer, size_t size, size_t count, FILE *stream)\n{\n   size_t retval = 0;\n   _lock_str(stream);   \t\/* lock stream *\/\n   __try\n   {\n      retval = _fwrite_nolock(buffer, size, count, stream);\n   }\n   __finally \n   {\n       _unlock_str(stream);   \t\/* unlock stream *\/\n   }\n   return retval;\n}\n<\/code><\/pre>\n
\nKui uskuda profiilerit, siis _fwrite_nolock\u2019ile kulub vaid 6% ajast, \u00fclej\u00e4\u00e4nud on overhead. Minu konkreetses jutuhoos on samal ajal vooluohutus ilmselge \u00fcleliigsus, selle ohverdan, asendades fwrite\u2019i nimetuse _fwrite_nolock<\/a><\/noindex> \u2014 isegi argumentide kasutamine ei ole vajalik. Kokkuv\u00f5tteks: see lihtne manipuleerimine v\u00e4hendas oluliselt tulemuse salvestamise kulusid, mis originaalses variandis moodustasid peaaegu poole ajakuludest. Muide, POSIX maailmas on olemas sarnane funktsioon \u2014 fwrite_unlocked<\/a><\/noindex>. \u00dcldiselt kehtib sama ka freadi kohta. Seega on paar #define'i abil v\u00f5imalik saada t\u00e4iesti platvormide vahel t\u00f6\u00f6tav lahendus ilma liigsete lukustusteta, kui need ei ole vajalikud (mis toimub \u00fcsna sageli).<\/p>\n
fwrite, _fwrite_nolock, setvbuf<\/h4>\n
\nL\u00e4hme originaalsest projektist eemale ja keskendume konkreetse juhtumi testimisele: suure faili (512 MB) salvestamine \u00e4\u00e4rmiselt v\u00e4ikeste portsjonitena \u2014 1 bait. Testis\u00fcsteem: AMD Ryzen 7 1700, 16 GB RAM, HDD 7200 rpm 64 MB vahem\u00e4lu, Windows 10 1809, binaar ehitatud 32-bitine, optimeerimised on sisse l\u00fclitatud, teek on staatiliselt lingitud.<\/p>\n
Katseproov eksperimentide l\u00e4biviimiseks:<\/p>\n
\n#include <chrono>\n#include <cstdio>\n#include <inttypes.h>\n#include <memory>\n\n#ifdef _MSC_VER\n#define fwrite_unlocked _fwrite_nolock\n#endif\n\nusing namespace std::chrono;\n\nint main()\n{\n    std::unique_ptr<FILE, int(*)(FILE*)> file(fopen(\"test.bin\", \"wb\"), fclose);\n    if (!file)\n        return 1;\n\n    constexpr size_t TEST_BUFFER_SIZE = 256 * 1024;\n    if (setvbuf(file.get(), nullptr, _IOFBF, TEST_BUFFER_SIZE) != 0)\n        return 2;\n\n    auto start = steady_clock::now();\n    const uint8_t b = 77;\n    constexpr size_t TEST_FILE_SIZE = 512 * 1024 * 1024;\n    for (size_t i = 0; i < TEST_FILE_SIZE; ++i)\n        fwrite_unlocked(&b, 1, sizeof(b), file.get());\n\n    auto end = steady_clock::now();\n    auto interval = duration_cast<microseconds>(end - start);\n    printf(\"Time: %lldn\", interval.count());\n\n    return 0;\n}\n<\/code><\/pre>\n
\nMuutujateks on TEST_BUFFER_SIZE ning paaril juhul asendame fwrite_unlocked fwrite'iga. Alustame fwrite juhtumist, ilma et oleksime selgelt bufferi suurust seadnud (kommenteerime v\u00e4lja setvbuf ja sellega seotud koodi): aeg 27048906 \u00b5s, kirjutamise kiirus \u2014 18.93 MB\/s. N\u00fc\u00fcd seadistame bufferi suuruseks 64 KB: aeg \u2014 25037111 \u00b5s, kiirus \u2014 20.44 MB\/s. Testime n\u00fc\u00fcd _fwrite_nolock'i setvbuf'i kutsumata: 7262221 \u00b5s, kiirus \u2014 70.5 MB\/s! <\/p>\n
Edasi katsetame bufferi suurusega (setvbuf):<\/p>\n
\"Failide
\n
\nAndmed on saadud 5 katse keskmistamisel, vigu ma ei arvestanud. Minu arvates on 93 MB\/s kirjutamine 1 baitiga tavalisele HDD-le v\u00e4ga hea tulemus, tuleb vaid valida optimaalne bufferi suurus (minu jaoks 256 KB \u2014 ideaalne) ja asendada fwrite _fwrite_nolock'i v\u00f5i fwrite_unlocked'iga (kui voogude ohutus muidugi ei ole vajalik). 
\nSarnane kehtib fread'i kohta sarnastes tingimustes. Kuna mul ei ole k\u00e4sitsi linuxi 'raudmasinat' (\u00fcheplaadilised ei kvalifitseeru), otsustasin l\u00e4bi viia piiratud eksperimendi virtuaalmasinas (Hyper-V, OpenSUSE 15, GCC 8.3.1) \u2014 seadusp\u00e4rasus on p\u00f5him\u00f5tteliselt sama: 'paljas' fwrite 20 MB\/s, fwrite + 256 KB buffer andis 23 MB\/s, fwrite_unlocked sama bufferiga \u2014 35 MB\/s (binaar 64-bitine, kompileeritud g++ -o2 -s -static-libgcc -static-libstdc++ fwrite_test.cpp -o fwrite_test).<\/p>\n
Eess\u00f5na<\/h4>\n
\nSelle artikli eesm\u00e4rk oli kirjeldada lihtsat ja t\u00f5husat meetodit, mis t\u00f6\u00f6tab paljudes olukordades (funktsioonidega _fwrite_nolock\/fwrite_unlocked, millega ma enne ei olnud kokku puutunud, ei ole nad just v\u00e4ga populaarsed \u2014 kuid asjata). Uue materjali osas ma pretentsiooni ei oma, kuid loodan, et artikkel osutub kogukonnale kasulikuks.<\/p>\n
Allikas: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c \u043d\u0430 \u0441\u0432\u0435\u0442\u0435 \u0442\u0430\u043a\u0430\u044f \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0438 \u043e\u0447\u0435\u043d\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0430\u044f \u0443\u0442\u0438\u043b\u0438\u0442\u0430 \u2014 BDelta, \u0438 \u0442\u0430\u043a \u0432\u044b\u0448\u043b\u043e, \u0447\u0442\u043e \u043e\u043d\u0430 \u043e\u0447\u0435\u043d\u044c \u0434\u0430\u0432\u043d\u043e \u0443\u043a\u043e\u0440\u0435\u043d\u0438\u043b\u0430\u0441\u044c \u0432 \u043d\u0430\u0448\u0435\u043c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u043e\u043c \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0435 (\u043f\u0440\u0430\u0432\u0434\u0430 \u0435\u0451 \u0432\u0435\u0440\u0441\u0438\u044e \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c \u043d\u0435 \u0443\u0434\u0430\u043b\u043e\u0441\u044c, \u043d\u043e \u043e\u043d\u0430 \u0442\u043e\u0447\u043d\u043e \u0431\u044b\u043b\u0430 \u043d\u0435 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0439 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0439). \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0435\u0451 \u043f\u043e \u043f\u0440\u044f\u043c\u043e\u043c\u0443 \u043d\u0430\u0437\u043d\u0430\u0447\u0435\u043d\u0438\u044e \u2014 \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u0438\u0435 \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u043f\u0430\u0442\u0447\u0435\u0439. \u0415\u0441\u043b\u0438 \u0432\u0437\u0433\u043b\u044f\u043d\u0443\u0442\u044c, \u0447\u0442\u043e \u0442\u0430\u043c \u0432 \u0440\u0435\u043f\u043e\u0437\u0438\u0442\u043e\u0440\u0438\u0438, \u2014 \u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u0441\u044f \u0441\u043b\u0435\u0433\u043a\u0430 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-42134","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Failide sisendi ja v\u00e4ljundi kiirus C\/C++-s, pingutamata | ProHoster","description":"Eess\u00f5na","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0423\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u0435 \u0444\u0430\u0439\u043b\u043e\u0432\u043e\u0433\u043e \u0432\u0432\u043e\u0434\u0430-\u0432\u044b\u0432\u043e\u0434\u0430 C\/C++, \u043d\u0435 \u043e\u0441\u043e\u0431\u043e \u043d\u0430\u043f\u0440\u044f\u0433\u0430\u044f\u0441\u044c | ProHoster","og:description":"\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c.","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-03-17T21:00:00+00:00","article:modified_time":"2020-02-18T10:44:56+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"42134","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 07:22:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:05:22","updated":"2026-01-22 07:22:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/42134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=42134"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/42134\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=42134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=42134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=42134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}