{"id":42134,"date":"2019-03-18T00:00:00","date_gmt":"2019-03-17T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas"},"modified":"2020-02-18T13:44:56","modified_gmt":"2020-02-18T10:44:56","slug":"uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","title":{"rendered":"C\/C++ failide sisestamise ja v\u00e4ljundi kiirus, ilma liialdamata","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"C\/C++<\/p>\n

Eess\u00f5na<\/h4>\n

\nMaailmas on olemas selline lihtne ja v\u00e4ga kasulik utiliit \u2014 BDelta<\/a><\/noindex>, ja nii on juhtunud, et see on ammusest ajast meie tootmisprotsessis juurdunud (kuigi selle versiooni installimine ei \u00f5nnestunud, polnud see kindlasti viimane saadaval). Kasutame seda otstarbekohaselt \u2014 binaarsete patch'ide loomine. Kui vaadata, mis seal hoidlas on, \u2014 tekib veidi kurbus: p\u00f5him\u00f5tteliselt on see ammu h\u00fcljatud ja palju on seal tugevalt vananenud (kunagi tegi sinna paar muudatust minu endine kolleeg, aga see oli ammu). \u00dches\u00f5naga, otsustasin seda ellu \u00e4ratada: tegin fork'i, viskasin minema selle, mida ei plaani kasutada, ja viisin projekti \u00fcle cmake<\/a><\/noindex>, inline'isin \u201ekuumad\u201c mikrofunktsioonid, kustutasin virnast suured massiivid (ja muutlikud massiivid, mis mind t\u00f5eliselt \u201eraevu ajavad\u201d), jooksutasin uuesti profilaatorit \u2014 ja sain teada, et umbes 40% ajast kulub fwrite<\/a><\/noindex>\u2026
\n<\/a><\/noindex><\/p>\n

Nii et kuidas on fwrite'iga?<\/h4>\n

\nSelles koodis kutsutakse fwrite (minu konkreetse testi juhul: 300 MB suuruste failide patchi loomine, sisend on t\u00e4ielikult m\u00e4lus) miljonite kordadega v\u00e4ikese suurusega puhvris. On selge, et see asi t\u00f5mbab pidurit, seega sooviks leida lahenduse, mis aitaks olukorda parandada, kuid soovimata rakendada erinevaid andmeallikaid ega as\u00fcnkroonset sisend-v\u00e4ljundit. Esimene m\u00f5te, mis p\u00e4he tuli, oli puhvri suuruse suurendamine.<\/p>\n

setvbuf(file, nullptr, _IOFBF, 64 * 1024)<\/code><\/pre>\n
\nkuid oluliselt paremat tulemust ei saanud (n\u00fc\u00fcd kulus fwrite'ile umbes 37% ajast) \u2014 see t\u00e4hendab, et probleem ei ole andmete sagedases kirjutamises kettale. Vaadates fwrite'i \u201ekapoti alla\u201d v\u00f5ib n\u00e4ha, et seal toimub FILE struktuuri lukustamine ja lukust vabastamine umbes nii (pseudo-kood, kogu anal\u00fc\u00fcs viidi l\u00e4bi Visual Studio 2017 all):<\/p>\n
\nsize_t fwrite (const void *buffer, size_t size, size_t count, FILE *stream)\n{\n   size_t retval = 0;\n   _lock_str(stream);   \/* lukusta voog *\/\n   __try\n   {\n      retval = _fwrite_nolock(buffer, size, count, stream);\n   }\n   __finally \n   {\n       _unlock_str(stream);   \/* vabasta lukustus *\/\n   }\n   return retval;\n}\n<\/code><\/pre>\n
\nKui uskuda profiilimist, siis _fwrite_nolock h\u00f5ivab vaid 6% ajast, \u00fclej\u00e4\u00e4nud on overhead. Minu konkreetses juhtumis on l\u00f5imede turvalisus ilmselt \u00fcleliigne, millest ma loobun, asendades fwrite v\u00e4ljakutsega _fwrite_nolock<\/a><\/noindex> \u2014 isegi argumentide kallal ei pea nuputama. Kokkuv\u00f5ttes: see lihtne manipulatsioon v\u00e4hendas tulemuse kirjutamise kulusid mitu korda, mis algses variandis moodustasid peaaegu poole ajakulu. Muide, POSIX-i maailmas on sarnane funktsioon \u2014 fwrite_unlocked<\/a><\/noindex>. \u00dcldiselt puudutab see sama ka freadi. Seega, paar #define abil on v\u00f5imalik saavutada t\u00e4iesti platvormidevaheline lahendus ilma liigsete lukustusteta, kui neid ei ole vaja (mis juhtub v\u00e4ga sageli).<\/p>\n
fwrite, _fwrite_nolock, setvbuf<\/h4>\n
\nHaarame originaalprojektist kaugemale ja tegeleme konkreetse juhtumi testimisega: suurte failide (512 MB) salvestamine \u00e4\u00e4rmiselt v\u00e4ikestes portsjonites \u2014 1 bait. Testis\u00fcsteem: AMD Ryzen 7 1700, 16 GB RAM, HDD 7200 rpm 64 MB vahem\u00e4lu, Windows 10 1809, binaar koostati 32-bitises versioonis, optimeerimine on sisse l\u00fclitatud, raamatukogu on staatiliselt lingitud.<\/p>\n
Eksperimendi l\u00e4biviimise n\u00e4idis:<\/p>\n
\n#include <chrono>\n#include <cstdio>\n#include <inttypes.h>\n#include <memory>\n\n#ifdef _MSC_VER\n#define fwrite_unlocked _fwrite_nolock\n#endif\n\nusing namespace std::chrono;\n\nint main()\n{\n    std::unique_ptr<FILE, int(*)(FILE*)> file(fopen(\"test.bin\", \"wb\"), fclose);\n    if (!file)\n        return 1;\n\n    constexpr size_t TEST_BUFFER_SIZE = 256 * 1024;\n    if (setvbuf(file.get(), nullptr, _IOFBF, TEST_BUFFER_SIZE) != 0)\n        return 2;\n\n    auto start = steady_clock::now();\n    const uint8_t b = 77;\n    constexpr size_t TEST_FILE_SIZE = 512 * 1024 * 1024;\n    for (size_t i = 0; i < TEST_FILE_SIZE; ++i)\n        fwrite_unlocked(&b, 1, sizeof(b), file.get());\n\n    auto end = steady_clock::now();\n    auto interval = duration_cast<microseconds>(end - start);\n    printf(\"Time: %lldn\", interval.count());\n\n    return 0;\n}\n<\/code><\/pre>\n
\nMuutujateks on TEST_BUFFER_SIZE, samuti asendame fwrite_unlocked juhul fwrite'iga. Alustame kirjutamisest ilma leidliku puhver suuruse m\u00e4\u00e4ramiseta (kommenteerime setvbuf ja sellega seotud koodi v\u00e4lja): aeg 27048906 \u03bcs, kirjutamise kiirus \u2014 18.93 MB\/s. N\u00fc\u00fcd seame puhvri suuruseks 64 KB: aeg \u2014 25037111 \u03bcs, kiirus \u2014 20.44 MB\/s. J\u00e4rgmiseks katsetame _fwrite_nolock'i kasutamist ilma setvbuf'i kutsumata: 7262221 \u03bcs, kiirus \u2014 70.5 MB\/s! <\/p>\n
Vaatame edasi puhvri suuruse (setvbuf) katsetamist:<\/p>\n
\"C\/C++
\n
\nAndmed on saadud 5 katse keskmise p\u00f5hjal, viga ei hakanud k\u00fcll arvutama. Minu arvates on 93 MB\/s andmete kirjutamisel 1 baidi kaupa tavalisele HDD-le v\u00e4ga hea tulemus; tuleb vaid valida optimaalse puhvri suurus (minu puhul 256 KB - just \u00f5igesti) ja asendada fwrite _fwrite_nolock'iga\/fwrite_unlocked'iga (kui j\u00e4rjepidevuse ohutus pole vajalik, loomulikult). 
\nSarnane fread'ile sarnastes tingimustes. Kuna \u201eraud\u201c masinat Linuxiga k\u00e4ep\u00e4rast ei ole (\u00fcheplaadilised arvutid ei loe), otsustasin teha piiratud katse virtuaalmasinas (Hyper-V, OpenSUSE 15, GCC 8.3.1) \u2014 seadusp\u00e4rasus on p\u00f5him\u00f5tteliselt sama: \u201epalja\u201c fwrite kiirus on 20 MB\/s, fwrite + 256 KB puhver andis 23 MB\/s, fwrite_unlocked sama puhvri jaoks \u2014 35 MB\/s (binaar 64-bitine, koostasin g++ -o2 -s -static-libgcc -static-libstdc++ fwrite_test.cpp -o fwrite_test).<\/p>\n
J\u00e4rels\u00f5na<\/h4>\n
\nSelle artikli eesm\u00e4rk oli kirjeldada lihtsat ja t\u00f5husat l\u00e4henemist, mis t\u00f6\u00f6tab paljudes olukordades (funktsioonidega _fwrite_nolock\/fwrite_unlocked ei olnud mul varem kokku puutunud, need ei ole v\u00e4ga populaarsed \u2014 ja kahju). Uudsuse osas ei pretendeeri, kuid loodan, et artikkel osutub kogukonnale kasulikuks.<\/p>\n
Allikas: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c \u043d\u0430 \u0441\u0432\u0435\u0442\u0435 \u0442\u0430\u043a\u0430\u044f \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0438 \u043e\u0447\u0435\u043d\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0430\u044f \u0443\u0442\u0438\u043b\u0438\u0442\u0430 \u2014 BDelta, \u0438 \u0442\u0430\u043a \u0432\u044b\u0448\u043b\u043e, \u0447\u0442\u043e \u043e\u043d\u0430 \u043e\u0447\u0435\u043d\u044c \u0434\u0430\u0432\u043d\u043e \u0443\u043a\u043e\u0440\u0435\u043d\u0438\u043b\u0430\u0441\u044c \u0432 \u043d\u0430\u0448\u0435\u043c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u043e\u043c \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0435 (\u043f\u0440\u0430\u0432\u0434\u0430 \u0435\u0451 \u0432\u0435\u0440\u0441\u0438\u044e \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c \u043d\u0435 \u0443\u0434\u0430\u043b\u043e\u0441\u044c, \u043d\u043e \u043e\u043d\u0430 \u0442\u043e\u0447\u043d\u043e \u0431\u044b\u043b\u0430 \u043d\u0435 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0439 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0439). \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0435\u0451 \u043f\u043e \u043f\u0440\u044f\u043c\u043e\u043c\u0443 \u043d\u0430\u0437\u043d\u0430\u0447\u0435\u043d\u0438\u044e \u2014 \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u0438\u0435 \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u043f\u0430\u0442\u0447\u0435\u0439. \u0415\u0441\u043b\u0438 \u0432\u0437\u0433\u043b\u044f\u043d\u0443\u0442\u044c, \u0447\u0442\u043e \u0442\u0430\u043c \u0432 \u0440\u0435\u043f\u043e\u0437\u0438\u0442\u043e\u0440\u0438\u0438, \u2014 \u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u0441\u044f \u0441\u043b\u0435\u0433\u043a\u0430 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-42134","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47 Failide sisendi ja v\u00e4ljundi kiirendamine C\/C++-s, pingutamata | ProHoster","description":"Eess\u00f5na. On olemas selline lihtne ja v\u00e4ga kasulik utiliit nagu BDelta, mis on juba pikka aega juurdunud meie tootmisprotsessis (kuigi tema versiooni ei \u00f5nnestunud installida, oli see kindlasti mitte viimane saadaval). Kasutame seda otstarbekalt \u2013 binaarsete plaastrite loomine. Kui vaadata, mis seal repos on, siis muutub see veidi...","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0423\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u0435 \u0444\u0430\u0439\u043b\u043e\u0432\u043e\u0433\u043e \u0432\u0432\u043e\u0434\u0430-\u0432\u044b\u0432\u043e\u0434\u0430 C\/C++, \u043d\u0435 \u043e\u0441\u043e\u0431\u043e \u043d\u0430\u043f\u0440\u044f\u0433\u0430\u044f\u0441\u044c | ProHoster","og:description":"\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c \u043d\u0430 \u0441\u0432\u0435\u0442\u0435 \u0442\u0430\u043a\u0430\u044f \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0438 \u043e\u0447\u0435\u043d\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0430\u044f \u0443\u0442\u0438\u043b\u0438\u0442\u0430 \u2014 BDelta, \u0438 \u0442\u0430\u043a \u0432\u044b\u0448\u043b\u043e, \u0447\u0442\u043e \u043e\u043d\u0430 \u043e\u0447\u0435\u043d\u044c \u0434\u0430\u0432\u043d\u043e \u0443\u043a\u043e\u0440\u0435\u043d\u0438\u043b\u0430\u0441\u044c \u0432 \u043d\u0430\u0448\u0435\u043c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u043e\u043c \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0435 (\u043f\u0440\u0430\u0432\u0434\u0430 \u0435\u0451 \u0432\u0435\u0440\u0441\u0438\u044e \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c \u043d\u0435 \u0443\u0434\u0430\u043b\u043e\u0441\u044c, \u043d\u043e \u043e\u043d\u0430 \u0442\u043e\u0447\u043d\u043e \u0431\u044b\u043b\u0430 \u043d\u0435 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0439 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0439). \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0435\u0451 \u043f\u043e \u043f\u0440\u044f\u043c\u043e\u043c\u0443 \u043d\u0430\u0437\u043d\u0430\u0447\u0435\u043d\u0438\u044e \u2014 \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u0438\u0435 \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u043f\u0430\u0442\u0447\u0435\u0439. \u0415\u0441\u043b\u0438 \u0432\u0437\u0433\u043b\u044f\u043d\u0443\u0442\u044c, \u0447\u0442\u043e \u0442\u0430\u043c \u0432 \u0440\u0435\u043f\u043e\u0437\u0438\u0442\u043e\u0440\u0438\u0438, \u2014 \u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u0441\u044f \u0441\u043b\u0435\u0433\u043a\u0430","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-03-17T21:00:00+00:00","article:modified_time":"2020-02-18T10:44:56+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"42134","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 07:22:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:05:22","updated":"2026-01-22 07:22:19"},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/42134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=42134"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/42134\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=42134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=42134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=42134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}