{"id":42134,"date":"2019-03-18T00:00:00","date_gmt":"2019-03-17T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas"},"modified":"2020-02-18T13:44:56","modified_gmt":"2020-02-18T10:44:56","slug":"uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","status":"publish","type":"post","link":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","title":{"rendered":"Ubrzanje unosa\/izlaza datoteka C\/C++, bez previ\u0161e napora","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Ubrzanje unosa\/izlaza datoteka C\/C++, bez previ\u0161e napora\" src=\"\/wp-content\/uploads\/2019\/03\/ed7e4c2b671c87b382572f08a411e009.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<h4>Predgovor<\/h4>\n<p>\nPostoji jednostavna i vrlo korisna alatka \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/jjwhitney\/BDelta\">BDelta<\/a><\/noindex>, i tako je ispalo da je ona ve\u0107 dugo ukorijenjena u na\u0161em proizvodnom procesu (iako nije bilo mogu\u0107e instalirati njenu verziju, bila je sigurno daleko od posljednje dostupne). Koristimo je za njenu namjenu \u2014 izradu binarnih zakrpa. Kada pogledam \u0161to se nalazi u repozitoriju, postajem pomalo tu\u017ean: zapravo je odavno zapu\u0161ten i mnoge informacije su zastarjele (neki od mojih biv\u0161ih kolega su nekada unosili promjene, ali to je bilo davno). Sve u svemu, odlu\u010dio sam o\u017eivjeti ovo: forkovao sam, izbacio \u0161to ne planiram koristiti, prebacio projekt na <noindex><a rel=\"nofollow\" href=\"https:\/\/cmake.org\/\">cmake<\/a><\/noindex>, uvrstio \u201evru\u0107e\u201d mikrofunkcije, uklonio velike nizove s steka (i nizove promjenljive du\u017eine, od kojih me otvoreno \u201epali\u201d), ponovo pokrenuo profajler \u2014 i saznao da se oko 40% vremena tro\u0161i na <noindex><a rel=\"nofollow\" href=\"http:\/\/www.cplusplus.com\/reference\/cstdio\/fwrite\/\">fwrite<\/a><\/noindex>\u2026<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h4>\u0160to je s fwrite?<\/h4>\n<p>\nU ovom kodu fwrite (u svom konkretnom testnom slu\u010daju: izrada zakrpe izme\u0111u sli\u010dnih 300 MB datoteka, ulazni podaci u potpunosti u memoriji) poziva se milijune puta s malim bufferom. O\u010digledno je da \u0107e to usporiti proces, pa bih nekako \u017eelio utjecati na to bez obzira na sve. Ne \u017eelim uvoditi razli\u010dite izvore podataka ili asinkrono ulaz\/izlaz, \u017eelio bih prona\u0107i jednostavnije re\u0161enje. Prvo \u0161to mi je palo na pamet \u2014 pove\u0107ati veli\u010dinu buffer-a<\/p>\n<pre><code class=\"cpp\">setvbuf(file, nullptr, _IOFBF, 64* 1024)<\/code><\/pre>\n<p>\nali nisam dobio zna\u010dajno pobolj\u0161anje rezultata (sada se na fwrite tro\u0161i oko 37% vremena) \u2014 zna\u010di da nije svejedno u \u010destom pisanju podataka na disk. Kada pogledam \u201eispod haube\u201d fwrite, mogu vidjeti da se unutar odvija zaklju\u010davanje\/otklju\u010davanje FILE strukture otprilike ovako (pseudokod, cijela analiza provedena je u Visual Studio 2017):<\/p>\n<pre><code class=\"cpp\">\nsize_t fwrite (const void *buffer, size_t size, size_t count, FILE *stream)\n{\n   size_t retval = 0;\n   _lock_str(stream);   \/* lock stream *\/\n   __try\n   {\n      retval = _fwrite_nolock(buffer, size, count, stream);\n   }\n   __finally \n   {\n       _unlock_str(stream);   \/* unlock stream *\/\n   }\n   return retval;\n}\n<\/code><\/pre>\n<p>\nAko vjerujem profajleru, na _fwrite_nolock otpada samo 6% vremena, ostatak \u2014 na overhead. U svom konkretnom slu\u010daju, sigurnost u vi\u0161e niti je o\u010digledna suvi\u0161nost, nju \u0107u \u017ertvovati, zamjenjuju\u0107i poziv fwrite s <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.microsoft.com\/en-us\/cpp\/c-runtime-library\/reference\/fwrite-nolock?view=vs-2017\">_fwrite_nolock<\/a><\/noindex> \u2014 \u010dak i s argumentima ne treba previ\u0161e filozofirati. Ukratko: ova jednostavna manipulacija zna\u010dajno je smanjila tro\u0161kove za zapis rezultata, koji su u izvornom varijanti iznosili gotovo polovicu vremenskih tro\u0161kova. Usput, u svijetu POSIX postoji sli\u010dna funkcija \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/linux.die.net\/man\/3\/fwrite_unlocked\">fwrite_unlocked<\/a><\/noindex>. Uistinu, to se odnosi i na fread. Tako se uz pomo\u0107 nekoliko #define mo\u017ee dobiti sasvim funkcionalno rje\u0161enje bez nepotrebnih zaklju\u010davanja u slu\u010daju da nisu potrebna (a to se \u010desto doga\u0111a).<\/p>\n<h4>fwrite, _fwrite_nolock, setvbuf<\/h4>\n<p>\nPogledajmo malo dalje od originalnog projekta i bavit \u0107emo se testiranjem konkretnog slu\u010daja: zapis velikog datoteke (512 MB) u izuzetno malim porcijama \u2014 u 1 bajt. Testni sustav: AMD Ryzen 7 1700, 16 GB RAM-a, HDD 7200 rpm 64 MB cache, Windows 10 1809, binarni je gra\u0111en za 32-bitni sustav, optimizacije uklju\u010dene, biblioteka je stati\u010dki povezivana.<\/p>\n<p>Uzorak za provo\u0111enje eksperimenta:<\/p>\n<pre><code class=\"cpp\">\n#include &lt;chrono&gt;\n#include &lt;cstdio&gt;\n#include &lt;inttypes.h&gt;\n#include &lt;memory&gt;\n\n#ifdef _MSC_VER\n#define fwrite_unlocked _fwrite_nolock\n#endif\n\nusing namespace std::chrono;\n\nint main()\n{\n    std::unique_ptr&lt;FILE, int(*)(FILE*)&gt; file(fopen(\"test.bin\", \"wb\"), fclose);\n    if (!file)\n        return 1;\n\n    constexpr size_t TEST_BUFFER_SIZE = 256 * 1024;\n    if (setvbuf(file.get(), nullptr, _IOFBF, TEST_BUFFER_SIZE) != 0)\n        return 2;\n\n    auto start = steady_clock::now();\n    const uint8_t b = 77;\n    constexpr size_t TEST_FILE_SIZE = 512 * 1024 * 1024;\n    for (size_t i = 0; i &lt; TEST_FILE_SIZE; ++i)\n        fwrite_unlocked(&amp;b, 1, sizeof(b), file.get());\n\n    auto end = steady_clock::now();\n    auto interval = duration_cast&lt;microseconds&gt;(end - start);\n    printf(\"Time: %lldn\", interval.count());\n\n    return 0;\n}\n<\/code><\/pre>\n<p>\nKao varijable koristit \u0107emo TEST_BUFFER_SIZE, a tako\u0111er za nekoliko slu\u010dajeva zamijenit \u0107emo fwrite_unlocked s fwrite. Po\u010dnimo s slu\u010dajem fwrite bez izri\u010ditog postavljanja veli\u010dine baferira (zakomentirat \u0107emo setvbuf i povezani kod): vrijeme 27048906 \u03bcs, brzina pisanja \u2014 18,93 MB\/s. Sada postavljamo veli\u010dinu bafera na 64 KB: vrijeme \u2014 25037111 \u03bcs, brzina \u2014 20,44 MB\/s. Sada \u0107emo testirati rad _fwrite_nolock bez poziva setvbuf: 7262221 \u03bcs, brzina \u2014 70,5 MB\/s! <\/p>\n<p>Dalje \u0107emo eksperimentirati s veli\u010dinom bafera (setvbuf):<\/p>\n<p><img decoding=\"async\" alt=\"Ubrzanje unosa\/izlaza datoteka C\/C++, bez previ\u0161e napora\" src=\"\/wp-content\/uploads\/2019\/03\/6d7998d805ec2e3f0a7905251c664e5c.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPodaci su dobiveni prosjekom od 5 eksperimenta, a nisam se pot trudio da izra\u010dunam pogre\u0161ke. Po mom mi\u0161ljenju, 93 MB\/s pri pisanju po 1 bajtu na obi\u010dnom HDD-u \u2014 to je vrlo dobar rezultat, samo treba odabrati optimalnu veli\u010dinu bafera (u mom slu\u010daju 256 KB \u2014 to je to\u010dno) i zamijeniti fwrite s _fwrite_nolock\/fwrite_unlocked (u slu\u010daju da nije potrebna sigurnost u vi\u0161e niti, naravno). <br \/>\nSli\u010dno je s fread u sli\u010dnim uvjetima. Budu\u0107i da nemam 'hardverski' stroj s linuxom pri ruci (jednoplatni ra\u010dunari se ne ra\u010dunaju), odlu\u010dio sam provesti ograni\u010deni eksperiment na virtualnoj ma\u0161ini (Hyper-V, OpenSUSE 15, GCC 8.3.1) \u2014 uzorak je zapravo isti: 'goloin' fwrite 20 MB\/s, fwrite + bafer od 256 KB dao je 23 MB\/s, fwrite_unlocked s istim baferom \u2014 35 MB\/s (binarno 64-bitno, sakupljeno g++ -o2 -s -static-libgcc -static-libstdc++ fwrite_test.cpp -o fwrite_test).<\/p>\n<h4>Epilog<\/h4>\n<p>\nCilj ovog \u010dlanka bio je opisati jednostavnu i u\u010dinkovitu tehniku koja se mo\u017ee koristiti u mnogim slu\u010dajevima (s funkcijama _fwrite_nolock\/fwrite_unlocked ranije nisam imao iskustva, nisu jako popularne \u2014 a \u0161teta). Ne tvrdim da je materijal nov, ali se nadam da \u0107e \u010dlanak biti koristan zajednici.<\/p>\n<p>Izvor: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/444036\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c \u043d\u0430 \u0441\u0432\u0435\u0442\u0435 \u0442\u0430\u043a\u0430\u044f \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0438 \u043e\u0447\u0435\u043d\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0430\u044f \u0443\u0442\u0438\u043b\u0438\u0442\u0430 \u2014 BDelta, \u0438 \u0442\u0430\u043a \u0432\u044b\u0448\u043b\u043e, \u0447\u0442\u043e \u043e\u043d\u0430 \u043e\u0447\u0435\u043d\u044c \u0434\u0430\u0432\u043d\u043e \u0443\u043a\u043e\u0440\u0435\u043d\u0438\u043b\u0430\u0441\u044c \u0432 \u043d\u0430\u0448\u0435\u043c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u043e\u043c \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0435 (\u043f\u0440\u0430\u0432\u0434\u0430 \u0435\u0451 \u0432\u0435\u0440\u0441\u0438\u044e \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c \u043d\u0435 \u0443\u0434\u0430\u043b\u043e\u0441\u044c, \u043d\u043e \u043e\u043d\u0430 \u0442\u043e\u0447\u043d\u043e \u0431\u044b\u043b\u0430 \u043d\u0435 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0439 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0439). \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0435\u0451 \u043f\u043e \u043f\u0440\u044f\u043c\u043e\u043c\u0443 \u043d\u0430\u0437\u043d\u0430\u0447\u0435\u043d\u0438\u044e \u2014 \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u0438\u0435 \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u043f\u0430\u0442\u0447\u0435\u0439. \u0415\u0441\u043b\u0438 \u0432\u0437\u0433\u043b\u044f\u043d\u0443\u0442\u044c, \u0447\u0442\u043e \u0442\u0430\u043c \u0432 \u0440\u0435\u043f\u043e\u0437\u0438\u0442\u043e\u0440\u0438\u0438, \u2014 \u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u0441\u044f \u0441\u043b\u0435\u0433\u043a\u0430 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-42134","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.4 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.4\" \/>\n\t\t<meta property=\"og:locale\" content=\"hr_HR\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0423\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u0435 \u0444\u0430\u0439\u043b\u043e\u0432\u043e\u0433\u043e \u0432\u0432\u043e\u0434\u0430-\u0432\u044b\u0432\u043e\u0434\u0430 C\/C++, \u043d\u0435 \u043e\u0441\u043e\u0431\u043e \u043d\u0430\u043f\u0440\u044f\u0433\u0430\u044f\u0441\u044c | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-03-17T21:00:00+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-02-18T10:44:56+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Pove\u0107anje brzine unosa\/izlaza datoteka C\/C++, bez previ\u0161e muke | ProHoster","description":"Predgovor.","canonical_url":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"hr_HR","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0423\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u0435 \u0444\u0430\u0439\u043b\u043e\u0432\u043e\u0433\u043e \u0432\u0432\u043e\u0434\u0430-\u0432\u044b\u0432\u043e\u0434\u0430 C\/C++, \u043d\u0435 \u043e\u0441\u043e\u0431\u043e \u043d\u0430\u043f\u0440\u044f\u0433\u0430\u044f\u0441\u044c | ProHoster","og:description":"\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c.","og:url":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-03-17T21:00:00+00:00","article:modified_time":"2020-02-18T10:44:56+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"42134","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 07:22:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:05:22","updated":"2026-01-22 07:22:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts\/42134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/comments?post=42134"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts\/42134\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/media?parent=42134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/categories?post=42134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/tags?post=42134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}