{"id":42134,"date":"2019-03-18T00:00:00","date_gmt":"2019-03-17T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas"},"modified":"2020-02-18T13:44:56","modified_gmt":"2020-02-18T10:44:56","slug":"uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","status":"publish","type":"post","link":"https:\/\/prohoster.info\/hu\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","title":{"rendered":"F\u00e1jlbeolvas\u00e1s gyors\u00edt\u00e1sa C\/C++-ban, k\u00fcl\u00f6n\u00f6sebb er\u0151fesz\u00edt\u00e9s n\u00e9lk\u00fcl","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"F\u00e1jlbeolvas\u00e1s<\/p>\n

Bevezet\u0151<\/h4>\n

\nA vil\u00e1gon l\u00e9tezik egy rendk\u00edv\u00fcl egyszer\u0171 \u00e9s hasznos eszk\u00f6z \u2014 BDelta<\/a><\/noindex>, \u00e9s \u00edgy alakult, hogy ez r\u00e9g\u00f3ta gy\u00f6keret vert a gy\u00e1rt\u00e1si folyamatunkban (b\u00e1r a verzi\u00f3j\u00e1t telep\u00edteni nem tudtuk, de biztosan nem a legfrissebb volt). K\u00f6zvetlen c\u00e9lj\u00e1ra haszn\u00e1ljuk \u2014 bin\u00e1ris patch-ek \u00e9p\u00edt\u00e9se. Ha r\u00e1n\u00e9z\u00fcnk, mi tal\u00e1lhat\u00f3 a t\u00e1rol\u00f3ban, kiss\u00e9 szomor\u00fav\u00e1 v\u00e1lik az ember: l\u00e9nyeg\u00e9ben r\u00e9g\u00f3ta elhagyott, \u00e9s sok dolog jelent\u0151sen elavult (valamikor p\u00e1r v\u00e1ltoztat\u00e1st bejuttatott ide egy egykori koll\u00e9g\u00e1m, de az r\u00e9g volt). Sz\u00f3val eld\u00f6nt\u00f6ttem, hogy \u00e9letre keltsem: forkoltam, kidobtam azt, amit nem tervezek haszn\u00e1lni, \u00e1t\u00e1ll\u00edtottam a projektet cmake<\/a><\/noindex>, be\u00e1gyaztam a \u201eforr\u00f3\u201d mikrofunkci\u00f3kat, elt\u00fcntettem a veremb\u0151l a nagy t\u00f6mb\u00f6ket (\u00e9s a v\u00e1ltoz\u00f3 m\u00e9ret\u0171 t\u00f6mb\u00f6ket, amikt\u0151l \u0151szint\u00e9n sz\u00f3lva \u201erobban\u00e1st\u201d kaptam), \u00fajra futtattam a profiloz\u00f3t \u2014 \u00e9s megtudtam, hogy k\u00f6r\u00fclbel\u00fcl az id\u0151 40%-\u00e1t elviszi a fwrite<\/a><\/noindex>\u2026
\n<\/a><\/noindex><\/p>\n

Mi a helyzet az fwrite-tel?<\/h4>\n

\nEbben a k\u00f3dban az fwrite (az \u00e9n konkr\u00e9t tesztem eset\u00e9ben: patch \u00e9p\u00edt\u00e9se k\u00f6zeli 300 MB f\u00e1jlok k\u00f6z\u00f6tt, a bemenetek teljesen a mem\u00f3ri\u00e1ban) milli\u00f3szor h\u00edv\u00f3dik meg kicsi m\u00e9ret\u0171 pufferral. Nyilv\u00e1nval\u00f3, hogy ez a dolog lass\u00edtani fog, \u00e9s ez\u00e9rt szeretn\u00e9m valahogy befoly\u00e1solni ezt a helyzetet. K\u00fcl\u00f6nf\u00e9le adatforr\u00e1sok, aszinkron I\/O bevezet\u00e9s\u00e9re egyel\u0151re nincs v\u00e1gyam, egyszer\u0171bb megold\u00e1st szerettem volna tal\u00e1lni. Az els\u0151, ami eszembe jutott \u2014 n\u00f6velni a buffer m\u00e9ret\u00e9t<\/p>\n

setvbuf(file, nullptr, _IOFBF, 64* 1024)<\/code><\/pre>\n
\nde l\u00e9nyeges javul\u00e1st nem \u00e9rtem el az eredm\u00e9nyben (most az fwrite k\u00f6r\u00fclbel\u00fcl az id\u0151 37%-\u00e1t tette ki) \u2014 teh\u00e1t a dolog m\u00e9gsem a gyakori adatr\u00f6gz\u00edt\u00e9sen m\u00falik. Ha \u201ea motorh\u00e1z al\u00e1\u201d n\u00e9z\u00fcnk, l\u00e1thatjuk, hogy az fwrite belsej\u00e9ben k\u00f6r\u00fclbel\u00fcl \u00edgy folyik a lock\/unlock a FILE strukt\u00far\u00e1val (pszeudok\u00f3d, az eg\u00e9sz elemz\u00e9s a Visual Studio 2017 alatt t\u00f6rt\u00e9nt):<\/p>\n
\nsize_t fwrite (const void *buffer, size_t size, size_t count, FILE *stream)\n{\n   size_t retval = 0;\n   _lock_str(stream);   \/* lez\u00e1r\u00e1s *\/\n   __try\n   {\n      retval = _fwrite_nolock(buffer, size, count, stream);\n   }\n   __finally \n   {\n       _unlock_str(stream);   \/* felold\u00e1s *\/\n   }\n   return retval;\n}\n<\/code><\/pre>\n
\nHa a profiloz\u00f3ra t\u00e1maszkodunk, a _fwrite_nolock mind\u00f6ssze az id\u0151 6%-\u00e1t teszi ki, a t\u00f6bbi pedig az overhead. Az \u00e9n konkr\u00e9t esetemben a sz\u00e1lbiztons\u00e1g nyilv\u00e1nval\u00f3an f\u00f6l\u00f6sleges, ezzel \u00e9n is \u00e1ldozatot hozok, lecser\u00e9lve az fwrite h\u00edv\u00e1st _fwrite_nolock<\/a><\/noindex> \u2014 m\u00e9g az \u00e9rvekkel se kell t\u00falbonyol\u00edtani. \u00d6sszegz\u00e9s\u00fcl: ez az egyszer\u0171 tr\u00fckk jelent\u0151sen cs\u00f6kkentette az eredm\u00e9nyek \u00edr\u00e1s\u00e1nak k\u00f6lts\u00e9geit, amelyek az eredeti v\u00e1ltozatban csaknem a teljes id\u0151r\u00e1ford\u00edt\u00e1s fel\u00e9t tett\u00e9k ki. Egy\u00e9bk\u00e9nt a POSIX vil\u00e1g\u00e1ban l\u00e9tezik egy hasonl\u00f3 funkci\u00f3 \u2014 fwrite_unlocked<\/a><\/noindex>. \u00c1ltal\u00e1noss\u00e1gban, a fread-re is ez vonatkozik. \u00cdgy p\u00e1r #define seg\u00edts\u00e9g\u00e9vel egy meglehet\u0151sen platformf\u00fcggetlen megold\u00e1st hozhatunk l\u00e9tre felesleges z\u00e1rol\u00e1sok n\u00e9lk\u00fcl, ha azok nem sz\u00fcks\u00e9gesek (\u00e9s ez el\u00e9g gyakran el\u0151fordul).<\/p>\n
fwrite, _fwrite_nolock, setvbuf<\/h4>\n
\nAbsztrah\u00e1ljunk az eredeti projektt\u0151l \u00e9s n\u00e9zz\u00fck meg egy konkr\u00e9t eset tesztel\u00e9s\u00e9t: egy nagy f\u00e1jl (512 MB) \u00edr\u00e1sa rendk\u00edv\u00fcl kis darabokban \u2014 1 b\u00e1jt. Teszt rendszer: AMD Ryzen 7 1700, 16 GB RAM, HDD 7200 rpm 64 MB cache, Windows 10 1809, a bin\u00e1ris 32 bitesre \u00e9p\u00fclt, optimaliz\u00e1lva van, a k\u00f6nyvt\u00e1r statikusan linkelve.<\/p>\n
K\u00eds\u00e9rlethez k\u00e9sz\u00fclt minta:<\/p>\n
\n#include <chrono>\n#include <cstdio>\n#include <inttypes.h>\n#include <memory>\n\n#ifdef _MSC_VER\n#define fwrite_unlocked _fwrite_nolock\n#endif\n\nusing namespace std::chrono;\n\nint main()\n{\n    std::unique_ptr<FILE, int(*)(FILE*)> file(fopen(\"test.bin\", \"wb\"), fclose);\n    if (!file)\n        return 1;\n\n    constexpr size_t TEST_BUFFER_SIZE = 256 * 1024;\n    if (setvbuf(file.get(), nullptr, _IOFBF, TEST_BUFFER_SIZE) != 0)\n        return 2;\n\n    auto start = steady_clock::now();\n    const uint8_t b = 77;\n    constexpr size_t TEST_FILE_SIZE = 512 * 1024 * 1024;\n    for (size_t i = 0; i < TEST_FILE_SIZE; ++i)\n        fwrite_unlocked(&b, 1, sizeof(b), file.get());\n\n    auto end = steady_clock::now();\n    auto interval = duration_cast<microseconds>(end - start);\n    printf(\"Time: %lldn\", interval.count());\n\n    return 0;\n}\n<\/code><\/pre>\n
\nA v\u00e1ltoz\u00f3k k\u00f6z\u00f6tt szerepel a TEST_BUFFER_SIZE, tov\u00e1bb\u00e1 n\u00e9h\u00e1ny esetben a fwrite_unlocked-ot fwrite-re cser\u00e9lj\u00fck. Kezdj\u00fck a fwrite esettel, an\u00e9lk\u00fcl, hogy kifejezetten be\u00e1ll\u00edtan\u00e1nk a pufferm\u00e9retet (komment\u00e1ljuk a setvbuf-ot \u00e9s a kapcsol\u00f3d\u00f3 k\u00f3dot): id\u0151 27048906 \u03bcs, \u00edr\u00e1si sebess\u00e9g \u2014 18.93 MB\/s. Most \u00e1ll\u00edtsuk be a pufferm\u00e9retet 64 KB-ra: id\u0151 \u2014 25037111 \u03bcs, sebess\u00e9g \u2014 20.44 MB\/s. Most tesztelj\u00fck a _fwrite_nolock-ot a setvbuf h\u00edv\u00e1sa n\u00e9lk\u00fcl: 7262221 \u03bcs, sebess\u00e9g \u2014 70.5 MB\/s! <\/p>\n
Folytassuk a pufferm\u00e9rettel kapcsolatos k\u00eds\u00e9rleteket (setvbuf):<\/p>\n
\"F\u00e1jlbeolvas\u00e1s
\n
\nAz adatok 5 k\u00eds\u00e9rlet \u00e1tlagol\u00e1s\u00e1val k\u00e9sz\u00fcltek, a hib\u00e1t nem sz\u00e1moltam. V\u00e9lem\u00e9nyem szerint a 93 MB\/s \u00edr\u00e1si sebess\u00e9g 1 b\u00e1jt\u00edr\u00e1s mellett egy hagyom\u00e1nyos HDD-n nagyon j\u00f3 eredm\u00e9ny, csup\u00e1n meg kell tal\u00e1lni az optim\u00e1lis pufferm\u00e9retet (az \u00e9n esetemben 256 KB \u2014 \u00e9ppen megfelel\u0151) \u00e9s sz\u00fcks\u00e9g eset\u00e9n a fwrite-t cser\u00e9lni _fwrite_nolock\/fwrite_unlocked-ra (ha nincs sz\u00fcks\u00e9g sz\u00e1lbiztons\u00e1gra, term\u00e9szetesen). 
\nHasonl\u00f3an a fread-re is ilyen k\u00f6r\u00fclm\u00e9nyek k\u00f6z\u00f6tt. Mivel \u201ehardveres\u201d Linux g\u00e9pem nincs k\u00e9zn\u00e9l (az egyenl\u0151re nem sz\u00e1m\u00edt), korl\u00e1tozott k\u00eds\u00e9rletet v\u00e9geztem egy virtu\u00e1lis g\u00e9pen (Hyper-V, OpenSUSE 15, GCC 8.3.1) \u2014 az \u00f6sszef\u00fcgg\u00e9s alapvet\u0151en ugyanez: a \u201enyers\u201d fwrite 20 MB\/s, fwrite + 256 KB-os pufferral 23 MB\/s-ot produk\u00e1lt, fwrite_unlocked ugyanolyan pufferral \u2014 35 MB\/s (a bin\u00e1ris 64 bites volt, g++ -o2 -s -static-libgcc -static-libstdc++ fwrite_test.cpp -o fwrite_test).<\/p>\n
Ut\u00f3sz\u00f3<\/h4>\n
\nA cikk \u00edr\u00e1s\u00e1nak c\u00e9lja egy egyszer\u0171 \u00e9s sok esetben hat\u00e9kony m\u00f3dszer bemutat\u00e1sa volt (kor\u00e1bban nem tal\u00e1lkoztam a _fwrite_nolock\/fwrite_unlocked funkci\u00f3kkal, nem t\u00fal n\u00e9pszer\u0171ek \u2013 pedig k\u00e1r). A tartalom \u00fajdons\u00e1g\u00e1ra nem tartok ig\u00e9nyt, de rem\u00e9lem, hogy a cikk hasznos lesz a k\u00f6z\u00f6ss\u00e9g sz\u00e1m\u00e1ra.<\/p>\n
Forr\u00e1s: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c \u043d\u0430 \u0441\u0432\u0435\u0442\u0435 \u0442\u0430\u043a\u0430\u044f \u043f\u0440\u043e\u0441\u0442\u0430\u044f \u0438 \u043e\u0447\u0435\u043d\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u0430\u044f \u0443\u0442\u0438\u043b\u0438\u0442\u0430 \u2014 BDelta, \u0438 \u0442\u0430\u043a \u0432\u044b\u0448\u043b\u043e, \u0447\u0442\u043e \u043e\u043d\u0430 \u043e\u0447\u0435\u043d\u044c \u0434\u0430\u0432\u043d\u043e \u0443\u043a\u043e\u0440\u0435\u043d\u0438\u043b\u0430\u0441\u044c \u0432 \u043d\u0430\u0448\u0435\u043c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0441\u0442\u0432\u0435\u043d\u043d\u043e\u043c \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0435 (\u043f\u0440\u0430\u0432\u0434\u0430 \u0435\u0451 \u0432\u0435\u0440\u0441\u0438\u044e \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c \u043d\u0435 \u0443\u0434\u0430\u043b\u043e\u0441\u044c, \u043d\u043e \u043e\u043d\u0430 \u0442\u043e\u0447\u043d\u043e \u0431\u044b\u043b\u0430 \u043d\u0435 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0435\u0439 \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0439). \u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u043c \u0435\u0451 \u043f\u043e \u043f\u0440\u044f\u043c\u043e\u043c\u0443 \u043d\u0430\u0437\u043d\u0430\u0447\u0435\u043d\u0438\u044e \u2014 \u043f\u043e\u0441\u0442\u0440\u043e\u0435\u043d\u0438\u0435 \u0431\u0438\u043d\u0430\u0440\u043d\u044b\u0445 \u043f\u0430\u0442\u0447\u0435\u0439. \u0415\u0441\u043b\u0438 \u0432\u0437\u0433\u043b\u044f\u043d\u0443\u0442\u044c, \u0447\u0442\u043e \u0442\u0430\u043c \u0432 \u0440\u0435\u043f\u043e\u0437\u0438\u0442\u043e\u0440\u0438\u0438, \u2014 \u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u0441\u044f \u0441\u043b\u0435\u0433\u043a\u0430 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-42134","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47F\u00e1jlok be- \u00e9s kimenet\u00e9nek felgyors\u00edt\u00e1sa C\/C++ nyelven, nem t\u00falfesz\u00fclve | ProHoster","description":"El\u0151sz\u00f3 Van.","canonical_url":"https:\/\/prohoster.info\/hu\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"hu_HU","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0423\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u0435 \u0444\u0430\u0439\u043b\u043e\u0432\u043e\u0433\u043e \u0432\u0432\u043e\u0434\u0430-\u0432\u044b\u0432\u043e\u0434\u0430 C\/C++, \u043d\u0435 \u043e\u0441\u043e\u0431\u043e \u043d\u0430\u043f\u0440\u044f\u0433\u0430\u044f\u0441\u044c | ProHoster","og:description":"\u041f\u0440\u0435\u0434\u0438\u0441\u043b\u043e\u0432\u0438\u0435 \u0415\u0441\u0442\u044c.","og:url":"https:\/\/prohoster.info\/hu\/blog\/administrirovanie\/uskorenie-fajlovogo-vvoda-vyvoda-c-c-ne-osobo-napryagayas","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-03-17T21:00:00+00:00","article:modified_time":"2020-02-18T10:44:56+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"42134","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 07:22:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:05:22","updated":"2026-01-22 07:22:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/posts\/42134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/comments?post=42134"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/posts\/42134\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/media?parent=42134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/categories?post=42134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/tags?post=42134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}