Mee ngwa ngwa C/C++ faịlụ I/O na-agbajighị ọsụsọ

Okwu mmalite

Enwere ngwa dị mfe na nke bara uru n'ụwa - BDelta, na o mere na ọ gbanyere mkpọrọgwụ na usoro mmepụta anyị ogologo oge (ọ bụ ezie na ọ gaghị ekwe omume ịwụnye ụdị ya, ma ọ bụghị nke ikpeazụ). Anyị na-eji ya maka ebumnuche ya - wuo patches ọnụọgụ abụọ. Ọ bụrụ na i lee ihe dị na ebe nchekwa ahụ, ọ na-aghọ ntakịrị mwute: n'ezie, a gbahapụrụ ya ogologo oge gara aga na ọtụtụ n'ime ya bụ ihe ochie (onye ọrụ mbụ m mere ọtụtụ ndezi ebe ahụ, ma nke ahụ bụ ogologo oge gara aga) . N'ozuzu, ekpebiri m ịkpọlite ​​okwu a: Agbaghara m, tụfuo ihe m na-achọghị iji mee ihe, kpaliri ọrụ ahụ na imma, inlined "ọkụ" microfunctions, wepụrụ nnukwu arrays si tojupụtara (na arrays nke agbanwe ogologo, nke n'eziokwu na-eme ka m "bọmbụ"), gbapụrụ na profiler ọzọ - wee chọpụta na ihe dị ka 40% nke oge na-eji na. dee...

Yabụ kedu ihe na-eme na fwrite?

Na koodu a, fwrite (n'ime ule m akọwapụtara: ịmepụta ihe nkpuchi n'etiti faịlụ 300 MB dị nso, data ntinye bụ kpamkpam na ebe nchekwa) a na-akpọ ọtụtụ nde ugboro na obere ihe nchekwa. N'ụzọ doro anya, ihe a ga-akwụsịlata, ya mere ọ ga-amasị m imetụ ihere a n'ụzọ ụfọdụ. Enweghị ọchịchọ ma mejuputa ụdị data dị iche iche, asynchronous I / O, Achọrọ m ịchọta ngwọta dị mfe. Ihe mbụ batara n'uche bụ ịbawanye nha ihe nchekwa

setvbuf(file, nullptr, _IOFBF, 64* 1024)

mana enwetaghị m mmụba dị ịrịba ama na nsonaazụ ya (ugbu a fwrite ruru ihe dị ka 37% nke oge) - nke pụtara na ọ bụghị okwu nke ide data ugboro ugboro na diski. N'ileghachi anya "n'okpuru mkpuchi" nke fwrite, ị nwere ike ịhụ na mkpọchi / imeghe FILE na-eme n'ime ihe dị ka nke a (pseudo-koodu, nyocha niile mere n'okpuru Visual Studio 2017):

size_t fwrite (const void *buffer, size_t size, size_t count, FILE *stream)
{
   size_t retval = 0;
   _lock_str(stream);   /* lock stream */
   __try
   {
      retval = _fwrite_nolock(buffer, size, count, stream);
   }
   __finally 
   {
       _unlock_str(stream);   /* unlock stream */
   }
   return retval;
}

Dị ka onye profaịlụ si kwuo, _fwrite_nolock na-aza naanị 6% nke oge, ndị ọzọ na-aga n'ihu. N'okwu nke m, nchekwa eriri doro anya karịrị akarị, yabụ m ga-achụ ya site na iji dochie oku fwrite. _fwrite_nolock - ọ dịghịkwa mkpa ka ị mara ihe na arụmụka. Mkpokọta: aghụghọ a dị mfe belatara ọnụ ahịa ịdekọ nsonaazụ ya, nke na ụdị izizi ahụ ruru ihe fọrọ nke nta ka ọ bụrụ ọkara nke oge ejiri. Site n'ụzọ, na ụwa POSIX enwere ọrụ yiri ya - fwrite_unlocked. N'ikwu okwu n'ozuzu, otu ihe ahụ na-emetụta fred. Ya mere, iji otu ụzọ #defines, ị nwere ike nweta ngwọta cross-platform kpamkpam na-enweghị mkpọchi na-enweghị isi ma ọ bụrụ na ọ dịghị mkpa (nke a na-eme ọtụtụ ugboro).

fwrite, _fwrite_nolock, setvbuf

Ka anyị pụọ na ọrụ mbụ ahụ ma lekwasị anya n'ịnwale otu ikpe kpọmkwem: ide nnukwu faịlụ (512 MB) n'ime obere iberibe—otu byte nke ọ bụla. Sistemụ nnwale: AMD Ryzen 7 1700, 16 GB RAM, 7200 rpm HDD, 64 MB cache. Windows 10 1809, e wuru binary dị ka 32-bit, a na-agba mbọ ime ka optimizations rụọ ọrụ, ọbá akwụkwọ ahụ jikọtara ya na statically.

Nlereanya maka nnwale:

#include <chrono>
#include <cstdio>
#include <inttypes.h>
#include <memory>

#ifdef _MSC_VER
#define fwrite_unlocked _fwrite_nolock
#endif

using namespace std::chrono;

int main()
{
    std::unique_ptr<FILE, int(*)(FILE*)> file(fopen("test.bin", "wb"), fclose);
    if (!file)
        return 1;

    constexpr size_t TEST_BUFFER_SIZE = 256 * 1024;
    if (setvbuf(file.get(), nullptr, _IOFBF, TEST_BUFFER_SIZE) != 0)
        return 2;

    auto start = steady_clock::now();
    const uint8_t b = 77;
    constexpr size_t TEST_FILE_SIZE = 512 * 1024 * 1024;
    for (size_t i = 0; i < TEST_FILE_SIZE; ++i)
        fwrite_unlocked(&b, 1, sizeof(b), file.get());

    auto end = steady_clock::now();
    auto interval = duration_cast<microseconds>(end - start);
    printf("Time: %lldn", interval.count());

    return 0;
}

Ngbanwe ndị a ga-abụ TEST_BUFFER_SIZE, ma maka ikpe ole na ole, anyị ga-eji fwrite dochie fwrite_unlocked. Ka anyị bido n'okwu fwrite na-edobeghi nha nchekwa nke ọma (kwupụta setvbuf na koodu metụtara): oge ​​27048906 µs, dee ọsọ - 18.93 MB/s. Ugbu a, ka anyị tọọ nha nchekwa na 64 KB: oge ​​- 25037111 μs, ọsọ - 20.44 Mb/s. Ugbu a, ka anyị nwalee ọrụ nke _fwrite_nolock na-akpọghị setvbuf: 7262221 µs, ọsọ - 70.5 Mb/s!

Na-esote, ka anyị jiri nha nchekwa (setvbuf) nwalee:

Enwetara data ahụ site na nkezi nnwale 5; adị m umengwụ ịgbakọ mmejọ ndị ahụ. Ma mụ onwe m, 93 MB / s mgbe ị na-ede 1 byte na HDD oge niile bụ nsonaazụ dị mma, naanị ị ga-ahọrọ nha nchekwa kacha mma (n'ọnọdụ m, 256 KB ziri ezi) wee dochie fwrite na _fwrite_nolock/fwrite_unlocked ( ma ọ bụrụ na nchekwa eri adịghị mkpa, n'ezie).
N'otu aka ahụ na fred na ọnọdụ ndị yiri ya. Ebe ọ bụ na enweghị m igwe ngwaike nwere Linux dị n'aka (kọmpụta otu bọọdụ anaghị agụta), ekpebiri m ịme nnwale dị oke na igwe mebere (Hyper-V, OpenSUSE 15, GCC 8.3.1) - na ụkpụrụ bụ, n'ụkpụrụ, otu: "gba ọtọ" fwrite 20 Mb / s, fwrite + 256 KB ihe nchekwa emepụtara 23 Mb/s, fwrite_unlocked na otu ihe nchekwa - 35 Mb/s (64-bit ọnụọgụ abụọ, gbakọtara g ++ -o2 - s -static-libgcc -static-libstdc++ fwrite_test. cpp -o fwrite_test).

Afterword

Ebumnuche nke ide edemede a bụ ịkọwa usoro dị mfe ma dị irè n'ọtụtụ ọnọdụ (Ahụtụbeghị m ọrụ _fwrite_nolock/fwrite_unlocked tupu, ha anaghị ewu ewu - ma n'efu). Anaghị m eme ka à ga-asị na ihe ahụ dị ọhụrụ, ma enwere m olileanya na isiokwu ahụ ga-aba uru nye obodo.

isi: www.habr.com