Haɓaka fayil ɗin C/C++ I/O ba tare da fasa gumi ba

Magana

Akwai irin wannan sauƙi kuma mai amfani mai amfani a cikin duniya - BDelta, kuma hakan ya faru da cewa an daɗe a cikin tsarin samar da mu (ko da yake ba zai yiwu a shigar da sigar sa ba, amma ba shakka ba shine na ƙarshe ba). Muna amfani da shi don manufar da aka yi niyya - gina faci na binary. Idan ka dubi abin da ke cikin ma'ajiyar, ya zama ɗan baƙin ciki: a gaskiya, an yi watsi da shi da dadewa kuma yawancinsa ya tsufa sosai (tsohon abokin aikina ya taɓa yin gyare-gyare da yawa a can, amma wannan ya daɗe). . Gabaɗaya, na yanke shawarar tayar da wannan al'amari: Na ƙi, na jefar da abin da ban yi niyyar amfani da shi ba, na motsa aikin zuwa kama, lined "zafi" microfunctions, cire manyan tsararru daga tari (da kuma tsararru na m tsawon, wanda a gaskiya ya sa ni "bam"), gudu da profiler sake - kuma gano cewa game da 40% na lokacin da aka kashe a kan. rubuta...

To me ke faruwa da fwrite?

A cikin wannan lambar, fwrite (a cikin takamaiman yanayin gwaji na: gina faci tsakanin fayilolin kusan 300 MB, bayanan shigarwa gabaɗaya a cikin ƙwaƙwalwar ajiya) ana kiransa miliyoyin sau tare da ƙaramin ƙarami. Babu shakka, wannan abu zai ragu, saboda haka zan so in yi tasiri ga wannan wulakanci. Har yanzu babu wani sha'awar aiwatar da nau'ikan tushen bayanai daban-daban, shigarwar asynchronous-fitarwa, Ina so in sami mafita mafi sauƙi. Abu na farko da ya zo a hankali shine ƙara girman buffer

setvbuf(file, nullptr, _IOFBF, 64* 1024)

amma ban sami ci gaba mai mahimmanci a sakamakon ba (yanzu fwrite ya kai kusan 37% na lokacin) - wanda ke nufin har yanzu ba batun rubuta bayanai akai-akai zuwa faifai ba. Duban "karkashin kaho" na fwrite, zaku iya ganin cewa tsarin kulle / buɗe FILE yana faruwa a cikin wani abu kamar wannan (lambar ƙira, an gudanar da duk bincike ƙarƙashin Visual Studio 2017):

size_t fwrite (const void *buffer, size_t size, size_t count, FILE *stream)
{
   size_t retval = 0;
   _lock_str(stream);   /* lock stream */
   __try
   {
      retval = _fwrite_nolock(buffer, size, count, stream);
   }
   __finally 
   {
       _unlock_str(stream);   /* unlock stream */
   }
   return retval;
}

A cewar Profiler, _fwrite_nolock yana lissafin kashi 6 kawai na lokacin, sauran yana kan gaba. A cikin yanayina na musamman, amincin zaren a fili ya wuce kima, don haka zan sadaukar da shi ta maye gurbin kiran fwrite tare da _fwrite_nolock - ba kwa buƙatar yin wayo da gardama. Jimlar: wannan magudi mai sauƙi ya rage farashin rikodin sakamakon, wanda a cikin asali na asali ya kai kusan rabin lokacin da aka kashe. Af, a cikin POSIX duniya akwai irin wannan aiki - fwrite_unlocked. Gabaɗaya magana, iri ɗaya ya shafi fread. Don haka, ta amfani da nau'i biyu na #bayyanarwa, zaku iya samun mafita ta giciye gaba ɗaya ba tare da makullin da ba dole ba idan ba lallai ba ne (kuma wannan yana faruwa sau da yawa).

rubuta, _fwrite_nolock, setvbuf

Bari mu koma ga aikin asali mu mai da hankali kan gwada wani takamaiman lamari: rubuta babban fayil (512 MB) a cikin ƙananan guntu-guntu—kowanne byte ɗaya. Tsarin gwaji: AMD Ryzen 7 1700, 16 GB RAM, 7200 rpm HDD, cache 64 MB. Windows 10 1809, an gina binary a matsayin 32-bit, an kunna ingantawa, ɗakin karatu yana da alaƙa ta static.

Misalin gwaji:

#include <chrono>
#include <cstdio>
#include <inttypes.h>
#include <memory>

#ifdef _MSC_VER
#define fwrite_unlocked _fwrite_nolock
#endif

using namespace std::chrono;

int main()
{
    std::unique_ptr<FILE, int(*)(FILE*)> file(fopen("test.bin", "wb"), fclose);
    if (!file)
        return 1;

    constexpr size_t TEST_BUFFER_SIZE = 256 * 1024;
    if (setvbuf(file.get(), nullptr, _IOFBF, TEST_BUFFER_SIZE) != 0)
        return 2;

    auto start = steady_clock::now();
    const uint8_t b = 77;
    constexpr size_t TEST_FILE_SIZE = 512 * 1024 * 1024;
    for (size_t i = 0; i < TEST_FILE_SIZE; ++i)
        fwrite_unlocked(&b, 1, sizeof(b), file.get());

    auto end = steady_clock::now();
    auto interval = duration_cast<microseconds>(end - start);
    printf("Time: %lldn", interval.count());

    return 0;
}

Masu canjin za su kasance TEST_BUFFER_SIZE, kuma na wasu lokuta za mu maye gurbin fwrite_unlocked da fwrite. Bari mu fara da shari'ar fwrite ba tare da saita girman buffer a sarari ba (yi sharhi game da setvbuf da lambar alaƙa): lokaci 27048906 µs, rubuta saurin - 18.93 MB/s. Yanzu bari mu saita girman buffer zuwa 64 KB: lokaci - 25037111 μs, gudun - 20.44 Mb/s. Yanzu bari mu gwada aikin _fwrite_nolock ba tare da kiran setvbuf: 7262221 µs, saurin - 70.5 Mb/s!

Na gaba, bari mu gwada da girman buffer (setvbuf):

An samo bayanan ta matsakaicin gwaje-gwaje 5; Na yi kasala don lissafta kurakuran. Amma ni, 93 MB/s lokacin rubuta 1 byte zuwa HDD na yau da kullun sakamako ne mai kyau, kawai kuna buƙatar zaɓar mafi girman girman buffer (a cikin akwati na, 256 KB daidai ne) kuma maye gurbin fwrite tare da _fwrite_nolock/fwrite_unlocked ( idan ba a buƙatar amincin zaren, ba shakka).
Hakanan tare da fread a cikin yanayi iri ɗaya. Tun da ba ni da na'ura mai amfani da Linux a hannu (kwamfutoci guda ɗaya ba su ƙidaya), na yanke shawarar gudanar da iyakataccen gwaji akan na'ura mai mahimmanci (Hyper-V, OpenSUSE 15, GCC 8.3.1) - the Tsarin shine, a ka'ida, iri ɗaya: "tsirara" fwrite 20 Mb/s, fwrite + 256 KB buffer samar da 23 Mb/s, fwrite_unlocked tare da buffer iri ɗaya - 35 Mb/s (binary 64-bit, haɗa g ++ -o2 - s -static-libgcc -static-libstdc++ fwrite_test. cpp -o fwrite_test).

Bayanword

Manufar rubuta wannan labarin shine don bayyana fasaha mai sauƙi kuma mai tasiri a yawancin lokuta (Ban taɓa cin karo da ayyukan _fwrite_nolock/fwrite_unlocked ba, ba su da farin jini sosai - amma a banza). Ba na yin kamar cewa kayan sabo ne, amma ina fatan labarin zai kasance da amfani ga al'umma.

source: www.habr.com