prehistory

Pali makina ogulitsa omwe adapangidwa ndithu. Mkati mwa Raspberry Pi ndi mawaya ena pa bolodi losiyana. Wolandira ndalama, wolandira bilu, malo osungira banki amalumikizidwa ... Chilichonse chimayendetsedwa ndi pulogalamu yodzilemba yokha. Mbiri yonse ya ntchito imalembedwa ku chipika pa flash drive (MicroSD), yomwe imafalitsidwa kudzera pa intaneti (pogwiritsa ntchito modemu ya USB) ku seva, kumene imasungidwa mu database. Zambiri zogulitsa zimayikidwa mu 1c, palinso mawonekedwe osavuta a intaneti owunikira, ndi zina zambiri.

Ndiko kuti, magazini ndiyofunikira - pakuwerengera ndalama (ndalama, malonda, ndi zina), kuyang'anira (zolephera zamitundu yonse ndi zochitika zina zamphamvu); Izi, wina anganene, ndizo zonse zomwe tili nazo za makinawa.

vuto

Ma drive a Flash amadziwonetsa ngati zida zosadalirika. Amalephera mokhazikika. Izi zimabweretsa kutsika kwa makina onse ndipo (ngati pazifukwa zina chipikacho sichinasamutsidwe pa intaneti) kutayika kwa data.

Ichi sichinali choyamba chogwiritsa ntchito ma drive ang'onoang'ono, izi zisanachitike panali pulojekiti ina yokhala ndi zida zopitilira zana, pomwe magaziniyi idasungidwa pa ma drive a USB flash, panalinso mavuto ndi kudalirika, nthawi zina kuchuluka kwa omwe adalephera. mwezi uli m'ma khumi ndi awiri. Tidayesa ma drive osiyanasiyana, kuphatikiza omwe ali ndi kukumbukira kwa SLC, ndipo mitundu ina ndi yodalirika kuposa ena, koma kusintha ma drive a flash sikunathetse vutoli.

Chonde chonde! Zakale! Ngati mulibe chidwi ndi "chifukwa", koma "motani", mukhoza kupita molunjika Pomaliza pake zolemba.

chisankho

Chinthu choyamba chimene chimabwera m'maganizo ndi: kusiya MicroSD, kukhazikitsa, mwachitsanzo, SSD, ndi boot kuchokera izo. Mwachidziwitso zotheka, mwina, koma ndi okwera mtengo, komanso osadalirika (adapta ya USB-SATA imawonjezedwa; ziwerengero zolephera za SSD za bajeti sizilimbikitsanso).

USB HDD nayonso sikuwoneka ngati yankho lokongola kwambiri.

Chifukwa chake, tasankha njira iyi: siyani kuyambika kuchokera ku MicroSD, koma muzigwiritsa ntchito powerenga-pokha, ndikusunga chipika cha opareshoni (ndi zidziwitso zina zapadera pagawo lina la hardware - nambala ya serial, ma sensor calibrations, etc.) kwinakwake. .

Mutu wa FS wowerengeka chabe wa raspberries waphunziridwa kale mkati ndi kunja, sindidzakhazikika pazambiri zoyendetsera nkhaniyi. (koma ngati pali chidwi, mwina ndilemba kankhani kakang'ono pamutuwu). Mfundo yokhayo yomwe ndikufuna kuti ndizindikire ndikuti zonse kuchokera pazochitika zaumwini komanso kuchokera ku ndemanga za omwe adazikhazikitsa kale, pali phindu lodalirika. Inde, ndizosatheka kuchotsa zowonongeka, koma kuchepetsa kwambiri mafupipafupi awo ndizotheka. Ndipo makhadi akukhala ogwirizana, zomwe zimapangitsa kuti m'malo mwake mukhale osavuta kwa ogwira ntchito.

Z Hardware

Panalibe kukayikira kwenikweni pa kusankha kwa mtundu wa kukumbukira - NOR Flash.
Zokangana:

kugwirizana kosavuta (nthawi zambiri mabasi a SPI, omwe mwakhala mukugwiritsa ntchito kale, kotero kuti palibe mavuto a hardware omwe akuwoneratu);
mtengo wotsika;
ndondomeko yoyendetsera ntchito (kukhazikitsa kuli kale mu Linux kernel, ngati mukufuna, mukhoza kutenga wina wachitatu, yemwe alipo, kapena kulemba nokha, mwamwayi zonse ndi zophweka);
kudalirika ndi zothandizira:
kuchokera ku deta yeniyeni: deta imasungidwa kwa zaka 20, 100000 kufufuta kuzungulira chipika chilichonse;
kuchokera kumagwero a chipani chachitatu: BER yotsika kwambiri, imanena kuti palibe chifukwa chowongolera zolakwika (ntchito zina zimaganizira ECC za NOR, koma nthawi zambiri zimatanthawuza MLC NOR; izi zimachitikanso).

Tiyeni tiyerekeze zofunikira za voliyumu ndi gwero.

Ndikufuna kuti deta itsimikizidwe kuti isungidwe kwa masiku angapo. Izi ndizofunikira kuti pakakhala vuto lililonse ndi kulumikizana, mbiri yamalonda siyitayika. Tidzayang'ana pa masiku 5, panthawiyi (ngakhale kuganizira kumapeto kwa sabata ndi tchuthi) vuto likhoza kuthetsedwa.

Pakalipano timasonkhanitsa pafupifupi 100kb ya zipika patsiku (zolemba 3-4 zikwi), koma pang'onopang'ono chiwerengerochi chikukula - tsatanetsatane ikuwonjezeka, zochitika zatsopano zikuwonjezeredwa. Kuphatikiza apo, nthawi zina pamakhala kuphulika (sensor ina imayamba spamming ndi zabwino zabodza, mwachitsanzo). Tiwerengera ma 10 ma 100 byte - megabytes patsiku.

Pazonse, 5MB ya data yoyera (yoponderezedwa bwino) imatuluka. Zambiri kwa iwo (kuyerekeza) 1MB ya data yantchito.

Ndiko kuti, tifunika chip 8MB ngati sitigwiritsa ntchito kukanikiza, kapena 4MB ngati tigwiritsa ntchito. Manambala enieni amtundu uwu wa kukumbukira.

Ponena za gwero: ngati tikukonzekera kuti chikumbukiro chonse chilembedwenso osaposa kamodzi masiku 5, ndiye kuti pazaka 10 zautumiki timapeza zozungulira zosakwana chikwi.
Ndiroleni ndikukumbutseni kuti wopanga akulonjeza chikwi zana.

Zambiri za NOR vs NAND

Lero, zachidziwikire, kukumbukira kwa NAND ndikotchuka kwambiri, koma sindikanagwiritsa ntchito ntchitoyi: NAND, mosiyana ndi NOR, imafunikira kugwiritsa ntchito ma code owongolera zolakwika, tebulo la midadada yoyipa, ndi zina zambiri, komanso miyendo ya NAND chips nthawi zambiri zimakhala zambiri.

Zoyipa za NOR ndi izi:

voliyumu yaying'ono (ndipo, motero, mtengo wapamwamba pa megabyte);
Kuthamanga kwapang'onopang'ono (makamaka chifukwa chakuti mawonekedwe a serial amagwiritsidwa ntchito, kawirikawiri SPI kapena I2C);
kufufuta pang'onopang'ono (kutengera kukula kwa chipika, zimatengera kachigawo kakang'ono mpaka masekondi angapo).

Zikuwoneka kuti palibe chovuta kwa ife, kotero tikupitiriza.

Ngati zambiri zili zosangalatsa, microcircuit yasankhidwa ku 25df321a (Komabe, izi ndizosafunika, pali ma analogue ambiri pamsika omwe amagwirizana ndi pinout ndi dongosolo lamalamulo; ngakhale tikufuna kukhazikitsa microcircuit kuchokera kwa wopanga wina ndi / kapena kukula kosiyana, chilichonse chimagwira ntchito popanda kusintha kodi).

Ndimagwiritsa ntchito dalaivala yomangidwa mu kernel ya Linux; pa Rasipiberi, chifukwa cha chithandizo chokuta mtengo cha chipangizo, chirichonse chiri chophweka - muyenera kuyika pamwamba pa /boot/overlays ndikusintha pang'ono /boot/config.txt.

Chitsanzo dts file

Kunena zowona, sindikutsimikiza kuti zalembedwa popanda zolakwika, koma zimagwira ntchito.

/*
 * Device tree overlay for at25 at spi0.1
 */

/dts-v1/;
/plugin/;

/ {
    compatible = "brcm,bcm2835", "brcm,bcm2836", "brcm,bcm2708", "brcm,bcm2709"; 

    /* disable spi-dev for spi0.1 */
    fragment@0 {
        target = <&spi0>;
        __overlay__ {
            status = "okay";
            spidev@1{
                status = "disabled";
            };
        };
    };

    /* the spi config of the at25 */
    fragment@1 {
        target = <&spi0>;
        __overlay__ {
            #address-cells = <1>;
            #size-cells = <0>;
            flash: m25p80@1 {
                    compatible = "atmel,at25df321a";
                    reg = <1>;
                    spi-max-frequency = <50000000>;

                    /* default to false:
                    m25p,fast-read ;
                    */
            };
        };
    };

    __overrides__ {
        spimaxfrequency = <&flash>,"spi-max-frequency:0";
        fastread = <&flash>,"m25p,fast-read?";
    };
};

Ndipo mzere wina mu config.txt

dtoverlay=at25:spimaxfrequency=50000000

Ndidzasiya kufotokozera za kulumikiza chip ku Raspberry Pi. Kumbali imodzi, sindine katswiri wa zamagetsi, kumbali ina, chirichonse pano ndi banal ngakhale kwa ine: microcircuit ili ndi miyendo 8 yokha, yomwe timafunikira nthaka, mphamvu, SPI (CS, SI, SO, SCK). ); milingo ndi yofanana ndi ya Raspberry Pi, palibe mawaya owonjezera omwe amafunikira - ingolumikizani mapini 6 omwe awonetsedwa.

Kupanga kwa vuto

Monga mwachizolowezi, mawu avuto amapitilira kangapo, ndipo zikuwoneka kwa ine kuti ndi nthawi yotsatira. Choncho tiyeni tiyime, phatikizani zomwe zalembedwa kale, ndi kumveketsa bwino zomwe zatsala m'mithunzi.

Chifukwa chake, tasankha kuti chipikacho chisungidwe mu SPI NOR Flash.

Kodi NOR Flash ndi chiyani kwa omwe sakudziwa?

Uku ndikukumbukira kosasunthika komwe mungathe kuchita maopaleshoni atatu:

Kuwerenga:
Kuwerenga kofala: timatumiza adilesi ndikuwerenga ma byte ambiri momwe timafunikira;
Lembani:
Kulembera ku NOR flash kumawoneka ngati kokhazikika, koma kumakhala ndi mawonekedwe amodzi: mutha kusintha 1 kukhala 0, koma osati mosemphanitsa. Mwachitsanzo, tikadakhala ndi 0x55 mu cell memory, ndiye kuti titalemba 0x0f kwa iyo, 0x05 idzasungidwa kale pamenepo. (onani tebulo pansipa);
Fufutani:
Zachidziwikire, tifunika kuchita ntchito yosiyana - sinthani 0 kukhala 1, izi ndi zomwe ntchito yofufutira ili. Mosiyana ndi ziwiri zoyamba, sizigwira ntchito ndi ma byte, koma ndi midadada (chocheperako chofufutira mu chip chosankhidwa ndi 4kb). Kufufuta kumawononga chipika chonse ndipo ndiyo njira yokhayo yosinthira 0 mpaka 1. Chifukwa chake, mukamagwira ntchito ndi flash memory, nthawi zambiri mumayenera kugwirizanitsa zida za data kumalire a block block.
Kujambula mu NOR Flash:

Data ya binary

Zinali
01010101

Zojambulidwa
00001111

Zakhala
00000101

Logi palokha imayimira mndandanda wa zolemba zautali wosiyanasiyana. Utali wanthawi zonse wa mbiriyo ndi pafupifupi ma byte 30 (ngakhale zolemba zomwe zimakhala ndi ma kilobytes angapo kutalika nthawi zina zimachitika). Pankhaniyi, timagwira nawo ntchito ngati ma byte, koma, ngati mukufuna, CBOR imagwiritsidwa ntchito mkati mwazolemba.

Kuphatikiza pa chipikacho, tiyenera kusunga zina "zokhazikitsira", zonse zosinthidwa osati: ID ya chipangizo china, kuwerengetsa kwa sensa, mbendera ya "chipangizo choyimitsidwa kwakanthawi", ndi zina zambiri.
Izi ndi mndandanda wa zolemba zamtengo wapatali, zomwe zimasungidwa mu CBOR. Tilibe zambiri za izi (ma kilobytes ochepa kwambiri), ndipo zimasinthidwa pafupipafupi.
Muzotsatira tidzazitcha kuti nkhani.

Ngati tikumbukira komwe nkhaniyi idayambira, ndikofunikira kwambiri kuonetsetsa kuti kusungidwa kwa data kodalirika komanso, ngati kuli kotheka, kugwira ntchito mosalekeza ngakhale pakagwa kulephera kwa hardware / katangale wa data.

Ndi magwero ati a mavuto omwe angaganizidwe?

Kuzimitsa panthawi yolemba/kufufuta ntchito. Izi zikuchokera m'gulu la "palibe chinyengo motsutsana ndi crowbar."
Zambiri kuchokera zokambirana pa stackexchange: mphamvu ikazimitsidwa ikugwira ntchito ndi kung'anima, zonse fufutani (zokhazikitsidwa ku 1) ndi kulemba (zokhazikitsidwa ku 0) zimatsogolera ku khalidwe losadziwika: deta ikhoza kulembedwa, kulembedwa pang'ono (kunena, tinasamutsa 10 bytes / 80 bits). , koma osati ma bits 45 okha omwe angalembedwe), ndizothekanso kuti ma bits ena adzakhala "pakatikati" (kuwerenga kungapangitse 0 ndi 1);
Zolakwika mu flash memory yokha.
BER, ngakhale yotsika kwambiri, sangakhale wofanana ndi ziro;
Zolakwika za basi
Zomwe zimafalitsidwa kudzera pa SPI sizitetezedwa mwanjira iliyonse, zolakwika zapang'ono pang'ono ndi zolakwika zamalumikizidwe zitha kuchitika - kutayika kapena kuyika ma bits (zomwe zimadzetsa kuwonongeka kwakukulu kwa data);
Zolakwa zina/zolakwika
Zolakwika mu code, rasipiberi glitches, kusokoneza mlendo ...

Ndapanga zofunikira, zomwe, mwa lingaliro langa, ndizofunikira kuti zitsimikizire kudalirika:

zolemba ziyenera kulowa mu flash memory nthawi yomweyo, zochedwa kulemba sizikuganiziridwa; - ngati cholakwika chichitika, chiyenera kuzindikiridwa ndikukonzedwa mwachangu momwe zingathere; - dongosololi liyenera, ngati kuli kotheka, kuchira zolakwika.
(chitsanzo kuchokera ku moyo "momwe sichiyenera kukhalira", zomwe ndikuganiza kuti aliyense wakumana nazo: pambuyo poyambiranso mwadzidzidzi, fayilo ya fayilo "yasweka" ndipo makina ogwiritsira ntchito sakuyamba)

Malingaliro, njira, malingaliro

Nditayamba kuganizira za vutoli, malingaliro ambiri adadutsa m'mutu mwanga, mwachitsanzo:

gwiritsani ntchito kuponderezana kwa data;
gwiritsani ntchito zida zanzeru za data, mwachitsanzo, kusunga mitu yolembera mosiyana ndi zolemba zokha, kuti ngati pali cholakwika muzolemba zilizonse, mutha kuwerenga zina zonse popanda vuto;
gwiritsani ntchito magawo ang'onoang'ono kuti muwongolere kumaliza kujambula pomwe mphamvu yazimitsidwa;
sungani macheke pa chilichonse;
gwiritsani ntchito zokhota zosamva phokoso.

Ena mwa malingalirowa adagwiritsidwa ntchito, pomwe ena adasankhidwa kuti asiyidwe. Tiyeni tipite mwadongosolo.

Kuphatikizika kwa data

Zochitika zomwe timalemba m'magaziniyi ndizofanana komanso zobwerezabwereza ("anaponya ndalama za ruble 5", "ndinasindikiza batani kuti musinthe", ...). Choncho, psinjika ayenera ndithu ogwira.

Kuponderezana kwapamutu ndikocheperako (purosesa yathu ndi yamphamvu kwambiri, ngakhale Pi yoyamba inali ndi core imodzi yokhala ndi ma frequency a 700 MHz, mitundu yaposachedwa imakhala ndi ma cores angapo ndi ma frequency a gigahertz), kusinthanitsa ndi zosungirako ndizotsika (kambiri. megabytes pamphindikati), kukula kwa zolembazo ndi zazing'ono. Nthawi zambiri, ngati kupsinjika kumakhudza magwiridwe antchito, kumakhala bwino. (zopanda kutsutsa, kungonena). Kuphatikiza apo, tilibe ophatikizidwa kwenikweni, koma Linux wamba - kotero kukhazikitsa sikuyenera kuyesayesa kwambiri (ndikokwanira kungolumikiza laibulale ndikugwiritsa ntchito zingapo kuchokera pamenepo).

Chidutswa cha chipikacho chinatengedwa kuchokera ku chipangizo chogwirira ntchito (1.7 MB, 70 zikwi zolembera) ndikuyang'ana kaye kuti chikhoza kutheka pogwiritsa ntchito gzip, lz4, lzop, bzip2, xz, zstd yomwe ilipo pa kompyuta.

gzip, xz, zstd adawonetsa zotsatira zofanana (40Kb).
Ndinadabwa kuti xz yapamwamba idadziwonetsa pano pamlingo wa gzip kapena zstd;
lzip yokhala ndi zosintha zosasintha idapereka zotsatira zoyipa pang'ono;
lz4 ndi lzop sanawonetse zotsatira zabwino kwambiri (150Kb);
bzip2 adawonetsa zotsatira zabwino modabwitsa (18Kb).

Choncho, deta ndi wothinikizidwa bwino kwambiri.
Chifukwa chake (ngati sitipeza zolakwika zakupha) padzakhala kuponderezana! Mwachidule chifukwa zambiri zitha kukwanira pagalimoto yomweyo.

Tiyeni tiganizire za kuipa kwake.

Vuto loyamba: tavomereza kale kuti mbiri iliyonse iyenera kupita ku flash. Kawirikawiri, wosunga zakale amasonkhanitsa deta kuchokera kumtsinje wolowetsa mpaka atasankha kuti ndi nthawi yolemba kumapeto kwa sabata. Tiyenera kulandira nthawi yomweyo chipika cha data ndikuchisunga mu kukumbukira kosasunthika.

Ndikuwona njira zitatu:

Tsindikani mbiri iliyonse pogwiritsa ntchito dikishonale ya mtanthauzira mawu m'malo mwa ma algorithms omwe takambirana pamwambapa.
Ndi njira yogwirira ntchito kwathunthu, koma sindimakonda. Kuwonetsetsa kuti kanikizidwe kabwinoko kapena kocheperako, dikishonale iyenera "kusinthidwa" kuti igwirizane ndi data yeniyeni; kusintha kulikonse kumapangitsa kuti mulingo wapakatikati ugwe mwatsoka. Inde, vutoli litha kuthetsedwa popanga buku latsopano la mtanthauzira mawu, koma ili ndi mutu - tifunika kusunga mitundu yonse ya dikishonale; m'cholemba chilichonse tifunika kuwonetsa mtundu wa mtanthauzira mawu womwe unatsindikiridwa...
Gwirani mbiri iliyonse pogwiritsa ntchito ma algorithms a "classical", koma osatengera enawo.
Ma aligorivimu oponderezedwa omwe akuganiziridwawo sanapangidwe kuti azigwira ntchito ndi zolemba za kukula uku (makumi a ma byte), kuchuluka kwa kuponderezana kudzakhala kochepera 1 (ndiko kuti, kuwonjezera voliyumu ya data m'malo mokakamiza);
Chitani FLUSH mukatha kujambula kulikonse.
Ma library ambiri ophatikizika ali ndi chithandizo cha FLUSH. Ili ndi lamulo (kapena gawo la njira yoponderezedwa), ikalandira zomwe wosunga zakale amapanga mtsinje woponderezedwa kuti ugwiritsidwe ntchito kubwezeretsa. onse deta yosakanizidwa yomwe yalandilidwa kale. Analogue yotere sync mumafayilo kapena commit mu sql.
Chofunikira ndichakuti ma compression otsatirawa azitha kugwiritsa ntchito dikishonale yosonkhanitsidwa ndipo kuchuluka kwa kuponderezana sikudzavutikira monga momwe zidalili kale.

Ndikuganiza kuti ndizodziwikiratu kuti ndasankha njira yachitatu, tiyeni tiwone mwatsatanetsatane.

Zapezeka nkhani yayikulu za FLUSH mu zlib.

Ndinayesa bondo pogwiritsa ntchito nkhaniyi, ndinatenga zolemba za 70 zikwi kuchokera ku chipangizo chenichenicho, ndi kukula kwa tsamba la 60Kb. (tibwereranso ku kukula kwatsamba mtsogolo) analandira:

Zambiri
Kupondereza gzip -9 (palibe FLUSH)
zlib ndi Z_PARTIAL_FLUSH
zlib ndi Z_SYNC_FLUSH

Mtundu, KB
1692
40
352
604

Poyang'ana koyamba, mtengo woperekedwa ndi FLUSH ndi wokwera kwambiri, koma kwenikweni tilibe chosankha - mwina kusapanikiza nkomwe, kapena kufinya (komanso mogwira mtima) ndi FLUSH. Sitiyenera kuiwala kuti tili ndi zolemba zikwi 70, redundancy yomwe idayambitsidwa ndi Z_PARTIAL_FLUSH ndi ma byte 4-5 okha pa mbiri iliyonse. Ndipo chiŵerengero cha kuponderezana chinakhala pafupifupi 5: 1, chomwe chiri choposa zotsatira zabwino kwambiri.

Zitha kudabwitsa, koma Z_SYNC_FLUSH ndiyo njira yabwino kwambiri yochitira FLUSH

Mukamagwiritsa ntchito Z_SYNC_FLUSH, mabayiti 4 omaliza pazolowera zilizonse azikhala 0x00, 0x00, 0xff, 0xff. Ndipo ngati timawadziwa, ndiye kuti sitiyenera kuwasunga, kotero kukula komaliza ndi 324Kb kokha.

Nkhani yomwe ndidalumikizana nayo ili ndi kufotokozera:

Chida chatsopano cha 0 chokhala ndi zopanda kanthu chimawonjezeredwa.

Mtundu wa 0 block wokhala ndi zopanda kanthu uli ndi:

mutu wa block-bit-bit;

0 mpaka 7 bits wofanana ndi ziro, kukwaniritsa ma byte;

mndandanda wa ma baiti anayi 00 00 FF FF.

Monga mukuwonera mosavuta, mu block yomaliza pamaso pa ma byte 4 awa pali kuyambira 3 mpaka 10 zero bits. Komabe, machitidwe awonetsa kuti pali zochepera 10 zero.

Zikuwonekeratu kuti midadada yayifupi yotereyi nthawi zambiri imakhala (nthawi zonse?) yosungidwa pogwiritsa ntchito chipika cha mtundu 1 (chidacho chokhazikika), chomwe chimatha ndi ziro 7, kupatsa zonse 10-17 zotsimikizika zero bits (ndi zina zonse zidzatero. kukhala ziro ndi chotheka cha pafupifupi 50%).

Chifukwa chake, pa data yoyeserera, mu 100% ya milandu pali ziro byte imodzi isanakwane 0x00, 0x00, 0xff, 0xff, ndipo mopitilira gawo limodzi mwa magawo atatu a milandu pali ziro byte ziwiri. (mwinamwake chowonadi ndichakuti ndimagwiritsa ntchito binary CBOR, ndipo ndikamagwiritsa ntchito mawu a JSON, midadada yamtundu wa 2 - block block ingakhale yofala kwambiri, motsatana, midadada yopanda ziro byte isanakwane 0x00, 0x00, 0xff, 0xff kukumana).

Ponseponse, pogwiritsa ntchito mayeso omwe alipo, ndizotheka kulowa muzochepera 250Kb za data yothinikizidwa.

Mutha kupulumutsa pang'ono pochita juggling pang'ono: pakadali pano timanyalanyaza kukhalapo kwa ziro zocheperako kumapeto kwa chipikacho, zocheperako kumayambiriro kwa chipikacho sizisinthanso...
Koma kenako ndinapanga chiganizo champhamvu chosiya, apo ayi pamlingo uwu nditha kupanga zolemba zanga.

Pazonse, kuchokera ku data yanga yoyesa ndinalandira ma byte 3-4 polemba, chiŵerengero cha kuponderezana chinakhala choposa 6: 1. Ndikhala woona mtima: Sindinayembekezere zotsatira zotere; m'malingaliro anga, chilichonse chabwino kuposa 2: 1 ndi zotsatira zomwe zimatsimikizira kugwiritsa ntchito kuponderezana.

Chilichonse chili bwino, koma zlib (deflate) ikadali yachikale, yoyenera komanso yachikale kwambiri. Zomwe zimangonena kuti 32Kb yomaliza ya mtsinje wosakanizidwa wa data imagwiritsidwa ntchito ngati dikishonale ikuwoneka yachilendo lero (ndiko kuti, ngati chipika cha data chili chofanana ndi chomwe chinali mumtsinje wa 40Kb wapitawo, ndiye kuti chidzayambanso kusungidwa, ndipo sizikutanthauza zomwe zidachitika kale). M'mafayilo amakono amakono, kukula kwa mtanthauzira mawu nthawi zambiri kumayesedwa mu megabytes osati ma kilobytes.

Chifukwa chake tikupitiliza maphunziro athu ang'onoang'ono osunga zakale.

Kenako tidayesa bzip2 (kumbukirani, popanda FLUSH idawonetsa chiwopsezo chosangalatsa cha pafupifupi 100: 1). Tsoka ilo, silinachite bwino kwambiri ndi FLUSH; kukula kwa deta yoponderezedwa kudakhala yayikulu kuposa zomwe sizimalumikizidwa.

Malingaliro anga pazifukwa zolephera

Libbz2 imapereka njira imodzi yokha yosinthira, yomwe ikuwoneka kuti ikuchotsa mtanthauzira mawu (ofanana ndi Z_FULL_FLUSH mu zlib); palibe zokamba za kukakamiza kulikonse pambuyo pa izi.

Ndipo womaliza kuyesedwa anali zstd. Kutengera magawo, imakanikiza mwina pamlingo wa gzip, koma mwachangu kwambiri, kapena kuposa gzip.

Tsoka, ndi FLUSH sizinachite bwino kwambiri: kukula kwa deta yoponderezedwa kunali pafupifupi 700Kb.

Я anafunsa funso pa tsamba la github la polojekitiyi, ndidalandira yankho loti muyenera kuwerengera mpaka ma 10 byte a data pagawo lililonse la data yoponderezedwa, yomwe ili pafupi ndi zotsatira zomwe zapezedwa; palibe njira yopezera deflate.

Ndidaganiza zosiya pakadali pano pakuyesa kwanga ndi osunga zakale (ndiroleni ndikukumbutseni kuti xz, lzip, lzo, lz4 sanadziwonetsere ngakhale pagawo loyesa popanda FLUSH, ndipo sindidaganizire ma aligorivimu achilendo kwambiri).

Tiyeni tibwerere ku zovuta zosunga zakale.

Yachiwiri (monga momwe amanenera kuti, osati phindu) vuto ndiloti deta yoponderezedwa ndi mtsinje umodzi, momwe nthawi zonse zimatchula zigawo zam'mbuyo. Chifukwa chake, ngati gawo la data lopanikizidwa likuwonongeka, sititaya gawo lokhalo la deta yosakanizidwa, komanso zonse zotsatila.

Pali njira yothetsera vutoli:

Pewani vuto kuti lisachitike - onjezani kubwereza kwa data yothinikizidwa, yomwe ingakuthandizeni kuzindikira ndikuwongolera zolakwika; tidzakambirana pambuyo pake;
Chepetsani zotsatira ngati vuto lichitika
Tanena kale kuti mutha kukakamiza chipika chilichonse cha data pawokha, ndipo vutolo lizimiririka palokha (kuwonongeka kwa data ya chipika chimodzi kungayambitse kutayika kwa data kokha pa chipikachi). Komabe, izi ndizovuta kwambiri zomwe kuponderezana kwa data sikungakhale kothandiza. Mosiyana kwambiri ndi izi: gwiritsani ntchito 4MB yonse ya chip yathu ngati chosungira chimodzi, zomwe zingatipatse kuponderezana kwakukulu, koma zotsatira zowopsa pakachitika chiphuphu cha data.
Inde, kulolerana kumafunika pankhani yodalirika. Koma tiyenera kukumbukira kuti tikupanga mawonekedwe osungira deta a kukumbukira kosasunthika ndi BER yotsika kwambiri komanso nthawi yodziwika yosungira deta ya zaka 20.

Pazoyeserera, ndidapeza kuti zotayika zochulukirapo kapena zocheperako pamlingo woponderezedwa zimayambira pama block a data osakwana 10 KB kukula.
Zinanenedwapo kale kuti kukumbukira komwe kumagwiritsidwa ntchito ndi tsamba; Sindikuwona chifukwa chomwe makalata a "tsamba limodzi - gawo limodzi la data compressed" sayenera kugwiritsidwa ntchito.

Ndiko kuti, kukula kwatsamba kocheperako ndi 16Kb (yokhala ndi malo osungira zambiri). Komabe, tsamba laling'ono ngatilo limapereka zoletsa zazikulu pakukula kwambiri.

Ngakhale sindikuyembekezerabe zolemba zazikulu kuposa ma kilobytes ochepa mu mawonekedwe oponderezedwa, ndinaganiza zogwiritsa ntchito masamba a 32Kb (pamasamba onse a 128 pa chip).

Chidule:

Timasunga deta yoponderezedwa pogwiritsa ntchito zlib (deflate);
Pazolowera zilizonse timayika Z_SYNC_FLUSH;
Pa mbiri iliyonse yopanikizidwa, timadula ma byte otsatsira (mwachitsanzo 0x00, 0x00, 0xff, 0xff); pamutu tikuwonetsa ma byte angati omwe tadula;
Timasunga deta mumasamba a 32Kb; pali mtsinje umodzi wa deta wothinikizidwa mkati mwa tsamba; Patsamba lililonse timayambanso kukanikiza.

Ndipo, ndisanamalize ndi kukakamiza, ndikufuna ndikuwonetseni kuti tili ndi ma byte ochepa chabe a data pa mbiri iliyonse, kotero ndikofunikira kwambiri kuti tisawonjezere zambiri zautumiki, ma byte aliwonse amawerengera apa.

Kusunga Mitu Yama Data

Popeza tili ndi zolembedwa zautali wosiyanasiyana, tifunika kudziwa mwanjira inayake kuyika/malire a zolembedwa.

Ndikudziwa njira zitatu:

Zolemba zonse zimasungidwa mumtsinje wopitirira, choyamba pali mutu wa zolemba womwe uli ndi kutalika kwake, ndiyeno zolembazo.
M'mawonekedwe awa, mitu yonse ndi deta zitha kukhala zautali wosiyanasiyana.
Kwenikweni, timapeza mndandanda wolumikizidwa womwe umagwiritsidwa ntchito nthawi zonse;
Mitu ndi zolembazo zimasungidwa mumitsinje yosiyana.
Pogwiritsa ntchito mitu yautali wokhazikika, timaonetsetsa kuti kuwonongeka kwa mutu umodzi sikukhudza ena.
Njira yofananira imagwiritsidwa ntchito, mwachitsanzo, m'mafayilo ambiri;
Zolemba zimasungidwa mumtsinje wosalekeza, malire a zolemba amatsimikiziridwa ndi chikhomo china (khalidwe / mndandanda wa zilembo zomwe zimaletsedwa mkati mwa midadada). Ngati pali cholembera mkati mwa cholembera, ndiye kuti timachisintha ndikutsata (kuthawa).
Njira yofananira imagwiritsidwa ntchito, mwachitsanzo, mu protocol ya PPP.

Ndifotokoza.

Zosankha 1:

Chilichonse ndi chophweka kwambiri apa: podziwa kutalika kwa zolembazo, tikhoza kuwerengera adiresi ya mutu wotsatira. Kotero timadutsa pamitu mpaka titakumana ndi malo odzaza ndi 0xff (malo aulere) kapena kumapeto kwa tsamba.

Zosankha 2:

Chifukwa cha kutalika kwa mbiri yakale, sitingathe kunena pasadakhale kuti ndi zolemba zingati (ndiye mitu) zomwe tidzafunikira patsamba lililonse. Mukhoza kufalitsa mitu ndi deta pamasamba osiyanasiyana, koma ndimakonda njira yosiyana: timayika mitu ndi deta pa tsamba limodzi, koma mitu (ya kukula kosalekeza) imachokera kumayambiriro kwa tsamba, ndi deta (ya kutalika kosiyana) imachokera kumapeto. "Atangokumana" (palibe malo okwanira olowera mwatsopano), timaona kuti tsamba ili latha.

Zosankha 3:

Palibe chifukwa chosungira utali kapena chidziwitso china chokhudza malo omwe deta ili pamutu; zolembera zomwe zikuwonetsa malire a zolemba ndizokwanira. Komabe, deta iyenera kukonzedwa polemba / kuwerenga.
Ndikagwiritsa ntchito 0xff ngati chikhomo (chomwe chimadzaza tsambalo pambuyo pofufutidwa), kotero kuti malo aulere sadzatengedwa ngati deta.

Kuyerekeza tebulo:

Zosankha 1
Zosankha 2
Zosankha 3

Kulekerera zolakwika
-
+
+

Kugwirizana
+
-
+

Kukhazikitsa zovuta
*
**
**

Chosankha 1 chili ndi vuto lalikulu: ngati mutu uliwonse wawonongeka, unyolo wonse wotsatira umawonongeka. The otsala options amakulolani kuti achire ena deta ngakhale zitawonongeka kwambiri.
Koma apa ndi koyenera kukumbukira kuti tinaganiza zosunga deta mu mawonekedwe oponderezedwa, choncho timataya deta yonse pa tsamba pambuyo pa mbiri "yosweka", kotero ngakhale pali minus patebulo, sitichita. ganizirani.

Kukhazikika:

mu njira yoyamba, tifunika kusunga utali wokha pamutu; ngati tigwiritsa ntchito manambala osinthika, ndiye kuti nthawi zambiri timatha kudutsa ndi baiti imodzi;
mu njira yachiwiri tiyenera kusunga adilesi yoyambira ndi kutalika; zolembazo ziyenera kukhala kukula kosalekeza, ndikuyerekeza ma byte 4 pa rekodi (ma byte awiri a kuchotsera, ndi ma byte awiri m'litali);
njira yachitatu imangofunika munthu mmodzi kuti asonyeze chiyambi cha kujambula, kuphatikizapo kujambula komweko kudzawonjezeka ndi 1-2% chifukwa cha chitetezo. Kawirikawiri, pafupifupi kufanana ndi njira yoyamba.

Poyambirira, ndidawona njira yachiwiri ngati yayikulu (ndipo ngakhale ndikulemba kukhazikitsa). Ndinazisiya pamene ndinaganiza zogwiritsa ntchito compression.

Mwina tsiku lina ndidzagwiritsabe ntchito njira yomweyi. Mwachitsanzo, ngati ndiyenera kuthana ndi kusungirako deta kwa sitima yomwe ikuyenda pakati pa Dziko Lapansi ndi Mars, padzakhala zofunikira zosiyana kwambiri za kudalirika, kuwala kwa cosmic, ...

Ponena za njira yachitatu: Ndidapereka nyenyezi ziwiri chifukwa chazovuta kukhazikitsa chifukwa sindimakonda kusokoneza ndi kutchingira, kusintha kutalika kwake, ndi zina zambiri. Inde, mwina ndikukondera, koma ndiyenera kulemba kachidindo - bwanji kudzikakamiza kuchita zomwe simukonda.

Chidule: Timasankha njira yosungiramo ngati maunyolo "mutu wokhala ndi kutalika - data yautali wosiyanasiyana" chifukwa chakuchita bwino komanso kugwiritsa ntchito mosavuta.

Kugwiritsa Ntchito Bit Fields Kuwunika Kupambana kwa Kulemba Ntchito

Sindikukumbukira tsopano komwe ndinapeza lingaliro, koma zikuwoneka motere:
Pazolowera zilizonse, timagawa ma bits angapo kuti tisunge mbendera.
Monga tanena kale, titachotsa ma bits onse amadzazidwa ndi 1s, ndipo titha kusintha 1 mpaka 0, koma osati mosemphanitsa. Chifukwa chake pa "mbendera sinakhazikitsidwe" timagwiritsa ntchito 1, chifukwa "mbendera yakhazikitsidwa" timagwiritsa ntchito 0.

Izi ndi zomwe kuyika mbiri yosinthika mu flash kungawonekere:

Khazikitsani mbendera "kujambula kwautali kwayamba";
Lembani kutalika;
Khazikitsani mbendera ya "kujambula kwa data";
Timalemba deta;
Khazikitsani mbendera ya "kujambula kwatha".

Kuphatikiza apo, tidzakhala ndi mbendera "yolakwika" pagulu lonse la mbendera 4.

Pankhaniyi, tili ndi zigawo ziwiri zokhazikika "1111" - kujambula sikunayambe ndipo "1000" - kujambula kunapambana; pakakhala kusokoneza kosayembekezereka kwa kujambula, tidzalandira mayiko apakati, omwe tingathe kuwazindikira ndikukonza.

Njirayi ndi yosangalatsa, koma imangoteteza kuphulika kwadzidzidzi kwa magetsi ndi zolephera zofanana, zomwe, ndithudi, ndizofunikira, koma izi siziri kutali (kapena ngakhale chachikulu) chifukwa cholephera zotheka.

Chidule: Tiyeni tipitirire kufunafuna yankho labwino.

Macheke

Macheke amathandizanso kutsimikizira (ndi kuthekera koyenera) kuti tikuwerenga ndendende zomwe ziyenera kulembedwa. Ndipo, mosiyana ndi magawo omwe takambirana pamwambapa, amagwira ntchito nthawi zonse.

Ngati tilingalira mndandanda wazomwe zingayambitse mavuto omwe takambirana pamwambapa, ndiye kuti checksum imatha kuzindikira cholakwika mosasamala kanthu za chiyambi chake. (kupatula, mwina, kwa alendo oyipa - atha kupanga cheke nawonso).

Chifukwa chake ngati cholinga chathu ndikuwonetsetsa kuti zomwe zidalipo sizili bwino, ma checksums ndi lingaliro labwino.

Kusankhidwa kwa algorithm pakuwerengera cheke sikunabweretse mafunso - CRC. Kumbali imodzi, masamu amatha kugwira zolakwika za 100%; Komano, pa data mwachisawawa, algorithm iyi nthawi zambiri imawonetsa kuthekera kwa kugundana osati kwakukulu kuposa malire amalingaliro. . Ikhoza kusakhala yofulumira kwambiri, komanso nthawi zonse imakhala yochepa kwambiri pa chiwerengero cha kugunda, koma ili ndi khalidwe lofunika kwambiri: mu mayesero omwe ndinakumana nawo, panalibe machitidwe omwe adalephera momveka bwino. Kukhazikika ndi khalidwe lalikulu pankhaniyi.

Chitsanzo cha kafukufuku wa volumetric: gawo 1, gawo 2 (zolumikizana ndi narod.ru, pepani).

Komabe, ntchito yosankha cheke sinamalize; CRC ndi banja lonse la macheke. Muyenera kusankha kutalika, ndiyeno sankhani polynomial.

Kusankha kutalika kwa checksum si funso lophweka monga momwe zimawonekera poyamba.

Ndiloleni ndifotokozere:
Tiyeni tikhale ndi kuthekera kwa cholakwika mu baiti iliyonse ndi checksum yabwino, tiyeni tiwerengere kuchuluka kwa zolakwika pa ma miliyoni miliyoni:

Data, byte
Checksum, bati
Zolakwika zosazindikirika
Kuzindikira zolakwika zabodza
Zonse zabodza zabwino

1
0
1000
0
1000

1
1
4
999
1003

1
2
.0
1997
1997

1
4
.0
3990
3990

10
0
9955
0
9955

10
1
39
990
1029

10
2
.0
1979
1979

10
4
.0
3954
3954

1000
0
632305
0
632305

1000
1
2470
368
2838

1000
2
10
735
745

1000
4
.0
1469
1469

Zingawoneke kuti chirichonse chiri chophweka - malingana ndi kutalika kwa deta yomwe ikutetezedwa, sankhani kutalika kwa checksum ndi zochepa zolakwika zolakwika - ndipo chinyengo chiri mu thumba.

Komabe, vuto limabwera ndi ma checksums achidule: ngakhale ali bwino pakuzindikira zolakwika pang'ono, amatha kuvomereza zomwe zachitika mwachisawawa kuti ndizolondola. Panali kale nkhani ya Habré yofotokoza vuto m'moyo weniweni.

Chifukwa chake, kuti mupange macheki osasinthika kukhala osatheka, muyenera kugwiritsa ntchito macheke omwe ali 32 bits kapena kutalika kwake. (kwautali wopitilira 64 bits, ntchito za cryptographic hashi zimagwiritsidwa ntchito).

Ngakhale kuti ndidalemba kale kuti tiyenera kusunga malo mwa njira zonse, tidzagwiritsabe ntchito 32-bit checksum (16 bits sikokwanira, mwayi wa kugunda ndi woposa 0.01%; ndi 24 bits, monga momwe amachitira. kunena, palibe pano kapena apo) .

Chotsutsa chingakhalepo apa: kodi tidasunga ma byte aliwonse posankha kukakamiza kuti tsopano tipereke ma byte 4 nthawi imodzi? Kodi sizingakhale bwino kusakanikiza kapena kuwonjezera cheke? Inde ayi, palibe psinjika sizikutanthauza, kuti sitifunikira kuwunika umphumphu.

Posankha polynomial, sitidzayambiranso gudumu, koma titenge CRC-32C yotchuka.
Khodi iyi imazindikira zolakwika 6 pamapaketi mpaka ma byte 22 (mwinamwake wofala kwambiri kwa ife), zolakwika 4 pang'ono pamapaketi mpaka 655 byte (komanso vuto wamba kwa ife), 2 kapena nambala yosamvetseka ya zolakwika pang'ono pamapaketi. kutalika kulikonse koyenera.

Ngati wina ali ndi chidwi ndi zambiri

Nkhani ya Wikipedia za CRC.

Zizindikiro za code crc-32c pa Webusayiti ya Koopman - mwina katswiri wamkulu wa CRC padziko lapansi.

В nkhani yake pali code ina yosangalatsa, yomwe imapereka magawo abwinoko pang'ono pamapaketi omwe ali ofunikira kwa ife, koma sindinawone kusiyana kwake kukhala kofunikira, ndipo ndinali wokhoza kusankha kachidindo kachitidwe m'malo mwa muyezo komanso wofufuzidwa bwino.

Komanso, popeza deta yathu yapanikizidwa, funso limadzuka: kodi tiyenera kuwerengera cheke cha data yoponderezedwa kapena yosakanizidwa?

Zotsutsana zokomera kuwerengera cheke cha data yosakanizidwa:

Pamapeto pake tifunika kuyang'ana chitetezo cha kusungirako deta - kotero timayang'ana mwachindunji (panthawi yomweyo, zolakwika zomwe zingatheke pakukhazikitsa kupanikizika / kusokoneza, kuwonongeka kwa kukumbukira kukumbukira, etc. kudzafufuzidwa);
The deflate algorithm mu zlib ili ndi kukhazikitsa kokhwima bwino komanso sayenera kugwa ndi "zokhotakhota" zolowera; Komanso, nthawi zambiri imatha kuzindikira zolakwika panjira yolowera, kuchepetsa mwayi wonse wosazindikira cholakwika (kuyesa ndikutembenuza pang'ono pang'ono, zlib idazindikira cholakwika. pafupifupi gawo limodzi mwa magawo atatu a milandu).

Zotsutsa zotsutsa kuwerengera cheke cha data yosakanizidwa:

CRC ndi "yogwirizana" makamaka chifukwa cha zolakwika zochepa zomwe zimakhala ndi kukumbukira kwa flash (kulakwitsa pang'ono mumtsinje woponderezedwa kungayambitse kusintha kwakukulu mumtsinje wotuluka, womwe, mwachidziwitso, "tingagwire" kugunda);
Sindimakonda lingaliro lopereka deta yomwe ingathe kusweka kwa decompressor, Angadziwe ndanimmene adzachita.

Muntchitoyi, ndinaganiza zopatuka ku mchitidwe womwe anthu ambiri amauvomereza wosunga cheke cha data yosakanizidwa.

Chidule: Timagwiritsa ntchito CRC-32C, timawerengera cheke kuchokera ku data mu mawonekedwe omwe amalembedwa kuti aziwunikira (pambuyo pa kupsinjika).

Kuperewera

Kugwiritsa ntchito zolembera zosafunikira sikumachotsa kutayika kwa data, komabe, kumatha (nthawi zambiri ndi maulamuliro ambiri) kumachepetsa mwayi wotayika kosabweza.

Titha kugwiritsa ntchito mitundu yosiyanasiyana ya redundancy kukonza zolakwika.
Manambala a Hamming amatha kukonza zolakwika pang'ono, ma code a Reed-Solomon, makopi angapo a data kuphatikiza macheke, kapena ma encodings ngati RAID-6 angathandize kubwezeretsanso deta ngakhale patakhala chiphuphu chachikulu.
Poyamba, ndinali wodzipereka ku kugwiritsidwa ntchito kofala koletsa zolakwika, koma kenako ndinazindikira kuti choyamba tiyenera kukhala ndi lingaliro la zolakwika zomwe tikufuna kudziteteza ku zolakwika, ndiyeno kusankha kukopera.

Tidanena kale kuti zolakwika ziyenera kugwidwa mwachangu momwe zingathere. Ndi pati pomwe tingakumane ndi zolakwika?

Kujambula kosamalizidwa (pazifukwa zina panthawi yojambulira mphamvu idazimitsidwa, Raspberry adawuma, ...)
Tsoka, pakachitika cholakwika chotere, chotsalira ndikunyalanyaza zolemba zosavomerezeka ndikuganizira zomwe zatayika;
Lembani zolakwika (pazifukwa zina, zomwe zidalembedwa ku flash memory sizinalembedwe)
Titha kuzindikira nthawi yomweyo zolakwika ngati titayesa kuwerenga tingomaliza kujambula;
Kusokonezeka kwa deta mu kukumbukira panthawi yosungira;
Zolakwika powerenga
Kuti mukonze, ngati cheke sichikugwirizana, ndikwanira kubwereza kuwerenga kangapo.

Ndiko kuti, zolakwika zokha za mtundu wachitatu (zivundi zodzidzimutsa za data panthawi yosungira) sizingawongoleredwe popanda kulembera zolakwika. Zikuoneka kuti zolakwika zoterozo zikadali zokayikitsa kwambiri.

Chidule: adaganiza zosiya zolemba zosafunikira, koma ngati opareshoni ikuwonetsa cholakwika cha chisankhochi, bwererani ku nkhaniyo (ndi ziwerengero zomwe zasonkhanitsidwa kale pazolephera, zomwe zingathandize kusankha mtundu woyenera wa zolemba).

Прочее

Zoonadi, mawonekedwe a nkhaniyo salola kuti tizilungamitsa chilichonse mumpangidwe (ndipo mphamvu zanga zatha), kotero ndifotokoza mwachidule mfundo zina zomwe sizinakhudzidwepo kale.

Anaganiza kuti masamba onse akhale "ofanana"
Ndiko kuti, sipadzakhala masamba apadera okhala ndi metadata, ulusi wosiyana, ndi zina zotero, koma m'malo mwake ulusi umodzi womwe umalembanso masamba onse motsatizana.
Izi zimatsimikizira ngakhale kuvala pamasamba, palibe mfundo imodzi yolephera, ndipo ndimangokonda;
Ndikofunikira kupereka kumasulira kwa mawonekedwe.
Mawonekedwe opanda nambala yamtundu pamutu ndi oyipa!
Ndikokwanira kuwonjezera gawo lomwe lili ndi Nambala Yamatsenga (siginecha) pamutu watsamba, zomwe zikuwonetsa mtundu wa mawonekedwe omwe agwiritsidwa ntchito. (Sindikuganiza kuti mwakuchita padzakhala ngakhale khumi ndi awiri a iwo);
Gwiritsani ntchito mutu wautali wosiyanasiyana pazolemba (zomwe zilipo zambiri), kuyesera kuti zikhale zazitali za 1 nthawi zambiri;
Kuti musindikize kutalika kwa mutu ndi kutalika kwa gawo lodulidwa la mbiri yopanikizidwa, gwiritsani ntchito zizindikiro za binary zautali wosiyana.

Anathandiza kwambiri jenereta ya pa intaneti Huffman kodi. M'mphindi zochepa chabe tinatha kusankha zizindikiro zautali wofunikira.

Kufotokozera za mawonekedwe osungira deta

Byte dongosolo

Minda yokulirapo kuposa imodzi imasungidwa mumtundu wa big-endian (network byte order), ndiye kuti, 0x1234 imalembedwa ngati 0x12, 0x34.

Pagination

Memory flash yonse imagawidwa m'masamba ofanana kukula.

Kukula kwa tsamba losasintha ndi 32Kb, koma osapitilira 1/4 ya kukula konse kwa memory chip (kwa chip 4MB, masamba 128 amapezedwa).

Tsamba lililonse limasunga zomwe zili patsamba lina (ndiko kuti, tsamba limodzi silimatchulanso za patsamba lina).

Masamba onse amawerengedwa mwadongosolo lachilengedwe (mongokwera ma adilesi), kuyambira ndi nambala 0 (tsamba ziro limayambira pa adilesi 0, tsamba loyamba limayamba pa 32Kb, tsamba lachiwiri limayamba pa 64Kb, ndi zina zambiri.)

Chipangizo chokumbukira chimagwiritsidwa ntchito ngati cyclic buffer (ring buffer), ndiye kuti, kulemba koyamba kumapita patsamba nambala 0, kenako nambala 1, ..., tikadzaza tsamba lomaliza, kuzungulira kwatsopano kumayamba ndikujambula kumapitilira patsamba zero. .

Mkati mwa tsamba

Kumayambiriro kwa tsamba, mutu wa tsamba la 4-byte umasungidwa, ndiye checksum yamutu (CRC-32C), ndiye zolemba zimasungidwa mumtundu wa "mutu, deta, checksum".

Mutu watsamba (wobiriwira wodetsedwa pachithunzichi) uli ndi:

magawo awiri a Magic Number (komanso chizindikiro cha mtunduwo)
kwa mtundu wamakono wamtundu womwe umawerengedwa ngati 0xed00 ⊕ номер страницы;
XNUMX-byte counter "Page Version" (nambala yozungulira yokumbukira kukumbukira).

Zolemba patsamba zimasungidwa mu mawonekedwe oponderezedwa (deflate algorithm imagwiritsidwa ntchito). Zolemba zonse patsamba limodzi zimapanikizidwa mu ulusi umodzi (dikishonale wamba imagwiritsidwa ntchito), ndipo kukakamiza kwatsamba lililonse kumayambanso. Ndiko kuti, kuti muchepetse mbiri iliyonse, zolemba zonse zam'mbuyomu kuchokera patsamba lino (ndi iyi yokha) ndiyofunikira.

Mbiri iliyonse idzapanikizidwa ndi mbendera ya Z_SYNC_FLUSH, ndipo kumapeto kwa mtsinje woponderezedwa padzakhala 4 byte 0x00, 0x00, 0xff, 0xff, mwina kutsogozedwa ndi ziro byte imodzi kapena ziwiri.
Timataya ndondomekoyi (4, 5 kapena 6 byte kutalika) tikamalemba ku flash memory.

Cholembera chamutu ndi 1, 2 kapena 3 mabayiti osungidwa:

pang'ono (T) kusonyeza mtundu wa zolemba: 0 - nkhani, 1 - chipika;
gawo lautali wosiyanasiyana (S) kuchokera ku 1 mpaka 7 bits, kufotokozera kutalika kwa mutu ndi "mchira" womwe uyenera kuwonjezeredwa ku mbiri ya decompression;
kutalika kwa mbiri (L).

S mtengo tebulo:

S
Kutalika kwamutu, mabayiti
Kutayidwa polemba, byte

0
1
5 (00 00 00 ff ff)

10
1
6 (00 00 00 00 ff ff)

110
2
4 (00 00 ff ff)

1110
2
5 (00 00 00 ff ff)

11110
2
6 (00 00 00 00 ff ff)

1111100
3
4 (00 00 ff ff)

1111101
3
5 (00 00 00 ff ff)

1111110
3
6 (00 00 00 00 ff ff)

Ndidayesa kufotokoza, sindikudziwa kuti zidakhala bwanji:

Yellow apa ikuwonetsa gawo la T, loyera la S, lobiriwira L (kutalika kwa deta yoponderezedwa mu byte), buluu deta yoponderezedwa, yofiira ma byte omaliza a deta yoponderezedwa yomwe sinalembedwe kuti ikumbukire.

Chifukwa chake, titha kulemba zolemba zam'mutu zautali wodziwika bwino (mpaka 63 + 5 byte mu mawonekedwe oponderezedwa) mu baiti imodzi.

Pambuyo pa mbiri iliyonse, cheke ya CRC-32C imasungidwa, momwe mtengo wosinthika wa cheke yam'mbuyomu umagwiritsidwa ntchito ngati mtengo woyamba (init).

CRC ili ndi "nthawi", njira yotsatirayi imagwira ntchito (kuphatikiza kapena kuchotsera pang'ono posintha): .
Izi ndiye kuti, timawerengera CRC ya ma byte am'mbuyomu ammutu ndi data patsamba lino.

Kutsatira molunjika checksum ndi mutu wa mbiri yotsatira.

Mutuwo umapangidwa m'njira yoti byte yake yoyamba nthawi zonse imakhala yosiyana ndi 0x00 ndi 0xff (ngati m'malo mwa byte yoyamba ya mutu timakumana ndi 0xff, ndiye izi zikutanthauza kuti ili ndi malo osagwiritsidwa ntchito; 0x00 imawonetsa cholakwika).

Chitsanzo ma algorithms

Kuwerenga kuchokera ku Flash Memory

Kuwerenga kulikonse kumabwera ndi cheke chekeni.
Ngati checksum sichikufanana, kuwerenga kumabwerezedwa kangapo ndikuyembekeza kuwerenga deta yolondola.

(izi ndizomveka, Linux sichimawerengedwa kuchokera ku NOR Flash, yoyesedwa)

Lembani ku flash memory

Timalemba deta.
Tiyeni tiwawerenge.

Ngati zomwe zawerengedwa sizikugwirizana ndi zomwe zalembedwa, timadzaza malowa ndi ziro ndikuwonetsa cholakwika.

Kukonzekera microcircuit yatsopano kuti igwire ntchito

Poyambitsa, mutu wokhala ndi mtundu 1 umalembedwa patsamba loyamba (kapena zero).
Pambuyo pake, mawu oyamba amalembedwa patsamba lino (ali ndi UUID ya makina ndi zosintha zosasintha).

Ndi zimenezo, kukumbukira kung'anima ndi wokonzeka kugwiritsidwa ntchito.

Kutsegula makina

Mukatsitsa, ma byte 8 oyamba a tsamba lililonse (mutu + CRC) amawerengedwa, masamba omwe ali ndi Nambala yamatsenga yosadziwika kapena CRC yolakwika amanyalanyazidwa.
Kuchokera pamasamba "olondola", masamba omwe ali ndi mawonekedwe apamwamba amasankhidwa, ndipo tsamba lomwe lili ndi chiwerengero chapamwamba limachotsedwa kwa iwo.
Zolemba zoyamba zimawerengedwa, kulondola kwa CRC ndi kukhalapo kwa mbendera ya "context" kumafufuzidwa. Ngati zonse zili bwino, tsamba ili limatengedwa kuti ndi lamakono. Ngati sichoncho, timabwereranso ku yapitayi mpaka titapeza tsamba la "live".
ndipo patsamba lomwe tapeza timawerenga zolemba zonse, zomwe timagwiritsa ntchito ndi mbendera ya "context".
Sungani dikishonale ya zlib (idzafunika kuti muwonjezere patsamba lino).

Ndizo zonse, kutsitsa kwatha, nkhaniyo imabwezeretsedwa, mutha kugwira ntchito.

Kuwonjezera Journal Entry

Timapanikiza zolembedwazo ndi mtanthauzira mawu wolondola, kutchula Z_SYNC_FLUSH. Tikuwona ngati zolembedwazo zikukwanira patsamba lapano.
Ngati sichikukwanira (kapena panali zolakwika za CRC patsamba), yambani tsamba latsopano (onani pansipa).
Timalemba zolemba ndi CRC. Ngati cholakwika chichitika, yambani tsamba latsopano.

Tsamba latsopano

Timasankha tsamba laulere lokhala ndi nambala yocheperako (tikuwona tsamba laulere kukhala tsamba lomwe lili ndi cheke cholakwika pamutu kapena ndi mtundu wocheperapo womwe ulipo). Ngati palibe masamba oterowo, sankhani tsamba lomwe lili ndi nambala yochepa kuchokera kwa omwe ali ndi mtundu wofanana ndi wapano.
Timachotsa tsamba losankhidwa. Timayang'ana zomwe zili mkati ndi 0xff. Ngati chinachake sichili bwino, tengani tsamba lotsatira laulere, ndi zina zotero.
Timalemba mutu pa tsamba lofufutidwa, choyamba cholowera ndi momwe zilili panopa, chotsatira ndicho cholembera chosalembedwa (ngati chilipo).

Kugwiritsa ntchito mawonekedwe

M'malingaliro mwanga, idakhala mawonekedwe abwino osungiramo zidziwitso zochulukirapo kapena zochepa (zolemba zomveka, JSON, MessagePack, CBOR, mwina protobuf) mu NOR Flash.

Zachidziwikire, mawonekedwewo ndi "ogwirizana" ndi SLC NOR Flash.

Siyenera kugwiritsidwa ntchito ndi ma BER apamwamba kwambiri monga NAND kapena MLC NOR (kodi kukumbukira koteroko kulipo kugulitsidwa? Ndaziwonapo zikutchulidwa m'ntchito zamakhodi owongolera).

Komanso, siziyenera kugwiritsidwa ntchito ndi zida zomwe zili ndi FTL yawo: USB flash, SD, MicroSD, etc (pazokumbukira zotere ndidapanga mawonekedwe okhala ndi tsamba la ma byte 512, siginecha koyambirira kwa tsamba lililonse ndi manambala apadera - nthawi zina zinali zotheka kubwezeretsanso zonse kuchokera pa "glitched" flash drive powerenga motsatizana).

Kutengera ndi ntchito, mawonekedwewo atha kugwiritsidwa ntchito popanda kusintha pama drive a Flash kuchokera ku 128Kbit (16Kb) kupita ku 1Gbit (128MB). Ngati mungafune, mutha kugwiritsa ntchito tchipisi tokulirapo, koma muyenera kusintha kukula kwa tsamba (Koma apa funso la kuthekera kwachuma lidayamba kale; mtengo wa voliyumu yayikulu NOR Flash sikulimbikitsa).

Ngati wina apeza mawonekedwe osangalatsa ndipo akufuna kuti agwiritse ntchito pulojekiti yotseguka, lembani, ndiyesera kupeza nthawi, kupukuta kachidindo ndikuyiyika pa github.

Pomaliza

Monga mukuwonera, pamapeto pake mawonekedwewo adakhala osavuta komanso ngakhale wotopetsa.

Ndizovuta kuwonetsa kusinthika kwa malingaliro anga m'nkhani, koma ndikhulupirireni: poyamba ndinkafuna kupanga chinthu chamakono, chosawonongeka, chokhoza kupulumuka ngakhale kuphulika kwa nyukiliya pafupi. Komabe, chifukwa (ndikukhulupirira) adapambanabe ndipo pang'onopang'ono zoyambira zidasinthira ku kuphweka komanso kuphatikizika.

Zingakhale kuti ndinalakwitsa? Inde, zedi. Zitha kukhala, mwachitsanzo, kuti tidagula gulu la ma microcircuit otsika kwambiri. Kapena pazifukwa zina zida sizingakwaniritse zoyembekeza zodalirika.

Kodi ndili ndi dongosolo la izi? Ndikuganiza kuti mutawerenga nkhaniyi simukukayikira kuti pali ndondomeko. Ndipo osati yekha.

Pazovuta kwambiri, mawonekedwewo adapangidwa ngati njira yogwirira ntchito komanso ngati "baluni yoyeserera".

Pakadali pano zonse zomwe zili patebulo zikuyenda bwino, kwenikweni tsiku lina yankho lidzatumizidwa (pafupifupi) pazida zana, tiyeni tiwone zomwe zimachitika pakuchita "kumenyana" (mwamwayi, ndikuyembekeza kuti mawonekedwewa amakupatsani mwayi wozindikira zolephera; kotero mutha kusonkhanitsa ziwerengero zonse). M'miyezi ingapo zidzakhala zotheka kupeza mfundo (ndipo ngati mulibe mwayi, ngakhale kale).

Ngati, kutengera zotsatira za kugwiritsidwa ntchito, mavuto akulu apezeka ndipo kuwongolera kumafunika, ndiye kuti ndikulemba za izi.

Mabuku

Sindinafune kupanga mndandanda wautali wotopetsa wa ntchito zomwe zagwiritsidwa ntchito; pambuyo pake, aliyense ali ndi Google.

Apa ndinaganiza zosiya mndandanda wa zomwe ndapeza zomwe zinkawoneka zosangalatsa kwambiri kwa ine, koma pang'onopang'ono zinasamukira kumutu wa nkhaniyi, ndipo chinthu chimodzi chinatsalira pamndandanda:

Zothandiza infgen kuchokera kwa wolemba zlib. Itha kuwonetsa bwino zomwe zili mu deflate/zlib/gzip archives. Ngati mukuyenera kuthana ndi mawonekedwe amkati amtundu wa deflate (kapena gzip), ndimalimbikitsa kwambiri.

Source: www.habr.com

Kukhazikitsa kwanga kotchingira mphete mu NOR flash