Ngelixa ndiphanda ukuzinza kokugcinwa kwedatha kwiinkqubo zamafu, ndagqiba ekubeni ndizivavanye ukuze ndiqinisekise ukuba ndiyaziqonda izinto ezisisiseko. I ukuze uqonde ukuba zeziphi iziqinisekiso malunga nokugcinwa kwedatha ezinzileyo (oko kukuthi, iqinisekisa ukuba idatha iya kufumaneka emva kokusilela kwenkqubo) kusinika iidiski ze-NMVe. Ndenze ezi zigqibo eziphambili zilandelayo: idatha kufuneka ithathelwe ingqalelo yonakele ukususela kumzuzu umyalelo wokubhala idatha unikezelwa kude kube ngumzuzu obhaliweyo kwindawo yokugcina. Nangona kunjalo, uninzi lweenkqubo ngolonwabo zisebenzisa iifowuni zenkqubo ukurekhoda idatha.
Kule post, ndiphonononga iindlela zokugcina eziqhubekayo ezinikezelwa yi-API yefayile ye-Linux. Kubonakala ngathi yonke into kufuneka ibe lula apha: inkqubo ibiza umyalelo write(), kwaye emva kokuba lo myalelo ugqityiwe, idatha iya kugcinwa ngokukhuselekileyo kwidiski. Kodwa write() ikhuphela kuphela idatha yesicelo kwi-kernel cache ebekwe kwi-RAM. Ukuze unyanzelise inkqubo ukuba ibhale idatha kwidiski, kufuneka usebenzise ezinye iindlela ezongezelelweyo.
Ngokubanzi, le nkcazelo yingqokelela yamanqaku anxulumene noko ndikufundileyo ngesihloko esinomdla kum. Ukuba sithetha ngokufutshane kakhulu malunga neyona nto ibalulekileyo, kuvela ukuba ukulungelelanisa ukugcinwa kwedatha ezinzileyo kufuneka usebenzise umyalelo fdatasync() okanye uvule iifayile ngeflegi O_DSYNC. Ukuba unomdla wokufunda ngakumbi malunga nokuba kwenzeka ntoni kwidatha kwindlela yayo ukusuka kwikhowudi ukuya kwidiski, jonga inqaku.
Iimpawu zokusebenzisa bhala () umsebenzi
Umnxeba wenkqubo write() ichazwe kumgangatho njengenzame yokubhala idatha kwisichazi sefayile. Emva kokugqiba ngempumelelo write() Imisebenzi yokufunda idatha kufuneka ibuyisele kanye iibytes ebezibhalwe ngaphambili, ukwenza oku nokuba idatha ifunyenwe kwezinye iinkqubo okanye imisonto ( icandelo elifanelekileyo lomgangatho wePOSIX). , kwicandelo malunga nendlela imisonto esebenzisana ngayo nemisebenzi yefayile eqhelekileyo, kukho inqaku elithi ukuba imisonto emibini nganye ibiza le misebenzi, ngoko umnxeba ngamnye kufuneka ubone nokuba zonke iziphumo ezichongiweyo zomnye umnxeba, okanye akukho nanye. iziphumo. Oku kukhokelela kwisigqibo sokuba yonke imisebenzi yefayile ye-I/O kufuneka ibambe isitshixo kwisixhobo abasebenza kuso.
Ngaba oku kuthetha ukuba utyando write() ngaba yiatom? Ukusuka kumbono wobugcisa, ewe. Ukufundwa kwedatha kufuneka kubuyise konke okanye akukho nto ibibhalwe ngayo write(). Kodwa utyando write(), ngokomgangatho, akuyomfuneko ukuba iphele ngokubhala yonke into ebicelwe ukuba ibhalwe. Uvumelekile ukuba abhale kuphela inxalenye yedatha. Umzekelo, sinokuba nemisonto emibini edibanisa i-1024 bytes nganye kwifayile echazwe yinkcazelo yefayile efanayo. Ukusuka kwimbono yomgangatho, umphumo owamkelekileyo uya kuba xa umsebenzi ngamnye wokubhala unokongeza i-byte enye kuphela kwifayile. Le misebenzi iya kuhlala i-athomu, kodwa emva kokuba igqityiwe, idatha abayibhale kwifayile iya kuxutywa. Ingxoxo enika umdla kakhulu kwesi sihloko sokuPhuphuma kweSitaki.
fsync () kunye fdatasync () imisebenzi
Eyona ndlela ilula yokugungxula idatha kwidiski kukufowunela umsebenzi . Lo msebenzi ucela inkqubo yokusebenza ukuba idlulise zonke iibhloko ezilungisiweyo ukusuka kwi-cache ukuya kwidiski. Oku kubandakanya yonke imethadatha yefayile (ixesha lokufikelela, ixesha lokuguqulwa kwefayile, njalo njalo). Ndiyakholelwa ukuba le metadata ayifane idingeke, ke ukuba uyazi ukuba ayibalulekanga kuwe, ungasebenzisa umsebenzi. fdatasync(). Ku kwi fdatasync() Kuthiwa ngexesha lokusebenza kwalo msebenzi, isixa esinjalo semetadata sigcinwa kwidiski "eyimfuneko ukuze kuqhutywe ngokuchanekileyo imisebenzi elandelayo yokufunda idatha." Kwaye le yeyona nto kanye uninzi lwezicelo ezikhathalelayo.
Enye ingxaki enokuthi ivele apha kukuba ezi ndlela aziqinisekisi ukuba ifayile iya kufumaneka emva kokusilela okunokwenzeka. Ngokukodwa, xa udala ifayile entsha, kufuneka ufowunele fsync() kuluhlu oluqulathe yona. Ngaphandle koko, emva kokusilela, kunokuvela ukuba le fayile ayikho. Isizathu soku kukuba kwi UNIX, ngenxa yosebenziso lwamakhonkco anzima, ifayile ingabakho kubalawuli abaninzi. Ngoko ke, xa ufowuna fsync() akukho ndlela yokuba ifayile yazi ukuba loluphi ulawulo lwedatha ekufuneka lugungxulwe kwidisk ( Unokufunda ngakumbi malunga noku). Kubonakala ngathi inkqubo yefayile ye-ext4 iyakwazi ukwenza faka isicelo fsync() kubalawuli abaqulathe iifayile ezihambelanayo, kodwa oku kusenokungabi njalo kwezinye iindlela zefayile.
Le ndlela inokusetyenziswa ngokwahlukileyo kwiindlela ezahlukeneyo zefayile. ndidla ngoku Ukufunda malunga nokuba yeyiphi imisebenzi yediski esetyenziswayo kwiinkqubo zefayile ze ext4 kunye ne XFS. Zombini zikhupha imiyalelo yokubhala rhoqo kwidisk kuzo zombini iziqulatho zefayile kunye nejenali yenkqubo yefayile, gungxula i-cache, kwaye uphume ngokwenza iFUA (Force Unit Access, ukubhala idatha ngokuthe ngqo kwidisk, ngokudlula i-cache) bhala kwijenali. Mhlawumbi benza oku ukuze baqinisekise ukuba intengiselwano yenzekile. Kwiidrive ezingaxhasi i-FUA, oku kubangela ukugungxulwa kwecache kabini. Imifuniselo yam yabonisa ukuba fdatasync() ngokukhawuleza kancinci fsync(). Uncedo blktrace ibonisa ukuba fdatasync() ihlala ibhala idatha encinci kwidiski (kwi-ext4 fsync() ubhala 20 KB, kwaye fdatasync() - 16 KB). Kwakhona, ndiye ndafumanisa ukuba i-XFS ikhawuleza kancinci kune-ext4. Kwaye apha ngoncedo blktrace ikwazile ukufumanisa ukuba fdatasync() igungxula idatha encinci kwidisk (4 KB kwi XFS).
Iimeko ezingaqondakaliyo ezivelayo xa usebenzisa i-fsync()
Ndiyakwazi ukucinga ngeemeko ezintathu ezingaqondakaliyo malunga fsync()endidibene nayo xa ndisenza.
Ityala lokuqala elinjalo lenzeka ngo-2008. Emva koko ujongano lweFirefox 3 luye lwaba ngumkhenkce ukuba inani elikhulu leefayile zibhalelwe kwidisk. Ingxaki yayikukuba ukuphunyezwa kwe-interface kusetyenziswa i-database ye-SQLite ukugcina ulwazi malunga nesimo sayo. Emva kwenguqu nganye eyenzekayo kwi-interface, umsebenzi wabizwa fsync(), eyanika iziqinisekiso ezilungileyo zokugcina idatha ezinzileyo. Kwinkqubo yefayile ye-ext3 esetyenzisiweyo, umsebenzi fsync() ulahle onke amaphepha "amdaka" kwisixokelelwano kwidisk, kwaye hayi nje lawo ayenxulumene nefayile ehambelanayo. Oku kuthetha ukuba ukucofa iqhosha kwiFirefox kunokubangela iimegabytes zedatha ukuba ibhalwe kwidisk magnetic, enokuthatha imizuzwana emininzi. Isisombululo kwingxaki, ngokokude ndiqonde imathiriyeli yayikukudlulisela umsebenzi kunye nesiseko sedatha kwimisebenzi yangasemva engahambelaniyo. Oku kuthetha ukuba ngaphambili iFirefox iphumeze iimfuno ezingqongqo zogcino kunokuba ibifuneka ngokwenene, kwaye iimpawu zenkqubo yefayile ye-ext3 ziye zayenza mandundu le ngxaki.
Ingxaki yesibini yenzeka ngo-2009. Emva koko, emva kokuwa kwenkqubo, abasebenzisi benkqubo entsha yefayile ye-ext4 baye bajongana nenyaniso yokuba iifayile ezininzi ezisandul 'ukwenziwa zazinobude be-zero, kodwa oku akuzange kwenzeke ngenkqubo yefayile ye-ext3 endala. Kumhlathi odlulileyo, ndithethe malunga nendlela i-ext3 egungxula ngayo idatha eninzi kwidiski, eyanciphisa izinto kakhulu. fsync(). Ukuphucula imeko, kwi-ext4 kuphela loo maphepha amdaka ahambelana nefayile ethile agungxulwa kwidiski. Kwaye idatha evela kwezinye iifayile ihlala kwinkumbulo ixesha elide kakhulu kune ext3. Oku kwenziwa ukuphucula ukusebenza (ngokungagqibekanga, idatha ihlala kwesi simo kangangemizuzwana engama-30, ungaqwalasela oku usebenzisa ; Ungafumana imathiriyeli eyongezelelweyo malunga noku). Oku kuthetha ukuba isixa esikhulu sedatha singalahleka ngokungenakubuyiseka emva kokusilela. Isisombululo kule ngxaki kukusebenzisa fsync() kwizicelo ezifuna ukuqinisekisa ukugcinwa kwedatha okuzinzileyo kunye nokukhusela kangangoko kunokwenzeka kwimiphumo yokungaphumeleli. Umsebenzi fsync() isebenza kakuhle kakhulu xa usebenzisa i-ext4 kunaxa usebenzisa i-ext3. Ukungalungi kwale ndlela kukuba ukusetyenziswa kwayo, njengangaphambili, kunciphisa ukuphunyezwa kweminye imisebenzi, njengokufakela iinkqubo. Bona iinkcukacha malunga noku ΠΈ .
Ingxaki yesithathu malunga fsync(), yaqala ngo-2018. Emva koko, ngaphakathi kwesakhelo seprojekthi yePostgreSQL, kwafunyaniswa ukuba ukuba umsebenzi fsync() idibana nempazamo, iphawula amaphepha "amdaka" njenge "clean". Ngenxa yoko, le minxeba ilandelayo fsync() Ayenzi nto ngamaphepha anjalo. Ngenxa yoku, amaphepha alungisiweyo agcinwa kwinkumbulo kwaye akaze abhalwe kwidiski. Le yintlekele yokwenene, ekubeni isicelo siya kucinga ukuba idatha ethile ibhaliwe kwidiski, kodwa ngokwenene ayiyi kuba. Ukusilela okunjalo fsync() zinqabile, isicelo kwiimeko ezinjalo asinakwenza phantse nto ukulwa ingxaki. Kwezi ntsuku, xa oku kusenzeka, iPostgreSQL kunye nezinye izicelo ziyawa. , kwizinto eziphathekayo "Ngaba ii-Applications ziyakwazi ukuBuyisa kwiiFayile ze-fsync?", Le ngxaki iphononongwa ngokubanzi. Okwangoku esona sisombululo kule ngxaki kukusebenzisa i-I/O eNgqo neflegi O_SYNC okanye ngendwe O_DSYNC. Ngale ndlela, inkqubo iya kuxela iimpazamo ezinokuthi zenzeke ngexesha lomsebenzi othile wokubhala, kodwa le ndlela ifuna ukuba isicelo silawule izithinteli ngokwazo. Funda ngakumbi ngale nto ΠΈ .
Ukuvula iifayile kusetyenziswa iiflegi ze-O_SYNC kunye ne-O_DSYNC
Makhe sibuyele kwingxoxo yeendlela zeLinux ezibonelela ngogcino lwedatha oluzinzileyo. Oko kukuthi, sithetha ngokusebenzisa iflegi O_SYNC okanye iflegi O_DSYNC xa uvula iifayile usebenzisa inkqubo yokufowuna . Ngale ndlela, umsebenzi ngamnye wokubhala idatha wenziwa ngokungathi emva komyalelo ngamnye write() inkqubo inikwa imiyalelo ngokufanelekileyo fsync() ΠΈ fdatasync(). Ku oku kubizwa ngokuba yi "Ngcaciso yeFayile ye-I/O eDityanisiweyo" kunye "nokuQeda uMfezeko weDatha". Inzuzo ephambili yale ndlela kukuba ukuqinisekisa ukuthembeka kwedatha, kufuneka wenze umnxeba omnye wenkqubo, kunokuba ezimbini (umzekelo - write() ΠΈ fdatasync()). Ukungalungi okuphambili kwale ndlela kukuba zonke zibhala usebenzisa i-descriptor yefayile ehambelanayo iya kulungelelaniswa, enokunciphisa ukukwazi ukucwangcisa ikhowudi yesicelo.
Ukusebenzisa i-I/O eNgqo kunye ne-O_DIRECT iflegi
Umnxeba wenkqubo open() ixhasa iflegi O_DIRECT, eyenzelwe ukugqitha i-cache yenkqubo yokusebenza ukwenza imisebenzi ye-I/O ngokunxibelelana ngqo nediski. Oku, kwiimeko ezininzi, kuthetha ukuba imiyalelo yokubhala ekhutshwe yiprogram iya kuguqulelwa ngokuthe ngqo kwimiyalelo ejoliswe ekusebenzeni ngediski. Kodwa, ngokubanzi, le sixhobo asiyondawo yemisebenzi fsync() okanye fdatasync(). Inyaniso kukuba idiski ngokwayo inako imiyalelo ehambelanayo yokubhala idatha. Kwaye, ukwenza izinto zibe mbi ngakumbi, kwezinye iimeko ezikhethekileyo imisebenzi ye-I / O eyenziwa xa kusetyenziswa iflegi O_DIRECT, kwimisebenzi yemveli ekhuselweyo. Eyona ndlela ilula yokusombulula le ngxaki kukusebenzisa iflegi ukuvula iifayile O_DSYNC, okuya kuthetha ukuba umsebenzi ngamnye wokubhala uya kulandelwa yifowuni fdatasync().
Kuye kwavela ukuba inkqubo yefayile ye-XFS isandula ukongeza "indlela ekhawulezayo" ye O_DIRECT|O_DSYNC-ukurekhoda idatha. Ukuba ibhloko iphinda ibhalwe kusetyenziswa O_DIRECT|O_DSYNC, ngoko XFS, endaweni yokugungxula indawo efihlakeleyo, izakwenza i FUA bhala umyalelo ukuba isixhobo siyayixhasa. Ndiyiqinisekisile le nto ngokusebenzisa usetyenziso blktrace kwi-Linux 5.4/Ubuntu 20.04 inkqubo. Le ndlela ifanele isebenze ngakumbi, ekubeni xa isetyenziswa, inani elincinci ledatha libhalwa kwidiski kwaye kusetyenziswe umsebenzi omnye, kunokuba ezimbini (ukubhala kunye nokugungxula i-cache). Ndifumene ikhonkco ku I-kernel ye-2018, esebenzisa le ndlela. Kukho ingxoxo apho malunga nokusebenzisa olu lungiselelo kwezinye iinkqubo zefayile, kodwa ngokokwazi kwam, i-XFS kuphela kwendlela yefayile exhasa oku ukuza kuthi ga ngoku.
sync_file_range () umsebenzi
I-Linux inomnxeba wenkqubo , ekuvumela ukuba ugungxule kuphela inxalenye yefayile kwidiski, kuneyo yonke ifayile. Le mnxeba iqalisa ukugungxulwa kwedatha engahambelaniyo kwaye ayilindi ukuba igqitywe. Kodwa kwisatifikethi sync_file_range() iqela kuthiwa "liyingozi kakhulu". Akukhuthazwa ukuyisebenzisa. Iimpawu kunye neengozi sync_file_range() ichazwe kakuhle kakhulu impahla. Ngokukodwa, le fowuni ibonakala isebenzisa i-RocksDB ukulawula xa i-kernel iphosa idatha engcolileyo kwidiski. Kodwa kwangaxeshanye, ukuqinisekisa ukugcinwa kwedatha okuzinzileyo, kuyasetyenziswa kwakhona fdatasync(). Ku I-RocksDB inezimvo ezinomdla kwesi sihloko. Umzekelo, kubonakala ngathi ifowuni sync_file_range() Xa usebenzisa i-ZFS, ayigungxuli idatha kwidiski. Amava andixelela ukuba ikhowudi engafane isetyenziswe kusenokwenzeka ukuba iqulathe iibugs. Ke ngoko, ndingacebisa ngokuchasene nokusebenzisa le ndlela yokufowuna ngaphandle kokuba kuyimfuneko.
Iifowuni zeSistim ezinceda ukuqinisekisa ukuzingisa kwedatha
Ndiye ndafikelela kwisigqibo sokuba kukho iindlela ezintathu ezinokuthi zisetyenziswe ukwenza imisebenzi ye-I / O eqinisekisa ukuzingisa kwedatha. Zonke zifuna umnxeba womsebenzi fsync() kulawulo apho ifayile yenziwe khona. Ezi ziindlela:
- Ukufowunela umsebenzi
fdatasync()okanyefsync()emva komsebenziwrite()(Kungcono ukusebenzisafdatasync()). - Ukusebenza ngesichazi sefayile sivulwe ngeflegi
O_DSYNCokanyeO_SYNC(ngcono - ngeflegiO_DSYNC). - Ukusetyenziswa komyalelo
pwritev2()ngeflegiRWF_DSYNCokanyeRWF_SYNC(ngokukhethekileyo ngeflegiRWF_DSYNC).
Amanqaku oMsebenzi
Andikhange ndilinganise ngononophelo ukusebenza kweendlela ezahlukeneyo endizivavanyileyo. Umahluko endiwuqapheleyo kwisantya somsebenzi wabo mncinane kakhulu. Oku kuthetha ukuba ndinokuba ndiphosakele, kwaye phantsi kweemeko ezahlukeneyo into enye inokuvelisa iziphumo ezahlukeneyo. Okokuqala, ndiza kuthetha ngento echaphazela ukusebenza ngakumbi, kwaye emva koko yintoni echaphazela ukusebenza kancinci.
- Idatha yefayile yokubhala ngaphezulu ikhawuleza kunokufaka idatha kwifayile (inzuzo yokusebenza ingaba yi-2-100%). Ukufaka idatha kwifayile kufuna utshintsho olongezelelweyo kwimethadatha yefayile, nasemva komnxeba wenkqubo
fallocate(), kodwa ubukhulu besi siphumo bunokwahluka. Ndincoma, ekusebenzeni kakuhle, ukubizafallocate()ukwaba kwangaphambili indawo efunekayo. Emva koko esi sithuba kufuneka sigcwaliswe ngokucacileyo ngo-zero kwaye sibizwefsync(). Oku kuya kuqinisekisa ukuba iibhloko ezingqinelanayo kwisistim yeefayili ziphawulwa njenge-"allocated" endaweni yokuba "zingabiwanga". Oku kunika kancinci (malunga ne-2%) ukuphuculwa komsebenzi. Ukongeza, ezinye iidiski zinokuba nofikelelo olucothayo lokuqala kwibhloko kunezinye. Oku kuthetha ukuba ukuzalisa indawo nge-zero kunokukhokelela ekuphuculeni okuphawulekayo (malunga ne-100%) ekusebenzeni. Ngokukodwa, oku kunokwenzeka ngeediski (Le yidatha engekho semthethweni, andikwazanga ukuyiqinisekisa). Okufanayo kuya kwindawo yokugcina (kwaye olu sele lulwazi olusemthethweni, olungqinisiswe luvavanyo). Nezinye iingcali zenze okufanayo , ezinxulumene neediski ezahlukeneyo. - Iifowuni ezincinci zenkqubo, iphezulu ukusebenza (inzuzo ingaba malunga ne-5%). Kubonakala ngathi lucelomngeni
open()ngeflegiO_DSYNCokanye ukufowunapwritev2()ngeflegiRWF_SYNCngokukhawuleza kunomnxebafdatasync(). Ndiyakrokrela ukuba ingongoma apha kukuba le ndlela idlala indima kwinto yokuba iifowuni ezimbalwa zenkqubo kufuneka zenziwe ukusombulula ingxaki efanayo (umnxeba omnye endaweni yezimbini). Kodwa umahluko ekusebenzeni kuncinci kakhulu, ngoko unokungawuhoyi ngokupheleleyo kwaye usebenzise into kwisicelo engayi kuyenza nzima ingqiqo yayo.
Ukuba unomdla kwisihloko sokugcinwa kwedatha ezinzileyo, nazi izinto eziluncedo:
- β amagqabantshintshi eziseko zeendlela zegalelo/imveliso.
- - ibali malunga nokuba kwenzekani kwidatha endleleni ukusuka kwisicelo ukuya kwidiski.
- - impendulo kumbuzo wokuba usetyenziswe nini
fsync()kubalawuli. Ukubeka oku ngokufutshane, kuvela ukuba kufuneka wenze oku xa udala ifayile entsha, kwaye isizathu sesi ncomo kukuba kwi-Linux kunokubakho iireferensi ezininzi kwifayile efanayo. - - nantsi inkcazo yendlela yokugcinwa kwedatha eqhubekayo kwi-SQL Server kwi-platform ye-Linux. Kukho uthelekiso olunomdla phakathi kweefowuni zeWindows kunye neLinux apha. Ndiphantse ndaqiniseka ukuba ibingumbulelo kwesi sixhobo ndifunde malunga nokwenza ngcono kwe-FUA ye-XFS.
Ngaba ulahlekelwe idatha ocinga ukuba igcinwe ngokukhuselekileyo kwidiski?
umthombo: www.habr.com