{"id":98090,"date":"2020-10-24T02:42:38","date_gmt":"2020-10-24T00:42:38","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/ustojchivoe-hranenie-dannyh-i-fajlovye-api-linux"},"modified":"2020-11-18T00:58:48","modified_gmt":"2020-11-17T22:58:48","slug":"ustojchivoe-hranenie-dannyh-i-fajlovye-api-linux","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/ustojchivoe-hranenie-dannyh-i-fajlovye-api-linux","title":{"rendered":"Ruajtje e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave dhe API skedar\u00ebsh Linux","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Un\u00eb, duke eksperimentuar q\u00ebndrueshm\u00ebrin\u00eb e ruajtjes s\u00eb t\u00eb dh\u00ebnave n\u00eb sistemet n\u00eb re, vendosa t\u00eb provoj veten, t\u00eb sigurohem q\u00eb kuptoj gj\u00ebrat bazike. Un\u00eb fillova me leximin e specifikimeve NVMe<\/a><\/noindex> p\u00ebr t\u00eb kuptuar se cilat garanci, n\u00eb lidhje me ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave (dometh\u00ebn\u00eb - garantimi q\u00eb t\u00eb dh\u00ebnat do t\u00eb jen\u00eb t\u00eb aksesueshme pas nj\u00eb d\u00ebshtimi t\u00eb sistemit), na ofrojn\u00eb disk\u00ebt NMVe. Un\u00eb b\u00ebra k\u00ebto p\u00ebrfundime kryesore: duhet t\u00eb mendohet se t\u00eb dh\u00ebnat jan\u00eb d\u00ebmtuar nga momenti kur jepet komanda p\u00ebr t\u00eb shkruar t\u00eb dh\u00ebnat, dhe deri n\u00eb momentin kur p\u00ebrfundon shkrimi i tyre n\u00eb pajisjen e informacionit. Megjithat\u00eb, n\u00eb shumic\u00ebn e programeve p\u00ebr shkrimin e t\u00eb dh\u00ebnave, p\u00ebrdoren thirrje p\u00ebr sistemin pa ndonj\u00eb problem.<\/p>\n

N\u00eb k\u00ebt\u00eb material, un\u00eb studioj mekanizmat e ruajtjes s\u00eb q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave, q\u00eb ofrohen nga API-t\u00eb e skedave t\u00eb Linux. Duket se k\u00ebtu gjith\u00e7ka duhet t\u00eb jet\u00eb e thjesht\u00eb: programi th\u00ebrret komand\u00ebn write()<\/code>, dhe pas p\u00ebrfundimit t\u00eb pun\u00ebs s\u00eb k\u00ebsaj komande, t\u00eb dh\u00ebnat do t\u00eb ruhen sigurt n\u00eb disk. Por write()<\/code> vet\u00ebm kopjon t\u00eb dh\u00ebnat e aplikacionit n\u00eb cache-in e b\u00ebrtham\u00ebs, q\u00eb ndodhet n\u00eb memorien operative. P\u00ebr t\u00eb detyruar sistemin t\u00eb shkruaj\u00eb t\u00eb dh\u00ebnat n\u00eb disk, duhen p\u00ebrdorur disa mekanizma shtes\u00eb.<\/p>\n

\"Ruajtje<\/a><\/noindex><\/p>\n

N\u00eb p\u00ebrgjith\u00ebsi, ky material p\u00ebrb\u00ebn nj\u00eb grumbull sh\u00ebnimesh, n\u00eb lidhje me at\u00eb q\u00eb kam m\u00ebsuar mbi tem\u00ebn q\u00eb m\u00eb intereson. N\u00ebse do t\u00eb fliste shum\u00eb shkurt p\u00ebr m\u00eb t\u00eb r\u00ebnd\u00ebsishmen, do t\u00eb rezultonte se p\u00ebr t\u00eb organizuar ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave, duhet t\u00eb p\u00ebrdorim komand\u00ebn fdatasync()<\/code> ose t\u00eb hapim skedar\u00ebt me flamurin O_DSYNC<\/code>. N\u00ebse jeni t\u00eb interesuar t\u00eb m\u00ebsoni n\u00eb detaje se \u00e7far\u00eb ndodh me t\u00eb dh\u00ebnat n\u00eb rrug\u00ebn nga kodi programor te disku, shikoni artikull.<\/a><\/noindex> artikullin.
\n<\/a><\/noindex><\/p>\n

Ve\u00e7orit\u00eb e p\u00ebrdorimit t\u00eb funksionit write()<\/h2>\n

\nThirrja sistemike write()<\/code> shtjellohet n\u00eb standardin IEEE POSIX<\/a><\/noindex> si nj\u00eb p\u00ebrpjekje p\u00ebr t\u00eb shkruar t\u00eb dh\u00ebnat n\u00eb nj\u00eb descriptor skedari. Pas p\u00ebrfundimit me sukses write()<\/code> t\u00eb dh\u00ebnat e leximit duhet t\u00eb kthejn\u00eb pik\u00ebrisht ato byte q\u00eb ishin shkruar m\u00eb par\u00eb, duke b\u00ebr\u00eb k\u00ebt\u00eb edhe n\u00eb rast se t\u00eb dh\u00ebnat qas\u00ebn nga procese ose rrjedha t\u00eb tjera (ja<\/a><\/noindex> pjesa p\u00ebrkat\u00ebse e standardit POSIX). K\u00ebtu<\/a><\/noindex>, n\u00eb seksionin q\u00eb i kushtohet nd\u00ebrveprimit t\u00eb rrjedhave me operacionet e zakonshme t\u00eb skedar\u00ebve, ka nj\u00eb sh\u00ebnim q\u00eb thot\u00eb se n\u00ebse secila nga dy rrjedhat th\u00ebrret k\u00ebto funksione, \u00e7do thirrje duhet t\u00eb shoh\u00eb ose t\u00eb gjitha pasojat e specifikuara q\u00eb vijn\u00eb si rezultat i ekzekutimit t\u00eb thirrjes tjet\u00ebr, ose t\u00eb mos shoh\u00eb fare asnj\u00eb pasoj\u00eb. Kjo lejon p\u00ebrfundimin se t\u00eb gjitha operacionet e skedarit p\u00ebr futje\/dalje duhet t\u00eb mbajn\u00eb bllokimin e burimit me t\u00eb cilin punojn\u00eb.<\/p>\n

A do t\u00eb thot\u00eb kjo se operacioni write()<\/code> \u00ebsht\u00eb atomar? Nga ana teknike - po. Operacionet e leximit t\u00eb t\u00eb dh\u00ebnave duhet t\u00eb kthejn\u00eb ose gjith\u00e7ka, ose asgj\u00eb nga ajo q\u00eb \u00ebsht\u00eb shkruar me write()<\/code>. Por operacioni write()<\/code>, sipas standardit, nuk \u00ebsht\u00eb domosdoshm\u00ebrisht i detyruar t\u00eb p\u00ebrfundoj\u00eb duke shkruar gjith\u00e7ka q\u00eb i \u00ebsht\u00eb propozuar t\u00eb shkruaj\u00eb. I lejohet t\u00eb kryej\u00eb vet\u00ebm nj\u00eb pjes\u00eb t\u00eb t\u00eb dh\u00ebnave. P\u00ebr shembull, mund t\u00eb kemi dy rrjedha, secila prej t\u00eb cilave bashkangjit 1024 byte n\u00eb nj\u00eb skedar, i cili p\u00ebrshkruhet nga nj\u00eb descriptor i t\u00eb nj\u00ebjtit skedar. Nga pik\u00ebpamja e standardit, rezultati kur secili nga operacionet e shkruar arrin t\u00eb bashkangjis\u00eb vet\u00ebm nj\u00eb byte n\u00eb skedarin \u00ebsht\u00eb i pranuesh\u00ebm. K\u00ebto operacione do t\u00eb mbeten atomare, por pasi t\u00eb p\u00ebrfundojn\u00eb, t\u00eb dh\u00ebnat e shkruara prej tyre n\u00eb skedar do t\u00eb jen\u00eb t\u00eb p\u00ebrziera. Ja<\/a><\/noindex> nj\u00eb diskutim shum\u00eb interesant n\u00eb k\u00ebt\u00eb tem\u00eb n\u00eb Stack Overflow.<\/p>\n

Funksionet fsync() dhe fdatasync()<\/h2>\n

\nM\u00ebnyra m\u00eb e thjesht\u00eb p\u00ebr t\u00eb rikthyer t\u00eb dh\u00ebnat n\u00eb disk \u00ebsht\u00eb duke thirrur funksionin fsync()<\/a><\/noindex>. Ky funksion k\u00ebrkon nga sistemi operativ q\u00eb t\u00eb transferoj\u00eb t\u00eb gjitha blloqet e modifikuara nga cache n\u00eb disk. Kjo p\u00ebrfshin gjithashtu t\u00eb gjith\u00eb metadata e skedarit (koha e qasjes, koha e modifikimit t\u00eb skedarit dhe k\u00ebshtu me radh\u00eb). Un\u00eb mendoj se nevoja p\u00ebr k\u00ebto metadata ndodh rrall\u00eb, prandaj, n\u00ebse e dini se ato nuk jan\u00eb t\u00eb r\u00ebnd\u00ebsishme p\u00ebr ju, mund t\u00eb p\u00ebrdorni funksionin fdatasync()<\/code>. N\u00eb ndihm\u00ebs<\/a><\/noindex> sip\u00ebr fdatasync()<\/code> thot\u00eb se gjat\u00eb pun\u00ebs s\u00eb k\u00ebtij funksioni, ruhet n\u00eb disk nj\u00eb volum metadata q\u00eb \"\u00ebsht\u00eb e nevojshme p\u00ebr p\u00ebrfundimin e sakt\u00eb t\u00eb operacioneve t\u00eb ardhshme t\u00eb leximit t\u00eb t\u00eb dh\u00ebnave.\" Dhe kjo \u00ebsht\u00eb ajo q\u00eb lidhet me shumic\u00ebn e aplikacioneve.<\/p>\n

Nj\u00eb nga problemet q\u00eb mund t\u00eb lindin k\u00ebtu \u00ebsht\u00eb se k\u00ebto mekanizma nuk garantojn\u00eb se skedari do t\u00eb jet\u00eb i zbuluesh\u00ebm pas nj\u00eb d\u00ebshtimi t\u00eb mundsh\u00ebm. N\u00eb ve\u00e7anti, kur krijohet nj\u00eb skedar i ri, \u00ebsht\u00eb e nevojshme t\u00eb thirret fsync()<\/code> p\u00ebr katalogun q\u00eb e p\u00ebrmban at\u00eb. P\u00ebrndryshe, pas nj\u00eb d\u00ebshtimi, mund t\u00eb ndodhte q\u00eb ky skedar t\u00eb mos ekzistoj\u00eb. Arsyeja e k\u00ebsaj \u00ebsht\u00eb se n\u00eb UNIX, p\u00ebr shkak t\u00eb p\u00ebrdorimit t\u00eb lidhjeve t\u00eb forta, nj\u00eb skedar mund t\u00eb ekzistoj\u00eb n\u00eb disa katalog\u00eb. Prandaj, kur thirret fsync()<\/code> p\u00ebr skedarin nuk ka m\u00ebnyr\u00eb p\u00ebr t\u00eb ditur se t\u00eb dh\u00ebnat e cilit katalog gjithashtu duhet t\u00eb shkarkohen n\u00eb disk (k\u00ebtu<\/a><\/noindex> p\u00ebr k\u00ebt\u00eb mund t\u00eb lexoni m\u00eb shum\u00eb). Duket se sistemi i skedar\u00ebve ext4 \u00ebsht\u00eb n\u00eb gjendje automatikisht<\/a><\/noindex> aplikohen fsync()<\/code> p\u00ebr katalog\u00ebt q\u00eb p\u00ebrmbajn\u00eb skedar\u00ebt p\u00ebrkat\u00ebs, por n\u00eb rastin e sistemeve t\u00eb tjera t\u00eb skedar\u00ebve, kjo mund t\u00eb mos jet\u00eb e v\u00ebrtet\u00eb.<\/p>\n

Ky mekaniz\u00ebm mund t\u00eb implementohet n\u00eb m\u00ebnyra t\u00eb ndryshme n\u00eb sisteme t\u00eb ndryshme t\u00eb skedar\u00ebve. Un\u00eb p\u00ebrdora blktrace<\/a><\/noindex> p\u00ebr t\u00eb m\u00ebsuar se cilat operacione diskesh p\u00ebrdoren n\u00eb sistemet e skedar\u00ebve ext4 dhe XFS. T\u00eb dy sistemet japin komanda standarde p\u00ebr shkrimin n\u00eb disk p\u00ebr p\u00ebrmbajtjen e skedar\u00ebve dhe p\u00ebr regjistrin e sistemit t\u00eb skedar\u00ebve, shkarkojn\u00eb cache-n dhe p\u00ebrfundojn\u00eb pun\u00ebn duke kryer nj\u00eb shkrim FUA (Force Unit Access, shkrimi i t\u00eb dh\u00ebnave direkt n\u00eb disk, duke anashkaluar cache) n\u00eb regjist\u00ebr. Probabilisht, ato veprojn\u00eb k\u00ebshtu p\u00ebr t\u00eb konfirmuar faktin e kryerjes s\u00eb operacionit. N\u00eb disqet q\u00eb nuk mb\u00ebshtesin FUA, kjo shkakton dy shkarkime t\u00eb cache-it. Eksperimentet e mia treguan se fdatasync()<\/code> disi m\u00eb shpejt fsync()<\/code>. Vegla blktrace<\/code> tregon se fdatasync()<\/code> zakonisht shkruan n\u00eb disk m\u00eb pak t\u00eb dh\u00ebna (n\u00eb ext4 fsync()<\/code> shkruan 20 KiB, dhe fdatasync()<\/code> \u2013 16 KiB). P\u00ebr m\u00eb tep\u00ebr, un\u00eb zbuloja se XFS \u00ebsht\u00eb pak m\u00eb i shpejt\u00eb se ext4. Dhe k\u00ebtu, me ndihm\u00ebn e blktrace<\/code> arriti t\u00eb kuptoj se fdatasync()<\/code> shk\u00ebput nga disku m\u00eb pak t\u00eb dh\u00ebna (4 KiB n\u00eb XFS).<\/p>\n

Situatat e paqart\u00eb, q\u00eb ndodhin gjat\u00eb p\u00ebrdorimit t\u00eb fsync()<\/h2>\n

\nMund t\u00eb kujtoj tre situata t\u00eb paqarta q\u00eb lidhen me fsync()<\/code>, me t\u00eb cilat u p\u00ebrballa n\u00eb praktik\u00eb.<\/p>\n

Rasti i par\u00eb ndodhi n\u00eb 2008. At\u00ebher\u00eb, nd\u00ebrfaqja Firefox 3 \"ngadal\u00ebsohej\" n\u00eb rast se po b\u00ebhej shkrimi n\u00eb disk i nj\u00eb numri t\u00eb madh skedar\u00ebsh. Problemi q\u00ebndronte n\u00eb faktin se n\u00eb implementimin e nd\u00ebrfaqes p\u00ebr ruajtjen e informacionit mbi gjendjen e saj p\u00ebrdorej nj\u00eb baz\u00eb t\u00eb dh\u00ebnash SQLite. Pas \u00e7do ndryshimi q\u00eb ndodhte n\u00eb nd\u00ebrfaqe, thirrej funksioni fsync()<\/code>, duke ofruar garanci t\u00eb mira p\u00ebr ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave. N\u00eb sistemin e skedar\u00ebve ext3 q\u00eb p\u00ebrdorej at\u00ebher\u00eb, funksioni fsync()<\/code> p\u00ebrmbante t\u00eb gjitha faqet \"e pista\" n\u00eb sistem, jo vet\u00ebm ato q\u00eb kishin lidhje me skedarin p\u00ebrkat\u00ebs. Kjo do t\u00eb thoshte se nj\u00eb klikim n\u00eb butonin n\u00eb Firefox mund t\u00eb inicjonte shkrimin e megabajtave t\u00eb t\u00eb dh\u00ebnave n\u00eb disqet magnetik, gj\u00eb q\u00eb mund t\u00eb zgjaste shum\u00eb sekonda. Zgjidhja e problemit, sa kam kuptuar nga k\u00ebt\u00eb<\/a><\/noindex> materiali, consistonte n\u00eb zhvendosjen e pun\u00ebs me baz\u00ebn e t\u00eb dh\u00ebnave n\u00eb detyra asinkrone n\u00eb sfond. Kjo do t\u00eb thot\u00eb se m\u00eb par\u00eb n\u00eb Firefox ishin realizuar k\u00ebrkesa m\u00eb t\u00eb forta p\u00ebr q\u00ebndrueshm\u00ebrin\u00eb e ruajtjes s\u00eb t\u00eb dh\u00ebnave, sesa kishte nevoj\u00eb p\u00ebr t\u00eb, dhe ve\u00e7orit\u00eb e sistemit t\u00eb skedar\u00ebve ext3 vet\u00ebm sa e kishin p\u00ebrkeq\u00ebsuar k\u00ebt\u00eb problem.<\/p>\n

E dyta ndodhi n\u00eb vitin 2009. At\u00ebher\u00eb, pas nj\u00eb d\u00ebshtimi t\u00eb sistemit, p\u00ebrdoruesit e sistemit t\u00eb ri t\u00eb skedar\u00ebve ext4 p\u00ebrballeshin me faktin se shum\u00eb skedar\u00eb t\u00eb sapo krijuar kishin gjat\u00eb zero, nd\u00ebrsa me sistemin m\u00eb t\u00eb vjet\u00ebr ext3 kjo nuk ndodhi. N\u00eb paragrafin e m\u00ebparsh\u00ebm, un\u00eb thash\u00eb se ext3 shkarkonte shum\u00eb t\u00eb dh\u00ebna n\u00eb disk, e cila ngadal\u00ebsoi shum\u00eb pun\u00ebn fsync()<\/code>. P\u00ebr t\u00eb p\u00ebrmir\u00ebsuar situat\u00ebn, n\u00eb ext4 shkarkohen n\u00eb disk vet\u00ebm ato faqe \"e pista\" q\u00eb i p\u00ebrkasin nj\u00eb skedari specifik. T\u00eb dh\u00ebnat e skedar\u00ebve t\u00eb tjer\u00eb mbeten n\u00eb kujtes\u00eb p\u00ebr nj\u00eb periudh\u00eb shum\u00eb m\u00eb t\u00eb gjat\u00eb, sesa me p\u00ebrdorimin e ext3. Kjo u b\u00eb p\u00ebr t\u00eb p\u00ebrmir\u00ebsuar performanc\u00ebn (n\u00eb m\u00ebnyr\u00eb t\u00eb paracaktuar, t\u00eb dh\u00ebnat q\u00ebndrojn\u00eb n\u00eb nj\u00eb gjendje t\u00eb till\u00eb p\u00ebr 30 sekonda, kjo mund t\u00eb konfiguroni me dirty_expire_centisecs<\/a><\/noindex>; k\u00ebtu<\/a><\/noindex> , mund t\u00eb gjeni materiale t\u00eb tjera mbi k\u00ebt\u00eb). Kjo do t\u00eb thot\u00eb se nj\u00eb volum i madh t\u00eb dh\u00ebnash mund t\u00eb humbas\u00eb p\u00ebrfundimisht pas nj\u00eb d\u00ebshtimi. Zgjidhja e k\u00ebtij problemi \u00ebsht\u00eb p\u00ebrdorimi i fsync()<\/code> n\u00eb aplikacionet q\u00eb duan t\u00eb garantojn\u00eb ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave dhe t\u00eb sigurtojn\u00eb ato nga pasojat e d\u00ebshtimeve. Funksioni fsync()<\/code> funksionon shum\u00eb m\u00eb efektivisht n\u00eb p\u00ebrdorimin e ext4, sesa n\u00eb p\u00ebrdorimin e ext3. Nj\u00eb disavantazh i k\u00ebtij qasje \u00ebsht\u00eb se aplikimi i saj, si m\u00eb par\u00eb, ngadal\u00ebson ekzekutimin e disa operacioneve, si instalimi i programeve. Detajet mbi k\u00ebt\u00eb shikoni k\u00ebtu<\/a><\/noindex> dhe k\u00ebtu<\/a><\/noindex>.<\/p>\n

Problemi i tret\u00eb, lidhur me fsync()<\/code>, ndodhi n\u00eb vitin 2018. At\u00ebher\u00eb, n\u00eb kuad\u00ebr t\u00eb projektit PostgreSQL, u zbuluar se n\u00ebse funksioni fsync()<\/code> p\u00ebrballet me nj\u00eb gabim, ai sh\u00ebnon faqet \"e pista\" si \"t\u00eb pastra\". Si pasoj\u00eb, thirrjet e ardhshme fsync()<\/code> nuk\u00eb b\u00ebhet asgj\u00eb me k\u00ebto faqe. P\u00ebr k\u00ebt\u00eb arsye, faqet e modifikuara ruajn\u00eb n\u00eb memorie dhe kurr\u00eb nuk shkruhen n\u00eb disk. Kjo \u00ebsht\u00eb nj\u00eb katastrof\u00eb e v\u00ebrtet\u00eb, pasi aplikacioni do t\u00eb mendoj\u00eb se disa t\u00eb dh\u00ebna jan\u00eb shkruar n\u00eb disk, nd\u00ebrsa n\u00eb t\u00eb v\u00ebrtet\u00eb nuk \u00ebsht\u00eb k\u00ebshtu. T\u00eb tilla d\u00ebshtime fsync()<\/code> ndodhin rrall\u00eb, aplikacioni n\u00eb k\u00ebto situata pothuajse nuk mund t\u00eb b\u00ebj\u00eb asgj\u00eb p\u00ebr t\u00eb luftuar problemin. N\u00eb dit\u00ebt e sotme, kur ndodh kjo, PostgreSQL dhe aplikacione t\u00eb tjera p\u00ebrfundojn\u00eb me d\u00ebshtim. K\u00ebtu<\/a><\/noindex>, n\u00eb materialin \"Mund t\u00eb Rikuperohen Aplikacionet nga D\u00ebshtimet e fsync?\", kjo problematike shqyrtohet n\u00eb t\u00eb gjitha detajet. Aktualisht, zgjidhja m\u00eb e mir\u00eb p\u00ebr k\u00ebt\u00eb problem \u00ebsht\u00eb p\u00ebrdorimi i Direct I\/O me flamurin O_SYNC<\/code> ose me flamurin O_DSYNC<\/code>. Me k\u00ebt\u00eb qasje, sistemi do t\u00eb raportoj\u00eb p\u00ebr gabimet q\u00eb mund t\u00eb ndodhin gjat\u00eb kryerjes s\u00eb operacioneve t\u00eb caktuara t\u00eb shkruarjes s\u00eb t\u00eb dh\u00ebnave, por kjo qasje k\u00ebrkon q\u00eb aplikacioni t\u00eb menaxhoj\u00eb vet\u00eb bufferat. Detajet rreth k\u00ebsaj lexoni k\u00ebtu<\/a><\/noindex> dhe k\u00ebtu<\/a><\/noindex>.<\/p>\n

Hapja e skedareve me p\u00ebrdorimin e flamujve O_SYNC dhe O_DSYNC<\/h2>\n

\nLe t\u00eb kthehemi n\u00eb diskutimin e mekanizmave Linux q\u00eb sigurojn\u00eb ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave. N\u00eb ve\u00e7anti, b\u00ebhet fjal\u00eb p\u00ebr p\u00ebrdorimin e flamurit O_SYNC<\/code> ose flamurit O_DSYNC<\/code> kur hapen skedar\u00ebt duke p\u00ebrdorur thirrjen sistemike open()<\/a><\/noindex>. Me k\u00ebt\u00eb qasje, \u00e7do operacion i shkruarjes s\u00eb t\u00eb dh\u00ebnave kryhet sikur pas \u00e7do urdhri write()<\/code> sistemi jep, p\u00ebrkat\u00ebsisht, komanda fsync()<\/code> dhe fdatasync()<\/code>. N\u00eb specifikimit POSIX<\/a><\/noindex> kjo quhet \"P\u00ebrfundimi i Integritetit t\u00eb Skedar\u00ebve t\u00eb Shkruarjes t\u00eb Sinkronizuar\" dhe \"P\u00ebrfundimi i Integritetit t\u00eb T\u00eb Dh\u00ebnave\". Avantazhi kryesor i k\u00ebsaj qasjeje \u00ebsht\u00eb se p\u00ebr t\u00eb siguruar integritetin e t\u00eb dh\u00ebnave nevojitet t\u00eb kryhen vet\u00ebm nj\u00eb thirrje sistemike, jo dy (p\u00ebr shembull \u2014 write()<\/code> dhe fdatasync()<\/code>). Disavantazhi kryesor i k\u00ebsaj qasjeje \u00ebsht\u00eb se t\u00eb gjitha operacionet e shkruarjes q\u00eb p\u00ebrdorin skedarin e p\u00ebrkat\u00ebssh\u00ebm do t\u00eb jen\u00eb t\u00eb sinkronizuara, t\u00eb cilat mund t\u00eb kufizojn\u00eb mund\u00ebsit\u00eb p\u00ebr strukturimin e kodit t\u00eb aplikacionit.<\/p>\n

P\u00ebrdorimi i Direct I\/O me flamurin O_DIRECT<\/h2>\n

\nThirrja sistemike open()<\/code> mb\u00ebshtet flamurin O_DIRECT<\/code>, i cili \u00ebsht\u00eb menduar p\u00ebr t\u00eb kryer operacione hyrjeje-dali, duke anashkaluar cache-n\u00eb e sistemit operativ dhe duke komunikuar drejtp\u00ebrdrejt me diskun. Kjo, n\u00eb shum\u00eb raste, do t\u00eb thot\u00eb se komandat e shkruarjes t\u00eb l\u00ebshuara nga programi do t\u00eb translen n\u00eb m\u00ebnyr\u00eb t\u00eb drejtp\u00ebrdrejt\u00eb n\u00eb komandat q\u00eb i drejtohen diskut. Por, n\u00eb p\u00ebrgjith\u00ebsi, ky mekaniz\u00ebm nuk \u00ebsht\u00eb nj\u00eb z\u00ebvend\u00ebsim p\u00ebr funksionet fsync()<\/code> ose fdatasync()<\/code>. Kjo \u00ebsht\u00eb p\u00ebr faktor\u00eb se disku vet\u00eb mund t\u00eb vonoj\u00eb ose t\u00eb cache-oj\u00eb<\/a><\/noindex> komandat p\u00ebrkat\u00ebse p\u00ebr regjistrimin e t\u00eb dh\u00ebnave. Dhe, \u00e7ka \u00ebsht\u00eb edhe m\u00eb keq, n\u00eb disa raste t\u00eb ve\u00e7anta operacionet e hyrjes-daljes, q\u00eb kryhen kur p\u00ebrdoret flagu O_DIRECT<\/code>, p\u00ebrcillen<\/a><\/noindex> n\u00eb operacione tradicionale t\u00eb tamponuara. M\u00eb leht\u00eb se sa t\u00eb zgjidh\u00ebsh k\u00ebt\u00eb problem \u00ebsht\u00eb t\u00eb p\u00ebrdor\u00ebsh p\u00ebr hapjen e skedar\u00ebve gjithashtu nj\u00eb flag O_DSYNC<\/code>, \u00e7ka do t\u00eb thot\u00eb se pas \u00e7do operacioni regjistrimi do t\u00eb ket\u00eb nj\u00eb thirrje fdatasync()<\/code>.<\/p>\n

Doli se n\u00eb sistemin e skedar\u00ebve XFS koh\u00ebt e fundit \u00ebsht\u00eb shtuar nj\u00eb \"rrug\u00eb e shpejt\u00eb\" p\u00ebr O_DIRECT|O_DSYNC<\/code>-regjistrimin e t\u00eb dh\u00ebnave. N\u00ebse ri-regjistron nj\u00eb bllok duke p\u00ebrdorur O_DIRECT|O_DSYNC<\/code>, at\u00ebher\u00eb XFS, n\u00eb vend q\u00eb t\u00eb pastroj\u00eb cache-n, do t\u00eb kryej\u00eb komand\u00ebn e regjistrimit FUA n\u00ebse pajisja e mb\u00ebshtet k\u00ebt\u00eb. Un\u00eb e kam verifikuar k\u00ebt\u00eb duke p\u00ebrdorur utilitarin blktrace<\/code> n\u00eb sistemin Linux 5.4\/Ubuntu 20.04. Ky qasje duhet t\u00eb jet\u00eb m\u00eb efikase, pasi me t\u00eb shkruhet nj\u00eb sasi minimale e t\u00eb dh\u00ebnave n\u00eb disk dhe gjithashtu aplikohet nj\u00eb operacion, jo dy (regjistrimi dhe pastrimi i cache-it). Kam gjetur nj\u00eb lidhje p\u00ebr patch<\/a><\/noindex> n\u00eb kernelin e vitit 2018, ku \u00ebsht\u00eb realizuar ky mekaniz\u00ebm. Aty ka nj\u00eb diskutim n\u00eb lidhje me aplikimin e k\u00ebsaj optimizimi edhe n\u00eb sisteme t\u00eb tjera skedari, por, p\u00ebr aq sa di, XFS \u00ebsht\u00eb p\u00ebr momentin e vetmja sistem skedari q\u00eb e mb\u00ebshtet k\u00ebt\u00eb.<\/p>\n

Funksioni sync_file_range()<\/h2>\n

\nN\u00eb Linux ka nj\u00eb thirrje sistemike sync_file_range()<\/a><\/noindex>, e cila lejon t\u00eb pastrohet n\u00eb disk vet\u00ebm nj\u00eb pjes\u00eb e skedarit, e jo t\u00ebr\u00eb skedari. Kjo thirrje iniciaton nj\u00eb pastrim asinkron t\u00eb t\u00eb dh\u00ebnave dhe nuk pret p\u00ebr p\u00ebrfundimin e tij. Por n\u00eb dokumentacionin p\u00ebr sync_file_range()<\/code> thuhet se kjo komand\u00eb \"\u00ebsht\u00eb shum\u00eb e rrezikshme\". Nuk rekomandohet p\u00ebrdorimi i saj. Karakteristikat dhe rreziqet sync_file_range()<\/code> jan\u00eb p\u00ebrshkruar shum\u00eb mir\u00eb n\u00eb k\u00ebt\u00eb<\/a><\/noindex> material. Sidomos, duket se kjo thirrje p\u00ebrdor RocksDB p\u00ebr t\u00eb menaxhuar kur b\u00ebrthama pastron t\u00eb dh\u00ebnat \"t\u00eb ndotura\" n\u00eb disk. Por n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, p\u00ebr t\u00eb siguruar ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave, p\u00ebrdoret gjithashtu fdatasync()<\/code>. N\u00eb kode<\/a><\/noindex> RocksDB ka komente interesante n\u00eb k\u00ebt\u00eb tem\u00eb. P\u00ebr shembull, duket se thirrja sync_file_range()<\/code> kur p\u00ebrdoret ZFS nuk e \u00e7on n\u00eb pastrimin e t\u00eb dh\u00ebnave n\u00eb disk. Eksperienca m\u00eb tregon se kodi, i cili p\u00ebrdoret rrall\u00eb, ndoshta p\u00ebrmban gabime. Prandaj, do t'ju rekomandoja t\u00eb mos p\u00ebrdoroni k\u00ebt\u00eb thirrje sistemike pa nj\u00eb nevoj\u00eb t\u00eb madhe.<\/p>\n

Thirrjet sistemike q\u00eb ndihmojn\u00eb p\u00ebr t\u00eb siguruar ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave<\/h2>\n

\nKam er n\u00eb p\u00ebrfundimin se p\u00ebr t\u00eb kryer operacione input\/output q\u00eb sigurojn\u00eb ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave, mund t\u00eb p\u00ebrdoren tre qasje. T\u00eb gjitha ato k\u00ebrkojn\u00eb thirrjen e funksionit fsync()<\/code> p\u00ebr direktorin\u00eb ku \u00ebsht\u00eb krijuar skedari. K\u00ebto jan\u00eb qasjet:<\/p>\n

    \n
  1. Thirrja e funksionit fdatasync()<\/code> ose fsync()<\/code> pas funksionit write()<\/code> (\u00ebsht\u00eb m\u00eb mir\u00eb t\u00eb p\u00ebrdoret fdatasync()<\/code>).<\/li>\n
  2. Puna me descriptorin e skedarit, i hapur me flag O_DSYNC<\/code> ose O_SYNC<\/code> (m\u00eb mir\u00eb \u2014 me flag O_DSYNC<\/code>).<\/li>\n
  3. P\u00ebrdorimi i komand\u00ebs pwritev2()<\/code> me flamurin RWF_DSYNC<\/code> ose RWF_SYNC<\/code> (preferohet \u2014 me flag RWF_DSYNC<\/code>).<\/li>\n<\/ol>\n

    <\/p>\n

    Sh\u00ebnime mbi performanc\u00ebn<\/h2>\n

    \nNuk kam b\u00ebr\u00eb matje t\u00eb kujdesshme t\u00eb performanc\u00ebs s\u00eb mekanizmave t\u00eb ndrysh\u00ebm q\u00eb kam studiuar. Diferencat e v\u00ebrejtura nga un\u00eb n\u00eb shpejt\u00ebsin\u00eb e pun\u00ebs s\u00eb tyre jan\u00eb mjaft t\u00eb vogla. Kjo do t\u00eb thot\u00eb se mund t\u00eb kem gabuar dhe se, n\u00eb kushte t\u00eb tjera, t\u00eb nj\u00ebjtat gj\u00ebra mund t\u00eb tregojn\u00eb rezultate t\u00eb tjera. Fillimisht, do t\u00eb flas p\u00ebr at\u00eb q\u00eb ndikon m\u00eb shum\u00eb n\u00eb performanc\u00eb, e m\u00eb pas p\u00ebr at\u00eb q\u00eb ndikon m\u00eb pak n\u00eb performanc\u00eb.<\/p>\n

      \n
    1. Rikthimi i t\u00eb dh\u00ebnave t\u00eb skedarit \u00ebsht\u00eb m\u00eb i shpejt\u00eb se bashkimi i t\u00eb dh\u00ebnave n\u00eb skedar (fitimi n\u00eb performanc\u00eb mund t\u00eb arrij\u00eb 2-100%). Bashkimi i t\u00eb dh\u00ebnave n\u00eb skedar k\u00ebrkon ndryshime t\u00eb tjera n\u00eb metadat\u00ebn e skedarit, madje pas thirrjes s\u00eb sistemit fallocate()<\/code>, por shkall\u00ebt e k\u00ebtij efekti mund t\u00eb ndryshojn\u00eb. Un\u00eb rekomandoj, p\u00ebr t\u00eb siguruar performanc\u00ebn m\u00eb t\u00eb mir\u00eb, t\u00eb thirret fallocate()<\/code> p\u00ebr rezervimin e hap\u00ebsir\u00ebs s\u00eb nevojshme. M\u00eb pas, kjo hap\u00ebsir\u00eb duhet t\u00eb plot\u00ebsohet qart\u00eb me zero dhe t\u00eb thirret fsync()<\/code>. Fal\u00eb k\u00ebsaj, blloket p\u00ebrkat\u00ebse n\u00eb sistemin e skedar\u00ebve do t\u00eb sh\u00ebnohen si \"rezervuar\", dhe jo si \"n\u00eb dispozicion\". Kjo ofron nj\u00eb p\u00ebrmir\u00ebsim t\u00eb vog\u00ebl (rreth 2%) n\u00eb performanc\u00eb. P\u00ebr m\u00eb tep\u00ebr, disa disqe mund t\u00eb ken\u00eb operacionin e par\u00eb t\u00eb aksesit n\u00eb bllok q\u00eb mund t\u00eb realizohet m\u00eb ngadal\u00eb se t\u00eb tjer\u00ebt. Kjo do t\u00eb thot\u00eb se plot\u00ebsimi i hap\u00ebsir\u00ebs me zero mund t\u00eb sjell\u00eb nj\u00eb p\u00ebrmir\u00ebsim t\u00eb ndjesh\u00ebm (rreth 100%) n\u00eb performanc\u00eb. N\u00eb ve\u00e7anti, di\u00e7ka e till\u00eb mund t\u00eb ndodh\u00eb me disqet AWS EBS<\/a><\/noindex> (kjo jan\u00eb t\u00eb dh\u00ebna jozyrtare, nuk kam arritur t'i konfirmoj ato). E nj\u00ebjta gj\u00eb vlen p\u00ebr ruajtjet GCP Persistent Disk<\/a><\/noindex> (dhe kjo \u00ebsht\u00eb informacion zyrtar, e konfirmuar nga provat). Specialist\u00eb t\u00eb tjer\u00eb kan\u00eb b\u00ebr\u00eb v\u00ebzhgime t\u00eb ngjashme n\u00eb lidhje me disqe t\u00eb ndryshme.<\/a><\/noindex>Sa m\u00eb pak thirrje sistemesh - aq m\u00eb e lart\u00eb \u00ebsht\u00eb performanca (fitimi mund t\u00eb arrij\u00eb rreth 5%). Duket se thirrja<\/li>\n
    2. ose thirrja open()<\/code> me flamurin O_DSYNC<\/code> \u00ebsht\u00eb m\u00eb e shpejt\u00eb se thirrja pwritev2()<\/code> me flamurin RWF_SYNC<\/code> thirrja m\u00eb e shpejt\u00eb fdatasync()<\/code>Dyshoj se kjo ka t\u00eb b\u00ebj\u00eb me faktin se me k\u00ebt\u00eb qasje, luan rol fakti q\u00eb p\u00ebr t\u00eb zgjidhur t\u00eb nj\u00ebjt\u00ebn detyr\u00eb nevojiten m\u00eb pak thirrje sistemike (nj\u00eb thirrje n\u00eb vend t\u00eb dyve). Por ndryshimi n\u00eb performanc\u00eb \u00ebsht\u00eb shum\u00eb i vog\u00ebl, prandaj ju mund ta injoroni at\u00eb dhe t\u00eb p\u00ebrdorni n\u00eb aplikacion at\u00eb q\u00eb nuk do ta komplikoje logjik\u00ebn e tij.<\/li>\n<\/ol>\n

      \nN\u00ebse jeni t\u00eb interesuar p\u00ebr tem\u00ebn e ruajtjes s\u00eb q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave \u2014 k\u00ebtu jan\u00eb disa materiale t\u00eb dobishme:<\/p>\n