{"id":98090,"date":"2020-10-24T02:42:38","date_gmt":"2020-10-24T00:42:38","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/ustojchivoe-hranenie-dannyh-i-fajlovye-api-linux"},"modified":"2020-11-18T00:58:48","modified_gmt":"2020-11-17T22:58:48","slug":"ustojchivoe-hranenie-dannyh-i-fajlovye-api-linux","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/ustojchivoe-hranenie-dannyh-i-fajlovye-api-linux","title":{"rendered":"Ruajtja e q\u00ebndrueshme e t\u00eb dh\u00ebnave dhe API-t\u00eb e skedar\u00ebve n\u00eb Linux","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Un\u00eb, duke hulumtuar q\u00ebndrueshm\u00ebrin\u00eb e ruajtjes s\u00eb t\u00eb dh\u00ebnave n\u00eb sistemet e cloud, vendosa t\u00eb verifikoj veten, t\u00eb sigurohem q\u00eb kuptoj gj\u00ebrat baz\u00eb. Un\u00eb fillova me leximin e specifikacionit NVMe<\/a><\/noindex> p\u00ebr t'u kuptuar me ato garancit\u00eb q\u00eb ofrojn\u00eb disqet NVMe p\u00ebr ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave (pra, garancit\u00eb q\u00eb t\u00eb dh\u00ebnat do t\u00eb jen\u00eb t\u00eb disponueshme pas nj\u00eb d\u00ebshtimi t\u00eb sistemit). Un\u00eb b\u00ebra k\u00ebto p\u00ebrfundime kryesore: duhet t\u00eb merret n\u00eb konsiderat\u00eb se t\u00eb dh\u00ebnat jan\u00eb t\u00eb d\u00ebmtuara q\u00eb nga momenti kur \u00ebsht\u00eb dh\u00ebn\u00eb urdhri p\u00ebr t\u00eb shkruar t\u00eb dh\u00ebnat dhe deri n\u00eb momentin kur p\u00ebrfundon shkrimi i tyre n\u00eb mediumin e informacionit. Megjithat\u00eb, n\u00eb shumic\u00ebn e programeve p\u00ebr shkrimin e t\u00eb dh\u00ebnave, thjesht p\u00ebrdoren thirrjet sistemike.<\/p>\n

N\u00eb k\u00ebt\u00eb material, un\u00eb hulumtoj mekanizmat e ruajtjes s\u00eb q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave q\u00eb ofrohen nga API-t\u00eb e skedar\u00ebve n\u00eb Linux. Duket se k\u00ebtu gjith\u00e7ka duhet t\u00eb jet\u00eb e thjesht\u00eb: programi th\u00ebrret komand\u00ebn write()<\/code>, dhe pasi p\u00ebrfundon puna e k\u00ebsaj komande, t\u00eb dh\u00ebnat do t\u00eb jen\u00eb ruajtur n\u00eb m\u00ebnyr\u00eb t\u00eb sigurt n\u00eb disk. Por write()<\/code> thjesht kopjon t\u00eb dh\u00ebnat e aplikacionit n\u00eb cache-n e b\u00ebrtham\u00ebs, i cili ndodhet n\u00eb memorien operative. P\u00ebr t\u00eb detyruar sistemin t\u00eb ruaj\u00eb t\u00eb dh\u00ebnat n\u00eb disk, nevojitet p\u00ebrdorimi i disa mekanizmave shtes\u00eb.<\/p>\n

\"Ruajtja<\/a><\/noindex><\/p>\n

N\u00eb p\u00ebrgjith\u00ebsi, ky material p\u00ebrfaq\u00ebson nj\u00eb grup sh\u00ebnimesh rreth asaj q\u00eb kam m\u00ebsuar p\u00ebr tem\u00ebn q\u00eb m\u00eb intereson. N\u00ebse flasim shum\u00eb shkurt mbi m\u00eb t\u00eb r\u00ebnd\u00ebsishmen, ajo \u00ebsht\u00eb se p\u00ebr t\u00eb organizuar ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave duhet t\u00eb p\u00ebrdoret komanda fdatasync()<\/code> ose t\u00eb hapen skedar\u00eb me flak\u00eb O_DSYNC<\/code>. N\u00ebse jeni t\u00eb interesuar t\u00eb dini n\u00eb detaje se \u00e7far\u00eb ndodh me t\u00eb dh\u00ebnat gjat\u00eb rrug\u00ebs nga kodi i programit n\u00eb disk, shikoni k\u00ebt\u00eb<\/a><\/noindex> artikullin.
\n<\/a><\/noindex><\/p>\n

Karakteristikat e p\u00ebrdorimit t\u00eb funksionit write()<\/h2>\n

\nThirrja e sistemit write()<\/code> jan\u00eb p\u00ebrcaktuar n\u00eb standardin IEEE POSIX<\/a><\/noindex> si nj\u00eb p\u00ebrpjekje p\u00ebr t\u00eb shkruar t\u00eb dh\u00ebna n\u00eb nj\u00eb t\u00eb drejta skedari. Pas p\u00ebrfundimit t\u00eb suksessh\u00ebm t\u00eb veprimit write()<\/code> operacionet e leximit t\u00eb t\u00eb dh\u00ebnave duhet t\u00eb kthejn\u00eb pik\u00ebrisht ato byte q\u00eb ishin shkruar m\u00eb par\u00eb, duke e b\u00ebr\u00eb k\u00ebt\u00eb edhe n\u00eb rast se t\u00eb dh\u00ebnat qasen nga procese ose rrjedha t\u00eb tjera (k\u00ebtu<\/a><\/noindex> pj\u00ebza p\u00ebrkat\u00ebse e standardit POSIX). K\u00ebtu<\/a><\/noindex>, n\u00eb seksionin q\u00eb trajton nd\u00ebrveprimin e rrjedhave me operacionet e zakonshme t\u00eb skedar\u00ebve ka nj\u00eb sh\u00ebnim ku thuhet se n\u00ebse secila nga dy rrjedhat th\u00ebrret k\u00ebto funksione, at\u00ebher\u00eb \u00e7do thirrje duhet t\u00eb shoh\u00eb ose t\u00eb gjitha pasojat e caktuara q\u00eb ndodhin p\u00ebr shkak t\u00eb ekzekutimit t\u00eb thirrjes tjet\u00ebr, ose t\u00eb mos shoh\u00eb asnj\u00eb pasoj\u00eb. Kjo lejon t\u00eb p\u00ebrfundohet se t\u00eb gjitha operacionet e skedar\u00ebve t\u00eb hyrjes\/daljes duhet t\u00eb mbajn\u00eb bllokimin e burimit me t\u00eb cilin punojn\u00eb.<\/p>\n

A do t\u00eb thot\u00eb kjo se operacioni write()<\/code> \u00ebsht\u00eb atomic? Nga perspektiva teknike \u2014 po. Operacionet e leximit t\u00eb t\u00eb dh\u00ebnave duhet t\u00eb kthejn\u00eb ose gjith\u00e7ka ose asgj\u00eb nga ajo q\u00eb \u00ebsht\u00eb shkruar me write()<\/code>. Por operacioni write()<\/code>, sipas standardit, nuk \u00ebsht\u00eb domosdoshm\u00ebrisht i detyruar t\u00eb p\u00ebrfundoj\u00eb duke shkruar gjith\u00e7ka q\u00eb iu ofrua p\u00ebr t'u shkruar. Ajo lejohet t\u00eb shkruaj\u00eb vet\u00ebm nj\u00eb pjes\u00eb t\u00eb t\u00eb dh\u00ebnave. P\u00ebr shembull, mund t\u00eb kemi dy rrjedha, secila nga t\u00eb cilat bashk\u00ebngjit 1024 byte n\u00eb nj\u00eb skedar q\u00eb p\u00ebrshkruhet nga e nj\u00ebjta e drejte skedari. Nga pik\u00ebpamja e standardit, \u00ebsht\u00eb e pranueshme nj\u00eb rezultat kur secila nga operacionet e shkrimit arrin t\u00eb bashk\u00ebngjis\u00eb vet\u00ebm nj\u00eb byte n\u00eb skedar. K\u00ebto operacione do t\u00eb mbeten atomike, por pasi t\u00eb p\u00ebrfundojn\u00eb, t\u00eb dh\u00ebnat q\u00eb ato shkruan n\u00eb skedar do t\u00eb jen\u00eb t\u00eb p\u00ebrziera. K\u00ebtu \u00ebsht\u00eb<\/a><\/noindex> ka nj\u00eb diskutim shum\u00eb interesant mbi k\u00ebt\u00eb tem\u00eb n\u00eb Stack Overflow.<\/p>\n

Funksionet fsync() dhe fdatasync()<\/h2>\n

\nM\u00ebnyra m\u00eb e thjesht\u00eb p\u00ebr t\u00eb shp\u00ebrndar\u00eb t\u00eb dh\u00ebnat n\u00eb disk \u00ebsht\u00eb thirrja e funksionit fsync()<\/a><\/noindex>. Ky funksion k\u00ebrkon nga sistemi operativ t\u00eb transferoj\u00eb t\u00eb gjith\u00eb blloqet e modifikuar nga cache n\u00eb disk. K\u00ebtu p\u00ebrfshihen gjithashtu t\u00eb gjitha metadata e skedarit (koha e aksesit, koha e modifikimit t\u00eb skedarit dhe k\u00ebshtu me radh\u00eb). Un\u00eb besoj se nevoja p\u00ebr k\u00ebto metadata ndodh rrall\u00eb, prandaj, n\u00ebse e dini se ato nuk jan\u00eb t\u00eb r\u00ebnd\u00ebsishme p\u00ebr ju, mund t\u00eb p\u00ebrdorni funksionin fdatasync()<\/code>. N\u00eb n\u00eb informacionin<\/a><\/noindex> p\u00ebr fdatasync()<\/code> thuhet se gjat\u00eb pun\u00ebs s\u00eb k\u00ebtij funksioni, ruhet n\u00eb disk nj\u00eb volum metadata q\u00eb \"\u00ebsht\u00eb e nevojshme p\u00ebr kryerjen e duhur t\u00eb operacioneve t\u00eb ardhshme t\u00eb leximit t\u00eb t\u00eb dh\u00ebnave\". Dhe kjo \u00ebsht\u00eb ajo q\u00eb i shqet\u00ebson shumic\u00ebn e aplikacioneve.<\/p>\n

Nj\u00eb nga problemet q\u00eb mund t\u00eb lindin k\u00ebtu \u00ebsht\u00eb se k\u00ebto mekanizma nuk garantojn\u00eb q\u00eb skedari mund t\u00eb zbulohet pas nj\u00eb d\u00ebshtimi t\u00eb mundsh\u00ebm. N\u00eb ve\u00e7anti, kur krijohet nj\u00eb skedar i ri, \u00ebsht\u00eb e nevojshme t\u00eb thirret fsync()<\/code> p\u00ebr direktorin\u00eb q\u00eb e p\u00ebrmban at\u00eb. Ndryshe, pas d\u00ebshtimit mund t\u00eb ndodh\u00eb q\u00eb ky skedar t\u00eb mos ekzistoj\u00eb. Arsyeja p\u00ebr k\u00ebt\u00eb \u00ebsht\u00eb se n\u00eb UNIX, p\u00ebr shkak t\u00eb p\u00ebrdorimit t\u00eb lidhjeve t\u00eb forta, skedari mund t\u00eb ekzistoj\u00eb n\u00eb disa direktori. Prandaj, kur thirret fsync()<\/code> nuk ka m\u00ebnyr\u00eb p\u00ebr t\u00eb zbuluar se cilit direktorje i p\u00ebrkasin t\u00eb dh\u00ebnat gjithashtu duhet t\u00eb shkarkohet n\u00eb disk (k\u00ebtu<\/a><\/noindex> mund t\u00eb lexoni m\u00eb shum\u00eb rreth k\u00ebsaj). Duket se sistema skedari ext4 \u00ebsht\u00eb n\u00eb gjendje automatikisht<\/a><\/noindex> t\u00eb aplikoni fsync()<\/code> t\u00eb ndihmoj\u00eb me direktor\u00ebt q\u00eb p\u00ebrmbajn\u00eb skedar\u00ebt p\u00ebrkat\u00ebs, por n\u00eb rastin e sistemeve t\u00eb tjera t\u00eb skedareve, kjo mund t\u00eb mos jet\u00eb e till\u00eb.<\/p>\n

Ky mekaniz\u00ebm mund t\u00eb jet\u00eb implementuar ndryshe n\u00eb sisteme t\u00eb ndryshme skedari. Kam p\u00ebrdorur blktrace<\/a><\/noindex> p\u00ebr t\u00eb zbuluar se cilat operacione disku p\u00ebrdoren n\u00eb sistemet e skedar\u00ebve ext4 dhe XFS. T\u00eb dy japin komandat e zakonshme p\u00ebr shkrimin n\u00eb disk p\u00ebr p\u00ebrmbajtjen e skedar\u00ebve dhe p\u00ebr jurnalin e sistemit t\u00eb skedar\u00ebve, shkarkojn\u00eb cache-n dhe p\u00ebrfundojn\u00eb pun\u00ebn, duke kryer shkrim FUA (Accessi i Nj\u00ebsis\u00eb Forcuar, shkrimi i t\u00eb dh\u00ebnave direkt n\u00eb disk, duke anashkaluar cache) n\u00eb jurnal. Ndoshta, ata veprojn\u00eb n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb p\u00ebr t\u00eb konfirmuar faktin e kryerjes s\u00eb operacionit. N\u00eb disk\u00ebt q\u00eb nuk mb\u00ebshtesin FUA, kjo shkakton dy shkarkime t\u00eb caches. Eksperimentet e mia treguan se fdatasync()<\/code> pak m\u00eb shpejt fsync()<\/code>. Utilitari blktrace<\/code> tregon se fdatasync()<\/code> zakonisht shkruan m\u00eb pak t\u00eb dh\u00ebna n\u00eb disk (n\u00eb ext4 fsync()<\/code> shkruan 20 KiB, nd\u00ebrsa fdatasync()<\/code> \u2014 16 KiB). P\u00ebr m\u00eb tep\u00ebr, kam zbuluar se XFS \u00ebsht\u00eb pak m\u00eb i shpejt\u00eb se ext4. Dhe k\u00ebtu me ndihm\u00ebn e blktrace<\/code> kam arritur t\u00eb zbuloj se fdatasync()<\/code> shkarkon m\u00eb pak t\u00eb dh\u00ebna n\u00eb disk (4 KiB n\u00eb XFS).<\/p>\n

Situatat e paqart\u00eb q\u00eb ndodhin kur p\u00ebrdoret fsync()<\/h2>\n

\nMund t\u00eb p\u00ebrmend tri situata t\u00eb paqarta, q\u00eb lidhen me fsync()<\/code>, me t\u00eb cilat jam p\u00ebrballur n\u00eb praktik\u00eb.<\/p>\n

Rasti i par\u00eb ndodhi n\u00eb vitin 2008. At\u00ebher\u00eb, nd\u00ebrfaqja e Firefox 3 \u201cngadal\u00ebsohej\u201d n\u00ebse po b\u00ebhej shkrim n\u00eb disk t\u00eb nj\u00eb numri t\u00eb madh skedari. Problemi ishte se n\u00eb implementimin e nd\u00ebrfaqes p\u00ebr ruajtjen e informacionit t\u00eb saj t\u00eb gjendjes ishte p\u00ebrdorur nj\u00eb baz\u00eb t\u00eb dh\u00ebnash SQLite. Pas \u00e7do ndryshimi q\u00eb ndodhte n\u00eb nd\u00ebrfaqe, thirrej funksioni fsync()<\/code>, i cili jepte garanci t\u00eb mira p\u00ebr ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave. N\u00eb sistemin e skedar\u00ebve ext3 n\u00eb p\u00ebrdorim asokohe, funksioni fsync()<\/code> shkarkonte n\u00eb disk t\u00eb gjitha \u201cfaqsat e ndotura\u201d n\u00eb sistem, dhe jo vet\u00ebm ato q\u00eb kishin lidhje me skedarin p\u00ebrkat\u00ebs. Kjo do t\u00eb thoshte se nj\u00eb klikim mbi butonin n\u00eb Firefox mund t\u00eb iniciativ\u00ebsonte shkrimin e megabajt-eve t\u00eb t\u00eb dh\u00ebnave n\u00eb diskun magnetik, \u00e7ka mund t\u00eb zgjas\u00eb disa sekonda. Zgjidhja e problemit, sa kam kuptuar nga ky<\/a><\/noindex> materiali, konsistonte n\u00eb zhvendosjen e pun\u00ebs me baz\u00ebn e t\u00eb dh\u00ebnave n\u00eb detyra asinkrone n\u00eb sfond. Kjo do t\u00eb thoshte se m\u00eb par\u00eb n\u00eb Firefox ishin implementuar k\u00ebrkesa m\u00eb rigoroze p\u00ebr ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave, sesa ishte realisht e nevojshme, dhe ve\u00e7orit\u00eb e sistemit t\u00eb skedar\u00ebve ext3 vet\u00ebm e kishin p\u00ebrkeq\u00ebsuar k\u00ebt\u00eb problem.<\/p>\n

E dyta ndodhi n\u00eb vitin 2009. At\u00ebher\u00eb, pas nj\u00eb d\u00ebshtimi t\u00eb sistemit, p\u00ebrdoruesit e sistemit t\u00eb ri t\u00eb skedar\u00ebve ext4 u p\u00ebrball\u00ebn me faktin se shum\u00eb skedar\u00eb t\u00eb sapokrijuar kishin gjat\u00ebsi zero, nd\u00ebrsa me sistemin e vjet\u00ebr t\u00eb skedar\u00ebve ext3 ndodhi ndryshe. N\u00eb paragrafin e kaluar thash\u00eb se ext3 shkarkonte shum\u00eb t\u00eb dh\u00ebna n\u00eb disk, gj\u00eb q\u00eb ngadal\u00ebsonte pun\u00ebn fsync()<\/code>. P\u00ebr t\u00eb p\u00ebrmir\u00ebsuar situat\u00ebn, n\u00eb ext4 shkarkohen n\u00eb disk vet\u00ebm faqet \u201ce ndotura\u201d q\u00eb lidhen me skedarin specifik. T\u00eb dh\u00ebnat e skedar\u00ebve t\u00eb tjer\u00eb mbeten n\u00eb memorie p\u00ebr nj\u00eb periudh\u00eb shum\u00eb m\u00eb t\u00eb gjat\u00eb kohore se sa duke p\u00ebrdorur ext3. Kjo u b\u00eb p\u00ebr t\u00eb p\u00ebrmir\u00ebsuar performanc\u00ebn (sip\u00ebrfaq\u00ebsisht t\u00eb dh\u00ebnat q\u00ebndrojn\u00eb n\u00eb k\u00ebt\u00eb gjendje 30 sekonda, kjo mund t\u00eb rregullohet p\u00ebrmes dirty_expire_centisecs<\/a><\/noindex>; k\u00ebtu<\/a><\/noindex> mund t\u00eb gjenden materiale shtes\u00eb rreth k\u00ebsaj). Kjo do t\u00eb thoshte se nj\u00eb volum i madh t\u00eb dh\u00ebnash mund t\u00eb humbas\u00eb p\u00ebrfundimisht pas nj\u00eb d\u00ebshtimi. Zgjidhja e k\u00ebtij problemi \u00ebsht\u00eb p\u00ebrdorimi i fsync()<\/code> n\u00eb aplikacionet q\u00eb kan\u00eb nevoj\u00eb t\u00eb sigurojn\u00eb ruajtje t\u00eb q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave dhe t'i mbrojn\u00eb maksimalisht ato nga pasojat e d\u00ebshtimeve. Funksioni fsync()<\/code> punon n\u00eb p\u00ebrdorimin e ext4 shum\u00eb m\u00eb efektivisht se n\u00eb p\u00ebrdorimin e ext3. Disavantazhi i k\u00ebtij qasje \u00ebsht\u00eb se aplikimi i saj, si m\u00eb par\u00eb, ngadal\u00ebson ekzekutimin e disa operacioneve, si instalimi i programeve. Detajet rreth k\u00ebsaj shihni k\u00ebtu<\/a><\/noindex> dhe k\u00ebtu<\/a><\/noindex>.<\/p>\n

Problemi i tret\u00eb q\u00eb lidhet me fsync()<\/code>, ndodhi n\u00eb vitin 2018. At\u00ebher\u00eb, n\u00eb kuad\u00ebr t\u00eb projektit PostgreSQL, u zbulua se n\u00ebse funksioni fsync()<\/code> n\u00eb rast t\u00eb nj\u00eb gabimi, e sh\u00ebnon \u201cfaqet e ndotura\u201d si \u201ct\u00eb pastra\u201d. Si rezultat, thirrjet e m\u00ebpasshme fsync()<\/code> nuk b\u00ebjn\u00eb asgj\u00eb me k\u00ebto faqe. P\u00ebr k\u00ebt\u00eb arsye, faqet e modifikuara mbahen n\u00eb memorie dhe kurr\u00eb nuk shkarkohen n\u00eb disk. Kjo \u00ebsht\u00eb nj\u00eb katastrof\u00eb e v\u00ebrtet\u00eb, pasi aplikacioni do t\u00eb mendoj\u00eb se disa t\u00eb dh\u00ebna jan\u00eb shkarkuar n\u00eb disk, nd\u00ebrsa n\u00eb t\u00eb v\u00ebrtet\u00eb kjo do t\u00eb mos jet\u00eb ashtu. K\u00ebto d\u00ebshtime fsync()<\/code> ndodhin rrall\u00eb, aplikacioni n\u00eb k\u00ebto situata gati nuk mund t\u00eb b\u00ebj\u00eb asgj\u00eb p\u00ebr t\u00eb luftuar problemin. N\u00eb dit\u00ebt e sotme, kur ndodh kjo, PostgreSQL dhe aplikacione t\u00eb tjera dalin menj\u00ebher\u00eb nga puna. K\u00ebtu<\/a><\/noindex>, n\u00eb materialin \"A mund t\u00eb rikuperohen aplikacionet nga d\u00ebshtimet e fsync?\", kjo \u00e7\u00ebshtje studiohet n\u00eb t\u00eb gjitha detajet. Aktualisht, zgjidhja m\u00eb e mir\u00eb p\u00ebr k\u00ebt\u00eb problem \u00ebsht\u00eb p\u00ebrdorimi i Direct I\/O me flagun O_SYNC<\/code> ose me flagun O_DSYNC<\/code>. Me k\u00ebt\u00eb qasje, sistemi do t\u00eb njoftoj\u00eb p\u00ebr gabimet q\u00eb mund t\u00eb ndodhin gjat\u00eb kryerjes s\u00eb operacioneve specifike t\u00eb shkrimit t\u00eb t\u00eb dh\u00ebnave, por ky qasje k\u00ebrkon q\u00eb aplikacioni t\u00eb administroj\u00eb vet\u00eb buffers. Detaje mbi k\u00ebt\u00eb lexoni k\u00ebtu<\/a><\/noindex> dhe k\u00ebtu<\/a><\/noindex>.<\/p>\n

Hapja e skedar\u00ebve duke p\u00ebrdorur flagjet O_SYNC dhe O_DSYNC<\/h2>\n

\nT\u00eb kthehemi n\u00eb diskutimin e mekanizmave Linux q\u00eb ofrojn\u00eb ruajtje t\u00eb q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave. Sakt\u00ebsisht, b\u00ebhet fjal\u00eb p\u00ebr p\u00ebrdorimin e flagut O_SYNC<\/code> ose flagut O_DSYNC<\/code> kur hapni skedar\u00eb duke p\u00ebrdorur thirrjen sistemore open()<\/a><\/noindex>. Me k\u00ebt\u00eb qasje, \u00e7do operacion shkrimi t\u00eb dh\u00ebnash kryhet sikur pas \u00e7do komande write()<\/code> sistemit i jepen, p\u00ebrkat\u00ebsisht, komandat fsync()<\/code> dhe fdatasync()<\/code>. N\u00eb specifikimeve POSIX<\/a><\/noindex> kjo quhet \"P\u00ebrfundimi i Integritetit t\u00eb Skedarit t\u00eb I\/O t\u00eb Sinkronizuar\" dhe \"P\u00ebrfundimi i Integritetit t\u00eb t\u00eb Dh\u00ebnave\". Avantazhi kryesor i k\u00ebtij qasjes \u00ebsht\u00eb se p\u00ebr t\u00eb siguruar integritetin e t\u00eb dh\u00ebnave, duhet t\u00eb kryhet vet\u00ebm nj\u00eb thirrje sistemore, e jo dy (p.sh. - write()<\/code> dhe fdatasync()<\/code>). Disavantazhi kryesor i k\u00ebtij qasjeje \u00ebsht\u00eb se t\u00eb gjitha operacionet e shkrimit q\u00eb p\u00ebrdorin skedarin p\u00ebrkat\u00ebs do t\u00eb jen\u00eb t\u00eb sinkronizuara, gj\u00eb q\u00eb mund t\u00eb kufizoj\u00eb mund\u00ebsit\u00eb p\u00ebr strukturuar kodin e aplikacionit.<\/p>\n

P\u00ebrdorimi i Direct I\/O me flagun O_DIRECT<\/h2>\n

\nThirrja e sistemit open()<\/code> mb\u00ebshtet flagun O_DIRECT<\/code>, i cili \u00ebsht\u00eb menduar p\u00ebr t\u00eb realizuar operacione input-output duke anashkaluar nd\u00ebrfaqen e memories s\u00eb sistemit, duke nd\u00ebrvepruar drejtp\u00ebrdrejt me diskun. Kjo, n\u00eb shum\u00eb raste, do t\u00eb thot\u00eb se komandat e shkrimit t\u00eb l\u00ebshuara nga programi do t\u00eb p\u00ebrkthehen drejtp\u00ebrdrejt n\u00eb komanda t\u00eb orientuara ndaj diskut. Por, n\u00eb p\u00ebrgjith\u00ebsi, ky mekaniz\u00ebm nuk \u00ebsht\u00eb nj\u00eb z\u00ebvend\u00ebsim p\u00ebr funksionet fsync()<\/code> ose fdatasync()<\/code>. Problemi \u00ebsht\u00eb se disku vet\u00eb mund t\u00eb vonoj\u00eb ose t\u00eb ruaj\u00eb<\/a><\/noindex> komandat p\u00ebrkat\u00ebse t\u00eb shkrimit t\u00eb t\u00eb dh\u00ebnave. Dhe, \u00e7far\u00eb \u00ebsht\u00eb edhe m\u00eb keq, n\u00eb disa raste t\u00eb ve\u00e7anta, operacionet e input-output q\u00eb realizohen duke p\u00ebrdorur flagun O_DIRECT<\/code>, p\u00ebrkthehen<\/a><\/noindex> n\u00eb operacione tradicionale me buffered. Problemi leht\u00eb mund t\u00eb zgjidhet duke p\u00ebrdorur edhe flagun O_DSYNC<\/code>, gj\u00eb q\u00eb do t\u00eb n\u00ebnkuptonte se pas \u00e7do operacioni shkrimi do t\u00eb kishte nj\u00eb thirrje fdatasync()<\/code>.<\/p>\n

Doli se n\u00eb sistemin e skedar\u00ebve XFS, s\u00eb fundmi \u00ebsht\u00eb shtuar nj\u00eb \"rrug\u00eb e shpejt\u00eb\" p\u00ebr O_DIRECT|O_DSYNC<\/code>-shkrimin e t\u00eb dh\u00ebnave. N\u00ebse nj\u00eb bllok rip\u00ebrs\u00ebritet duke p\u00ebrdorur O_DIRECT|O_DSYNC<\/code>, at\u00ebher\u00eb XFS, n\u00eb vend q\u00eb t\u00eb zbraz\u00eb memorjen, do t\u00eb kryej\u00eb komand\u00ebn FUA-shkrimi vet\u00ebm n\u00eb rast se pajisja e mb\u00ebshtet. E verifikova k\u00ebt\u00eb duke p\u00ebrdorur utilitarin blktrace<\/code> n\u00eb sistemin Linux 5.4\/Ubuntu 20.04. Kjo qasje duhet t\u00eb jet\u00eb m\u00eb efikase, pasi gjat\u00eb saj shkruhen pak sa m\u00eb shum\u00eb t\u00eb dh\u00ebna n\u00eb disk dhe p\u00ebrdoret nj\u00eb operacion, e jo dy (shkrimi dhe zbrazja e memorjes). Gjeta nj\u00eb lidhje n\u00eb patch<\/a><\/noindex> n\u00eb kernelin e vitit 2018, ku \u00ebsht\u00eb realizuar ky mekaniz\u00ebm. Aty ka nj\u00eb diskutim n\u00eb lidhje me aplikimin e k\u00ebsaj optimizimi edhe n\u00eb sisteme t\u00eb tjera skedar\u00ebsh, por deri ku di un\u00eb, XFS \u00ebsht\u00eb p\u00ebr momentin sistemi i vet\u00ebm i skedar\u00ebve q\u00eb e mb\u00ebshtet k\u00ebt\u00eb.<\/p>\n

Funksioni sync_file_range()<\/h2>\n

\nN\u00eb Linux ka nj\u00eb thirrje sistemore sync_file_range()<\/a><\/noindex>, e cila lejon zbrazjen n\u00eb disk vet\u00ebm t\u00eb nj\u00eb pjese t\u00eb skedarit, e jo t\u00eb gjith\u00eb skedarit. Kjo thirrje iniciaton nj\u00eb zbrazje asinkrone t\u00eb t\u00eb dh\u00ebnave dhe nuk pret p\u00ebr p\u00ebrfundimin e saj. Por n\u00eb sh\u00ebnimin e sync_file_range()<\/code> thuhet se kjo komand\u00eb \u00ebsht\u00eb \"shum\u00eb e rrezikshme\". P\u00ebrdorimi i saj nuk rekomandohet. Ve\u00e7orit\u00eb dhe rreziqet sync_file_range()<\/code> jan\u00eb p\u00ebrshkruar shum\u00eb mir\u00eb n\u00eb k\u00ebt\u00eb<\/a><\/noindex> material. N\u00eb ve\u00e7anti, duket se kjo thirrje p\u00ebrdor RocksDB p\u00ebr t\u00eb menaxhuar se kur b\u00ebrthama zbraz \"t\u00eb dh\u00ebnat e ndotura\" n\u00eb disk. Por gjithashtu aty, p\u00ebr t\u00eb siguruar ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave, p\u00ebrdoret edhe fdatasync()<\/code>. N\u00eb kodin<\/a><\/noindex> RocksDB ka komente interesante n\u00eb k\u00ebt\u00eb tem\u00eb. P\u00ebr shembull, duket se thirrja sync_file_range()<\/code> duke p\u00ebrdorur ZFS nuk \u00e7on n\u00eb zbrazjen e t\u00eb dh\u00ebnave n\u00eb disk. Eksperienca m\u00eb tregon se kodi, i cili p\u00ebrdoret rrall\u00eb, mund t\u00eb p\u00ebrmbaj\u00eb gabime. Prandaj do t\u00eb rekomandoja q\u00eb t\u00eb mos e p\u00ebrdorni k\u00ebt\u00eb thirrje sistemore pa nj\u00eb nevoj\u00eb t\u00eb madhe.<\/p>\n

Thirrjet sistemore q\u00eb ndihmojn\u00eb n\u00eb sigurimin e ruajtjes s\u00eb q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave<\/h2>\n

\nArrita n\u00eb p\u00ebrfundimin se p\u00ebr t\u00eb realizuar operacione input-output q\u00eb sigurojn\u00eb ruajtjen e q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave, mund t\u00eb p\u00ebrdoren tre qasje. T\u00eb gjitha ato k\u00ebrkojn\u00eb thirrjen e funksionit fsync()<\/code> p\u00ebr direktorine ku \u00ebsht\u00eb krijuar skedari. K\u00ebto jan\u00eb k\u00ebto qasje:<\/p>\n

    \n
  1. Thirrja e funksionit fdatasync()<\/code> ose fsync()<\/code> pas funksionit write()<\/code> (m\u00eb mir\u00eb t\u00eb p\u00ebrdoret fdatasync()<\/code>).<\/li>\n
  2. Puna me skedarin e hapur me flagun O_DSYNC<\/code> ose O_SYNC<\/code> (m\u00eb mir\u00eb \u2014 me flagun O_DSYNC<\/code>).<\/li>\n
  3. P\u00ebrdorimi i komand\u00ebs pwritev2()<\/code> me flamur RWF_DSYNC<\/code> ose RWF_SYNC<\/code> (preferohet \u2014 me flagun RWF_DSYNC<\/code>).<\/li>\n<\/ol>\n

    <\/p>\n

    Sh\u00ebnime mbi performanc\u00ebn<\/h2>\n

    \nNuk e kam b\u00ebr\u00eb nj\u00eb matje t\u00eb kujdesshme t\u00eb performanc\u00ebs s\u00eb mekanizmave t\u00eb ndrysh\u00ebm q\u00eb kam shqyrtuar. Diferencat e v\u00ebrejtura nga un\u00eb n\u00eb shpejt\u00ebsin\u00eb e funksionimit t\u00eb tyre jan\u00eb t\u00eb vogla. Kjo do t\u00eb thot\u00eb se mund t\u00eb kem gabime, dhe se n\u00ebn kushte t\u00eb tjera, t\u00eb nj\u00ebjt\u00ebn gj\u00eb mund t\u00eb tregoj\u00eb rezultate t\u00eb tjera. S\u00eb pari, do t\u00eb flas p\u00ebr ato q\u00eb kan\u00eb nj\u00eb ndikim m\u00eb t\u00eb madh n\u00eb performanc\u00eb dhe pastaj p\u00ebr ato q\u00eb ndikojn\u00eb m\u00eb pak.<\/p>\n

      \n
    1. Rikthimi i t\u00eb dh\u00ebnave t\u00eb skedarit \u00ebsht\u00eb m\u00eb i shpejt\u00eb se sa bashkimi i t\u00eb dh\u00ebnave n\u00eb nj\u00eb skedar (fitimi n\u00eb performanc\u00eb mund t\u00eb jet\u00eb 2-100%). Bashkimi i t\u00eb dh\u00ebnave n\u00eb nj\u00eb skedar k\u00ebrkon ndryshime shtes\u00eb n\u00eb metadatat e skedarit, madje edhe pas thirrjes s\u00eb sistemit. fallocate()<\/code>, por shkalla e k\u00ebtij efekti mund t\u00eb ndryshoj\u00eb. Un\u00eb rekomandoj q\u00eb p\u00ebr t\u00eb siguruar performanc\u00ebn m\u00eb t\u00eb mir\u00eb, t\u00eb thirret fallocate()<\/code> p\u00ebr t\u00eb rezervuar hap\u00ebsir\u00ebn e nevojshme. M\u00eb pas, kjo hap\u00ebsir\u00eb duhet t\u00eb mbushet qart\u00eb me zero dhe t\u00eb thirret fsync()<\/code>. Fal\u00eb k\u00ebsaj, blloqet p\u00ebrkat\u00ebse n\u00eb sistemin e skedar\u00ebve do t\u00eb sh\u00ebnohen si \"t\u00eb rezervuara\", jo si \"t\u00eb pa rezervuara\". Kjo ofron nj\u00eb p\u00ebrmir\u00ebsim t\u00eb vog\u00ebl (rreth 2%) n\u00eb performanc\u00eb. P\u00ebr m\u00eb tep\u00ebr, disa disqe mund t\u00eb ken\u00eb operacionin e par\u00eb t\u00eb akseses n\u00eb bllok m\u00eb t\u00eb ngadalsh\u00ebm se t\u00eb tjerat. Kjo do t\u00eb thot\u00eb se mbushja e hap\u00ebsir\u00ebs me zero mund t\u00eb \u00e7oj\u00eb n\u00eb nj\u00eb p\u00ebrmir\u00ebsim t\u00eb konsideruesh\u00ebm (rreth 100%) n\u00eb performanc\u00eb. Kjo mund t\u00eb ndodh\u00eb ve\u00e7an\u00ebrisht me disqet AWS EBS<\/a><\/noindex> (kjo \u00ebsht\u00eb informacion jozyrtar, nuk kam mundur ta konfirmoj). E nj\u00ebjta gj\u00eb vlen edhe p\u00ebr ruajtjet GCP Persistent Disk<\/a><\/noindex> (dhe kjo \u00ebsht\u00eb tashm\u00eb informacion zyrtar, i konfirmuar nga testet). Spikat\u00ebs t\u00eb tjer\u00eb kan\u00eb b\u00ebr\u00eb t\u00eb nj\u00ebjtat v\u00ebzhgime<\/a><\/noindex>, q\u00eb lidhen me disqe t\u00eb ndryshme.<\/li>\n
    2. Sa m\u00eb pak thirrje sistemike, aq m\u00eb e lart\u00eb performanca (fitimi mund t\u00eb arrij\u00eb rreth 5%). Duket se thirrja open()<\/code> me flamur O_DSYNC<\/code> ose thirrja pwritev2()<\/code> me flamur RWF_SYNC<\/code> \u00ebsht\u00eb m\u00eb e shpejt\u00eb se thirrja fdatasync()<\/code>. Dyshoj se kjo ndodh p\u00ebr shkak se me k\u00ebt\u00eb qasje, roli luhet nga fakti se p\u00ebr t\u00eb zgjidhur nj\u00eb detyr\u00eb t\u00eb nj\u00ebjt\u00eb, duhet t\u00eb kryhen m\u00eb pak thirrje sistemike (nj\u00eb thirrje n\u00eb vend t\u00eb dy). Por diferenca n\u00eb performanc\u00eb \u00ebsht\u00eb shum\u00eb e vog\u00ebl, k\u00ebshtu q\u00eb ju mund ta injoroni dhe t\u00eb p\u00ebrdorni at\u00eb q\u00eb nuk do t\u00eb \u00e7onte n\u00eb komplikuarjen e logjik\u00ebs s\u00eb aplikacionit.<\/li>\n<\/ol>\n

      \nN\u00ebse jeni t\u00eb interesuar p\u00ebr tem\u00ebn e ruajtjes s\u00eb q\u00ebndrueshme t\u00eb t\u00eb dh\u00ebnave - ja disa materiale t\u00eb dobishme:<\/p>\n