Pogwira ntchito ndi ma data akuluakulu, vuto la kusowa kwa malo a disk nthawi zina likhoza kuwuka. Njira imodzi yothetsera vutoli ndi kuponderezana, chifukwa chake, pazida zomwezo, mutha kukwanitsa kuwonjezera kuchuluka kosungirako. M'nkhaniyi, tiwona momwe kuponderezana kwa data kumagwirira ntchito ku Apache Ignite. Nkhaniyi ingofotokoza njira zophatikizira za disk zomwe zakhazikitsidwa mkati mwazogulitsa. Njira zina zophatikizira deta (pa netiweki, kukumbukira), kaya zakhazikitsidwa kapena ayi, sizikhalabe pamlingo.
Chifukwa chake, ndi kulimbikira kumathandizidwa, chifukwa cha kusintha kwa data mu cache, Ignite imayamba kulembera ku diski:
- Zomwe zili mu cache
- Lembani Patsogolo Logi (pambuyo pake WAL)
Pakhala pali makina a WAL compression kwa nthawi yayitali, yotchedwa WAL compaction. Apache Ignite 2.8 yomwe yatulutsidwa posachedwapa inayambitsa njira zina ziwiri zomwe zimakulolani kuti musindikize deta pa disk: kupondaponda kwa tsamba la disk pokanikiza zomwe zili mu cache ndi WAL tsamba lachithunzithunzi la tsamba la WAL kukakamiza zolemba zina za WAL. Zambiri za njira zitatuzi pansipa.
Tsitsani tsamba la Disk
Kodi ntchito
Choyamba, tiyeni tiwone mwachidule momwe Ignite imasungira deta. Kukumbukira kwatsamba kumagwiritsidwa ntchito posungira. Kukula kwa tsamba kumayikidwa koyambirira kwa node ndipo sikungasinthidwe pambuyo pake; komanso, kukula kwa tsamba kuyenera kukhala mphamvu ziwiri komanso kuchuluka kwa block system yamafayilo. Masamba amalowetsedwa mu RAM kuchokera pa disk ngati pakufunika; kukula kwa data pa disk kumatha kupitilira kuchuluka kwa RAM yomwe yaperekedwa. Ngati palibe malo okwanira mu RAM kuti muyike tsamba kuchokera pa disk, masamba akale, osagwiritsidwanso ntchito adzachotsedwa ku RAM.
Deta imasungidwa pa diski motere: fayilo yosiyana imapangidwa pagawo lililonse la gulu la cache; mufayilo iyi, masamba amawonekera limodzi ndi linzake pokwera index. Tsamba lathunthu lili ndi chizindikiritso cha gulu la cache, nambala yogawa, ndi index yamasamba mufayilo. Chifukwa chake, pogwiritsa ntchito chizindikiritso cha masamba onse, titha kudziwa mwapadera fayilo ndi zomwe zili mufayilo yatsamba lililonse. Mutha kuwerenga zambiri za kukumbukira papage mu nkhani ya Apache Ignite Wiki: .
Makina osindikizira a disk page, monga momwe mungaganizire kuchokera ku dzina, amagwira ntchito pa tsamba. Makinawa akayatsidwa, deta mu RAM imasinthidwa monga momwe ilili, popanda kukakamiza, koma masamba akasungidwa kuchokera ku RAM kupita ku diski, amapanikizidwa.
Koma kukanikiza tsamba lililonse palokha si njira yothetsera vutoli; muyenera kuchepetsa kukula kwa mafayilo omwe atsatira. Ngati kukula kwatsamba sikunakhazikitsidwenso, sitingathenso kulemba masamba ku fayilo imodzi ndi ina, chifukwa izi zitha kuyambitsa mavuto angapo:
- Pogwiritsa ntchito mndandanda wamasamba, sitingathe kuwerengera momwe zilili mufayilo.
- Sizidziwikiratu zoyenera kuchita ndi masamba omwe sali kumapeto kwa fayilo ndikusintha kukula kwake. Ngati tsambalo likucheperachepera, malo omwe adamasulidwawo amatha. Ngati tsamba likuwonjezeka, muyenera kuyang'ana malo atsopano mu fayilo yake.
- Ngati tsamba likuyenda ndi ma byte angapo omwe sali kuchuluka kwa block system yamafayilo, ndiye kuti kuwerenga kapena kulemba pamafunika kukhudza chipika chimodzi cha fayilo, chomwe chingayambitse kuwonongeka kwa magwiridwe antchito.
Kuti mupewe kuthetsa mavutowa pamlingo wake womwe, kukanikiza kwa tsamba la disk mu Apache Ignite kumagwiritsa ntchito makina amafayilo otchedwa sparse files. Fayilo yocheperako ndi imodzi yomwe zigawo zodzaza ziro zimatha kulembedwa ngati "mabowo". Pankhaniyi, palibe midadada yamafayilo omwe adzapatsidwe kuti asungire mabowowa, zomwe zimapangitsa kuti pakhale ndalama pa disk space.
Ndizomveka kuti kuti mumasulire chipika cha fayilo, kukula kwa dzenje kuyenera kukhala kwakukulu kuposa kapena kofanana ndi chipika cha fayilo, chomwe chimayika malire owonjezera pa kukula kwa tsamba ndi Apache Ignite: kuti kuponderezana kukhale ndi zotsatira zilizonse, kukula kwatsamba kuyenera kukhala kokulirapo kuposa kukula kwa chipika cha fayilo. Ngati kukula kwa tsamba kuli kofanana ndi kukula kwa chipika, ndiye kuti sitingathe kumasula chipika chimodzi, chifukwa kuti timasule chipika chimodzi, tsamba lopanikizidwa liyenera kukhala ndi ma byte 0. Ngati kukula kwa tsamba kuli kofanana ndi kukula kwa midadada 2 kapena 4, titha kale kumasula chipika chimodzi ngati tsamba lathu litapanikizidwa mpaka 50% kapena 75% motsatana.
Chifukwa chake, kufotokozera komaliza kwa momwe makinawo amagwirira ntchito: Polemba tsamba ku diski, kuyesa kumapangidwa kukakamiza tsambalo. Ngati kukula kwa tsamba lopanikizidwa kumalola kuti fayilo imodzi kapena zingapo zimasulidwe, ndiye kuti tsambalo limalembedwa mu mawonekedwe oponderezedwa, ndipo "bowo" limapangidwa m'malo mwa midadada yomasulidwa (kuyimba kwadongosolo kumachitidwa. fallocate() ndi mbendera ya nkhonya). Ngati kukula kwa tsamba lopanikizidwa sikulola kuti midadada imasulidwe, tsambalo limasungidwa monga momwe liliri, losapindika. Masamba onse amawerengeredwa mofanana ndi popanda kukanikiza, mwa kuchulukitsa mndandanda wamasamba ndi kukula kwa tsamba. Palibe kusamutsa masamba komwe kumafunikira panokha. Zosintha zamasamba, monga popanda kukakamiza, zimagwera pamalire a midadada yamafayilo.
Pakukhazikitsa kwapano, Ignite imatha kugwira ntchito ndi mafayilo ochepa pansi pa Linux OS; motere, kukanikiza kwa tsamba la disk kumatha kuthandizidwa mukamagwiritsa ntchito Ignite pa makina ogwiritsira ntchito.
Ma aligorivimu ophatikizika omwe angagwiritsidwe ntchito pakupopera masamba a disk: ZSTD, LZ4, Snappy. Kuphatikiza apo, pali njira yogwiritsira ntchito (SKIP_GARBAGE), momwe malo osagwiritsidwa ntchito patsambalo amaponyedwa kunja osagwiritsa ntchito kukanikiza pa data yotsalayo, yomwe imachepetsa katundu pa CPU poyerekeza ndi ma aligorivimu omwe adalembedwa kale.
Zokhudza Kuchita
Tsoka ilo, sindinachite zoyezera zenizeni pamayimidwe enieni, chifukwa sitikukonzekera kugwiritsa ntchito makinawa popanga, koma titha kuganiza mozama komwe tidzaluza komanso komwe tidzapambane.
Kuti tichite izi, tiyenera kukumbukira momwe masamba amawerengedwa ndi kulembedwa akafika:
- Mukamachita ntchito yowerengera, imafufuzidwa koyamba mu RAM; ngati kusaka sikunapambane, tsambalo limayikidwa mu RAM kuchokera ku disk ndi ulusi womwewo womwe umawerengera.
- Ntchito yolemba ikachitika, tsamba la RAM limalembedwa ngati lakuda, koma tsambalo silimasungidwa ku diski nthawi yomweyo ndi ulusi womwe ukulemba. Masamba onse odetsedwa amasungidwa ku diski pambuyo pake munjira yoyang'anira mu ulusi wosiyana.
Chifukwa chake zotsatira zake pakuwerenga ntchito ndi:
- Positive (disk IO), chifukwa cha kuchepa kwa chiwerengero cha midadada yowerengera mafayilo.
- Negative (CPU), chifukwa cha katundu wowonjezera wofunikira ndi makina ogwiritsira ntchito kuti agwire ntchito ndi mafayilo ochepa. Ndizothekanso kuti ma IO owonjezera adzawonekera apa kuti asungire mafayilo ocheperako (mwatsoka, sindikudziwa tsatanetsatane wa momwe mafayilo ochepa amagwirira ntchito).
- Negative (CPU), chifukwa chofuna kutsitsa masamba.
- Palibe chokhudza kulemba ntchito.
- Zokhudza ntchito yoyang'anira (chilichonse apa chikufanana ndi ntchito zowerengera):
- Positive (disk IO), chifukwa cha kuchepa kwa chiwerengero cha mafayilo olembedwa.
- Negative (CPU, mwina disk IO), chifukwa chogwira ntchito ndi mafayilo ochepa.
- Negative (CPU), chifukwa chofuna kusindikiza masamba.
Ndi mbali iti ya sikelo yomwe ingapendeke sikelo? Zonsezi zimatengera chilengedwe, koma ndimakonda kukhulupirira kuti kupanikizana kwa tsamba la disk kungayambitse kuwonongeka kwa machitidwe ambiri. Kuphatikiza apo, mayeso pa ma DBMS ena omwe amagwiritsa ntchito njira yofananira ndi mafayilo ochepera amawonetsa kutsika kwa magwiridwe antchito akamakanizidwa.
Momwe mungayambitsire ndikusintha
Monga tafotokozera pamwambapa, mtundu wocheperako wa Apache Ignite womwe umathandizira kupsinjika kwa tsamba la disk ndi 2.8 ndipo makina ogwiritsira ntchito a Linux okha ndi omwe amathandizidwa. Yambitsani ndi kukonza motere:
- Payenera kukhala gawo la kuponderezana koyaka m'kalasi. Mwachikhazikitso, ili mu kugawa kwa Apache Ignite mu libs/chosankha chowongolera ndipo sichikuphatikizidwa munjira yakalasi. Mutha kusuntha chikwatucho kumtunda umodzi kupita ku libs ndiyeno mukachiyendetsa kudzera pa ignite.sh chidzayatsidwa.
- Kulimbikira kuyenera kuyatsidwa (Kuyatsidwa kudzera
DataRegionConfiguration.setPersistenceEnabled(true)). - Kukula kwatsamba kuyenera kukhala kokulirapo kuposa kukula kwa block system (mutha kuyiyika pogwiritsa ntchito
DataStorageConfiguration.setPageSize()). - Pa cache iliyonse yomwe deta yake ikufunika kukanikizidwa, muyenera kukonza njira yoponderezedwa ndi (mwakufuna) mulingo woponderezedwa (njira).
CacheConfiguration.setDiskPageCompression() , CacheConfiguration.setDiskPageCompressionLevel()).
Kusinthana kwa WAL
Kodi ntchito
WAL ndi chiyani ndipo chifukwa chiyani ikufunika? Mwachidule kwambiri: ichi ndi chipika chomwe chili ndi zochitika zonse zomwe pamapeto pake zimasintha kusungirako masamba. Zimafunika makamaka kuti muthe kuchira ngati wagwa. Opaleshoni iliyonse, isanapereke ulamuliro kwa wogwiritsa ntchito, iyenera kaye kulemba chochitika mu WAL, kuti ikalephera, itha kuseweredwa mu chipika ndikubwezeretsanso ntchito zonse zomwe wogwiritsa ntchitoyo adayankhira bwino, ngakhale izi zitachitika. analibe nthawi yowonetsera kusungirako masamba pa diski (kale pamwambapa Zafotokozedwa kuti kulembera kwenikweni ku sitolo yamasamba kumachitika mu njira yotchedwa "checkpointing" ndi kuchedwa kwina ndi ulusi wosiyana).
Zolemba mu WAL zimagawidwa kukhala zomveka komanso zakuthupi. Boolean ndiye makiyi ndipo amadzikonda okha. Zathupi - zikuwonetsa zosintha zamasamba omwe ali patsamba sitolo. Ngakhale zolemba zomveka zitha kukhala zothandiza pamilandu ina, zolemba zakuthupi zimafunikira kuti zibwezeretsedwe pakagwa ngozi ndipo zolembedwa zimangofunika kuyambira pomwe zidachitika bwino. Pano sitingafotokoze mwatsatanetsatane chifukwa chake zimagwira ntchito motere, koma omwe ali ndi chidwi atha kuloza ku nkhani yomwe yatchulidwa kale pa Apache Ignite Wiki: .
Nthawi zambiri pamakhala zolembedwa zingapo zomveka. Izi ndiye kuti, mwachitsanzo, kuyika kumodzi mu cache kumakhudza masamba angapo pamakumbukiro atsamba (tsamba lomwe lili ndi deta yokha, masamba okhala ndi indexes, masamba omwe ali ndi mindandanda yaulere). M'mayeso ena opangira, ndidapeza kuti zolemba zakuthupi zimakhala ndi 90% ya fayilo ya WAL. Komabe, amafunikira kwakanthawi kochepa kwambiri (mwachisawawa, nthawi yapakati pa macheke ndi mphindi 3). Zingakhale zomveka kuchotsa deta iyi pambuyo potaya kufunika kwake. Izi ndi zomwe WAL compaction mechanism imachita: imachotsa zolemba zakuthupi ndikukankhira zolemba zotsalira zotsalira pogwiritsa ntchito zip, pomwe kukula kwa fayilo kumachepetsedwa kwambiri (nthawi zina ndi makumi).
Mwakuthupi, WAL imakhala ndi zigawo zingapo (10 mwachisawawa) za kukula kokhazikika (64MB mwachisawawa), zomwe zimalembedwa mozungulira. Gawo lapano likangodzazidwa, gawo lotsatira limaperekedwa ngati lapano, ndipo gawo lodzazidwa limakopera ku archive ndi ulusi wosiyana. WAL compaction imagwira ntchito kale ndi magawo osungidwa. Komanso, monga ulusi wosiyana, imayang'anira kuchitidwa kwa cheke ndikuyamba kukanikiza m'magawo osungidwa omwe zolemba zakuthupi sizikufunikanso.
Zokhudza Kuchita
Popeza WAL compaction imayenda ngati ulusi wosiyana, sikuyenera kukhala ndi zotsatira zachindunji pazomwe zikuchitika. Koma imayikabe zowonjezera zakumbuyo pa CPU (compression) ndi disk (kuwerenga gawo lililonse la WAL kuchokera pankhokwe ndikulemba magawo oponderezedwa), kotero ngati dongosololi likuyenda pamlingo wake waukulu, zingayambitsenso kuwonongeka kwa magwiridwe antchito.
Momwe mungayambitsire ndikusintha
Mutha kuloleza WAL compaction pogwiritsa ntchito malowo WalCompactionEnabled в DataStorageConfiguration (DataStorageConfiguration.setWalCompactionEnabled(true)). Komanso, pogwiritsa ntchito njira ya DataStorageConfiguration.setWalCompactionLevel(), mutha kukhazikitsa mulingo wopondereza ngati simukukhutira ndi mtengo wokhazikika (BEST_SPEED).
Kuphatikizika kwachithunzi cha tsamba la WAL
Kodi ntchito
Tapeza kale kuti mu zolemba za WAL zimagawidwa kukhala zomveka komanso zakuthupi. Pakusintha kulikonse patsamba lililonse, mbiri ya WAL yakuthupi imapangidwa mu kukumbukira masamba. Zolemba zakuthupi, nawonso, zimagawidwa m'magulu awiri: mbiri yazithunzi za tsamba ndi mbiri ya delta. Nthawi zonse tikasintha china chake patsamba ndikuchichotsa kuchoka paukhondo kupita kumalo odetsedwa, tsamba lathunthu latsambali limasungidwa mu WAL (tsamba lachidule latsamba). Ngakhale titasintha baiti imodzi yokha mu WAL, mbiriyo idzakhala yokulirapo pang'ono kuposa kukula kwa tsamba. Ngati tisintha chinachake pa tsamba lodetsedwa kale, ndiye kuti zolemba za delta zimapangidwira mu WAL, zomwe zimasonyeza kusintha kokha poyerekeza ndi momwe tsamba lapitalo likuyendera, koma osati tsamba lonse. Popeza kukonzanso tsamba kuchokera ku zonyansa kupita ku ukhondo kumachitika panthawi yoyang'anira, atangoyamba cheke, pafupifupi zolemba zonse zakuthupi zimangokhala ndi zithunzi zamasamba (popeza masamba onse atangoyamba kumene cheke ndi oyera) , ndiye pamene tikuyandikira cheke chotsatira, kagawo kakang'ono ka delta kumayamba kukula ndikukonzanso kachiwiri kumayambiriro kwa cheke chotsatira. Kuyeza m'mayesero ena opangidwa kunawonetsa kuti gawo lazithunzi zamasamba muzolemba zonse zakuthupi zimafikira 2%.
Lingaliro la kuphatikizika kwazithunzi za WAL ndikusindikiza zithunzi zamasamba pogwiritsa ntchito chida chokonzekera masamba (onani kuphatikizika kwa tsamba la disk). Panthawi imodzimodziyo, mu WAL, zolemba zimasungidwa motsatizana muzowonjezera-zokha ndipo palibe chifukwa chomangirira zolemba kumalire a midadada yamafayilo, kotero apa, mosiyana ndi makina osindikizira a disk page, sitifunikira mafayilo ochepa zonse; motero, makinawa sagwira ntchito pa OS Linux yokha. Kuonjezera apo, zilibe kanthu kwa ife kuti tinatha kufinya bwanji tsambali. Ngakhale titamasula 1 byte, izi ndi zotsatira zabwino kale ndipo titha kusunga deta yothinikizidwa mu WAL, mosiyana ndi kuphatikizika kwa tsamba la disk, komwe timasunga tsamba lopanikizidwa pokhapokha titamasula chipika choposa 1 fayilo.
Masamba ndi deta yophatikizika kwambiri, gawo lawo mu voliyumu yonse ya WAL ndilokwera kwambiri, kotero popanda kusintha mawonekedwe a fayilo ya WAL titha kuchepetsa kwambiri kukula kwake. Kuponderezana, kuphatikizapo zolemba zomveka, kungafune kusintha kwa maonekedwe ndi kutayika kogwirizana, mwachitsanzo, kwa ogula akunja omwe angakhale ndi chidwi ndi zolemba zomveka, koma sizingabweretse kuchepa kwakukulu kwa fayilo.
Mofanana ndi kuphatikizika kwa tsamba la disk, kukakamiza kwazithunzi za WAL kutha kugwiritsa ntchito ZSTD, LZ4, Snappy compression algorithms, komanso SKIP_GARBAGE mode.
Zokhudza Kuchita
Sizovuta kuzindikira kuti kuthandizira mwachindunji kukakamiza kwazithunzi za WAL kumangokhudza ulusi womwe umalemba deta ku kukumbukira kwatsamba, ndiye kuti, ulusi womwe umasintha deta mu cache. Kuwerenga zolemba zakuthupi kuchokera ku WAL kumachitika kamodzi kokha, panthawi yomwe node imakwezedwa pambuyo pa kugwa (ndipo pokhapokha ngati itagwa panthawi yoyang'ana).
Izi zimakhudza ulusi womwe umasintha deta motere: timakhala ndi zotsatira zoipa (CPU) chifukwa cha kufunika kokanikizira tsamba nthawi zonse musanalembe ku disk, ndi zotsatira zabwino (disk IO) chifukwa cha kuchepetsa kuchuluka kwa deta yolembedwa. Choncho, chirichonse chiri chophweka apa: ngati ntchito ya dongosolo ili yochepa ndi CPU, timapeza kuwonongeka pang'ono, ngati kuli kochepa ndi disk I / O, timapeza kuwonjezeka.
Mosalunjika, kuchepetsa kukula kwa WAL kumakhudzanso (zabwino) mitsinje yomwe imataya magawo a WAL mu archive ndi WAL compaction mitsinje.
Mayesero enieni a ntchito m'malo athu pogwiritsa ntchito deta yopangidwa adawonetsa kuwonjezeka pang'ono (kuchuluka kwawonjezeka ndi 10% -15%, latency inatsika ndi 10% -15%).
Momwe mungayambitsire ndikusintha
Mtundu Wocheperako wa Apache Ignite: 2.8. Yambitsani ndi kukonza motere:
- Payenera kukhala gawo la kuponderezana koyaka m'kalasi. Mwachikhazikitso, ili mu kugawa kwa Apache Ignite mu libs/chosankha chowongolera ndipo sichikuphatikizidwa munjira yakalasi. Mutha kusuntha chikwatucho kumtunda umodzi kupita ku libs ndiyeno mukachiyendetsa kudzera pa ignite.sh chidzayatsidwa.
- Kulimbikira kuyenera kuyatsidwa (Kuyatsidwa kudzera
DataRegionConfiguration.setPersistenceEnabled(true)). - Ma compression mode ayenera kukhazikitsidwa pogwiritsa ntchito njira
DataStorageConfiguration.setWalPageCompression(), kukanikiza kumayimitsidwa mwachisawawa (DISABLED mode). - Optionally, mukhoza kukhazikitsa psinjika mlingo ntchito njira
DataStorageConfiguration.setWalPageCompression(), onani javadoc ya njira yamtengo wapatali pamtundu uliwonse.
Pomaliza
Njira zoponderezedwa za data mu Apache Ignite zitha kugwiritsidwa ntchito paokha, koma kuphatikiza kulikonse ndikovomerezeka. Kumvetsetsa momwe amagwirira ntchito kumakupatsani mwayi wodziwa momwe angagwiritsire ntchito ntchito zanu mdera lanu komanso zomwe muyenera kusiya mukazigwiritsa ntchito. Kuphatikizika kwa tsamba la Disk kudapangidwa kuti kupangitse kusungirako kwakukulu ndipo kumatha kupereka chiŵerengero chapakatikati. Kuphatikizika kwazithunzi za WAL kumapereka kupsinjika kwa mafayilo a WAL, ndipo kungapangitsenso magwiridwe antchito. WAL compaction sichidzakhala ndi zotsatira zabwino pa ntchito, koma idzachepetsa kukula kwa mafayilo a WAL momwe mungathere pochotsa zolemba zakuthupi.
Source: www.habr.com