Ekwindla ka-2019, kwenzeka umcimbi okade ulindelwe eqenjini le-Mail.ru Cloud iOS. I-database eyinhloko yokugcinwa okuqhubekayo kwesimo sohlelo lokusebenza isiphenduke into engavamile emhlabeni jikelele Isizindalwazi Esinemephu Yenkumbulo Yombani (LMDB). Ngaphansi kokusikwa, ukunaka kwakho kumenywa ekubuyekezweni kwayo okuningiliziwe ezingxenyeni ezine. Okokuqala, ake sikhulume ngezizathu zokukhetha okunjalo okungeyona into encane futhi enzima. Bese sidlulela ekucatshangelweni kwemikhomo emithathu enhliziyweni yesakhiwo se-LMDB: amafayela afakwe kumephu yenkumbulo, isihlahla se-B +, indlela yokukopisha ekubhaleni ukuze kusetshenziswe ukuthengiselana kanye nokuguqula izinto eziningi. Ekugcineni, i-dessert - ingxenye ewusizo. Kuyo, sizobheka indlela yokuklama nokusebenzisa i-schema eyisisekelo enamathebula ambalwa, okuhlanganisa nenkomba eyodwa, ngaphezulu kwe-API yenani lokhiye wezinga eliphansi.

Okuqukethwe

1. Ukuqaliswa Kokukhuthaza
2. Ukubeka i-LMDB
3. Imikhomo emithathu LMDB
   3.1. Umkhomo #1. Amafayela anemephu yenkumbulo
   3.2. Umkhomo #2. B+-isihlahla
   3.3. Umkhomo #3. kopisha-ngokubhala
4. Ukuklama i-schema sedatha phezu kwe-API yenani elingukhiye
   4.1. Izifinyezo eziyisisekelo
   4.2. Ukumodela Kwethebula
   4.3. Ukumodela ubudlelwano phakathi kwamathebula

1. Ukuqaliswa kokugqugquzela

Kanye ngonyaka, ngo-2015, sasinakekela ukuthatha imethrikhi, ukuthi kukangakhi isixhumi esibonakalayo sohlelo lwethu lokusebenza sibambezeleka. Asizange nje sikwenze lokhu. Sinezikhalazo eziningi mayelana neqiniso lokuthi kwesinye isikhathi uhlelo lokusebenza luyayeka ukuphendula ezenzweni zomsebenzisi: izinkinobho azicindezelwa, izinhlu aziskroli, njll. Mayelana nemishini yezilinganiso utshele ku-AvitoTech, ngakho-ke lapha nginikeza ukuhleleka kwezinombolo kuphela.

Imiphumela yokulinganisa yaba ishawa elibandayo kithi. Kuvele ukuthi izinkinga ezibangelwa amaqhwa ziningi kakhulu kunezinye. Uma, ngaphambi kokuqaphela leli qiniso, inkomba eyinhloko yekhwalithi yekhwalithi yayikhululekile, emva kokugxila kugudluziwe kwiqhwa mahhala.

Sewakhile ideshibhodi eneziqandisi futhi esechithe ubuningi и ikhwalithi ukuhlaziywa kwezimbangela zabo, isitha esikhulu sacaca - i-logic yebhizinisi esindayo eyenziwa emculweni oyinhloko wesicelo. Ukusabela okungokwemvelo kuleli hlazo kwaba isifiso esivuthayo sokulihlohla emisebenzini. Ukuze uthole isixazululo esihlelekile sale nkinga, siphendukele ekwakhiweni kwezintambo eziningi okusekelwe kubadlali abangasindi. Nginikezele ukujwayela kwakhe emhlabeni we-iOS imicu emibili ku-twitter ehlangene kanye isihloko ngoHabre. Njengengxenye yendaba yamanje, ngifuna ukugcizelela lezo zici zesinqumo ezibe nomthelela ekukhethweni kwesizindalwazi.

Imodeli yomlingisi wenhlangano yesistimu ithatha ukuthi i-multithreading iba ingqikithi yayo yesibili. Imodeli yezinto ezikulo zithanda ukweqa imingcele. Futhi abakwenzi lokhu ngezinye izikhathi futhi kwezinye izindawo, kodwa cishe njalo futhi yonke indawo.

Isizindalwazi singenye yezingxenye eziyisisekelo kumdwebo owethulwe. Umsebenzi wayo oyinhloko ukusebenzisa iphethini enkulu Isizindalwazi Sabiwe. Uma emhlabeni webhizinisi isetshenziselwa ukuhlela ukuvumelanisa idatha phakathi kwezinsizakalo, khona-ke esimweni sokwakhiwa komlingisi, idatha phakathi kwemicu. Ngakho-ke, besidinga isizindalwazi esinjalo, ukusebenza ngaso endaweni enemicu eminingi kungabangi ngisho nobunzima obuncane. Ikakhulukazi, lokhu kusho ukuthi izinto ezithathwe kuyo kumele okungenani ziphephe, futhi zingaguquleki nhlobo. Njengoba wazi, lokhu kokugcina kungasetshenziswa kanyekanye kusuka emicu eminingana ngaphandle kokusebenzisa noma yiluphi uhlobo lokukhiya, olunomphumela onenzuzo ekusebenzeni.

Isici sesibili esibalulekile esibe nomthelela ekukhethweni kwesizindalwazi kwakuyi-API yethu yamafu. Igqugquzelwe indlela ye-git yokuvumelanisa. Njengaye sasihlose i-API yokuqala engaxhunyiwe ku-inthanethi, okubonakala kungaphezu kokufanele amaklayenti amafu. Kwacatshangwa ukuthi bazokhipha kanye kuphela isimo esigcwele sefu, bese ukuvumelanisa kuningi lezimo kuzokwenzeka ngoshintsho oluqhubekayo. Maye, lokhu kungenzeka kusesezindaweni zethiyori kuphela, futhi ekusebenzeni, amaklayenti awafundile ukuthi angasebenza kanjani ngama-patches. Kunezizathu eziningi zenhloso yalokhu, okuthi, ukuze singabambezeli isingeniso, sizoshiya ngaphandle kubakaki. Manje okuthakazelisa kakhulu imiphumela efundisayo yesifundo mayelana nokuthi kwenzekani lapho i-API ithi "A" futhi umthengi wayo akazange athi "B".

Ngakho-ke, uma ucabanga i-git, okuthi, lapho ikhipha umyalo wokudonsa, esikhundleni sokusebenzisa ama-patches kusifinyezo sendawo, iqhathanise isimo sayo esigcwele neseva egcwele, khona-ke uzoba nombono onembile wokuthi ukuvumelanisa kanjani. kwenzeka kumaklayenti wamafu. Kulula ukuqagela ukuthi ukuze kusetshenziswe kuyadingeka ukwaba izihlahla ezimbili ze-DOM enkumbulweni ngolwazi lwe-meta mayelana nawo wonke amaseva namafayela endawo. Kuvela ukuthi uma umsebenzisi egcina amafayela ayizinkulungwane ezingu-500 efwini, bese ukuvumelanisa, kuyadingeka ukuphinda udale futhi ucekele phansi izihlahla ezimbili ezinama-node ayizigidi ezingu-1. Kodwa inodi ngayinye iyinhlanganisela equkethe igrafu yezihloko. Ngalokhu kukhanya, imiphumela yephrofayela bekulindeleke. Kwavela ukuthi ngisho ngaphandle kokucabangela i-algorithm yokuhlanganisa, yona kanye inqubo yokudala bese ucekela phansi inani elikhulu lezinto ezincane kubiza isenti elihle kakhulu. yemibhalo yomsebenzisi. Ngenxa yalokho, silungisa umbandela wesibili obalulekile ekukhetheni isizindalwazi - ikhono lokuqalisa imisebenzi ye-CRUD ngaphandle kokwabiwa okuguquguqukayo kwezinto.

Ezinye izidingo zingezendabuko, futhi uhlu lwazo olugcwele lumi kanje.

1. Ukuphepha kwentambo.
2. Ukucubungula okuningi. Kuqondiswa isifiso sokusebenzisa isibonelo sesizindalwazi esifanayo ukuvumelanisa isimo hhayi phakathi kwemicu kuphela, kodwa futhi phakathi kohlelo lokusebenza oluyinhloko nezandiso ze-iOS.
3. Ikhono lokumela amabhizinisi agciniwe njengezinto ezingaguquleki
4. Ukushoda kwezabelo eziguqukayo ngaphakathi kwemisebenzi ye-CRUD.
5. Ukwesekwa Kokwenziwe Kwezakhiwo Eziyisisekelo I-ACIDAmagama angukhiye: i-athomu, ukungaguquguquki, ukuzihlukanisa nokuthembeka.
6. Isivinini kumacala aziwa kakhulu.

Ngaleli sethi lezidingo, i-SQLite ibiyisinqumo esihle futhi namanje. Nokho, njengengxenye yocwaningo lwezinye izindlela, ngathola incwadi "Ukuqala nge-LevelDB". Ngaphansi kobuholi bakhe, kwabhalwa ibhentshimakhi eqhathanisa ijubane lomsebenzi nezingosi zolwazi ezihlukene ezimeni zangempela zamafu. Umphumela udlule obekulindelwe. Ezimweni ezidume kakhulu - ukuthola ikhesa ohlwini oluhleliwe lwawo wonke amafayela kanye nohlu oluhleliwe lwawo wonke amafayela ohla lwemibhalo olunikeziwe - i-LMDB ibonakale ishesha izikhathi ezingu-10 kune-SQLite. Ukukhetha kwaba sobala.

2. Ukuma kwe-LMDB

I-LMDB iwumtapo wolwazi, mncane kakhulu (imigqa engu-10K kuphela) esebenzisa isendlalelo esiyisisekelo esiphansi kakhulu solwazi - ukugcinwa.

Umdwebo ongenhla ubonisa ukuthi ukuqhathanisa i-LMDB ne-SQLite, esebenzisa amazinga aphezulu kakhulu, ngokuvamile akulungile kune-SQLite ene-Core Data. Kungaba okulungile nakakhulu ukucaphuna izinjini ezifanayo zokulondoloza njengezimbangi ezilinganayo - i-BerkeleyDB, i-LevelDB, i-Sophia, i-RocksDB, njll. Kukhona nokuthuthuka lapho i-LMDB isebenza njengengxenye yenjini yokugcina ye-SQLite. Ukuhlolwa kokuqala okunjalo ngo-2012 wachitha umbhali LMDB Howard Chu. Imiphumela kuvele kwaba mnandi kangangoba igxathu lakhe lathathwa ngabashisekeli be-OSS, bathola ukuqhubeka kwalo ebusweni be-OSS. LumoSQL. NgoJanuwari 2020 umbhali wale phrojekthi nguDen Shearer kwethulwe ku-LinuxConfAu.

Ukusetshenziswa okuyinhloko kwe-LMDB kunjengenjini yolwazi lohlelo lokusebenza. Umtapo wolwazi ukweleta ukubonakala kwawo kubathuthukisi I-OpenLDAP, ababenganelisekile kakhulu nge-BerkeleyDB njengesisekelo sephrojekthi yabo. Ukusunduza kude nomtapo wolwazi othobekile i-btree, U-Howard Chu wakwazi ukudala enye yezindlela ezithandwayo kakhulu zesikhathi sethu. Unikele ngombiko wakhe omuhle kakhulu kule ndaba, kanye nesakhiwo sangaphakathi se-LMDB. "I-Lightning Memory-mapped Database". Leonid Yuriev (aka yilo) evela ku-Positive Technologies enkulumweni yakhe ku-Highload 2015 "Injini ye-LMDB ingumpetha okhethekile". Kuyo, ukhuluma nge-LMDB esimweni somsebenzi ofanayo wokusebenzisa i-ReOpenLDAP, futhi i-LevelDB isivele ingaphansi kokugxekwa okuqhathaniswayo. Njengomphumela wokuqaliswa, i-Positive Technologies yaze yathola nemfoloko ethuthukayo I-MDBX izici ezihlwabusayo kakhulu, ukulungiselelwa kanye ukulungiswa kweziphazamisi.

I-LMDB ivame ukusetshenziswa njengendawo yokugcina futhi. Isibonelo, isiphequluli seMozilla Firefox wakhetha yona ngenani lezidingo, futhi, kusukela kunguqulo 9, Xcode ekhethwayo i-SQLite yayo yokugcina izinkomba.

Injini iphinde yabamba iqhaza emhlabeni wokuthuthukiswa kwamaselula. Iminonjana yokusetshenziswa kwayo kungaba ukuthola kuklayenti le-iOS leTelegram. I-LinkedIn iqhubekele phambili futhi yakhetha i-LMDB njengendawo yokugcina okuzenzakalelayo yohlaka lwayo lokulondolozwa kwedatha yasekhaya, i-Rocket Data, mayelana nayo. utshele esihlokweni sango-2016.

I-LMDB ilwela ngempumelelo indawo elangeni endaweni eshiywe yiBerkeleyDB ngemuva kwenguquko ngaphansi kokulawulwa kwe-Oracle. Umtapo wolwazi uthandwa ngenxa yejubane nokwethembeka kwawo, ngisho nalapho uqhathaniswa nohlobo lwawo. Njengoba wazi, azikho izidlo zasemini zamahhala, futhi ngithanda ukugcizelela ukuhwebelana okuzodingeka ubhekane nakho lapho ukhetha phakathi kwe-LMDB ne-SQLite. Umdwebo ongenhla ubonisa ngokucacile ukuthi isivinini esikhulisiwe sifinyelelwa kanjani. Okokuqala, asizikhokheli izendlalelo ezengeziwe zokukhipha ngaphezulu kwesitoreji sediski. Yiqiniso, ekwakhiweni okuhle, awukwazi ukwenza ngaphandle kwazo, futhi nakanjani zizovela kukhodi yesicelo, kodwa zizoba zincane kakhulu. Ngeke babe nezici ezingadingeki kuhlelo lokusebenza oluthile, isibonelo, usekelo lwemibuzo ngolimi lwe-SQL. Okwesibili, kuyenzeka ukuthi kusetshenziswe ngokufanele ukwenziwa kwemephu yemisebenzi yohlelo lokusebenza kwizicelo zokugcinwa kwediski. Uma SQLite emsebenzini wami ivela ezidingweni ezimaphakathi zohlelo lokusebenza olumaphakathi, khona-ke wena, njengomthuthukisi wohlelo lokusebenza, uzazi kahle izimo eziyinhloko zomthwalo. Ukuze uthole isixazululo esikhiqiza kakhudlwana, kuzodingeka ukhokhe ithegi yentengo yakho kokubili ukuthuthukiswa kwesixazululo sokuqala kanye nokusekelwa kwaso okulandelayo.

3. Imikhomo emithathu LMDB

Ngemva kokubuka i-LMDB ngeso lenyoni, sekuyisikhathi sokujula. Izigaba ezintathu ezilandelayo zizonikelwa ekuhlaziyweni kwemikhomo eyinhloko lapho ukwakhiwa kwesitoreji kuncike khona:

1. Amafayela anemephu yenkumbulo njengendlela yokusebenza ngediski nokuvumelanisa izakhiwo zedatha yangaphakathi.
2. I-B+-tree njengenhlangano yesakhiwo sedatha egciniwe.
3. Kopisha-ubhala njengendlela yokuhlinzeka ngezakhiwo ze-ACID zokuthengiselana kanye nokuguqulwa okuningi.

3.1. Umkhomo #1. Amafayela anemephu yenkumbulo

Amafayela anemephu yenkumbulo ayisici sokwakha esibaluleke kakhulu kangangokuthi aze avele egameni lenqolobane. Izinkinga zokugcinwa kwesikhashana kanye nokuvumelanisa ukufinyelela kulwazi olugciniwe zisemseni wesistimu yokusebenza. I-LMDB ayiqukethe izinqolobane ngaphakathi kwayo. Lesi yisinqumo esihlakaniphile sombhali, njengoba ukufunda idatha ngokuqondile kumafayela amephu kukuvumela ukuthi usike amakhona amaningi ekusetshenzisweni kwenjini. Ngezansi kukude nohlu oluphelele lwabanye babo.

1. Ukugcina ukungaguquguquki kwedatha kusitoreji lapho usebenza nayo kusuka ezinqubweni ezimbalwa kuba umthwalo wesistimu yokusebenza. Esigabeni esilandelayo, lo mshini uxoxwa kabanzi nangezithombe.
2. Ukungabi khona kwama-caches kuyikhulula ngokuphelele i-LMDB ye-overhead ehlobene nokwabiwa okuguquguqukayo. Ukufunda idatha ekusebenzeni kusetha isikhombisi ekhelini elilungile kumemori ebonakalayo futhi akukho okunye. Kuzwakala njengephupho, kodwa emthonjeni wenqolobane, zonke izingcingo ze-calloc zigxile emsebenzini wokumisa indawo yokugcina.
3. Ukungabikho kwezinqolobane kuphinde kusho ukungabikho kwezingidi ezihlobene nokuvumelanisa ukuze ufinyelele kuzo. Abafundi, okungaba khona inombolo engafanele ngesikhathi esifanayo, abahlangabezi i-mutex eyodwa lapho besendleleni eya kudatha. Ngenxa yalokhu, isivinini sokufunda sinokukalwa komugqa okufanelekile ngokwenani lama-CPU. Ku-LMDB, imisebenzi yokulungisa kuphela evumelaniswayo. Kungaba khona umbhali oyedwa kuphela ngesikhathi.
4. Ubuncane be-caching kanye ne-logic yokuvumelanisa ilondoloza ikhodi ohlotsheni oluyinkimbinkimbi kakhulu lwamaphutha ahlobene nokusebenza endaweni enezintambo eziningi. Kube nezifundo ezimbili zedatha ethokozisayo engqungqutheleni ye-Usenix OSDI 2014: "Zonke Izinhlelo Zefayela Azenziwanga Ngokulinganayo: Enkingeni Yokwenza Izicelo Ezingaguquki Ukuphahlazeka" и Imininingwane Yokuhlukumeza Yokuzijabulisa Nenzuzo. Kubo ungathola ulwazi mayelana kokubili ukwethembeka okungakaze kubonwe kwe-LMDB, kanye nokuqaliswa okungenasici kwezakhiwo ze-ACID zokuthengiselana, okudlula ku-SQLite efanayo.
5. I-minimalism ye-LMDB ivumela ukumelwa komshini kwekhodi yayo ukuthi kubekwe ngokuphelele kunqolobane ye-L1 yephrosesa enezici zesivinini eziwumphumela.

Ngeshwa, ku-iOS, amafayela afakwe kumephu yenkumbulo awamnandi ngendlela ebesingathanda ngayo. Ukuze ukhulume ngobubi obuhlobene nabo ngokuqaphela ngokwengeziwe, kuyadingeka ukukhumbula izimiso ezijwayelekile zokusebenzisa le ndlela kumasistimu okusebenza.

Ulwazi olujwayelekile mayelana namafayela emephu yememori

Ngohlelo lokusebenza ngalunye olusebenzisekayo, isistimu yokusebenza ihlobanisa ibhizinisi elibizwa ngokuthi inqubo. Inqubo ngayinye yabelwa amakheli ahlukahlukene lapho ibeka khona yonke into eyidingayo ukuze isebenze. Amakheli aphansi aqukethe izigaba ezinekhodi nedatha enekhodi eqinile nezisetshenziswa. Okulandelayo kuza ibhulokhi ekhulayo ekhuphukayo yesikhala sekheli esiguqukayo, eyaziwa kakhulu kithi njengenqwaba. Iqukethe amakheli ezinkampani ezivela ngesikhathi sokusebenza kohlelo. Phezulu kunendawo yenkumbulo esetshenziswa isitaki sohlelo lokusebenza. Ingakhula noma inciphe, ngamanye amazwi, ubukhulu bayo bubuye bube nemvelo eguquguqukayo. Ukuze isitaki nenqwaba zingasunduzana futhi ziphazamise enye nenye, zihlukaniswa ezindaweni ezihlukene zesikhala sekheli Kukhona imbobo phakathi kwezingxenye ezimbili eziguquguqukayo phezulu nangaphansi. Amakheli akulesi sigaba esimaphakathi asetshenziswa isistimu yokusebenza ukuze ahlotshaniswe nenqubo yamabhizinisi ahlukahlukene. Ikakhulukazi, ingafaka imephu isethi ethile eqhubekayo yamakheli kufayela elikudiski. Ifayela elinjalo libizwa ngokuthi ifayela elinemephu yememori

Isikhala samakheli esabelwe inqubo sikhulu. Ngokwetiyori, inani lamakheli linqunyelwe kuphela usayizi wesikhombi, onqunywa ukujula kancane kwesistimu. Uma inkumbulo yomzimba yabelwe yona 1-in-1, khona-ke inqubo yokuqala ibizokhuphula yonke i-RAM, futhi bekungeke kube khona umbuzo wanoma yikuphi ukwenza izinto eziningi.

Nokho, siyazi ngokuzibonela ukuthi amasistimu wokusebenza wesimanje angasebenzisa izinqubo eziningi ngendlela ofuna ngayo ngesikhathi esisodwa. Lokhu kungenzeka ngenxa yokuthi banikeza inkumbulo eningi ezinkambisweni kuphela ephepheni, kodwa empeleni balayisha kumemori eyinhloko kuphela leyo ngxenye edingekayo lapha futhi manje. Ngakho-ke, inkumbulo ehlobene nenqubo ibizwa ngokuthi i-virtual.

Uhlelo lokusebenza luhlela inkumbulo ebonakalayo nengokwenyama ibe amakhasi anosayizi othile. Lapho nje ikhasi elithile lememori ebonakalayo lidingeka, isistimu yokusebenza ililayisha kumemori yomzimba bese ibeka izincwadi phakathi kwabo etafuleni elikhethekile. Uma zingekho izikhala zamahhala, elinye lamakhasi alayishwe ngaphambilini likopishwa kudiski, futhi eliceliwe lithatha indawo yalo. Le nqubo, esizobuyela kuyo maduze, ibizwa ngokuthi ukushintshanisa. Umfanekiso ongezansi ubonisa inqubo echazwe. Kuyo, ikhasi A elinekheli elithi 0 lalayishwa futhi labekwa ekhasini lenkumbulo eliyinhloko elinekheli elithi 4. Leli qiniso labonakala ethebula lezincwadi kunombolo yeseli 0.​

Ngamafayela emephu yememori, indaba ifana ncamashi. Ngokunengqondo, abekwe ngokuqhubekayo futhi ngokuphelele endaweni yamakheli abonakalayo. Kodwa-ke, bangena ekhasini lememori ebonakalayo ngekhasi futhi ngokufunwa kuphela. Ukuguqulwa kwamakhasi anjalo kuvumelaniswa nefayela elikudiski. Ngakho-ke, ungenza ifayela le-I / O, ngokumane usebenze ngamabhayithi enkumbulweni - zonke izinguquko zizodluliselwa ngokuzenzakalelayo yi-kernel yesistimu yokusebenza kufayela lokuqala.
A
Isithombe esingezansi sibonisa ukuthi i-LMDB ivumelanisa kanjani isimo sayo lapho isebenza nesizindalwazi kusuka ezinqubweni ezihlukene. Ngokwenza imephu inkumbulo ebonakalayo yezinqubo ezihlukene kufayela elifanayo, sibophezela isistimu yokusebenza ukuthi ivumelanise ngokuqhubekayo amabhulokhi athile ezikhala zawo zamakheli enye nenye, kulapho i-LMDB ibukeka khona.
A

I-nuance ebalulekile ukuthi i-LMDB ilungisa ifayela ledatha ngokuzenzakalelayo ngokusebenzisa indlela yokushaya ucingo yesistimu, futhi ifayela ngokwalo libonisa ngemodi yokufunda kuphela. Le ndlela inemiphumela emibili ebalulekile.

Umphumela wokuqala uvamile kuwo wonke amasistimu wokusebenza. Ingqikithi yayo iwukwengeza isivikelo emonakalweni ongaqondile kusizindalwazi ngekhodi engalungile. Njengoba wazi, imiyalelo esebenzisekayo yenqubo imahhala ukufinyelela idatha kusuka noma yikuphi endaweni yayo yekheli. Ngesikhathi esifanayo, njengoba sisanda kukhumbula, ukubonisa ifayela kwimodi yokufunda-bhala kusho ukuthi noma yimuphi umyalelo ungawushintsha ngaphezu kwalokho. Uma enza lokhu ngephutha, ezama, ngokwesibonelo, ukubhala phezu kwento ethile ohlwini olungekho, khona-ke ngale ndlela angakwazi ukuguqula ngephutha ifayela elifakwe kuleli kheli, okuzoholela ekonakaleni kusizindalwazi. Uma ifayela liboniswa ngemodi yokufunda kuphela, lapho-ke umzamo wokushintsha indawo yekheli elihambisana nalo uzoholela ekuphahlazekeni kohlelo ngesignali. SIGSEGV, futhi ifayela lizohlala linjalo.

Umphumela wesibili usuvele uqondile ku-iOS. Umbhali nanoma yimiphi eminye imithombo ayisho ngokusobala, kodwa ngaphandle kwayo, i-LMDB ibingeke ifanelekele ukusebenza kulolu hlelo lokusebenza lweselula. Isigaba esilandelayo sinikezelwe ekucabangeni kwayo.

Imininingwane yamafayela emephu yememori ku-iOS

Ngo-2018, kwaba nombiko omuhle kakhulu ku-WWDC I-iOS Memory Deep Dive. Itshela ukuthi ku-iOS wonke amakhasi atholakala enkumbulweni yomzimba ayingxenye eyodwa yezinhlobo ezi-3: angcolile, acindezelwe futhi ahlanzekile.

Inkumbulo ehlanzekile iqoqo lamakhasi angashintshwa ngokuphephile akhishwe kunkumbulo engokwenyama. Idatha eziqukethwe zingaphinda zilayishwe emithonjeni yazo yoqobo njengoba kudingeka. Amafayela emephu enkumbulo yokufunda kuphela awela kulesi sigaba. I-iOS ayesabi ukulayisha amakhasi afakwe kumephu efayeleni kusuka kumemori nganoma yisiphi isikhathi, ngoba aqinisekisiwe ukuthi azovumelaniswa nefayela elikwidiski.
A
Wonke amakhasi ashintshiwe angena kumemori engcolile, kungakhathaliseki ukuthi atholakala kuphi. Ikakhulukazi, amafayela afakwe kumephu yenkumbulo ashintshwe ngokubhalela inkumbulo ebonakalayo ehlotshaniswa nawo nawo azohlukaniswa ngale ndlela. Ivula i-LMDB enefulegi MDB_WRITEMAP, ngemva kokwenza izinguquko kuyo, ungazibonela ngokwakho. . . 

Lapho nje uhlelo lokusebenza luqala ukuthatha inkumbulo eningi kakhulu, i-iOS icindezela amakhasi ayo angcolile. Iqoqo lememori ehlala amakhasi angcolile nacindezelwe yilokho okubizwa ngokuthi inkumbulo yonyawo lohlelo lokusebenza. Uma ifinyelela inani elithile lomkhawulo, i-daemon yesistimu ye-OOM killer iza ngemva kwenqubo futhi iyinqamule ngenkani. Lokhu ukukhethekile kwe-iOS uma kuqhathaniswa nezinhlelo zokusebenza zedeskithophu. Ngokuphambene, ukwehlisa isigxivizo sememori ngokushintshanisa amakhasi ukusuka kumemori ebonakalayo kuya kudiski akunikeziwe ku-iOS. Umuntu angaqagela kuphela mayelana nezizathu. Mhlawumbe inqubo yokuhambisa amakhasi ashukumisayo kudiski nangemuva idla amandla kakhulu kumadivayisi eselula, noma i-iOS igcina insiza yokubhala kabusha amaseli kumadiski e-SSD, noma mhlawumbe abaklami abanelisekile ngokusebenza jikelele kohlelo, lapho konke eshintshiwe njalo. Noma ngabe kunjalo, iqiniso lihlala likhona.

Izindaba ezinhle, esezishiwo ngaphambili, ukuthi i-LMDB ayisebenzisi indlela ye-mmap ngokuzenzakalelayo ukuze ibuyekeze amafayela. Lokhu kulandela ukuthi idatha enikeziwe ihlukaniswa njengememori ehlanzekile yi-iOS futhi ayifaki isandla kunkumbulo. Lokhu kungaqinisekiswa kusetshenziswa ithuluzi le-Xcode elibizwa nge-VM Tracker. Isithombe-skrini esingezansi sibonisa isimo sememori ebonakalayo yohlelo lokusebenza lwe-iOS Cloud phakathi nokusebenza. Ekuqaleni, izimo ze-LMDB ezi-2 zaqalwa kuyo. Eyokuqala yayivunyelwe ukubeka ifayela layo ku-1GiB yememori ebonakalayo, eyesibili - 512MiB. Naphezu kweqiniso lokuthi kokubili isitoreji sithatha inani elithile lememori ehlala khona, akekho kubo ofaka isandla kusayizi ongcolile.

Manje sekuyisikhathi sezindaba ezimbi. Ngenxa yomshini wokushintshisana kumasistimu wokusebenza wedeskithophu engu-64-bit, inqubo ngayinye ingathatha indawo yamakheli engokoqobo njengoba isikhala esikhululekile ku-hard disk sivumela ukushintshana kwayo okungaba khona. Ukushintsha ukushintsha ngokuminyanisa ku-iOS kunciphisa kakhulu umkhawulo wetiyori. Manje zonke izinqubo eziphilayo kufanele zingene kumemori eyinhloko (funda i-RAM), futhi zonke ezingalingani zingaphansi kokuqedwa ngempoqo. Kushiwo njengalokhu ngenhla bika, futhi imibhalo esemthethweni. Njengomphumela, i-iOS ikhawulela kakhulu inani lememori elitholakalayo ukuze linikezwe nge-mmap. Lapha lapha ungabheka imikhawulo ye-empirical enanini lememori elinganikezwa kumadivayisi ahlukene usebenzisa lolu cingo lwesistimu. Kumamodeli esimanje kakhulu ama-smartphones, i-iOS iye yaba nesandla esivulekile ngamagigabhayithi angu-2, futhi ezinguqulweni eziphezulu ze-iPad - ngo-4. Ngokwenza, yiqiniso, kufanele ugxile kumamodeli wedivayisi asekelwayo, lapho yonke into idabukisayo kakhulu. Okubi nakakhulu, uma ubheka isimo sememori yohlelo lokusebenza ku-VM Tracker, uzothola ukuthi i-LMDB ikude neyodwa ethi inkumbulo efakwe kumephu. Izingcezu ezinhle zidliwa abahlinzeki besistimu, amafayela wensiza, izinhlaka zezithombe, nezinye izilwane ezidla ezinye ezincane.

Njengomphumela wokuhlola ku-Cloud, siqhamuke namanani alandelayo okonakala enkumbulo anikezwe i-LMDB: amamegabhayithi angu-384 kumadivayisi angu-32-bit kanye no-768 kwengama-64-bit. Ngemuva kokuthi le volumu isiphelile, noma yimiphi imisebenzi yokulungisa iqala ukuqeda ngekhodi MDB_MAP_FULL. Siyawabona amaphutha anjalo ekuqapheni kwethu, kodwa mancane ngokwanele ukuthi anganakwa kulesi sigaba.

Isizathu esingabonakali sokusetshenziswa kwenkumbulo ngokweqile ngokugcinwa kungaba ukuthengiswa kwesikhathi eside. Ukuze siqonde ukuthi lezi zenzakalo ezimbili zihlobana kanjani, kuzosisiza ukuba sicabangele imikhomo emibili ye-LMDB esele.

3.2. Umkhomo #2. B+-isihlahla

Ukuze ulingise amathebula ngaphezulu kwesitolo senani elingukhiye, imisebenzi elandelayo kufanele ibe khona ku-API yaso:

1. Ifaka into entsha.
2. Sesha i-elementi ngokhiye othile.
3. Isusa i-elementi.
4. Phinda izikhathi ezibalulekile ngokulandelana kwazo.

Isakhiwo sedatha esilula esingasebenzisa kalula yonke imisebenzi emine isihlahla sosesho esinambambili. Ngayinye yamanodi ayo iwukhiye ohlukanisa yonke isethi yokhiye bengane ibe izihlahlana ezimbili ezincane. Kwesokunxele kukhona lezo ezincane kunomzali, futhi kwesokudla - lezo ezinkulu. Ukuthola isethi e-odwe yokhiye kufinyelelwa ngenye yezindlela zokunqamula izihlahla zakudala

Izihlahla kanambambili zinezithiyo ezimbili ezibalulekile ezizivimbela ekusebenzeni njengesakhiwo sedatha yediski. Okokuqala, izinga lebhalansi yabo alinakubikezelwa. Kunengozi enkulu yokuthola izihlahla lapho ukuphakama kwamagatsha ahlukene kungahluka kakhulu, okwenza kube kubi kakhulu inkimbinkimbi ye-algorithmic yokusesha uma kuqhathaniswa nalokho okulindelekile. Okwesibili, ukuchichima kokuxhumanisa phakathi kwamanodi kuphuca izihlahla kanambambili indawo enkumbulweni.Amanodi avaliwe (ngokwezixhumanisi phakathi kwawo) angatholakala emakhasini ahluke ngokuphelele kumemori ebonakalayo. Ngenxa yalokho, ngisho nokunqamula amanodi amaningana angomakhelwane esihlahleni kungase kudinge ukuvakashela inani eliqhathanisekayo lamakhasi. Lokhu kuyinkinga ngisho nalapho sikhuluma ngokusebenza kwezihlahla kanambambili njengesakhiwo sedatha yememori, njengoba amakhasi ajikelezayo njalo kunqolobane yeprosesa ayishibhile. Uma kukhulunywa ngokunyusa amakhasi ahlobene nama-node kusuka kudiski, izinto ziba zimbi kakhulu. edabukisayo.

Izihlahla ze-B, eziwukuvela kwezihlahla kanambambili, zixazulula izinkinga ezishiwo endimeni yangaphambilini. Okokuqala, bayazilinganisela. Okwesibili, indawo ngayinye yazo ihlukanisa isethi yokhiye bezingane hhayi ku-2, kodwa ibe amasethi amancane a-Odelwe ngu-M, futhi inombolo engu-M ingaba nkulu kakhulu, ngokulandelana kwamakhulu ambalwa noma izinkulungwane.

Ngalokho:

1. I-node ngayinye inenani elikhulu lezikhiye ezivele zi-odwe futhi izihlahla ziphansi kakhulu.
2. Isihlahla sithola indawo yendawo ngenkumbulo, njengoba okhiye abasondelene ngokwenani batholakala ngokwemvelo eduze komunye nomunye endaweni eyodwa noma engumakhelwane.
3. Yehlisa inani lezindawo zokuhamba lapho wehla esihlahleni phakathi nomsebenzi wokusesha.
4. Yehlisa inani lamanodi okuqondiwe afundelwe imibuzo yobubanzi, njengoba ngayinye yazo isivele iqukethe inani elikhulu lokhiye aba-odwe.

I-LMDB isebenzisa okuhlukile kwesihlahla B esibizwa ngokuthi isihlahla B+ ukuze igcine idatha. Umdwebo ongenhla ubonisa izinhlobo ezintathu zamanodi eziwaqukethe:

1. Phezulu kunempande. Akwenziwa lutho ngaphandle komqondo wesizindalwazi esingaphakathi kwendawo yokugcina izinto. Esimweni esisodwa se-LMDB, ungakha izizindalwazi eziningi ezabelana ngendawo yekheli elikumephu elibonakalayo. Ngamunye wabo uqala ezimpandeni zawo.
2. Ezingeni eliphansi kunamahlamvu (iqabunga). Yizo futhi yizo kuphela eziqukethe amapheya enani elingukhiye agcinwe kusizindalwazi. Ngendlela, lokhu kuyisici esiyingqayizivele sezihlahla ze-B +. Uma isihlahla esingu-B esivamile sigcina izingxenye zenani kumanodi awo wonke amazinga, khona-ke ukuhlukahluka kuka-B+ kuba kwephansi kakhulu. Ngemva kokulungisa leli qiniso, kulokhu okulandelayo sizobiza uhlobo oluncane lwesihlahla esisetshenziswe ku-LMDB nje ngesihlahla esingu-B.
3. Phakathi kwempande namaqabunga, kunamazinga obuchwepheshe angu-0 noma ngaphezulu anama-node okuzulazula (igatsha). Umsebenzi wabo ukuhlukanisa isethi yezihluthulelo ezihlungiwe phakathi kwamaqabunga.

Ngokomzimba, ama-node angamabhulokhi enkumbulo yobude obunqunywe kusengaphambili. Ubukhulu bawo buwubukhulu obuphindiwe bamakhasi enkumbulo ohlelweni lokusebenza, esikhulume ngalo ngenhla. Isakhiwo se-node siboniswe ngezansi. Iheda iqukethe imininingwane ye-meta, esobala kakhulu, ngokwesibonelo, i-checksum. Okulandelayo kuza ulwazi mayelana nama-offset, lapho amaseli anedatha atholakala khona. Iqhaza ledatha kungaba okhiye, uma sikhuluma ngamanodi okuzulazula, noma wonke amapheya enani likakhiye esimweni samaqabunga. Ungafunda kabanzi mayelana nesakhiwo samakhasi emsebenzini "Ukuhlolwa Kwezitolo Ezibalulekile Ezisebenza Kakhulu".

Ngemva kokubhekana nokuqukethwe kwangaphakathi kwamanodi ekhasi, sizoqhubeka nokumela isihlahla se-LMDB B ngendlela eyenziwe lula ngaleli fomu elilandelayo.

Amakhasi anamanodi ahlelwa ngokulandelana kudiski. Amakhasi anenombolo ephezulu atholakala ngasekupheleni kwefayela. Ikhasi elibizwa nge-meta (ikhasi le-meta) liqukethe ulwazi mayelana nama-offsets, angasetshenziswa ukuthola izimpande zazo zonke izihlahla. Lapho ifayela livuliwe, i-LMDB iskena ikhasi lefayela ngekhasi ukusuka ekugcineni kuye ekuqaleni lapho ifuna ikhasi lemetha elivumelekile futhi ithola imininingwane ekhona ngalo.

Manje, njengoba sinombono wesakhiwo esinengqondo nesingokomzimba senhlangano yedatha, singaqhubeka nokucabangela umkhomo wesithathu we-LMDB. Kungenxa yosizo lwayo ukuthi zonke izinguquko zesitoreji zenzeka ngokuthengiselana futhi ngokuhlukana komunye nomunye, ukunikeza i-database iyonke futhi impahla yokuguqula okuningi.

3.3. Umkhomo #3. kopisha-ngokubhala

Eminye imisebenzi ye-B-tree ihlanganisa ukwenza uchungechunge lonke lwezinguquko kumanodi ayo. Isibonelo esisodwa ukungeza ukhiye omusha endaweni esesifinyelele umthamo wayo omkhulu. Kulokhu, kuyadingeka, okokuqala, ukuhlukanisa i-node ibe kabili, futhi okwesibili, ukwengeza isixhumanisi ku-node entsha ephothiwe kumzali wayo. Le nqubo ingaba yingozi kakhulu. Uma ngesizathu esithile (ukuphahlazeka, ukuphela kukagesi, njll.) ingxenye kuphela yezinguquko ezivela ochungechungeni yenzeka, khona-ke isihlahla sizohlala esimweni esingaguquki.

Isixazululo esisodwa sendabuko sokwenza isizindalwazi sibekezelele amaphutha ukwengeza isakhiwo sedatha esekwe kudiski eyengeziwe, ilogi yokwenziwayo, eyaziwa nangokuthi ilogi yokubhala phambili (WAL), eduze kwesihlahla B. Ifayela, ekupheleni kwalo, ngokuqinile ngaphambi kokuguqulwa kwe-B-tree ngokwayo, ukusebenza okuhlosiwe kubhaliwe. Ngakho-ke, uma inkohlakalo yedatha itholwa ngesikhathi sokuzixilonga, isizindalwazi sithintana nelogi ukuze izihlanze.

I-LMDB ikhethe indlela ehlukile njengendlela yokubekezelela iphutha, ebizwa ngokuthi ikhophi-on-write. Ingqikithi yayo iwukuthi esikhundleni sokubuyekeza idatha ekhasini elikhona, iqala ngokuyikopisha ngokuphelele futhi yenze zonke izinguquko sezivele zikhona ekhophi.

Ngaphezu kwalokho, ukuze idatha ebuyekeziwe itholakale, kuyadingeka ukushintsha isixhumanisi ku-node eseyakamuva endaweni yomzali maqondana nayo. Njengoba nayo idinga ukulungiselwa lokhu, iphinde ikopishwe ngaphambilini. Inqubo iqhubeka ngokuphindaphindiwe yonke indlela eya empandeni. Idatha esekhasini le-meta ingeyokugcina ukushintsha

Uma ngokuzumayo inqubo iphahlazeka ngesikhathi senqubo yokuvuselela, khona-ke noma ikhasi elisha le-meta ngeke lidalwe, noma ngeke libhalwe kudiski kuze kube sekupheleni, futhi i-checksum yayo izobe ingalungile. Kulezi zimo ezimbili, amakhasi amasha ngeke afinyeleleke futhi amadala ngeke athinteke. Lokhu kuqeda isidingo sokuthi i-LMDB ibhale ilogi kusengaphambili ukuze igcine ukuhambisana kwedatha. I-De facto, isakhiwo sokugcinwa kwedatha kudiski, echazwe ngenhla, ngesikhathi esisodwa sithatha umsebenzi wayo. Ukungabikho kwelogi yokwenziwayo ecacile kungenye yezici ze-LMDB, ehlinzeka ngesivinini sokufunda idatha ephezulu.

Ukwakhiwa okuwumphumela, okubizwa nge-append-only B-tree, ngokwemvelo kunikeza ukuhlukaniswa kokwenziwe kanye nokuguqulwa okuningi. Ku-LMDB, okwenziwayo ngakunye okuvuliwe kunempande yesihlahla esesikhathini samanje ehlotshaniswa nayo. Uma nje umsebenzi ungakaqedwa, amakhasi esihlahla esihlotshaniswa naso ngeke aphinde ashintshwe noma asetshenziswe kabusha ezinguqulweni ezintsha zedatha. Ngakho-ke, ungasebenza inqobo nje uma uthanda ngesethi yedatha ebifaneleka ngesikhathi umsebenzi ovuliwe, ngisho noma isitoreji siqhubeka nokubuyekezwa ngalesi sikhathi. Lona umnyombo wokwenza i-multiversioning, okwenza i-LMDB ibe umthombo wedatha ofanele wabathandekayo bethu UICollectionView. Ngemva kokuvula umsebenzi, awudingi ukukhulisa inkumbulo yohlelo lokusebenza, ukhiphe idatha yamanje uyifaka esakhiweni esithile senkumbulo, wesaba ukushiywa ungenalutho. Lesi sici sihlukanisa i-LMDB ku-SQLite efanayo, engakwazi ukuziqhayisa ngokuhlukaniswa okuphelele okunjalo. Ngemva kokuvula ukuthengiselana okubili kokugcina nokususa irekhodi elithile ngaphakathi kweyodwa yazo, irekhodi elifanayo ngeke lisatholakala phakathi kwelesibili elisele.

I-flip side yohlamvu lwemali ukusetshenziswa okungaba okuphezulu kakhulu kwememori ebonakalayo. Isilayidi sibonisa ukuthi isakhiwo sedathabheyisi sizobukeka kanjani uma silungiswa ngesikhathi esisodwa nge-3 evulekile yokufunda imisebenzi ebheka izinguqulo ezahlukene zesizindalwazi. Njengoba i-LMDB ingakwazi ukuphinda isebenzise amanodi afinyeleleka kusukela ezimpandeni ezihlotshaniswa nokwenziwe kwangempela, isitoreji asinakho ukukhetha ngaphandle kokwaba enye impande yesine kumemori futhi siphinde sihlanganise amakhasi aguquliwe ngaphansi kwayo.

Lapha ngeke kube yinto engafaneleki ukukhumbula isigaba samafayela afakwe kumephu yenkumbulo. Kubonakala sengathi ukusetshenziswa okwengeziwe kwememori ebonakalayo akufanele kusikhathaze kakhulu, ngoba akufaki isandla kumemori yohlelo lokusebenza. Kodwa-ke, ngesikhathi esifanayo, kwaphawulwa ukuthi i-iOS iyancisha kakhulu ekunikezeni, futhi asikwazi ukunikeza isifunda se-LMDB esingu-1 terabyte kuseva noma ideskithophu esuka ehlombe lika-master futhi singacabangi ngalesi sici nhlobo. Uma kungenzeka, kufanele uzame ukugcina impilo yonke yemisebenzi ibe mfushane ngangokunokwenzeka.

4. Ukuklama i-schema sedatha phezu kwe-API yenani elingukhiye

Ake siqale ukuhlukanisa i-API ngokubheka izifinyezo eziyisisekelo ezinikezwe i-LMDB: indawo kanye nemininingwane yolwazi, okhiye namanani, okwenziwayo kanye nezikhombisi.

Inothi elimayelana nokufakwa kuhlu kwamakhodi

Yonke imisebenzi ku-API yomphakathi ye-LMDB ibuyisela umphumela womsebenzi wayo ngendlela yekhodi yephutha, kodwa kukho konke ukufakwa kuhlu okulandelanayo ukuhlola kwayo kukhishiwe ngenxa yokufingqa. Empeleni, sasebenzisa eyethu ikhodi ukuze sihlanganyele nekhosombe. imfoloko I-C++ wrappers lmdbxx, lapho amaphutha enziwa khona njengokuhlukile kwe-C++.

Njengendlela eshesha kakhulu yokuxhuma i-LMDB kuphrojekthi ye-iOS noma ye-macOS, nginikeza i-CocoaPod yami I-POSLMDB.

4.1. Izifinyezo eziyisisekelo

Imvelo

isakhiwo MDB_env iyinqolobane yesimo sangaphakathi se-LMDB. Umndeni wemisebenzi enesiqalo mdb_env ikuvumela ukuthi ulungiselele ezinye zezindawo zayo. Esimweni esilula, ukuqaliswa kwenjini kubukeka kanje.

mdb_env_create(env);​
mdb_env_set_map_size(*env, 1024 * 1024 * 512)​
mdb_env_open(*env, path.UTF8String, MDB_NOTLS, 0664);

Kuhlelo lokusebenza lwe-Mail.ru Cloud, siguqule amanani azenzakalelayo amapharamitha amabili kuphela.

Esokuqala siwusayizi wesikhala sekheli elibonakalayo ifayela lesitoreji elifakwe kuso imephu. Ngeshwa, nakudivayisi efanayo, inani elithile lingahluka kakhulu ukusuka ekuqaliseni ukuya kokusebenza. Ukunaka lesi sici se-iOS, sikhetha inani eliphezulu lesitoreji ngokushintshashintsha. Isuka enanini elithile, ihlukana ngokulandelana kuze kube umsebenzi mdb_env_open ngeke ibuyise umphumela ngaphandle ENOMEM. Ngokombono, kunendlela ehlukile - okokuqala beka inkumbulo encane enjini, bese kuthi lapho amaphutha etholwa. MDB_MAP_FULL, kwandise. Nokho, inameva kakhulu. Isizathu ukuthi inqubo yokwenza kabusha imemori usebenzisa umsebenzi mdb_env_set_map_size yenza wonke amabhizinisi angasebenzi (izikhombisi, okwenziwayo, okhiye namanani) atholwe enjinini ngaphambi kwesikhathi. Ukubalwa koshintsho olunjalo lwezehlakalo kukhodi kuzoholela enkingeni yayo ebalulekile. Uma, noma kunjalo, inkumbulo ebonakalayo ithandeka kakhulu kuwe, khona-ke lokhu kungase kube isizathu sokubheka imfoloko eye yahamba phambili. I-MDBX, lapho phakathi kwezici ezimenyezelwe kukhona “ukulungiswa kosayizi wesizindalwazi okuzenzakalelayo okuku-the-fly”.

Ipharamitha yesibili, inani elizenzakalelayo elingasifanelanga, lilawula izinsimbi zokuqinisekisa ukuphepha kwentambo. Ngeshwa, okungenani ku-iOS 10, kunezinkinga ngosekelo lwesitoreji sendawo. Ngenxa yalesi sizathu, esibonelweni esingenhla, indawo yokugcina ivulwa ngefulegi MDB_NOTLS. Ngaphezu kwalokho, kwakudingeka futhi imfoloko I-C++ wrapper lmdbxxukusika okuguquguqukayo kanye nakulesi sibaluli.

Amadokhumenti

I-database iyisibonelo esihlukile se-B-tree esikhulume ngaso ngenhla. Ukuvulwa kwayo kwenzeka ngaphakathi kokuthengiselana, okungenzeka ekuqaleni kubonakale kuyinqaba.

MDB_txn *txn;​
MDB_dbi dbi;​
mdb_txn_begin(env, NULL, MDB_RDONLY, &txn);​
mdb_dbi_open(txn, NULL, MDB_CREATE, &dbi);​
mdb_txn_abort(txn);

Ngempela, okwenziwayo ku-LMDB kuyinhlangano yokugcina, hhayi isizindalwazi esithile. Lo mqondo ukuvumela ukuthi wenze imisebenzi ye-athomu ezinkampanini ezitholakala kuzigcinilwazi ezahlukene. Ngokombono, lokhu kuvula amathuba okuba amatafula okumodela ngendlela yolwazi oluhlukene, kodwa ngake ngahamba ngenye indlela, echazwe ngokuningiliziwe ngezansi.

Okhiye namagugu

isakhiwo MDB_val imodela umqondo wakho kokubili ukhiye kanye nenani. Indawo yokugcina ayinalo ulwazi mayelana ne-semantics yabo. Kuye, into ehlukile iwuxhaxha nje lwamabhayithi osayizi othile. Usayizi omkhulu wokhiye ungamabhayithi angu-512.

typedef struct MDB_val {​
    size_t mv_size;​
    void *mv_data;​
} MDB_val;​​

Isitolo sisebenzisa isiqhanisi ukuze sihlunge okhiye ngendlela ekhuphukayo. Uma ungayishintshi ngeyakho, khona-ke ezenzakalelayo izosetshenziswa, ezihlunga nge-byte byte ngokulandelana kwe-lexicographic.

Ukuthengiselana

Idivayisi yokwenziwayo ichazwa ngokuningiliziwe ku isahluko esedlule, ngakho-ke lapha ngizophinda izici zabo eziyinhloko ngomugqa omfushane:

1. Ukusekela zonke izakhiwo eziyisisekelo I-ACIDAmagama angukhiye: i-athomu, ukungaguquguquki, ukuzihlukanisa nokuthembeka. Angikwazi ukusiza kodwa ngiphawule ukuthi ngokuya ngokuqina ku-macOS ne-iOS kunesiphazamisi esilungisiwe ku-MDBX. Ungafunda kabanzi kwezabo README.
2. Indlela yokwenza imicu eminingi ichazwa uhlelo "lombhali oyedwa / abafundi abaningi". Ababhali bayavimbana, kodwa ababavimbi abafundi. Abafundi abavimbeli ababhali noma omunye nomunye.
3. Ukusekela kokwenziwayo okufakwe esidlekeni.
4. Ukusekelwa kwe-Multiversion.

I-Multiversioning ku-LMDB kuhle kangangokuthi ngifuna ukuyikhombisa ngesenzo. Ikhodi engezansi ibonisa ukuthi okwenziwayo ngakunye kusebenza kanye ncamashi nenguqulo yesizindalwazi eyayifanelekile ngesikhathi sokuvulwa kwayo, ihlukaniswe ngokuphelele kuzo zonke izinguquko ezalandela. Ukuqalisa inqolobane nokwengeza irekhodi lokuhlola kuyo akunanzuzo, ngakho le mikhuba ishiywa ngaphansi komchithi.

Yengeza okufakiwe kokuhlola

MDB_env *env;
MDB_dbi dbi;
MDB_txn *txn;

mdb_env_create(&env);
mdb_env_open(env, "./testdb", MDB_NOTLS, 0664);

mdb_txn_begin(env, NULL, 0, &txn);
mdb_dbi_open(txn, NULL, 0, &dbi);
mdb_txn_abort(txn);

char k = 'k';
MDB_val key;
key.mv_size = sizeof(k);
key.mv_data = (void *)&k;

int v = 997;
MDB_val value;
value.mv_size = sizeof(v);
value.mv_data = (void *)&v;

mdb_txn_begin(env, NULL, 0, &txn);
mdb_put(txn, dbi, &key, &value, MDB_NOOVERWRITE);
mdb_txn_commit(txn);

MDB_txn *txn1, *txn2, *txn3;
MDB_val val;

// Открываем 2 транзакции, каждая из которых смотрит
// на версию базы данных с одной записью.
mdb_txn_begin(env, NULL, 0, &txn1); // read-write
mdb_txn_begin(env, NULL, MDB_RDONLY, &txn2); // read-only

// В рамках первой транзакции удаляем из базы данных существующую в ней запись.
mdb_del(txn1, dbi, &key, NULL);
// Фиксируем удаление.
mdb_txn_commit(txn1);

// Открываем третью транзакцию, которая смотрит на
// актуальную версию базы данных, где записи уже нет.
mdb_txn_begin(env, NULL, MDB_RDONLY, &txn3);
// Убеждаемся, что запись по искомому ключу уже не существует.
assert(mdb_get(txn3, dbi, &key, &val) == MDB_NOTFOUND);
// Завершаем транзакцию.
mdb_txn_abort(txn3);

// Убеждаемся, что в рамках второй транзакции, открытой на момент
// существования записи в базе данных, её всё ещё можно найти по ключу.
assert(mdb_get(txn2, dbi, &key, &val) == MDB_SUCCESS);
// Проверяем, что по ключу получен не абы какой мусор, а валидные данные.
assert(*(int *)val.mv_data == 997);
// Завершаем транзакцию, работающей хоть и с устаревшей, но консистентной базой данных.
mdb_txn_abort(txn2);

Ngokuzithandela, ngincoma ukuzama iqhinga elifanayo nge-SQLite futhi ubone ukuthi kwenzekani.

I-Multiversioning iletha izinzuzo ezinhle kakhulu empilweni yonjiniyela we-iOS. Usebenzisa lesi sici, ungakwazi ukulungisa kalula futhi ngokwemvelo izinga lokubuyekezwa komthombo wedatha kumafomu esikrini ngokusekelwe kokucatshangelwa kokuhlangenwe nakho komsebenzisi. Isibonelo, ake sithathe isici esinjalo sohlelo lokusebenza lwe-Mail.ru Cloud njengokulayisha ngokuzenzakalelayo okuqukethwe kugalari yemidiya yesistimu. Ngoxhumano oluhle, iklayenti liyakwazi ukwengeza izithombe ezimbalwa ngomzuzwana kuseva. Uma ubuyekeza ngemva kokulanda ngakunye UICollectionView ngokuqukethwe kwemidiya efwini lomsebenzisi, ungakhohlwa mayelana nama-fps angama-60 nokuskrola okubushelelezi phakathi nale nqubo. Ukuze uvimbele ukubuyekezwa kwesikrini njalo, udinga ukuthi ngandlela thize ukhawulele izinga lokushintsha kwedatha ngokwesisekelo UICollectionViewDataSource.

Uma isizindalwazi singakusekeli ukuguqulwa okuningi futhi ikuvumela ukuthi usebenze kuphela nesimo samanje, bese udala isifinyezo sedatha esizinzile, udinga ukusikopisha kwesinye isakhiwo sedatha yenkumbulo noma kuthebula lesikhashana. Noma iyiphi yalezi zindlela ibiza kakhulu. Esimeni sokugcinwa kwenkumbulo, sithola kokubili izindleko zememori ezibangelwa ukugcina izinto ezakhiwe kanye nezindleko zesikhathi ezihlobene noshintsho olungadingekile lwe-ORM. Ngokuqondene netafula lesikhashana, lokhu kuyinjabulo ebiza kakhulu, okunengqondo kuphela ezimweni ezingezona ezincane.

I-Multiversioning LMDB ixazulula inkinga yokugcina umthombo wedatha ozinzile ngendlela enhle kakhulu. Kwanele nje ukuvula umsebenzi kanye ne-voila - size sikuqedele, isethi yedatha iqinisekisiwe ukuthi izolungiswa. Ingqondo yezinga layo lokubuyekeza manje isisezandleni zosendlalelo sephrezentheshini, ngaphandle kwezinsiza ezibalulekile.

Iziqalekiso

Izikhombisi zihlinzeka ngendlela yokuphindaphinda okuhlelekile phezu kwamapheya enani elingukhiye ngokunqamula isihlahla esingu-B. Ngaphandle kwabo, bekungeke kwenzeke ukumodela ngempumelelo amatafula kusizindalwazi, manje esiphendukela kuwo.

4.2. Ukumodela Kwethebula

Isakhiwo soku-oda esiyinhloko sikuvumela ukuthi wakhe ukufinyezwa kwezinga eliphezulu njengetafula phezu kwezinto ezinqanyuliwe eziyisisekelo. Ake sicabangele le nqubo esibonelweni sethebula eliyinhloko leklayenti lefu, lapho ulwazi mayelana nawo wonke amafayela namafolda omsebenzisi lugcinwe khona.

I-Schema yethebula

Esinye sezimo ezivamile lapho isakhiwo setafula elinesihlahla sefolda okufanele silolwe khona ukukhetha zonke izici ezitholakala ngaphakathi kohla lwemibhalo olunikeziwe. Imodeli enhle yenhlangano yedatha yemibuzo ephumelelayo yalolu hlobo ithi Uhlu Lokujwayela. Ukuze uyisebenzise phezu kwesitoreji senani elingukhiye, kuyadingeka ukuhlela okhiye bamafayela namafolda ngendlela yokuthi aqoqwe ngokusekelwe ekubeni yinkomba yomzali. Ngaphezu kwalokho, ukuze ubonise okuqukethwe kwenkomba ngendlela ejwayelekile kumsebenzisi weWindows (amafolda kuqala, bese amafayela, womabili ahlelwa ngama-alfabhethi), kuyadingeka ukufaka izinkambu ezengeziwe ezihambisanayo kukhiye.

Isithombe esingezansi sibonisa ukuthi, ngokusekelwe emsebenzini, ukumelwa kokhiye njengohlu lwamabhayithi kungase kubukeke kanjani. Okokuqala, amabhayithi anesikhombi sohla lwemibhalo lomzali (obomvu) abekwa, bese kuba nohlobo (oluluhlaza okotshani), futhi asevele asemsilani onegama (oluhlaza okwesibhakabhaka). Njengoba ahlungwa ngesiqhathanisi esizenzakalelayo se-LMDB ngokulandelana kwe-lexicographic, ahlelwa ngo indlela edingekayo. Okhiye bokunqamula ngokulandelana abanesiqalo esibomvu esifanayo basinikeza amanani ahlotshaniswa nabo ngendlela okufanele baboniswe ngayo kusixhumi esibonakalayo somsebenzisi (kwesokudla), ngaphandle kokudinga noma yikuphi ukucubungula ngemva kokucubungula okwengeziwe.

Ukulinganisa Okhiye Namagugu

Ziningi izindlela zokwenza ama-serializing izinto emhlabeni jikelele. Njengoba besingenayo enye imfuneko ngaphandle kwesivinini, sizikhethele esheshayo ngangokunokwenzeka - indawo yokulahla inkumbulo ethathwe yisibonelo sesakhiwo solimi C. Ngakho, ukhiye wesici sohla lwemibhalo ungalingiswa ngesakhiwo esilandelayo. NodeKey.

typedef struct NodeKey {​
    EntityId parentId;​
    uint8_t type;​
    uint8_t nameBuffer[256];​
} NodeKey;

Zokonga NodeKey esidingweni sokugcina entweni MDB_val beka i-pointer kudatha ekhelini lokuqala kwesakhiwo, futhi ubale usayizi wabo nomsebenzi sizeof.

MDB_val serialize(NodeKey * const key) {
    return MDB_val {
        .mv_size = sizeof(NodeKey),
        .mv_data = (void *)key
    };
}

Esahlukweni sokuqala senqubo yokukhetha isizindalwazi, ngikhulume ngokunciphisa ukwaba okuguquguqukayo njengengxenye yemisebenzi ye-CRUD njengento ebalulekile yokukhetha. Ikhodi Yomsebenzi serialize ikhombisa ukuthi, endabeni ye-LMDB, ingagwenywa kanjani ngokuphelele lapho amarekhodi amasha efakwa kusizindalwazi. Amalungu afanayo angenayo amabhayithi asuka kuseva aqale aguqulwe abe izakhiwo zesitaki, bese aphonswa kancane kusitoreji. Njengoba kunikezwe ukuthi azikho futhi izabelo eziguqukayo ngaphakathi kwe-LMDB, ungathola isimo esihle ngezindinganiso ze-iOS - sebenzisa inkumbulo yesitaki kuphela ukuze usebenze nedatha kusukela kunethiwekhi kuye kudiski!

Uku-oda okhiye ngesiqhathanisi kanambambili

Ukuhlobana kwe-oda elingukhiye kunikezwa umsebenzi okhethekile obizwa ngokuthi isiqhathanisi. Njengoba injini ingazi lutho nge-semantics yamabhayithi ewaqukethe, isiqhathanisi esizenzakalelayo asinakho ukukhetha ngaphandle kokuhlela okhiye ngokulandelana kwe-lexicographic, besebenzisa ukuqhathanisa kwabo kwe-byte-by-byte. Ukuyisebenzisa ukuhlela izakhiwo kufana nokushefa ngembazo eqoshiwe. Nokho, ezimweni ezilula, ngithola le ndlela iyamukeleka. Okunye kuchazwe ngezansi, kodwa lapha ngizophawula amaraki ambalwa ahlakazekile endleleni.

Into yokuqala okufanele uyikhumbule ukumelwa kwenkumbulo yezinhlobo zedatha yakudala. Ngakho, kuwo wonke amadivayisi we-Apple, okuguquguqukayo okuphelele kugcinwa ngefomethi Endian omncane. Lokhu kusho ukuthi ibhayithi ebaluleke kakhulu izoba kwesokunxele, futhi ngeke ukwazi ukuhlunga izinombolo usebenzisa ukuqhathanisa kwazo kwe-byte-by-byte. Isibonelo, ukuzama ukwenza lokhu ngeqoqo lezinombolo ukusuka ku-0 kuye ku-511 kuzoholela kumphumela olandelayo.

// value (hex dump)
000 (0000)
256 (0001)
001 (0100)
257 (0101)
...
254 (fe00)
510 (fe01)
255 (ff00)
511 (ff01)

Ukuze kuxazululwe le nkinga, izinombolo kufanele zigcinwe kukhiye ngefomethi efanele isiqhathanisi sebhayithi. Imisebenzi evela emndenini izosiza ukwenza uguquko oludingekayo. hton* (ngokuqondene htons izinombolo zebhayithi ephindwe kabili kusukela kusibonelo).

Ifomethi yokumela iyunithi yezinhlamvu ezinhlelweni, njengoba wazi, iphelele umlando. Uma i-semantics yezintambo, kanye nombhalo wekhodi osetshenziselwa ukuwamela enkumbulweni, uphakamisa ukuthi kungase kube ne-byte engaphezu kweyodwa ngohlamvu ngalunye, kungcono ukushiya ngokushesha umqondo wokusebenzisa isiqhanisi esizenzakalelayo.

Into yesibili okufanele uyikhumbule izimiso zokuqondanisa i-struct field compiler. Ngenxa yabo, ama-byte anamanani kadoti angakhiwa enkumbulweni phakathi kwezinkambu, okuyinto, yiqiniso, ephula ukuhlelwa kwe-byte. Ukuze uqede udoti, kufanele umemezele izinkambu ngendlela echazwe ngokuqinile, ugcine imithetho yokuqondanisa engqondweni, noma usebenzise isici esimemezelweni sesakhiwo. packed.

Ukuhleleka kukakhiye ngesiqhanisi sangaphandle

I-logic yokuqhathanisa eyinhloko ingase ibonakale iyinkimbinkimbi kakhulu kumqhathanisi kanambambili. Esinye sezizathu eziningi ukuba khona kwezinkambu zobuchwepheshe ngaphakathi kwezakhiwo. Ngizofanekisa ukuvela kwazo esibonelweni sokhiye osevele ujwayelekile kithi wesici sohla lwemibhalo.

typedef struct NodeKey {​
    EntityId parentId;​
    uint8_t type;​
    uint8_t nameBuffer[256];​
} NodeKey;

Kubo bonke ubulula bayo, ezikhathini eziningi idla inkumbulo eningi kakhulu. Isihloko sebhafa singamabhayithi angu-256, nakuba ngokwesilinganiso samagama wefayela namafolda awavamile ukweqa izinhlamvu ezingu-20-30.

Enye yezindlela ezijwayelekile zokuthuthukisa usayizi werekhodi ukulinquma ukuze lilingane nosayizi walo wangempela. Ingqikithi yayo iwukuthi okuqukethwe kuzo zonke izinkambu zobude obuguquguqukayo kugcinwa ku-buffer ekupheleni kwesakhiwo, futhi ubude bazo bugcinwa eziguquguqukayo ezihlukene. Ngokuvumelana nale ndlela, ukhiye NodeKey iguqulwa ngendlela elandelayo.

typedef struct NodeKey {​
    EntityId parentId;​
    uint8_t type;​
    uint8_t nameLength;​
    uint8_t nameBuffer[256];​
} NodeKey;

Ngaphezu kwalokho, ngesikhathi sokwenziwa kwe-serial, akucaciswanga njengosayizi wedatha sizeof sonke isakhiwo, kanye nosayizi wazo zonke izinkambu ubude obugxilile kanye nosayizi wengxenye esetshenziswe ngempela yebhafa.

MDB_val serialize(NodeKey * const key) {
    return MDB_val {
        .mv_size = offsetof(NodeKey, nameBuffer) + key->nameLength,
        .mv_data = (void *)key
    };
}

Njengomphumela wokwenza kabusha, sithole ukonga okukhulu esikhaleni esithathwe okhiye. Nokho, ngenxa yomkhakha wezobuchwepheshe nameLength, isiqhathanisi kanambambili esimisiwe asisafanele ukuqhathanisa ukhiye. Uma singalimiseleli sifake elethu, ubude begama buzoba yinto ebaluleke kakhulu ekuhlungeni kunegama ngokwalo.

I-LMDB ivumela isizindalwazi ngasinye ukuthi sibe nomsebenzi wayo obalulekile wokuqhathanisa. Lokhu kwenziwa ngokusebenzisa uhlelo mdb_set_compare ngokuqinile ngaphambi kokuvula. Ngezizathu ezisobala, isizindalwazi asikwazi ukushintshwa kukho konke ukuphila kwayo. Kokufakayo, isiqhathanisi sithola okhiye ababili ngefomethi kanambambili, futhi ekuphumeni sibuyisela umphumela wokuqhathanisa: ngaphansi kuka-(-1), okukhulu kuno-(1) noma okulingana (0). I-pseudocode ye NodeKey kubukeka kanjalo.

int compare(MDB_val * const a, MDB_val * const b) {​
    NodeKey * const aKey = (NodeKey * const)a->mv_data;​
    NodeKey * const bKey = (NodeKey * const)b->mv_data;​
    return // ...
}​

Inqobo nje uma bonke okhiye abakusizindalwazi bewuhlobo olufanayo, kusemthethweni ukusakaza ngokungenamibandela ukumelwa kwebhayithi kuhlobo lwesakhiwo sohlelo lokhiye. Kukhona i-nuance eyodwa lapha, kodwa kuzoxoxwa ngayo kancane esigatshaneni esithi "Ukufunda Amarekhodi".

Ukwenziwa Kwevalue

Ngokhiye bamarekhodi agciniwe, i-LMDB isebenza ngokujulile. Ziqhathaniswa nomunye nomunye ngaphakathi kohlaka lwanoma yikuphi ukusebenza kwesicelo, futhi ukusebenza kwesixazululo sonke kuncike ngesivinini somqhathanisi. Ezweni elikahle, isifanisi esizenzakalelayo kanambambili kufanele sanele ukuqhathanisa okhiye, kodwa uma bekumele ngempela usebenzise esakho, inqubo yokususa okhiye kuyo kufanele isheshe ngangokunokwenzeka.

Isizindalwazi asinantshisekelo ikakhulukazi engxenyeni ye-Value yerekhodi (inani). Ukuguqulwa kwayo kusuka ekumeleleni kwebhayithi kuya entweni kwenzeka kuphela uma isivele idingwa ikhodi yohlelo lokusebenza, isibonelo, ukuyibonisa esikrinini. Njengoba lokhu kwenzeka kuyaqabukela uma kuqhathaniswa, izidingo zejubane lale nqubo azibucayi kangako, futhi ekusetshenzisweni kwayo sikhululeke kakhulu ukugxila ekululameni.Isibonelo, ukwenza uchungechunge lwemethadatha mayelana namafayela angakalandwa, sisebenzisa. NSKeyedArchiver.

NSData *data = serialize(object);​
MDB_val value = {​
    .mv_size = data.length,​
    .mv_data = (void *)data.bytes​
};

Nokho, kunezikhathi lapho ukusebenza kubalulekile. Isibonelo, lapho silondoloza imininingwane ye-meta mayelana nesakhiwo sefayela lefu lomsebenzisi, sisebenzisa into efanayo yokulahla inkumbulo. Okugqamile komsebenzi wokukhiqiza ukumelwa kwawo okulandelanayo iqiniso lokuthi izici zohla lwemibhalo zimodelwa ukulandelana kwesigaba.​

Ukuze kusetshenziswe ngolimi C, izinkambu ezithile zezindlalifa zikhishwa zibe izakhiwo ezihlukene, futhi ukuxhumana kwazo nesisekelo kucaciswe ngensimu yohlobo lwenyunyana. Okuqukethwe kwangempela kwenyunyana kucaciswa ngohlobo lwesibaluli sobuchwepheshe.

typedef struct NodeValue {​
    EntityId localId;​
    EntityType type;​
    union {​
        FileInfo file;​
        DirectoryInfo directory;​
    } info;​
    uint8_t nameLength;​
    uint8_t nameBuffer[256];​
} NodeValue;​

Ukwengeza nokubuyekeza okufakiwe

Ukhiye we-serialized kanye nenani kungangezwa esitolo. Kulokhu, umsebenzi usetshenziswa mdb_put.

// key и value имеют тип MDB_val​
mdb_put(..., &key, &value, MDB_NOOVERWRITE);

Esigabeni sokumisa, inqolobane ingavunyelwa noma ivinjelwe ukugcina amarekhodi amaningi ngokhiye ofanayo.​ Uma ukuphindwaphindwa kokhiye kunqatshelwe, lapho-ke ufaka irekhodi, unganquma ukuthi ukubuyekeza irekhodi eselivele likhona kuvunyelwe noma cha. Uma ukukhwabanisa kungenzeka kuphela ngenxa yephutha kukhodi, ungaqinisekisa ngakho ngokucacisa ifulegi. NOOVERWRITE.

Ukufunda Amarekhodi

Umsebenzi wokufunda amarekhodi ku-LMDB ngu mdb_get. Uma ipheya enani elingukhiye imelwe izakhiwo ezilahliwe ngaphambilini, le nqubo ibukeka kanje.

NodeValue * const readNode(..., NodeKey * const key) {​
    MDB_val rawKey = serialize(key);​
    MDB_val rawValue;​
    mdb_get(..., &rawKey, &rawValue);​
    return (NodeValue * const)rawValue.mv_data;​
}

Uhlu olwethulwe lubonisa ukuthi ukuhlelwa kabusha ngokulahlwa kwezakhiwo kukuvumela kanjani ukuthi ukhiphe izabelo eziguqukayo hhayi kuphela lapho ubhala, kodwa lapho ufunda idatha. Ithathwe kumsebenzi mdb_get i-pointer ibheka ngqo ekhelini lememori ebonakalayo lapho isizindalwazi sigcina khona ukumelwa kwe-byte kwento. Eqinisweni, sithola uhlobo lwe-ORM, cishe mahhala luhlinzeka ngesivinini esikhulu kakhulu sokufunda idatha. Ngabo bonke ubuhle bendlela, kuyadingeka ukukhumbula izici eziningana ezihambisana nayo.

1. Ngokwenziwayo kokufunda kuphela, isikhombi esakhiweni senani siqinisekiswa ukuthi sizohlala sisebenza kuze kube yilapho okwenziwayo kuvalwa. Njengoba kuphawuliwe ngaphambili, amakhasi esihlahla sika-B lapho into ehlala khona, ngenxa yomgomo wokukopisha-ngokubhala, ahlala engashintshile inqobo nje uma okungenani okwenziwayo okukodwa kubhekisela kuwo. Ngesikhathi esifanayo, ngokushesha nje lapho umsebenzi wokugcina ohlobene nazo usuqediwe, amakhasi angaphinda asetshenziswe kudatha entsha. Uma kudingekile ukuthi izinto zisinde emsebenzini ozidalile, kuzomele zikopishwe.
2. Ngomsebenzi wokubhala funda, isikhombisi senani lesakhiwo esiwumphumela sizosebenza kuze kube yinqubo yokuqala yokulungisa (ukubhala noma ukususa idatha).
3. Nakuba isakhiwo NodeValue hhayi ngokugcwele, kodwa kunqunyiwe (bheka isigatshana esithi "Okhiye boku-oda ngesiqhanisi sangaphandle"), ngokusebenzisa i-pointer, ungakwazi ukufinyelela kalula izinkambu zayo. Into esemqoka ukuthi ungayihoxisi!
4. Akunakwenzeka ukuthi uguqule isakhiwo ngokusebenzisa i-pointer etholiwe. Zonke izinguquko kufanele zenziwe kuphela ngokusebenzisa indlela mdb_put. Kodwa-ke, ngesifiso sokwenza lokhu, ngeke kusebenze, ngoba indawo yenkumbulo lapho lesi sakhiwo sitholakala khona imephu yemodi yokufunda kuphela.
5. Mepha kabusha ifayela endaweni yekheli lenqubo ukuze, isibonelo, ukwandise ubukhulu bosayizi wesitoreji usebenzisa umsebenzi mdb_env_set_map_size iqeda ngokuphelele konke okwenziwayo kanye nezinhlangano ezihlobene ngokujwayelekile kanye nezinkomba zokufunda izinto ikakhulukazi.

Okokugcina, esinye isici sinobuqili kangangokuthi ukudalulwa kwengqikithi yaso akungeni kwelinye iphuzu elilodwa nje. Esahlukweni esiku-B-tree, nginikeze umdwebo wenhlangano yamakhasi ayo enkumbulweni. Kuyalandela ukuthi ikheli lesiqalo sebhafa elinedatha ye-serialized lingaba ngokunganaki nhlobo. Ngenxa yalokhu, i-pointer kubo, etholakala esakhiweni MDB_val futhi ukuphonswa kusikhombisi esakhiweni ngokuvamile akuhambelanisiwe. Ngasikhathi sinye, izakhiwo zamanye ama-chips (esimweni se-iOS, lena i-armv7) zidinga ukuthi ikheli lanoma iyiphi idatha libe ukuphindaphinda kosayizi wegama lomshini, noma, ngamanye amazwi, ubuncane besistimu. (ye-armv7, lokhu ngamabhithi angama-32). Ngamanye amazwi, ukuhlinzwa okufana *(int *foo)0x800002 phezu kwabo kufana nokuphunyuka futhi kuholela ekubulaweni ngesinqumo EXC_ARM_DA_ALIGN. Kunezindlela ezimbili zokugwema isiphetho esinjalo.

Esokuqala siwukukopisha idatha esakhiweni esiqondaniswe esaziwayo ngaphambili. Isibonelo, kusiqhathanisi sangokwezifiso, lokhu kuzobonakala kanje.

int compare(MDB_val * const a, MDB_val * const b) {
    NodeKey aKey, bKey;
    memcpy(&aKey, a->mv_data, a->mv_size);
    memcpy(&bKey, b->mv_data, b->mv_size);
    return // ...
}

Enye indlela iwukwazisa umdidiyeli kusenesikhathi ukuthi izakhiwo ezinokhiye nevelu zingase zingaqondani kusetshenziswa isibaluli. aligned(1). Ku-ARM umphumela ofanayo ungaba zuza nokusebenzisa isibaluli esigcwele. Uma kucatshangelwa ukuthi futhi kunomthelela ekwandiseni isikhala esithathwe isakhiwo, le ndlela ibonakala ingcono kimina, nakuba приводит ukwandisa izindleko zokusebenza zokufinyelela idatha.

typedef struct __attribute__((packed)) NodeKey {
    uint8_t parentId;
    uint8_t type;
    uint8_t nameLength;
    uint8_t nameBuffer[256];
} NodeKey;

Ibanga lemibuzo

Ukuphindaphinda phezu kweqembu lamarekhodi, i-LMDB ihlinzeka ngokufushaniswa kwekhesa. Ake sibheke ukuthi singasebenza kanjani ngayo sisebenzisa isibonelo setafula elinemethadatha yamafu yomsebenzisi esivele siyijwayele.

Njengengxenye yokubonisa uhlu lwamafayela ohlwini lwemibhalo, udinga ukuthola zonke izihluthulelo ezihlotshaniswa nazo amafayela ezingane namafolda. Ezigatshaneni ezedlule, sihlele okhiye NodeKey ukuze ziqale zi-odwe nge-ID yazo yohla lwemibhalo yomzali. Ngakho-ke, ngobuchwepheshe, umsebenzi wokuthola okuqukethwe kwefolda uncishisiwe ekubekeni ikhesa emngceleni ongaphezulu weqembu lezikhiye ezinesiqalo esinikeziwe, okulandelwa ukuphindaphinda emngceleni ophansi.

Ungathola okuboshwe phezulu "ebunzini" ngokusesha okulandelanayo. Ukuze wenze lokhu, ikhesa ibekwe ekuqaleni kwalo lonke uhlu lokhiye ku-database bese inyuswa kuze kube yilapho ukhiye onesikhombi sohla lwemibhalo uvela ngezansi kwawo. Le ndlela inezingqinamba ezi-2 ezisobala:

1. Ubunkimbinkimbi bomugqa bokusesha, nakuba, njengoba wazi, ezihlahleni zizonke futhi ikakhulukazi esihlahleni esingu-B, kungenziwa ngesikhathi se-logarithmic.
2. Ngeze, wonke amakhasi andulela elifiswayo aphakanyiswa kusuka kufayela kuya kumemori eyinhloko, ebiza kakhulu.

Ngenhlanhla, i-LMDB API inikeza indlela ephumelelayo yokubeka ikhesa ekuqaleni. Ukuze wenze lokhu, udinga ukwakha ukhiye ivelu yawo eyaziwa ngokuthi ingaphansi noma ilingana nokhiye otholakala engxenyeni engenhla yesikhawu. Isibonelo, maqondana nohlu kumfanekiso ongenhla, singenza ukhiye lapho insimu parentId izolingana no-2, futhi bonke abanye bagcwaliswe ngoziro. Ukhiye onjalo ogcwaliswe kancane unikezwa okokufaka komsebenzi mdb_cursor_get ekhombisa ukusebenza MDB_SET_RANGE.

NodeKey upperBoundSearchKey = {​
    .parentId = 2,​
    .type = 0,​
    .nameLength = 0​
};​
MDB_val value, key = serialize(upperBoundSearchKey);​
MDB_cursor *cursor;​
mdb_cursor_open(..., &cursor);​
mdb_cursor_get(cursor, &key, &value, MDB_SET_RANGE);

Uma umkhawulo ongaphezulu weqembu lokhiye utholakala, siyaphinda-ke kuze kube yilapho sihlangana noma ukhiye nomunye. parentId, noma izikhiye ngeke ziphele nhlobo.

do {​
    rc = mdb_cursor_get(cursor, &key, &value, MDB_NEXT);​
    // processing...​
} while (MDB_NOTFOUND != rc && // check end of table​
         IsTargetKey(key));    // check end of keys group​​

Yini enhle, njengengxenye yokuphindaphinda usebenzisa mdb_cursor_get, asitholi kuphela ukhiye, kodwa futhi nenani. Uma, ukuze kugcwaliseke izimo zokukhetha, kudingekile ukuhlola, phakathi kwezinye izinto, izinkambu ezivela engxenyeni yenani lerekhodi, khona-ke zifinyeleleka ngempela kuzo ngaphandle kokuthinta okwengeziwe.

4.3. Ukumodela ubudlelwano phakathi kwamathebula

Kuze kube manje, sikwazile ukucabangela zonke izici zokuklama nokusebenza ngesizindalwazi sethebula elilodwa. Singasho ukuthi ithebula isethi yamarekhodi ahlungiwe ahlanganisa amapheya enani elingukhiye ohlobo olufanayo. Uma ubonisa ukhiye njengonxande kanye nenani elihlobene nalo njengebhokisi, uthola umdwebo obonakalayo wesizindalwazi.

A

Nokho, ekuphileni kwangempela, akuvamile ukuba ukwazi ukuphila ngegazi elincane kangaka. Ngokuvamile ku-database kuyadingeka, okokuqala, ukuba namatafula amaningana, futhi okwesibili, ukwenza okukhethiwe kuwo ngokulandelana okuhlukile kukhiye oyinhloko. Lesi sigaba sokugcina sigxile ezindabeni zokudalwa kwazo kanye nokuxhumana kwazo.

Amathebula enkomba

Uhlelo lokusebenza lwamafu lunesigaba esithi "Igalari". Ibonisa okuqukethwe kwemidiya kusuka kulo lonke ifu, ihlelwa ngedethi. Ukuze kusetshenziswe kahle ukukhetha okunjalo, eduze kwetafula elikhulu, udinga ukudala elinye ngohlobo olusha lokhiye. Azoqukatha inkambu enedethi ifayela elidalwe ngayo, ezosebenza njengemibandela yokuhlunga eyinhloko. Ngenxa yokuthi okhiye abasha babhekisela kudatha efanayo njengokhiye kuthebula elingaphansi, babizwa ngokuthi okhiye bezinkomba. Agqanyiswe ngowolintshi esithombeni esingezansi.

Ukuze kuhlukaniswe okhiye bamathebula ahlukene kwelinye ngaphakathi kwesizindalwazi esifanayo, i-tableId yenkambu yobuchwepheshe eyengeziwe yengezwe kuwo wonke. Ngokukwenza kube okuhamba phambili kakhulu ekuhleleni, sizoqoqa okhiye kuqala ngamathebula, futhi kakade ngaphakathi kwamathebula - ngokwemithetho yethu.​

Ukhiye wenkomba ubhekisela kudatha efanayo njengokhiye oyinhloko. Ukuqaliswa okuqondile kwalesi sakhiwo ngokuhlotshaniswa nayo ikhophi yenani lengxenye yokhiye oyinhloko kuncane kakhulu kusuka ezindaweni ezimbalwa zokubuka ngesikhathi esisodwa:

1. Ngokombono ogcwele isikhala, imethadatha ingaba inothe kakhulu.
2. Ngokombono wokusebenza, njengoba lapho ubuyekeza imethadatha yenodi, kuzodingeka ubhale phezu kokhiye ababili.
3. Ngokombono wokusekelwa kwekhodi, ngemva kwakho konke, uma sikhohlwa ukuvuselela idatha yesinye sezihluthulelo, sizothola iphutha elicashile lokungaguquki kwedatha kusitoreji.

Okulandelayo, sizocabangela ukuthi singawaqeda kanjani la maphutha.

Inhlangano yobudlelwano phakathi kwamathebula

Iphethini ifaneleka kahle ekuxhumaniseni ithebula lenkomba nelikhulu "ukhiye njengevelu". Njengoba igama layo lisho, ingxenye yenani yerekhodi lenkomba iyikhophi yenani lokhiye oyinhloko. Le ndlela iqeda zonke izinkinga ezibalwe ngenhla ezihambisana nokugcina ikhophi yengxenye yenani lerekhodi eliyinhloko. Imali kuphela ukuthi ukuze uthole inani ngokhiye wenkomba, udinga ukwenza imibuzo emi-2 kusizindalwazi esikhundleni sowodwa. Ngokuhlelekile, i-schema yedatha ewumphumela imi kanje.

Enye iphethini yokuhlela ubudlelwano phakathi kwamatafula ngu "ukhiye ongasasebenzi". Ingqikithi yayo iwukwengeza izimfanelo ezengeziwe kukhiye, ezidingekile hhayi ukuhlunga, kodwa ukuze udale kabusha ukhiye ohlobene. Kukhona izibonelo zangempela zokusetshenziswa kwayo kuhlelo lokusebenza lwe-Mail.ru Cloud, nokho, ukuze ugweme ukucwila ngaphakathi umongo wezinhlaka ezithile ze-iOS, ngizonikeza inkohliso, kodwa isibonelo esiqondakala kakhudlwana.

Amaklayenti eselula e-Cloud anekhasi elibonisa wonke amafayela namafolda umsebenzisi abelane ngawo nabanye abantu. Njengoba kunamafayela anjalo ambalwa uma kuqhathaniswa, futhi kunolwazi oluningi oluqondile mayelana nokukhangisa okuhlotshaniswa nawo (kubani abanikezwa ukufinyelela, ngamalungelo maphi, njll.), ngeke kube okunengqondo ukulithwesa umthwalo ngenani lengxenye ukungena etafuleni elikhulu. Nokho, uma ufuna ukubonisa amafayela anjalo ungaxhunyiwe ku-inthanethi, kusadingeka uwagcine ndawana thize. Isixazululo esingokwemvelo ukudala itafula elihlukile kulo. Kumdwebo ongezansi, ukhiye wayo ufakwe kuqala ngo-"P", futhi isimeli "propname" singathathelwa indawo inani eliqondile "ulwazi lomphakathi".​

Yonke imethadatha eyingqayizivele, okwakhiwe ngayo ithebula elisha, iyiswa engxenyeni yenani yerekhodi. Ngesikhathi esifanayo, angifuni ukuphinda idatha mayelana namafayela namafolda asevele agcinwe kuthebula eliyinhloko. Kunalokho, idatha engasadingeki yengezwa kukhiye othi "P" ngendlela "ye-node ID" nezinkambu "zesitembu sesikhathi". Ngenxa yabo, ungakha ukhiye wenkomba, ongathola ngawo ukhiye oyinhloko, lapho, ekugcineni, ungathola imethadatha ye-node.

Isiphetho

Siyihlola kahle imiphumela yokusetshenziswa kwe-LMDB. Ngemuva kwalokho, inani lokumiswa kwezicelo lehle ngo-30%.

Imiphumela yomsebenzi owenziwe ithole impendulo ngaphandle kwethimba le-iOS. Njengamanje, esinye sezigaba eziyinhloko "Zamafayela" kuhlelo lokusebenza lwe-Android naso sesishintshele ekusebenziseni i-LMDB, kanti ezinye izingxenye zisendleleni. Ulimi C, lapho kusetshenziswa khona isitoreji senani elingukhiye, kwaba usizo oluhle ukuze ekuqaleni kwenziwe uhlelo lokusebenza lubophezele kulo inkundla enqamulayo ngolimi lwe-C ++. Ukuze kuxhunywe okungenazihibe komtapo wezincwadi we-C ++ onekhodi yesikhulumi ku-Objective-C ne-Kotlin, kusetshenziswe umshini wokukhiqiza ikhodi. I-Djinni kusuka ku-Dropbox, kodwa leyo enye indaba.

Source: www.habr.com