sumo
Ndakapa chirevo ichi muChirungu pamusangano weGopherCon Russia 2019 muMoscow uye muchiRussia pamusangano muNizhny Novgorod. Isu tiri kutaura nezve bitmap index - isinganyanyi kuwanda pane B-muti, asi isinganyanyi kunakidza. Kugovera hurukuro pamusangano muChirungu uye zvinyorwa zvinyorwa muRussia.
Tichatarisa kuti bitmap index inoshanda sei, kana iri nani, kana yakaipisisa kupfuura mamwe ma indexes, uye mumamiriro ezvinhu api inokurumidza zvikuru kupfuura ivo; Ngationei kuti ndeapi maDBMS ane mukurumbira atova nebitmap indexes; Ngatiedzei kunyora zvedu muna Go. Uye "ye dessert" isu tichashandisa akagadzirira-akagadzirwa maraibhurari kugadzira yedu yepamusoro-nekukurumidza nyanzvi dhatabhesi.
Ndinovimba chaizvo kuti mabasa angu achave anobatsira uye anonakidza kwauri. Enda!
Nhanganyaya
Mhoroi mose! Dzave XNUMX manheru uye tese takaneta zvakanyanya. Nguva yakanaka yekutaura nezve inofinha dhatabhesi index theory, handiti? Usanetseke, ini ndichave nemitsara miviri yekodhi kodhi pano nepapo. π
Majee ese parutivi, chirevo chakazara neruzivo, uye isu hatina nguva yakawanda. Saka ngatitangei.
Nhasi ndichataura pamusoro pezvinotevera:
- chii chinonzi indexes;
- chii chinonzi bitmap index;
- painoshandiswa uye paisisiri kushandiswa uye nei;
- Kuitwa kuri nyore muGo uye kunetsa kudiki nemuunganidzi;
- zvishoma zvishoma zviri nyore, asi zvakanyanya kuita zvinobudirira muGo assembler;
- "matambudziko" e bitmap indexes;
- kushandiswa kuripo.
Saka chii chinonzi indexes?
Iyo index ndeye yakaparadzana data chimiro chatinochengetedza nekuvandudza kuwedzera kune iyo huru data. Inoshandiswa kukurumidza kutsvaga. Pasina ma indexes, kutsvaga kunoda kuenda kuburikidza nedata zvachose (maitiro anonzi akazara scan), uye maitiro aya ane mutsara algorithmic kuoma. Asi dhatabhesi anowanzo aine huwandu hukuru hwe data uye mutsara kuoma kunononoka. Zvakanakisa, isu taizowana logarithmic kana yekugara imwe.
Iyi inyaya yakaomesesa, yakazadzwa nehunyanzvi uye kutengeserana, asi mushure mekutarisa makumi emakore ekuvandudza dhatabhesi uye tsvagiridzo, ndinoda kutaura kuti kune nzira shoma dzinoshandiswa zvakanyanya kugadzira dhatabhesi indexes.
Nzira yekutanga ndeye hierarchically kuderedza nzvimbo yekutsvaga, kugovera nzvimbo yekutsvaga muzvikamu zviduku.
Tinowanzoita izvi tichishandisa marudzi akasiyana emiti. Muenzaniso ungave bhokisi hombe rezvinhu muwadhiropu yako iyo ine mabhokisi madiki ezvigadzirwa zvakakamurwa kuva misoro yakasiyana. Kana uchida zvekushandisa, ungangozvitsvaga mubhokisi rinoti "Zvishandiso" pane iro rinoti "Makuki," handiti?
Nzira yechipiri ndeyekusarudza nekukurumidza chinhu chaunoda kana boka rezvinhu. Isu tinoita izvi mune hashi mepu kana reverse indexes. Kushandisa mamepu ehashi kwakafanana nemuenzaniso wapfuura, asi pachinzvimbo chebhokisi remabhokisi, une boka remabhokisi madiki ezvinhu zvekupedzisira muwadhiropu yako.
Nzira yechitatu ndeyekubvisa kudiwa kwekutsvaga. Isu tinoita izvi tichishandisa Bloom mafirita kana cuckoo mafirita. Iwo ekutanga anopa mhinduro ipapo ipapo, achikuchengetedza kubva pakutsvaga.
Nzira yekupedzisira ndeyekushandisa zvizere simba rose iro hardware yemazuva ano inotipa. Izvi ndizvo chaizvo zvatinoita mu bitmap indexes. Ehe, kana tichivashandisa isu dzimwe nguva tinoda kuenda kuburikidza neiyo index yese, asi isu tinozviita zvakanyanya.
Sezvandakataura, musoro wedatabase indexes wakakura uye uzere nekukanganisika. Izvi zvinoreva kuti dzimwe nguva tinogona kushandisa nzira dzakati wandei panguva imwe chete: kana tichida kukurumidzira kutsvaga zvakanyanya, kana tichida kuvhara marudzi ese ekutsvaga.
Nhasi ndichataura nezve nzira isingazivikanwe yeizvi - bitmap indexes.
Ini ndini ani wekutaura nezvenyaya iyi?
Ini ndinoshanda sechikwata chinotungamira kuBadoo (pamwe iwe unonyatsoziva chimwe chigadzirwa chedu, Bumble). Isu tatova nevashandisi vanopfuura mamirioni mazana mana pasirese uye akawanda maficha anovasarudzira iwo mutambo wakanyanya kwavari. Isu tinoita izvi tichishandisa tsika masevhisi, kusanganisira bitmap indexes.
Saka chii chinonzi bitmap index?
Bitmap indexes, sezvinoreva zita racho, shandisa bitmaps kana bitsets kuti uise index yekutsvaga. Kubva pakuona kweziso reshiri, indekisi iyi ine imwe kana akawanda akadai mabitmaps anomiririra chero masangano (sevanhu) nemidziyo yavo kana maparameter (zera, ruvara rweziso, nezvimwewo), uye algorithm inoshandisa bit Operations (AND, OR, NOT. ) kupindura mubvunzo wekutsvaga.
Tinoudzwa kuti bitmap indexes akanyatsokodzera uye anoita zvakanyanya kuzviitiko uko kune tsvakiridzo dzinosanganisa mibvunzo kune dzakawanda dzakaderera makadhiinality columns (funga "ruvara rweziso" kana "chimiro chewanano" maringe nechimwe chinhu senge "chinhambwe kubva pakati peguta" ). Asi ini ndicharatidza gare gare kuti vanoshanda zvakanaka kune yakakwirira makadhiinality makoramu zvakare.
Ngatitarisei muenzaniso wakapusa weiyo bitmap index.
Fungidzira kuti isu tine runyorwa rwemaresitorendi eMoscow ane mabhinari zvivakwa seizvi:
- pedyo metro;
- pane private parking;
- kune veranda (ine terrace);
- iwe unogona kuchengetedza tafura (inobvuma kuchengetedza);
- yakakodzera kune vanodya miriwo (vegan friendly);
- zvinodhura (zvinodhura).
Ngatipei resitorendi yega yega nhamba yekutevedzana kubva pa0 uye tigovere ndangariro ye6 bitmaps (imwe yehunhu hwega hwega). Isu tichazozadza ma bitmaps aya zvichienderana nekuti resitorendi ine pfuma iyi here kana kuti kwete. Kana resitorendi 4 ine veranda, ipapo bit No.
Zvino isu tine yakapfava bitmap index inogoneka, uye isu tinogona kuishandisa kupindura mibvunzo senge:
- βNdiratidze maresitorendi asingadyi nyamaβ;
- "Ndiratidze maresitorendi asingadhure ane veranda paunogona kuchengetera tafura."
Sei? Ngationei. Chikumbiro chekutanga chiri nyore kwazvo. Zvese zvatinoda kuita kutora iyo "vegetarian hushamwari" bitmap uye kuishandura kuita runyorwa rwemaresitorendi ane mabheti akaburitswa.
Chikumbiro chechipiri chakanyanya kuoma. Isu tinofanirwa kushandisa iyo NOT bitmap pane "inodhura" bitmap kuti tiwane runyoro rwemaresitorendi asingadhure, zvino UYE neiyo "ndinogona kubhuka tafura" bitmap uye UYE mhedzisiro ine "kune veranda" bitmap. Iyo inoguma bitmap ichange iine rondedzero yezvivakwa zvinosangana neyedu maitiro. Mumuenzaniso uyu, iyi ingori resitorendi yeYunost.
Pane dzidziso yakawanda inobatanidzwa, asi usazvinetse, tichaona iyo kodhi munguva pfupi.
Bitmap indexes inoshandiswa kupi?
Kana iwe Google bitmap indexes, 90% yemhinduro ichave ine hukama neOracle DB neimwe nzira. Asi mamwe maDBMS pamwe anotsigirawo chinhu chinotonhorera chakadaro, handiti? Kwete saizvozvo.
Ngatipfuurei rondedzero yevakuru vanofungirwa.
MySQL haisati yatsigira bitmap indexes, asi pane Proposal inokurudzira kuwedzera iyi sarudzo ().
PostgreSQL haitsigire bitmap indexes, asi inoshandisa yakapfava bitmaps uye bit mashandiro kusanganisa mibairo yekutsvaga pane akawanda akawanda indexes.
Tarantool ine bitset indexes uye inotsigira nyore kutsvaga pazviri.
Redis ine nyore bitfields) pasina kukwanisa kuvatsvaga.
MongoDB haisati yatsigira bitmap indexes, asi pane zvakare Proposal inoratidza kuti iyi sarudzo iwedzerwe.
Elasticsearch inoshandisa bitmaps mukati).
- Asi muvakidzani mutsva akaonekwa mumba medu: Pilosa. Iri ibhuku idzva risiri rehukama rakanyorwa muGo. Iyo ine chete bitmap indexes uye inogadzika zvese pazviri. Tichazotaura nezvazvo gare gare.
Implementation muna Go
Asi nei bitmap indexes isingawanzo shandiswa? Ndisati ndapindura mubvunzo uyu, ndinoda kukuratidza maitiro ekuita yakapusa bitmap index muGo.
Bitmaps inongova zvidimbu zve data. MuGo, ngatishandisei byte zvimedu zveizvi.
Tine bitmap imwe yehunhu hweresitorendi, uye chidimbu chega chega mubitmap chinoratidza kuti imwe resitorendi ine chivakwa ichi here kana kuti kwete.
Tichada mabasa maviri emubatsiri. Imwe ichashandiswa kuzadza bitmaps edu neasina kurongeka data. Random, asi neimwe mukana wekuti resitorendi ine midziyo yega yega. Semuenzaniso, ndinotenda kuti kune maresitorendi mashoma muMoscow kwausingagone kuchengetedza tafura, uye zvinoratidzika kwandiri kuti inenge 20% yezvivakwa zvakakodzerwa nevanodya zvirimwa.
Chechipiri basa rinoshandura bitmap kuita runyorwa rwemaresitorendi.
Kuti tipindure mubvunzo wekuti βNdiratidze maresitorendi asingadhure ane patio uye anogona kubhuka,β tinoda mavhisisheni maviri: KWETE uye UYE.
Tinogona kurerutsa kodhi yedu zvishoma nekushandisa yakanyanya kuomarara UYE kwete mushandisi.
Tine mabasa echimwe nechimwe chezviito izvi. Vese vari vaviri vanopinda nepakati pezvimedu, tora izvo zvinoenderana kubva kune imwe neimwe, zvisanganise pamwe nekuita zvishoma oparesheni uye isa mhedzisiro muchikamu chinobuda.
Uye ikozvino isu tinogona kushandisa yedu bitmaps uye mabasa kupindura mubvunzo wekutsvaga.
Kuita hakuna kukwira zvakanyanya, kunyangwe mabasa acho ari nyore uye isu takachengeta mari yakawanda nekusadzosera chidimbu chitsva chinobuda pese paidanwa basa.
Mushure mekuita zvishoma zveprofile ne pprof, ndakaona kuti Go compiler yaishaya imwe yakapusa asi yakakosha optimization: basa inlining.
Icho chokwadi ndechekuti Go compiler inotya zvishwe zvinopinda nepakati, uye zvinoramba zvine mutsindo kuita inline mabasa ane zvishwe zvakadaro.
Asi ini handisi kutya uye ndinogona kunyengedza mugadziri nekushandisa goto pachinzvimbo chechiuno, senge mumazuva akanaka ekare.
Uye, sezvauri kuona, ikozvino mugadziri achafara inline basa redu! Nekuda kweizvozvo, isu tinokwanisa kuchengetedza anenge maviri mamicroseconds. Kusaipa!
Yechipiri bhodhoro iri nyore kuona kana iwe ukanyatsotarisisa pagungano rinobuda. Iyo compiler yakawedzera cheki muganho cheki mukati chaimo chedu chinopisa loop. Icho chokwadi ndechekuti Go mutauro wakachengeteka, mugadziri anotya kuti nharo dzangu nhatu (zvimedu zvitatu) ndezvehukuru hwakasiyana. Mushure mezvose, ipapo pachava neinoretical mukana wekuitika kweinonzi buffer mafashama.
Ngativimbisei muumbi nekuratidza kuti zvidimbu zvese zvakaenzana. Tinogona kuita izvi nekuwedzera cheki iri nyore pakutanga kwebasa redu.
Achiona izvi, muunganidzi anosvetuka cheki nemufaro, uye tinopedzisira tachengeta mamwe mazana mashanu nanoseconds.
Mabhutu mahombe
Zvakanaka, isu takakwanisa kudzvanya kumwe kuita kunze kwekuita kwedu kuri nyore, asi mhedzisiro iyi yatonyanya kuipa kupfuura zvinogoneka nehardware yazvino.
Zvese zvatinoita ndeyekutanga bit mashandiro, uye ma processors edu anoaita nemazvo. Asi, zvinosuruvarisa, isu "tinodyisa" processor yedu nezvidimbu zvidiki zvebasa. Mabasa edu anoita mashandiro pane byte-by-byte hwaro. Isu tinogona nyore nyore kugadzirisa kodhi yedu kushanda ne8-byte chunks tichishandisa UInt64 zvimedu.
Sezvauri kuona, shanduko diki iyi yakawedzera chirongwa chedu kasere nekuwedzera saizi yebatch kasere. Kuwana kunogona kunzi kune mutsara.
Implementation in assembler
Asi uku hakusi kuguma. Mapurosesa edu anogona kushanda nemachunks e16, 32 uye kunyange 64 bytes. Zvakadai "zvakafara" mashandiro anonzi single instruction multiple data (SIMD; one instruction, many data), uye maitiro ekushandura kodhi kuti ashandise maoperation akadaro anonzi vectorization.
Nehurombo, iyo Go compiler iri kure nekunaka pavectorization. Parizvino, iyo chete nzira yekuvector Go kodhi kutora uye kuisa aya mavhisi nemaoko uchishandisa Go assembler.
Go assembler chikara chinoshamisa. Iwe unogona kunge uchiziva kuti mutauro wegungano chinhu chakasungirirwa zvakanyanya kune mavakirwo ekombuta yauri kunyorera, asi handizvo zviri muGo. Go assembler yakafanana neIRL (mutauro wekumiririra wepakati) kana mutauro wepakati: inongoita yakazvimirira papuratifomu. Rob Pike akapa kuita kwakanakisa pane iyi nyaya makore akati wandei apfuura paGopherCon muDenver.
Mukuwedzera, Go inoshandisa isina kujairika Plan 9 fomati, iyo inosiyana kubva kune inogamuchirwa kazhinji AT&T uye Intel mafomati.
Zvakachengeteka kutaura kuti kunyora Go assembler nemaoko hakusi kunonyanya kunakidza.
Asi, nerombo rakanaka, kwatove nematurusi maviri epamusoro-soro anotibatsira kunyora Go assembler: PeachPy uye avo. Zvese zvinoshandiswa zvinogadzira Go assembler kubva kumusoro-level kodhi yakanyorwa muPython uye Go, zvichiteerana.
Zvishandiso izvi zvinorerutsa zvinhu zvakaita sekugovewa kwerejista, kunyora zvishwe, uye kazhinji kurerutsa maitiro ekupinda munyika yehurongwa hwegungano muGo.
Tichashandisa avo, saka zvirongwa zvedu zvinenge zviri zvenguva dzose Go zvirongwa.
Izvi ndizvo zvakaita muenzaniso wakapfava weavo chirongwa chinotaridzika. Tine main () function, inotsanangura mukati mayo iyo Add() function, zvinoreva kuwedzera nhamba mbiri. Pane mabasa emubatsiri pano kuti atore parameter nemazita uye uwane imwe yemahara uye akakodzera processor marejista. Imwe neimwe processor inoshanda ine inoenderana basa pane avo, sezvinoonekwa muADDQ. Pakupedzisira, tinoona basa rekubatsira kuchengetedza kukosha kunoguma.
Nekufona enda gadzira, isu tichaita chirongwa pane avo uye semhedzisiro, mafaera maviri achagadzirwa:
- add.s ine kodhi inoguma muGo assembler;
- stub.go ine basa misoro yekubatanidza nyika mbiri: Enda uye assembler.
Zvino zvataona zvinoita avo uye sei, ngatitarisei mabasa edu. Ndakashandisa ese scalar uye vector (SIMD) shanduro dzemabasa.
Ngatitarisei kune scalar shanduro kutanga.
Senge mumuenzaniso wapfuura, isu tiri kukumbira yemahara uye inoshanda general chinangwa rejista, isu hatifanire kuverenga maoffset uye saizi yenharo. avo vanotiitira zvose izvi.
Isu taimboshandisa mavara uye goto (kana kusvetuka) kuvandudza mashandiro uye kunyengedza Go compiler, asi ikozvino tiri kuzviita kubva pakutanga. Chiripo ndechekuti matenderedzwa ipfungwa yepamusoro-soro. Muassembler, isu tine mavara chete uye kusvetuka.
Iyo kodhi yasara inofanira kunge yatoziva uye inonzwisisika. Isu tinotevedzera loop ine mavara uye kusvetuka, tora chidimbu chidiki che data kubva kune edu maviri zvidimbu, tozvisanganisa nea bit operation (UYE KWETE mune iyi kesi) uye toisa mhedzisiro muchikamu chinobuda. Zvose.
Izvi ndizvo zvinoita seyekupedzisira assembler kodhi. Hatina kufanira kuverenga maoffset nemasaizi (akaiswa mugirini) kana kuchengeta marejista aishandiswa (akaiswa mutsvuku).
Kana tikaenzanisa kushanda kwemusangano wemutauro wegungano nekushanda kwekushanda kwakanakisisa muGo, tichaona kuti zvakafanana. Uye izvi zvinotarisirwa. Mushure mezvose, isu hatina kuita chero chinhu chakakosha - isu takangoburitsa zvaizoitwa neGo compiler.
Nehurombo, hatigone kumanikidza muunganidzi kuti aise mumutsara mabasa edu akanyorwa mumutauro wegungano. Iyo Go compiler parizvino haina chimiro chakadaro, kunyangwe paine chikumbiro chekuwedzera kwenguva yakati rebei.
Ichi ndicho chikonzero zvisingabviri kuwana chero rubatsiro kubva kumabasa maduku mumutauro wegungano. Isu tinofanirwa kunyora mabasa makuru, kana kushandisa iyo nyowani yemasvomhu/bits package, kana kunzvenga mutauro wekuunganidza.
Ngatitarisei kune vector shanduro dzemabasa edu.
Pamuenzaniso uyu, ndafunga kushandisa AVX2, saka tichashandisa maoperation anoshanda pa32-byte chunks. Chimiro chekodhi chakafanana chaizvo neiyo scalar vhezheni: kurodha paramita, kukumbira yemahara yakagovaniswa rejista, nezvimwe.
Imwe hunyanzvi ndeyekuti yakakura vector mashandiro anoshandisa akasarudzika marejista. Kana iri 32-byte chunks, aya marejista ane prefixed neY. Ichi ndicho chikonzero iwe uchiona YMM () basa mukodhi. Dai ndanga ndichishandisa AVX-512 ine 64-bit chunks, prefix ingadai iri Z.
Yechipiri hunyanzvi ndeyekuti ndakafunga kushandisa optimization inonzi loop unrolling, zvinoreva kuita sere loop oparesheni nemaoko ndisati ndasvetuka kusvika pakutanga kwechiuno. Iyi optimization inoderedza huwandu hwematavi mukodhi, uye inogumira nenhamba yemahara marejista aripo.
Zvakanaka, zvakadini nekuita? Akanaka! Isu takawana kukurumidza kwekanomwe kana tichienzanisa neyakanakisa Go mhinduro. Zvinoshamisa, handiti?
Asi kunyangwe kuita uku kunogona kukwidziridzwa nekushandisa AVX-512, prefetching kana JIT (inongova-mu-nguva compiler) yemubvunzo wekuronga. Asi iyi ichokwadi musoro wemushumo wakasiyana.
Matambudziko ne bitmap indexes
Zvino zvatatotarisa kuita kuri nyore kweiyo bitmap index muGo uye yakanyanya kugadzirwa mumutauro wegungano, ngatitaurei kuti sei bitmap indexes isingawanzo shandiswa.
Mapepa ekare anotaura matambudziko matatu ane bitmap indexes, asi mapepa matsva uye ini ndinopokana kuti haasisina basa. Hatisi kuzonyura zvakadzama mune rimwe nerimwe rematambudziko aya, asi tichaatarisa pamusoro pawo.
Dambudziko repamusoro makadhiinality
Saka, tinoudzwa kuti bitmap indexes inokodzera chete minda ine low cardinality, kureva, iyo ine mashoma mashoma (semuenzaniso, murume kana ziso ruvara), uye chikonzero ndechekuti iyo yakajairwa inomiririra yeminda yakadaro (imwe. bit per value) kana iri high cardinality, inotora nzvimbo yakawandisa uye, uyezve, idzi bitmap indexes dzichave dzisina kunaka (kashoma) kuzadzwa.
Dzimwe nguva tinogona kushandisa chinomiririra chakasiyana, sechiyero chatinoshandisa kumiririra nhamba. Asi kwaive kuuya kwekumanikidza algorithms kwakachinja zvese. Mumakumi emakore apfuura, masayendisiti nevatsvaguri vauya nenhamba huru yekumanikidza algorithms yebitmaps. Chavo chikuru chakanakira ndechekuti hapana chikonzero chekudzikisa bitmaps kuita bhiti mashandiro - isu tinokwanisa kuita bit operation zvakanangana neakamanikidzwa bitmaps.
Munguva pfupi yapfuura, nzira dzakasanganiswa dzakatanga kuoneka, senge mabitmaps anotinhira. Ivo panguva imwe chete vanoshandisa matatu akasiyana anomiririra mabitmaps - bitmaps ivo, arrays uye anonzi bit run - uye chiyero pakati pavo kuti kuwedzere kuita uye kuderedza ndangariro kushandiswa.
Iwe unogona kuwana anotinhira bitmaps mune anonyanya kufarirwa maapplication. Patova nenhamba huru yekushandiswa kwemitauro yakasiyana-siyana yekuronga, kusanganisira zvinopfuura zvitatu zveGo.
Imwe nzira inogona kutibatsira kubata nepamusoro makadhiinality inonzi binning. Fungidzira une munda unomiririra kureba kwemunhu. Hurefu inhamba yepoindi inoyangarara, asi isu vanhu hatizvifungi saizvozvo. Kwatiri hapana musiyano pakati pehurefu 185,2 cm uye 185,3 cm.
Zvinoitika kuti isu tinogona kuunganidza maitiro akafanana mumapoka mukati me1 cm.
Uye kana isu tichizivawo kuti vanhu vashoma vashoma vapfupi kupfuura masendimita makumi mashanu uye kureba kupfuura 50 cm, saka isu tinokwanisa chaizvo kushandura munda nekadhini risingaperi mumunda une kadhidhi yeanosvika mazana maviri.
Ehe, kana zvichidikanwa, tinogona kuita kumwe kusefa mushure.
High Bandwidth Dambudziko
Dambudziko rinotevera ne bitmap indexes nderekuti kuvagadziridza kunogona kudhura zvakanyanya.
Databases inofanirwa kukwanisa kugadzirisa data uku paine mazana emamwe mibvunzo ari kutsvaga data. Tinoda makiyi kudzivirira matambudziko nekuwana data panguva imwe chete kana mamwe matambudziko ekugovana. Uye uko kune kukiyi hombe, pane dambudziko - kukiya kukakavara, kana ichi chivharo chikava bhodhoro.
Dambudziko iri rinogona kugadziriswa kana kutenderedzwa nekushandisa sharding kana kushandisa vhezheni indexes.
Sharding chinhu chiri nyore uye chinozivikanwa. Iwe unogona kugovera bitmap index sezvaungangoita chero imwe data. Panzvimbo pekiyi hombe, iwe uchawana boka rekiyi madiki uye nekudaro ubvise kukakavara kwekiyi.
Nzira yechipiri yekugadzirisa dambudziko ndeye kushandisa shanduro dze indexed. Unogona kuva nekopi imwe yeindekisi yaunoshandisa pakutsvaga kana kuverenga, uye imwe yaunoshandisa pakunyora kana kuvandudza. Uye kamwe mune imwe nguva yenguva (semuenzaniso, kamwe chete 100 ms kana 500 ms) unoadzokorora uye uchichinjanisa. Ehe, nzira iyi inongoshanda mune zviitiko apo application yako inogona kubata zvishoma yakarembera yekutsvaga index.
Aya maitiro maviri anogona kushandiswa panguva imwe chete: unogona kuva nesharded versioned index.
Mimwe mibvunzo yakaoma
Dambudziko rekupedzisira ne bitmap indexes nderekuti tinoudzwa kuti havana kunyatsokodzera mhando dzemibvunzo yakaoma, senge span mibvunzo.
Chokwadi, kana ukafunga nezvazvo, maoperation akaita se AND, OR, etc. haana kunyatsokodzera mibvunzo a la "Ndiratidze mahotera ane mitengo yemakamuri kubva pamadhora mazana maviri kusvika kumazana matatu husiku."
Mhinduro isina njere uye isina hungwaru ingave yekutora mibairo pamutengo wedhora rega rega wosanganisa nea bitwise OR oparesheni.
Mhinduro iri nani zvishoma ndeye kushandisa mapoka. Semuenzaniso, mumapoka emadhora makumi mashanu. Izvi zvaizoita kuti basa redu rikurumidze neka50.
Asi dambudziko rinogadziriswa nyore nyore nekushandisa maonero akagadzirwa zvakananga kune rudzi urwu rwekukumbira. Mumapepa esainzi inonzi range-encoded bitmaps.
Muchiratidziro ichi, isu hatisi kungoisa imwe diki kune imwe kukosha (somuenzaniso, 200), asi isa kukosha uku uye zvese zvakakwirira. 200 uye pamusoro. Zvakafanana ne300: 300 uye pamusoro. Zvichingoenda zvakadaro.
Tichishandisa chinomiririra ichi, tinokwanisa kupindura rudzi urwu rwemubvunzo wekutsvaga nekupfuura nendekisi kaviri chete. Kutanga, tichawana runyoro rwemahotera umo mukamuri inodhura zvishoma kana madhora mazana matatu, uye tozobvisa kubva mairi ayo ane mutengo wekamuri iri shoma kana $300. Ready.
Iwe uchashamisika, asi kunyange geoqueries inogoneka uchishandisa bitmap indexes. Icho chinongedzo ndechekushandisa chimiro chejometri chakakomberedza kurongeka kwako nemufananidzo wejometri. Semuenzaniso, S2 kubva kuGoogle. Nhamba yacho inofanira kuve inomiririra muchimiro chemitsetse mitatu kana kupfuura inopindirana inogona kuverengwa. Nenzira iyi tinokwanisa kushandura geoquery yedu kuita mibvunzo yakati wandei "pamwe nemukaha" (pamwe nemitsetse ine nhamba iyi).
Yakagadzirira mhinduro
Ndinovimba ndakakufarirai zvishoma uye ikozvino mune chimwe chishandiso chinobatsira mune yako arsenal. Kana iwe uchizoda kuita chinhu chakadai, iwe unozoziva nzira yekutarisa.
Nekudaro, havasi vese vane nguva, moyo murefu, kana zviwanikwa zvekugadzira bitmap indexes kubva kutanga. Kunyanya epamberi, uchishandisa SIMD, semuenzaniso.
Sezvineiwo, pane akati wandei akagadzirira-akagadzirwa mhinduro kukubatsira.
Kudzvova bitmaps
Chekutanga, kune iyo yakafanana inodzvova bitmaps raibhurari yandakatotaura nezvayo. Iyo ine ese anodiwa midziyo uye bit mashandiro auchazoda kugadzira yakazara-yakazara bitmap index.
Nehurombo, parizvino, hapana kana yeGo ekuita inoshandisa SIMD, zvinoreva kuti Go maitirwo haanyanyi kuita pane C kuita, semuenzaniso.
Pilosa
Chimwe chigadzirwa chinogona kukubatsira iPilosa DBMS, iyo, chaizvoizvo, ine bitmap indexes chete. Iyi imhinduro itsva, asi iri kukunda mwoyo nekumhanya kukuru.
Pilosa inoshandisa kurira bitmaps mukati uye inokupa iwe kugona kuzvishandisa, inorerutsa uye inotsanangura zvinhu zvese zvandataura pamusoro: mapoka, encoded-encoded bitmaps, pfungwa yemunda, nezvimwe.
Ngatitarisei nekukurumidza muenzaniso wekushandisa Pilosa kupindura mubvunzo waunotoziva nawo.
Muenzaniso wakafanana chaizvo nezvawakamboona. Isu tinogadzira mutengi kune sevha yePilosa, gadzira index uye minda inodiwa, tobva tazadza minda yedu nechero data ine zvingangoitika uye, pakupedzisira, ita iyo yakajairika mubvunzo.
Mushure meizvozvo, isu tinoshandisa NOT pane "inodhura" munda, tozopesanisa mhedzisiro (kana AND iyo) ne "terrace" munda uye ne "reservations" munda. Uye pakupedzisira, tinowana mhedzisiro yekupedzisira.
Ndinovimba chaizvo kuti mune ramangwana rinoonekwa iyi mhando nyowani ichaonekwawo muDBMSs seMySQL nePostgreSQL - bitmap indexes.
mhedziso
Kana usati wabatwa nehope, ndatenda. Ini ndaifanira kubata muchidimbu pamisoro yakawanda nekuda kwenguva shoma, asi ndinovimba kuti hurukuro yacho yaibatsira uye pamwe ichikurudzira.
Bitmap indexes zvakanaka kuziva nezvazvo, kunyangwe iwe usingazvide izvozvi. Ngaave chimwe chishandiso mubhokisi rako rematurusi.
Isu takatarisa akasiyana maitiro ekuita eGo uye zvinhu izvo Go compiler haisati yanyatsobata. Asi izvi zvinobatsira chaizvo kune wese Go programmer kuziva.
Ndizvo chete zvandaida kukuudza. Ndatenda!
Source: www.habr.com