intshayelelo
Ndinike le ngxelo ngesiNgesi kwinkomfa yeGopherCon Russia 2019 eMoscow nakwisiRashiya kwindibano eyayiseNizhny Novgorod. Sithetha ngesalathiso se-bitmap - engaphantsi kwe-B-tree, kodwa akukho nto inomdla. Ukwabelana iintetho kwinkomfa ngesiNgesi kunye nemibhalo ebhaliweyo ngesiRashiya.
Siza kujonga indlela isalathisi se-bitmap sisebenza ngayo, xa singcono, xa sibi kakhulu kunezinye izalathisi, kwaye kweziphi iimeko ezikhawuleza kakhulu kunabo; Makhe sibone ukuba zeziphi iiDBMS ezidumileyo esele zinezalathisi zebitmap; Masizame ukubhala ezethu kuGo. Kwaye "kwidizethi" siya kusebenzisa amathala eencwadi asele elungile ukwenza eyethu idatabase ekhethekileyo ekhawulezayo.
Ndiyathemba ukuba imisebenzi yam iya kuba luncedo kwaye inomdla kuwe. Hamba!
Intshayelelo
Molweni nonke! Yintsimbi yesithandathu ngorhatya kwaye sonke sidinwe kakhulu. Ixesha elihle lokuthetha malunga nethiyori yesalathiso sedatha ekruqulayo, akunjalo? Ungakhathazeki, ndizakuba nemigca emibini yekhowudi yomthombo apha naphaya. π
Zonke iziqhulo ecaleni, ingxelo izele lulwazi, kwaye asinaxesha elininzi. Ngoko masiqalise.
Namhlanje ndiza kuthetha ngoku kulandelayo:
- zithini izalathi;
- yintoni isalathisi sebitmap;
- apho isetyenziswa khona nalapho INGASETYENZISWA kwaye ngoba;
- ukuphunyezwa okulula kwi-Go kunye nomzabalazo omncinci kunye nomqambi;
- ilula kancinane, kodwa uphumezo olunemveliso ngakumbi kwi-Go assembler;
- "iingxaki" zezalathisi ze-bitmap;
- uphumezo olukhoyo.
Ke zithini izalathi?
Isalathiso sisakhiwo sedatha esahlukileyo esisigcinayo kwaye sihlaziye ukongeza kwidatha ephambili. Isetyenziselwa ukukhawulezisa ukukhangela. Ngaphandle kwezalathisi, ukukhangela kuya kufuna ukuba kudlule idata ngokupheleleyo (inkqubo ebizwa ngokuba yi-scan epheleleyo), kwaye le nkqubo inobunzima be-algorithmic yomgama. Kodwa ugcino lwedatha luhlala luqulathe isixa esikhulu sedatha kwaye ukuntsonkotha komgca kucotha kakhulu. Ngokufanelekileyo, siya kufumana i-logarithmic okanye eqhubekayo.
Esi sihloko esinzima kakhulu, esizaliswe ngobuqili kunye nokurhweba, kodwa emva kokujonga amashumi eminyaka yophuhliso lwesiseko sedatha kunye nophando, ndikulungele ukuthetha ukuba kukho iindlela ezimbalwa ezisetyenziswa ngokubanzi ekudaleni izalathi zedatha.
Indlela yokuqala kukunciphisa ngokusemthethweni indawo yokukhangela, ukwahlula indawo yokukhangela kwiindawo ezincinci.
Ngokuqhelekileyo sikwenza oku sisebenzisa iintlobo ezahlukeneyo zemithi. Umzekelo uya kuba yibhokisi enkulu yezixhobo kwi-closet yakho equlethe iibhokisi ezincinci zezixhobo ezahlulwe kwizihloko ezahlukeneyo. Ukuba ufuna imathiriyeli, mhlawumbi uya kuzijonga kwibhokisi ethi "IiMathiriyeli" kunaleyo ethi "Iinkuki," akunjalo?
Indlela yesibini kukukhetha ngoko nangoko into efunekayo okanye iqela lezinto. Oku sikwenza kwiimephu ze-hash okanye izalathisi ezibuyela umva. Ukusebenzisa iimephu ze-hash kufana kakhulu nomzekelo wangaphambili, kodwa endaweni yebhokisi yeebhokisi, unebhokisi leebhokisi ezincinci zezinto zokugqibela kwikhabhathi yakho.
Indlela yesithathu kukuphelisa imfuneko yokukhangela. Senza oku sisebenzisa izihluzi zeBloom okanye izihluzi ze cuckoo. Ezokuqala zinika impendulo ngoko nangoko, zikusindisa ekubeni uphendle.
Indlela yokugqibela kukusebenzisa ngokupheleleyo onke amandla esiwafumana kwihardware yanamhlanje. Yile nto kanye siyenzayo kwizalathisi ze-bitmap. Ewe, xa sizisebenzisa ngamanye amaxesha kufuneka sidlule kwisalathiso sonke, kodwa siyenza kakuhle kakhulu.
Njengoko benditshilo, isihloko sezalathisi zedatha sikhulu kwaye sigcwele ukuthobela. Oku kuthetha ukuba ngamanye amaxesha sinokusebenzisa iindlela ezininzi ngexesha elinye: ukuba sifuna ukukhawulezisa uphendlo ngakumbi, okanye ukuba sifuna ukugubungela zonke iintlobo zophendlo ezinokwenzeka.
Namhlanje ndiza kuthetha ngeyona ndlela incinci eyaziwayo kwezi - izalathi ze-bitmap.
Ndingubani mna ukuba ndithethe ngalo mxholo?
Ndisebenza njengeqela elikhokelayo eBadoo (mhlawumbi uqhelene nenye imveliso yethu, iBumble). Sele sinabasebenzisi abangaphezu kwezigidi ezingama-400 kwihlabathi liphela kunye nezinto ezininzi ezibakhethela owona mdlalo ubalaseleyo. Senza oku sisebenzisa iinkonzo zesiko, kuquka izalathisi ze-bitmap.
Ke yintoni i-bitmap index?
Izalathisi zeBitmap, njengoko igama licebisa, sebenzisa i-bitmaps okanye i-bitsets ukuphumeza isalathiso sokukhangela. Ukusuka kwimboniselo yeso lentaka, esi salathiso siqulathe enye okanye ezininzi zebitmaps ezimele naziphi na iziko (ezinjengabantu) kunye neempawu zabo okanye iiparamitha (ubudala, umbala wamehlo, njalo njalo.), kunye ne-algorithm esebenzisa i-bit operations (KUNYE, OKANYE, HAYI ) ukuphendula umbuzo wokukhangela.
Sixelelwa ukuba izalathi ze-bitmap zifaneleke kakhulu kwaye zisebenza kakhulu kwiimeko apho kukho uphendlo oludibanisa imibuzo kwiikholamu ezininzi ezisezantsi zekhadinality (cinga "umbala wamehlo" okanye "imo yomtshato" ngokuchasene nento efana "nomgama ukusuka kumbindi wesixeko" ). Kodwa ndiza kubonisa kamva ukuba basebenza kakuhle kwiikholamu eziphezulu zekhadinali ngokunjalo.
Makhe sijonge owona mzekelo ulula wesalathisi se-bitmap.
Khawufane ucinge ukuba sinoluhlu lweendawo zokutyela zaseMoscow ezineempawu zokubini ezifana nezi:
- kufutshane nemetro;
- kukho indawo yokupaka yabucala;
- kukho iveranda (ine terrace);
- ungagcina itafile (yamkela ugcino);
- ilungele abatya imifuno (i-vegan friendly);
- kuyabiza (ixabiso eliphezulu).
Makhe sinike irestyu nganye inombolo yolandelelwano oluqala ku-0 kwaye sabele inkumbulo kwii-bitmaps ezi-6 (enye kuphawu ngalunye). Siza kukhupha ezi bitmaps ngokuxhomekeke ekubeni indawo yokutyela inalo propati okanye hayi. Ukuba i-restaurant ye-4 ine-veranda, ngoko ke i-bit No.
Ngoku sineyona index yebitmap ilula, kwaye sinokuyisebenzisa ukuphendula imibuzo efana nale:
- βNdibonise iindawo zokutyela ezifanelekileyo kwimifunoβ;
- βNdibonise iindawo zokutyela ezingabizi kakhulu ezineveranda apho unokugcine khona itafile.β
Njani? Makhe sijonge. Isicelo sokuqala silula kakhulu. Ekuphela kwento ekufuneka siyenze kukuthatha i-bitmap "elungele imifuno" kwaye siyiguqule ibe luluhlu lweendawo zokutyela ezivezwe amasuntswana.
Isicelo sesibini sinzima ngakumbi. Kufuneka sisebenzise i-HAYI i-bitmap kwi-bitmap "ebizayo" ukufumana uluhlu lweendawo zokutyela ezingabizi kakhulu, emva koko KUNYE kunye ne-bitmap ethi "ndingabhukisha itafile" kunye KUNYE nesiphumo "kukho i-veranda" bitmap. Isiphumo se-bitmap siya kuba noluhlu lweendawo ezihlangabezana nazo zonke iikhrayitheriya zethu. Kulo mzekelo, le yindawo yokutyela yaseYunost kuphela.
Kukho ithiyori eninzi ebandakanyekayo, kodwa ungakhathazeki, siza kubona ikhowudi kungekudala.
Zisetyenziswa phi izalathi zebitmap?
Ukuba izalathisi ze-bitmap zikaGoogle, i-90% yeempendulo ziya kunxulumana ne-Oracle DB ngendlela enye okanye enye. Kodwa ezinye ii-DBMS mhlawumbi zikwaxhasa into epholileyo, akunjalo? Hayi ncma.
Masingene kuluhlu lwabarhanelwa abaphambili.
I-MySQL ayikaxhasi izalathi zebitmap, kodwa kukho isiphakamiso esicebisa ukongeza olu khetho ().
I-PostgreSQL ayizixhasi izalathisi ze-bitmap, kodwa isebenzisa ii-bitmap ezilula kunye nokusebenza kwe-bit ukudibanisa iziphumo zokukhangela kwezinye izalathisi ezininzi.
I-Tarantool inezalathisi ze-bitset kwaye ixhasa uphendlo olulula kuzo.
I-Redis ine-bitfields elula) ngaphandle kokukwazi ukuzikhangela.
IMongoDB ayikaxhasi izalathi zebitmap, kodwa kukwakho isiphakamiso esicebisa ukuba olu khetho longezwe.
I-Elasticsearch isebenzisa ii-bitmaps ngaphakathi).
- Kodwa ummelwane omtsha uye wavela endlwini yethu: Pilosa. Le yidatabase entsha engeyiyo eyobudlelwane ebhalwe kwiGo. Iqulethe kuphela izalathi ze-bitmap kwaye isekela yonke into kuzo. Siza kuthetha ngayo kamva.
Ukuphunyezwa kwi-Go
Kodwa kutheni izalathi zebitmap zingafane zisetyenziswe? Phambi kokuba uphendule lo mbuzo, ndingathanda ukukubonisa indlela yokumisela isalathisi esilula kakhulu se-bitmap kwi-Go.
Iimephu zeBitmaps ngokwenyani ziziqwenga zedatha. Ku-Go, masisebenzise izilayi ze-byte kule nto.
Sine-bitmap enye yophawu lwendawo yokutyela enye, kwaye isuntswana ngalinye kwi-bitmap libonisa ukuba indawo yokutyela ethile inale propati okanye ayikho.
Siya kufuna imisebenzi emibini yomncedisi. Enye iya kusetyenziselwa ukugcwalisa i-bitmaps yethu ngedatha engakhethiyo. Ngokungacwangciswanga, kodwa kunethuba elithile lokuba indawo yokutyela inepropati nganye. Ngokomzekelo, ndiyakholelwa ukuba kukho iindawo zokutyela ezimbalwa kakhulu eMoscow apho awukwazi ukugcina itafile, kwaye kubonakala kum ukuba malunga ne-20% yeendawo ezisetyenzisiweyo zifanelekile kwimifuno.
Umsebenzi wesibini uya kuguqula i-bitmap ibe luluhlu lweendawo zokutyela.
Ukuphendula umbuzo "Ndibonise iindawo zokutyela ezingabizi kakhulu ezinepatio kwaye zingenza ubhukisho," sidinga imisebenzi emibini: HAYI kunye AND.
Singayenza lula ikhowudi yethu kancinci ngokusebenzisa eyona nto inzima YAYE HAYI umqhubi.
Sinemisebenzi yalo msebenzi ngamnye. Bobabini bahamba ngeesilayi, bathathe izinto ezihambelanayo nganye, badibanise kunye nokusebenza kancinci kwaye babeke isiphumo kwisilayi esisiphumo.
Kwaye ngoku sinokusebenzisa i-bitmaps yethu kunye nemisebenzi ukuphendula umbuzo wokukhangela.
Ukusebenza akukho phezulu, nangona imisebenzi ilula kakhulu kwaye sigcine imali eninzi ngokungabuyisi isilayi esitsha esinesiphumo rhoqo xa umsebenzi ubizwa.
Emva kokwenza iprofayile encinci kunye ne-pprof, ndiye ndaqaphela ukuba iGo compiler ilahlekile enye elula kakhulu kodwa ebaluleke kakhulu: ukusebenza ngaphakathi.
Inyani yeyokuba umqokeleli weGo woyika kakhulu iilophu ezihamba kwizilayi, kwaye ukwala ngokuthe ngqo ukwenza imisebenzi engaphakathi equlathe loo loops.
Kodwa andoyiki kwaye ndiyakwazi ukukhohlisa umqambi ngokusebenzisa i-goto endaweni yelophu, njengakwimihla yakudala.
Kwaye, njengoko ubona, ngoku umqambi uya kuhambelana nomsebenzi wethu ngovuyo! Ngenxa yoko, siyakwazi ukugcina malunga ne-2 microseconds. Akukubanga!
I-bottleneck yesibini kulula ukubona ukuba ujonge ngokusondeleyo kwimveliso yendibano. Umqambi wongeze umda wokujonga umda kanye ngaphakathi kweyona loop yethu ishushu. Inyani kukuba i-Go lulwimi olukhuselekileyo, umqambi uyesaba ukuba iingxabano zam ezintathu (izilayi ezithathu) zinobukhulu obahlukeneyo. Emva kwayo yonke loo nto, kuya kubakho ithiyori enokwenzeka yokwenzeka kwento ebizwa ngokuba yi-buffer overflow.
Masiqinisekise umqokeleli ngokumbonisa ukuba zonke izilayi zilingana. Singakwenza oku ngokongeza itshekhi elula ekuqaleni komsebenzi wethu.
Ukubona oku, umqokeleli weqa isheke ngovuyo, kwaye ekugqibeleni sigcina enye i-500 nanoseconds.
Iibhula ezinkulu
Kulungile, sikwazile ukucudisa ukusebenza okuthile kuphumezo lwethu olulula, kodwa esi siphumo sibi kakhulu kunokuba sinokwenzeka ngehardware yangoku.
Ekuphela kwento esiyenzayo kukusetyenzwa kancinci, kwaye abaqhubekekisi bethu bazenza ngokufanelekileyo. Kodwa, ngelishwa, "sondla" iprosesa yethu ngamaqhekeza amancinci omsebenzi. Imisebenzi yethu yenza imisebenzi ngokwe-byte-by-byte. Sinokwenza ngokulula ukwenza ikhowudi yethu ukuze sisebenze ngee-8-byte chunks sisebenzisa izilayi ze-UInt64.
Njengoko ubona, olu tshintsho luncinci lukhawulezisa inkqubo yethu ngamaxesha asibhozo ngokunyusa ubungakanani bebhetshi ngamaxesha asibhozo. Ukuzuza kunokuthiwa kumgca.
Ukuphunyezwa kwi-assembler
Kodwa oku ayisosiphelo. Iiprosesa zethu zinokusebenza kunye neziqwenga ze-16, 32 kunye ne-64 bytes. Imisebenzi enjalo "ebanzi" ibizwa ngokuba yi-single instruction multiple data (SIMD; umyalelo omnye, idatha eninzi), kunye nenkqubo yokuguqula ikhowudi ukuze isebenzise imisebenzi enjalo ibizwa ngokuba yi-vectorization.
Ngelishwa, i-Go compiler ikude kakhulu kwi-vectorization. Okwangoku, ekuphela kwendlela yokwenza ikhowudi yeGo kukuthatha kwaye ubeke le misebenzi ngesandla usebenzisa iGo assembler.
Go assembler sisilo esingaqhelekanga. Mhlawumbi uyazi ukuba ulwimi lwendibano yinto ebotshelelwe kakhulu kuyilo lwekhompyuter oyibhalelayo, kodwa akunjalo kuGo. I-Go assembler ifana ne-IRL (ulwimi oluphakathi) okanye ulwimi oluphakathi: liqonga elizimeleyo. URob Pike unike umsebenzi obalaseleyo kwesi sihloko kwiminyaka eliqela eyadlulayo eGopherCon eDenver.
Ukongeza, i-Go isebenzisa ifomathi yePlani ye-9 engaqhelekanga, eyahlukileyo kwi-AT & T kunye ne-Intel eyamkelwe ngokubanzi.
Kukhuselekile ukuthi ukubhala Go assembler ngesandla ayiyona nto imnandi.
Kodwa, ngethamsanqa, sele kukho izixhobo ezibini ezikumgangatho ophezulu ezisinceda sibhale iGo assembler: PeachPy kunye ne-avo. Zombini eziluncedo zenza iGo assembler ukusuka kwikhowudi ephezulu ebhalwe kwiPython kunye neGo, ngokulandelelanayo.
Ezi zinto ziluncedo zenza lula izinto ezifana nolwabiwo lwerejista, ukubhala iilophu, kwaye ngokubanzi zenze lula inkqubo yokungena kwihlabathi lenkqubo yendibano kwiGo.
Siza kusebenzisa avo, ngoko ke iinkqubo zethu ziya kuba phantse rhoqo iinkqubo Go.
Le yindlela obona ngayo umzekelo olula wenkqubo ye-avo ibonakala ngathi. Sine main () umsebenzi, ochaza ngaphakathi kwayo Dibanisa () umsebenzi, intsingiselo yawo kukudibanisa amanani amabini. Kukho imisebenzi yomncedisi apha ukufumana iparameters ngegama kwaye ufumane enye yeerejista zamahhala nezifanelekileyo. Umsebenzi ngamnye weprosesa unomsebenzi ohambelanayo kwi-avo, njengoko kubonwa kwi-ADDQ. Okokugqibela, sibona umsebenzi womncedisi wokugcina ixabiso lesiphumo.
Ngokubiza i-go velisa, siya kuqhuba inkqubo kwi-avo kwaye ngenxa yoko, iifayile ezimbini ziya kuveliswa:
- dibanisa ngekhowudi enesiphumo kwi-Go assembler;
- stub.go kunye neeheader zomsebenzi ukudibanisa amazwe amabini: Yiya kwaye uhlanganise.
Ngoku sele siyibonile into eyenziwa yiavo kwaye njani, makhe sijonge imisebenzi yethu. Ndiphumeze zombini iinguqulelo ze-scalar kunye ne-vector (i-SIMD) yemisebenzi.
Makhe sijonge kwiinguqulelo ze-scalar kuqala.
Njengoko kumzekelo wangaphambili, sicela irejista yenjongo ekhululekile kwaye esebenzayo, akukho mfuneko yokubala i-offsets kunye nobukhulu beengxabano. uvo usenzela yonke lento.
Sasidla ngokusebenzisa iilebhile kunye ne-goto (okanye ukutsiba) ukuphucula ukusebenza kunye nokukhohlisa i-Go compiler, kodwa ngoku siyenza kwasekuqaleni. Inqaku lelokuba imijikelo ingqikelelo yenqanaba eliphezulu. Kwi-assembler, sineeleyibhile kunye nokutsiba kuphela.
Ikhowudi eseleyo kufuneka sele iqhelekile kwaye iyaqondakala. Silinganisa i-loop kunye neelebhile kunye nokutsiba, thatha iqhekeza elincinci ledatha kwizilayi zethu ezibini, zidibanise kunye nokusebenza kancinci (KWAYE HAYI kule meko) kwaye emva koko ubeke isiphumo kwisiqwenga esisiphumo. Konke.
Yile ndlela ikhowudi yokugqibela yokuhlanganisa ijongeka ngayo. Akuzange kufuneke ukuba sibale ii-offsets kunye nobukhulu (zibonakaliswe eluhlaza) okanye silandele iirejista ezisetyenzisiweyo (ezibhalwe ngombala obomvu).
Ukuba sithelekisa ukusebenza kokuphunyezwa kolwimi lwendibano kunye nokusebenza kokuphunyezwa okungcono kakhulu kwi-Go, siya kubona ukuba kuyafana. Kwaye oku kulindeleke. Ngapha koko, akukho nto ikhethekileyo siyenzileyo- siphinde savelisa into eza kwenziwa ngumqokeleli weGo.
Ngelishwa, asinakunyanzela umqokeleli ukuba afake umgca wemisebenzi yethu ebhalwe ngolwimi lwendibano. Umqokeleli weGo okwangoku akanalo uphawu olunjalo, nangona bekukho isicelo sokuyongeza ixesha elide.
Kungenxa yoko le nto kungenakwenzeka ukufumana nayiphi na inzuzo kwimisebenzi encinci kulwimi lwendibano. Kufuneka nokuba sibhale imisebenzi emikhulu, okanye sisebenzise imathematika entsha/amasuntswana ipakethe, okanye sigqithe ulwimi lokuhlanganisa.
Ngoku makhe sijonge kwiinguqulelo zevector zemisebenzi yethu.
Kulo mzekelo, ndigqibe ekubeni ndisebenzise i-AVX2, ngoko ke siya kusebenzisa imisebenzi esebenza kwii-32-byte chunks. Ukwakhiwa kwekhowudi kufana kakhulu nenguqulo ye-scalar: ukulayisha iiparitha, ukucela irejista ekwabelwana ngayo yamahhala, njl.
Enye into entsha kukuba imisebenzi ye-vector ebanzi isebenzisa iirejista ezikhethekileyo ezibanzi. Kwimeko ye 32-byte chunks, ezi ziirejista ezifakwe kuqala ngo Y. Kungenxa yoko le nto ubona i YMM () umsebenzi kwikhowudi. Ukuba bendisebenzisa i-AVX-512 ene-64-bit chunks, isimaphambili besiya kuba ngu-Z.
Utshintsho lwesibini kukuba ndigqibe ekubeni ndisebenzise ulungiselelo olubizwa ngokuba yi-loop unrolling, okuthetha ukwenza imisebenzi esibhozo yelophu ngesandla phambi kokutsibela ekuqaleni kweluphu. Oku kulungiswa kunciphisa inani lamasebe kwikhowudi, kwaye lilinganiselwe inani leerejista zamahhala ezikhoyo.
Kulungile, kuthekani malunga nokusebenza? Umhle! Sifezekise isantya esimalunga namaxesha asixhenxe xa kuthelekiswa nesona sisombululo silungileyo Go. Kuyachukumisa, akunjalo?
Kodwa nolu phumezo lunokubakho ukukhawuleziswa ngokusebenzisa i-AVX-512, ukulandwa kwangaphambili okanye iJIT (umqokeleli wexesha nje) kumcwangcisi wemibuzo. Kodwa ngokuqinisekileyo esi sisihloko sengxelo eyahlukileyo.
Iingxaki ngezalathisi zebitmap
Ngoku sele sijonge ukuphunyezwa okulula kwesalathiso sebitmap kwiGo kunye nemveliso ngakumbi kulwimi lwendibano, ekugqibeleni masithethe malunga nokuba kutheni izalathisi zebitmap zingafane zisetyenziswe.
Amaphepha amadala akhankanya iingxaki ezintathu ngezalathisi ze-bitmap, kodwa amaphepha amatsha kwaye ndiyaphikisana ukuba awasekho. Asiyi kuntywila nzulu kwingxaki nganye kwezi, kodwa siya kuzijonga nje phezulu.
Ingxaki yekhadinali ephezulu
Ke, sixelelwa ukuba izalathisi zebitmap zilungele kuphela amasimi anekhadinali ephantsi, oko kukuthi, lawo anamaxabiso ambalwa (umzekelo, isini okanye umbala wamehlo), kwaye isizathu sikukuba ukumelwa okuqhelekileyo kwemimandla enjalo (enye bitmap ngexabiso) kwimeko yekhadinali ephezulu, iyakuthatha indawo eninzi kakhulu kwaye, ngaphezu koko, ezi indexes zebitmap ziyakugcwala (kunqabile) zizaliswe.
Ngamanye amaxesha sinokusebenzisa umelo olwahlukileyo, olufana nolo mgangatho siwusebenzisayo ukumela amanani. Kodwa yayikukufika kwe-algorithms yoxinzelelo eyatshintsha yonke into. Kule minyaka idlulileyo, izazinzulu kunye nabaphandi baye beza nenani elikhulu le-algorithms yoxinzelelo lwee-bitmaps. Inzuzo yabo ephambili kukuba akukho mfuneko yokucofa i-bitmaps ukwenza imisebenzi ye-bit - singenza imisebenzi ye-bit ngokuthe ngqo kwiimephu ze-bitmaps.
Kutshanje, iindlela ezixubeneyo ziye zaqala ukubonakala, ezinje nge-bitmaps egqumayo. Ngaxeshanye basebenzisa imiboniso emithathu eyahlukeneyo yee-bitmaps-i-bitmaps ngokwabo, i-arrays kunye nento ebizwa ngokuba yi-bit runs-kunye nokulinganisela phakathi kwabo ukwandisa ukusebenza kunye nokunciphisa ukusetyenziswa kwememori.
Unokufumana ii-bitmaps ezigqumayo kwezona zicelo zidumileyo. Sele kukho inani elikhulu lophunyezo kwiilwimi ezahlukeneyo zokucwangcisa, kubandakanywa ukuphunyezwa kweGo.
Enye indlela enokusinceda sijongane nekhadinali ephezulu ibizwa ngokuba yi-binning. Khawucinge ukuba unentsimi emele ubude bomntu. Ukuphakama linani lendawo edadayo, kodwa thina bantu asiyicingi ngolo hlobo. Kithina akukho mahluko phakathi kobude be-185,2 cm kunye ne-185,3 cm.
Kuyavela ukuba sinokudibanisa amaxabiso afanayo ngokwamaqela ngaphakathi kwe-1 cm.
Kwaye ukuba nathi siyazi ukuba abantu abambalwa kakhulu bafutshane kune-50 cm kwaye bade kune-250 cm, ngoko sinokuyijika intsimi kunye ne-cardinal engapheliyo kwintsimi ene-cardinality malunga ne-200 values.
Ngokuqinisekileyo, ukuba kuyimfuneko, singenza ukucoca okongeziweyo emva koko.
Ingxaki yobubanzi obuphezulu
Ingxaki elandelayo ngezalathisi ze-bitmap kukuba ukuzihlaziya kunokubiza kakhulu.
Oovimba beenkcukacha kufuneka babe nakho ukuhlaziya idatha ngelixa amakhulu eminye imibuzo ekhangela idatha. Sidinga izitshixo ukuze siphephe iingxaki ngokufikelela kwedatha ngaxeshanye okanye ezinye iingxaki zokwabelana. Kwaye apho kukho iqhaga elikhulu, kukho ingxaki-ingxabano yokutshixa, xa esi sitshixo siba ngumqobo.
Le ngxaki ingasonjululwa okanye ijikelezwe ngokusebenzisa i-sharding okanye ngokusebenzisa izalathisi eziguqulelweyo.
Ukwabelana yinto elula kwaye eyaziwayo. Unokwahlula isalathisi se-bitmap njengoko ubuya kwenza nayiphi na enye idatha. Endaweni yesitshixo esinye esikhulu, uya kufumana iqela lezitshixo ezincinci kwaye ngaloo ndlela ukhuphe ingxabano yokutshixa.
Indlela yesibini yokusombulula ingxaki kukusebenzisa izalathisi eziguqulelweyo. Unokuba nekopi enye yesalathiso oyisebenzisela ukukhangela okanye ukufunda, kunye nenye oyisebenzisela ukubhala okanye ukuhlaziya. Kwaye kanye ngexesha elithile (umzekelo, kanye qho ngo-100 ms okanye 500 ms) uyaziphinda kwaye uzitshintshe. Ngokuqinisekileyo, le ndlela isebenza kuphela kwiimeko apho isicelo sakho sinokusingatha isalathisi sokukhangela esishiyekileyo.
Ezi ndlela zimbini zingasetyenziswa ngaxeshanye: unokuba nesalathiso soguqulelo olwahluliweyo.
Imibuzo entsokothileyo
Ingxaki yokugqibela ngezalathisi ze-bitmap kukuba sixelelwa ukuba azilungelwanga kakuhle kwiindidi ezintsokothileyo zemibuzo, njengemibuzo ye-span.
Ewe, ukuba ucinga ngayo, imisebenzi encinci efana ne-AND, OKANYE, njl.njl
Isisombululo esingenangqondo nesingesobulumko kakhulu iyakuba kukuthatha iziphumo zexabiso ngalinye ledola kwaye uzidibanise nomsebenzi kancinane OKANYE.
Isisombululo esingcono kancinci kukusebenzisa ukwahlulahlula. Umzekelo, kumaqela e-50 yeedola. Oku kuya kuyikhawulezisa inkqubo yethu ngamaxesha angama-50.
Kodwa ingxaki iphinda isombululwe ngokulula ngokusebenzisa umbono owenziwe ngokukodwa ngolu hlobo lwesicelo. Kumaphepha enzululwazi ibizwa ngokuba yi-range-encoded bitmaps.
Kulo mboniso, asimiseli nje into enye kwixabiso elithile (umzekelo, i-200), kodwa setha eli xabiso kunye nayo yonke into ephezulu. 200 nangaphezulu. Okufanayo kwi-300: 300 nangaphezulu. Kwaye nangokunjalo.
Ngokusebenzisa olu phawu, sinokuphendula olu hlobo lombuzo wokukhangela ngokunqumla isalathisi kabini nje. Okokuqala, siya kufumana uluhlu lweehotele apho igumbi libiza ngaphantsi okanye i-$ 300, kwaye emva koko siya kususa kuyo apho ixabiso legumbi lingaphantsi okanye i-$ 199. Ulungile.
Uya kumangaliswa, kodwa neegeoqueries ziyenzeka usebenzisa izalathisi zebitmap. Ubuqhetseba kukusebenzisa ukubonakaliswa kwejometri ejikeleze ulungelelwaniso lwakho kunye nomfanekiso wejometri. Umzekelo, i-S2 evela kuGoogle. Umzobo kufuneka ukwazi ukumela ngendlela yemigca emithathu okanye ngaphezulu enqumlayo enokuthi ifakwe iinombolo. Ngale ndlela sinokujika i-geoquery yethu ibe yimibuzo eliqela βkunye nomsantsaβ (kunye nale migca ineenombolo).
Izisombululo ezilungeleyo
Ndiyathemba ukuba ndinomdla kancinci kwaye ngoku unesinye isixhobo esiluncedo kwi-arsenal yakho. Ukuba ukhe wafuna ukwenza into enje, uya kuyazi ukuba yeyiphi na indlela omawujonge ngayo.
Nangona kunjalo, ayinguye wonke umntu onexesha, umonde, okanye izixhobo zokudala izalathisi ze-bitmap ukusuka ekuqaleni. Ngokukodwa abaphambili ngakumbi, usebenzisa i-SIMD, umzekelo.
Ngethamsanqa, kukho izisombululo ezininzi esele zenziwe ukukunceda.
I-bitmaps egqumayo
Okokuqala, kukho kwaloo bitmaps ilayibrari egqumayo esele ndithetha ngayo. Iqulethe zonke izikhongozeli eziyimfuneko kunye nokusebenza kancinci oza kuyidinga ukwenza isalathiso sebitmap esipheleleyo.
Ngelishwa, okwangoku, akukho nanye yokuphunyezwa kwe-Go isebenzisa i-SIMD, oku kuthetha ukuba ukuphunyezwa kwe-Go akwenzeki kangako kunokuphunyezwa kwe-C, umzekelo.
Pilosa
Enye imveliso enokukunceda yiPilosa DBMS, leyo, eneneni, inezalathisi ze-bitmap kuphela. Esi sisisombululo esitsha, kodwa siphumelela iintliziyo ngesantya esikhulu.
I-Pilosa isebenzisa i-bitmaps egqumayo ngaphakathi kwaye ikunika ukukwazi ukuzisebenzisa, yenza lula kwaye ichaze zonke izinto endithethe ngazo ngasentla: ukuhlelwa, uluhlu lwe-bitmaps, ingcamango yentsimi, njl.
Makhe sijonge ngokukhawuleza kumzekelo wokusebenzisa iPilosa ukuphendula umbuzo osele uqhelene nawo.
Umzekelo uyafana kakhulu nalowo wawubonile ngaphambili. Senza umxhasi kumncedisi wePilosa, senze isalathisi kunye nemimandla efunekayo, emva koko ugcwalise amasimi ethu ngedatha engahleliweyo kunye namathuba kwaye, ekugqibeleni, senze umbuzo oqhelekileyo.
Emva koko, sisebenzisa NOT kwibala "elibizayo", emva koko sidibanise umphumo (okanye KUNYE nayo) kunye nentsimi ye "terrace" kunye nendawo "yogcino". Kwaye ekugqibeleni, sifumana umphumo wokugqibela.
Ndiyathemba ukuba kwixesha elizayo olu didi lutsha lwesalathiso luza kuvela kwakhona kwii-DBMS ezifana ne-MySQL kunye ne-PostgreSQL - i-bitmap indexes.
isiphelo
Ukuba awukafiki enkosi. Kwafuneka ndichukumise ngokufutshane kwizihloko ezininzi ngenxa yexesha elincinci, kodwa ndiyathemba ukuba intetho yayiluncedo kwaye mhlawumbi iyakhuthaza.
Izalathisi zeBitmap zilungile ukwazi ngazo, nokuba awuzifuni ngoku. Mazibe sesinye isixhobo kwibhokisi yakho yezixhobo.
Sijonge iindlela ezahlukeneyo zokusebenzisa i-Go kunye nezinto umqokeleli we-Go engaziphathi kakuhle okwangoku. Kodwa oku kuluncedo ngokupheleleyo kuye wonke umdwelisi weGosa ukuba azi.
Nantso kuphela into ebendifuna ukukuxelela yona. Enkosi!
umthombo: www.habr.com