mawu oyamba
Ndidapereka lipoti ili mu Chingerezi pamsonkhano wa GopherCon Russia 2019 ku Moscow komanso mu Chirasha pamsonkhano ku Nizhny Novgorod. Tikulankhula za index ya bitmap - yocheperako kuposa mtengo wa B, koma osasangalatsa. Kugawana zolankhula pamsonkhano mu Chingerezi ndi zolembedwa mu Chirasha.
Tidzawona momwe bitmap index imagwirira ntchito, ikakhala yabwino, ikakhala yoyipa kuposa ma index ena, komanso momwe zimakhalira mwachangu kuposa iwo; Tiyeni tiwone ma DBMS omwe ali kale ndi ma index a bitmap; Tiyeni tiyese kulemba zathu mu Go. Ndipo "za mchere" tidzagwiritsa ntchito malaibulale opangidwa kale kuti tipange nkhokwe yathu yachangu kwambiri.
Ndikukhulupirira kuti ntchito zanga zidzakhala zothandiza komanso zosangalatsa kwa inu. Pitani!
Mau oyamba
Moni nonse! Ndi XNUMX koloko madzulo ndipo tonse tatopa kwambiri. Nthawi yabwino yoti mulankhule za chiphunzitso chotopetsa cha database, sichoncho? Osadandaula, ndikhala ndi mizere ingapo yamagwero apa ndi apo. π
Kupatula nthabwala zonse, lipotilo ndi lodzaza ndi chidziwitso, ndipo tilibe nthawi yochuluka. Choncho tiyeni tiyambe.
Lero ndikamba izi:
- ma index ndi chiyani;
- kodi index ya bitmap ndi chiyani;
- kumene imagwiritsiridwa ntchito ndi kumene SIKUDWEREKEDWA ndi chifukwa chiyani;
- kukhazikitsa kosavuta mu Go ndikulimbana pang'ono ndi wolemba;
- zosavuta pang'ono, koma kukhazikitsa kopindulitsa kwambiri mu Go assembler;
- "zovuta" za bitmap indexes;
- zomwe zilipo kale.
Ndiye ma index ndi chiyani?
Mlozera ndi dongosolo lapadera la data lomwe timasunga ndikusintha kuwonjezera pa data yayikulu. Amagwiritsidwa ntchito kufulumizitsa kusaka. Popanda ma index, kusaka kungafune kudutsa mu data yonse (njira yotchedwa scan scan), ndipo njirayi imakhala ndi zovuta za algorithmic. Koma nkhokwe nthawi zambiri zimakhala ndi data yochulukirapo komanso zovuta zama mzere ndizochedwa kwambiri. Moyenera, titha kupeza logarithmic kapena yokhazikika.
Uwu ndi mutu wovuta kwambiri, wodzazidwa ndi zidziwitso ndi kusinthanitsa, koma nditayang'ana zaka makumi angapo zakukula kwa database ndi kafukufuku, ndili wokonzeka kunena kuti pali njira zochepa zomwe zimagwiritsidwa ntchito popanga ma index a database.
Njira yoyamba ndiyo kuchepetsa mwadongosolo malo osaka, kugawa malo osaka kukhala ang'onoang'ono.
Nthawi zambiri timachita izi pogwiritsa ntchito mitengo yamitundu yosiyanasiyana. Chitsanzo chingakhale bokosi lalikulu la zipangizo mu chipinda chanu chomwe chili ndi mabokosi ang'onoang'ono a zipangizo zogawidwa m'mitu yosiyanasiyana. Ngati mukufuna zipangizo, mungayang'ane m'bokosi lomwe limati "Zida" osati lomwe limati "Macookie," eti?
Njira yachiwiri ndikusankha nthawi yomweyo chinthu chomwe mukufuna kapena gulu lazinthu. Timachita izi mumapu a hashi kapena ma index a reverse. Kugwiritsa ntchito mapu a hashi ndikofanana kwambiri ndi chitsanzo cham'mbuyomu, koma m'malo mwa bokosi la mabokosi, muli ndi mabokosi ang'onoang'ono azinthu zomaliza m'chipinda chanu.
Njira yachitatu ndiyo kuthetsa kufunika kofufuza. Timachita izi pogwiritsa ntchito zosefera za Bloom kapena zosefera za cuckoo. Oyamba amayankha nthawi yomweyo, kukupulumutsani kuti musafufuze.
Njira yotsiriza ndiyo kugwiritsa ntchito mokwanira mphamvu zonse zomwe hardware yamakono imatipatsa. Izi ndi zomwe timachita muzolozera za bitmap. Inde, tikamazigwiritsa ntchito nthawi zina timafunika kudutsa mlozera wonse, koma timachita bwino kwambiri.
Monga ndanenera, mutu wama index a database ndi waukulu komanso wodzaza ndi zosagwirizana. Izi zikutanthauza kuti nthawi zina titha kugwiritsa ntchito njira zingapo nthawi imodzi: ngati tikufuna kufulumizitsa kusaka kwambiri, kapena ngati tikufuna kusaka mitundu yonse yosaka.
Lero ndilankhula za njira yosadziwika bwino ya izi - ma index a bitmap.
Ndine ndani kuti ndiyankhule pamutuwu?
Ndimagwira ntchito ngati gulu lotsogolera ku Badoo (mwina mumadziwa zambiri zazinthu zathu zina, Bumble). Tili kale ndi ogwiritsa ntchito opitilira 400 miliyoni padziko lonse lapansi komanso zinthu zambiri zomwe zimawasankhira omwe ali abwino kwambiri. Timachita izi pogwiritsa ntchito misonkhano, kuphatikiza ma index a bitmap.
Ndiye kodi index ya bitmap ndi chiyani?
Ma index a Bitmap, monga momwe dzinalo likusonyezera, gwiritsani ntchito bitmaps kapena bitsets kuti mugwiritse ntchito index yosaka. Kuchokera m'maso mwa mbalame, cholozerachi chimakhala ndi mapu a bitmap amodzi kapena angapo oyimira mabungwe aliwonse (monga anthu) ndi katundu kapena magawo awo (zaka, mtundu wamaso, ndi zina zotero), ndi ndondomeko yogwiritsira ntchito bitmap (NDI, KAPENA, OSATI). ) kuti muyankhe funso losaka.
Timauzidwa kuti ma index a bitmap ndi oyenerera bwino komanso ochita bwino kwambiri ngati pali zofufuza zomwe zimaphatikiza mafunso pamipingo yotsika kwambiri (ganizirani "mtundu wamaso" kapena "m'banja" motsutsana ndi "mtunda kuchokera pakati pa mzinda"). Koma ndikuwonetsa pambuyo pake kuti amagwiranso ntchito bwino pamagawo apamwamba kwambiri.
Tiyeni tiwone chitsanzo chosavuta cha bitmap index.
Tangoganizani kuti tili ndi mndandanda wamalesitilanti aku Moscow omwe ali ndi zinthu zingapo monga izi:
- pafupi ndi metro;
- pali malo oimikapo magalimoto payekha;
- pali khonde (ali ndi bwalo);
- mukhoza kusunga tebulo (kuvomereza kusungitsa);
- oyenera osadya zamasamba (okonda zamasamba);
- okwera mtengo (okwera mtengo).
Tiyeni tipatse malo odyera aliwonse nambala yotsatizana kuyambira 0 ndikugawa kukumbukira kwa 6 bitmaps (imodzi pamtundu uliwonse). Tidzadzaza ma bitmapswa kutengera ngati malo odyera ali ndi malowa kapena ayi. Ngati malo odyera 4 ali ndi veranda, ndiye pang'ono No.
Tsopano tili ndi cholozera chosavuta kwambiri cha bitmap, ndipo titha kuchigwiritsa ntchito kuyankha mafunso monga:
- βNdiwonetseni malo odyera okonda zamasambaβ;
- Ndiwonetseni malo odyera otsika mtengo okhala ndi khonde momwe mungasungire tebulo.
Bwanji? Tiyeni tiwone. Pempho loyamba ndi losavuta kwambiri. Zomwe tikuyenera kuchita ndikutenga "bitmap wokonda zamasamba" ndikusandutsa mndandanda wamalesitilanti omwe mabala awo amawonekera.
Pempho lachiwiri ndi lovuta kwambiri. Tiyenera kugwiritsa ntchito NOT bitmap pa bitmap "yokwera mtengo" kuti tipeze mndandanda wa malo odyera otsika mtengo, ndiye NDIPO ndi "kodi ndingasungitse tebulo" bitmap ndi NDI zotsatira zake ndi "pali khonde" bitmap. Zotsatira za bitmap zidzakhala ndi mndandanda wa malo omwe amakwaniritsa zofunikira zathu zonse. Mu chitsanzo ichi, awa ndi malo odyera a Yunost okha.
Pali malingaliro ambiri okhudzidwa, koma musadandaule, tiwona code posachedwa.
Kodi ma index a bitmap amagwiritsidwa ntchito pati?
Ngati inu Google bitmap indexes, 90% ya mayankho adzakhala okhudzana ndi Oracle DB mu njira imodzi. Koma ma DBMS ena mwina amathandiziranso chinthu chabwino chotere, sichoncho? Osati kwenikweni.
Tiyeni tidutse mndandanda wa omwe akukayikira.
MySQL sikuthandizira ma index a bitmap, koma pali Malingaliro owonjezera izi ().
PostgreSQL sichirikiza ma index a bitmap, koma imagwiritsa ntchito ma bitmap osavuta ndi magwiridwe antchito pang'ono kuphatikiza zotsatira zosaka pama index ena angapo.
Tarantool ili ndi ma index a bitset ndipo imathandizira kusaka kosavuta pa iwo.
Redis ili ndi magawo osavuta) popanda kutha kuzifufuza.
MongoDB sikuthandizira ma index a bitmap, koma palinso Pempho losonyeza kuti izi ziwonjezedwa.
Elasticsearch imagwiritsa ntchito bitmaps mkati).
- Koma mnansi watsopano waonekera mβnyumba mwathu: Pilosa. Ili ndi nkhokwe yatsopano yosagwirizana ndi ubale yolembedwa mu Go. Lili ndi ma index a bitmap okha ndipo amakhazikitsa chilichonse pa iwo. Tikambirana pambuyo pake.
Kukhazikitsa mu Go
Koma chifukwa chiyani ma index a bitmap sagwiritsidwa ntchito kawirikawiri? Ndisanayankhe funsoli, ndikufuna ndikuwonetseni momwe mungagwiritsire ntchito index ya bitmap yosavuta kwambiri mu Go.
Bitmaps kwenikweni ndi zidutswa za data. Mu Go, tiyeni tigwiritse ntchito magawo a byte pa izi.
Tili ndi bitmap imodzi ya malo odyera amodzi, ndipo chidutswa chilichonse mu bitmap chimawonetsa ngati malo odyera ena ali ndi malowa kapena ayi.
Tidzafunika ntchito ziwiri zothandizira. Mmodzi adzagwiritsidwa ntchito kudzaza bitmaps athu ndi deta mwachisawawa. Mwachisawawa, koma ndi mwayi wina kuti malo odyera ali ndi katundu aliyense. Mwachitsanzo, ndikukhulupirira kuti pali malo odyera ochepa kwambiri ku Moscow komwe simungathe kusungira tebulo, ndipo zikuwoneka kwa ine kuti pafupifupi 20% ya malowa ndi oyenera odya zamasamba.
Ntchito yachiwiri idzasintha bitmap kukhala mndandanda wa malo odyera.
Kuti tiyankhe funso lakuti "Ndiwonetseni malo odyera otsika mtengo omwe ali ndi khonde ndipo akhoza kusungitsa malo," tifunika maopaleshoni awiri: OSATI ndi NDI.
Titha kufewetsa kachidindo kathu pang'ono pogwiritsa ntchito wovuta kwambiri NDI OSATI.
Tili ndi ntchito pa chilichonse mwazochitazi. Onse a iwo amadutsa mu magawo, atenge zinthu zomwe zikugwirizana ndi aliyense, aziphatikiza ndi ntchito pang'ono ndikuyika zotsatira mu kagawo kotsatira.
Ndipo tsopano titha kugwiritsa ntchito ma bitmaps athu ndi ntchito kuti tiyankhe funso losaka.
Kuchita sikuli kokwera kwambiri, ngakhale kuti ntchitozo ndi zophweka kwambiri ndipo tinasunga ndalama zambiri posabwezera kagawo katsopano kamene kanayitanitsidwa.
Nditapanga mbiri pang'ono ndi pprof, ndidazindikira kuti Go compiler idasowa kukhathamiritsa kophweka koma kofunikira kwambiri: kuyika ntchito.
Chowonadi ndi chakuti Go compiler imawopa kwambiri malupu omwe amadutsa mu magawo, ndipo amakana mwatsatanetsatane ntchito zomwe zili ndi malupu oterowo.
Koma sindikuwopa ndipo nditha kupusitsa wopangayo pogwiritsa ntchito goto m'malo mwa loop, monga m'masiku akale abwino.
Ndipo, monga mukuwonera, tsopano wophatikizayo adzasangalala ndi ntchito yathu! Zotsatira zake, timatha kusunga pafupifupi 2 microseconds. Osayipa kwenikweni!
Botolo lachiwiri ndi losavuta kuwona ngati muyang'ana mozama pamisonkhano. Wopangayo adawonjezera cheke cham'malire mkati mwa loop yathu yotentha kwambiri. Chowonadi ndi chakuti Go ndi chilankhulo chotetezeka, wolembayo akuwopa kuti mikangano yanga itatu (magawo atatu) ndi yamitundu yosiyanasiyana. Kupatula apo, ndiye kuti padzakhala mwayi wongoyerekeza wa zochitika zomwe zimatchedwa kuti buffer kusefukira.
Tiyeni titsimikizire wopangayo powonetsa kuti magawo onse ndi ofanana. Titha kuchita izi powonjezera cheke chosavuta kumayambiriro kwa ntchito yathu.
Powona izi, wophatikizayo amadumpha cheke mosangalala, ndipo pamapeto pake timasunga ma nanoseconds ena 500.
Mabala akuluakulu
Chabwino, tidakwanitsa kufinya magwiridwe antchito pakukhazikitsa kwathu kosavuta, koma zotsatira zake ndizoyipa kwambiri kuposa momwe zingathere ndi zida zamakono.
Zomwe timachita ndi ntchito zoyambira, ndipo mapurosesa athu amazichita bwino kwambiri. Koma, mwatsoka, "timadyetsa" purosesa yathu ndi ntchito zazing'ono kwambiri. Ntchito zathu zimagwira ntchito pafupipafupi. Titha kusintha khodi yathu mosavuta kuti tigwire ntchito ndi 8-byte chunks pogwiritsa ntchito magawo a UInt64.
Monga mukuonera, kusintha kwakung'ono kumeneku kunafulumizitsa pulogalamu yathu kasanu ndi katatu powonjezera kukula kwa batch ndi kasanu ndi katatu. Kupindula kunganenedwe kukhala mzere.
Kukonzekera mu assembler
Koma awa si mathero. Mapurosesa athu amatha kugwira ntchito ndi chunks 16, 32 komanso 64 byte. Ntchito "zozama" zotere zimatchedwa single instruction multiple data (SIMD; malangizo amodzi, deta zambiri), ndi njira yosinthira code kuti igwiritse ntchito ntchito zoterezi imatchedwa vectorization.
Tsoka ilo, Go compiler ndiyabwino kwambiri pa vectorization. Pakadali pano, njira yokhayo yosinthira Go code ndikutenga ndikuyika izi pamanja pogwiritsa ntchito Go assembler.
Go assembler ndi chilombo chachilendo. Mwinamwake mukudziwa kuti chinenero cha msonkhano ndi chinachake chomwe chimagwirizanitsidwa kwambiri ndi kamangidwe ka kompyuta yomwe mukulembera, koma sizili choncho mu Go. Go assembler ili ngati IRL (chilankhulo choyimira pakati) kapena chilankhulo chapakatikati: ndiyodziyimira pawokha. Rob Pike adachita bwino kwambiri pamutuwu zaka zingapo zapitazo ku GopherCon ku Denver.
Kuphatikiza apo, Go imagwiritsa ntchito mawonekedwe osazolowereka a Plan 9, omwe amasiyana ndi mawonekedwe ovomerezeka a AT&T ndi Intel.
Ndizomveka kunena kuti kulemba Go assembler ndi dzanja sikosangalatsa kwambiri.
Koma, mwamwayi, pali zida ziwiri zapamwamba zomwe zimatithandiza kulemba Go assembler: PeachPy ndi avo. Zida zonsezi zimapanga Go assembler kuchokera pama code apamwamba olembedwa mu Python ndi Go, motsatana.
Zidazi zimathandizira zinthu monga kugawira ma registry, kulemba malupu, komanso kufewetsa njira yolowera mdziko la pulogalamu yapagulu mu Go.
Tidzagwiritsa ntchito avo, kotero kuti mapulogalamu athu azikhala pafupifupi mapulogalamu a Go.
Izi ndi zomwe chitsanzo chosavuta cha pulogalamu ya avo chikuwoneka. Tili ndi main () ntchito, yomwe imatanthawuza mkati mwawokha ntchito ya Add () , tanthauzo lake ndikuwonjezera manambala awiri. Pali ntchito zothandizira pano kuti mupeze magawo ndi dzina ndikupeza imodzi mwazolembetsa zaulere komanso zoyenera. Ntchito iliyonse ya purosesa imakhala ndi ntchito yofananira pa avo, monga tawonera mu ADDQ. Pomaliza, tikuwona ntchito yothandizira kusunga mtengo wotsatira.
Poyimba go panga, tipanga pulogalamuyi pa avo ndipo chifukwa chake, mafayilo awiri adzapangidwa:
- add.s ndi nambala yomwe ikubwera mu Go assembler;
- stub.go yokhala ndi mitu yolumikizira maiko awiriwa: Pitani ndikuphatikiza.
Tsopano popeza tawona zomwe avo amachita ndi momwe, tiyeni tiwone ntchito zathu. Ndidagwiritsa ntchito mitundu yonse ya scalar ndi vector (SIMD).
Tiyeni tiwone mitundu ya scalar poyamba.
Monga m'chitsanzo cham'mbuyomo, tikupempha kaundula wa cholinga chaulere komanso chovomerezeka, sitiyenera kuwerengera miyeso ndi kukula kwa mikangano. avo amatichitira zonsezi.
Tinkakonda kugwiritsa ntchito zilembo ndi goto (kapena kudumpha) kuti tiwongolere magwiridwe antchito ndikupusitsa Go compiler, koma tsopano tikuchita kuyambira pachiyambi. Mfundo ndi yakuti mikombero ndi lingaliro lapamwamba. Mu assembler, timangokhala ndi zilembo ndi kulumpha.
Khodi yotsalayo iyenera kukhala yodziwika bwino komanso yomveka. Timatsanzira chipika chokhala ndi zilembo ndikudumphira, timatenga kachidutswa kakang'ono kuchokera ku magawo athu awiri, kuwaphatikiza ndi opareshoni pang'ono (NDIPO OSATI munkhaniyi) ndikuyika zotsatira zake mugawo lotsatira. Zonse.
Izi ndi momwe code yophatikizira yomaliza imawonekera. Sitinafunikire kuwerengetsa kuchuluka kwake ndi makulidwe (owonetsedwa mobiriwira) kapena kutsatira zolembera zomwe zagwiritsidwa ntchito (zowunikira mofiira).
Ngati tifanizira magwiridwe antchito a chilankhulo cha msonkhano ndi magwiridwe antchito abwino kwambiri mu Go, tiwona kuti ndizofanana. Ndipo izi zikuyembekezeredwa. Kupatula apo, sitinachite chilichonse chapadera - tangopanganso zomwe Go compiler angachite.
Tsoka ilo, sitingakakamize wosonkhanitsa kuti agwirizane ndi ntchito zathu zolembedwa m'chinenero cha msonkhano. The Go compiler pakadali pano ilibe mawonekedwe otere, ngakhale pakhala pempho loti muwonjezere kwa nthawi yayitali.
Ichi ndichifukwa chake sikutheka kupeza phindu lililonse kuchokera kuzinthu zing'onozing'ono muchinenero cha msonkhano. Tiyenera kulemba ntchito zazikulu, kapena kugwiritsa ntchito masamu/bits phukusi latsopano, kapena kulambalala chilankhulo chophatikiza.
Tiyeni tsopano tiwone mitundu ya vekitala ya ntchito zathu.
Pachitsanzo ichi, ndinaganiza zogwiritsa ntchito AVX2, kotero tidzagwiritsa ntchito ma 32-byte chunks. Mapangidwe a code ndi ofanana kwambiri ndi mtundu wa scalar: kutsitsa magawo, kupempha kaundula waulere, ndi zina.
Chimodzi mwazinthu zatsopano ndikuti ma vector ambiri amagwiritsa ntchito ma regista apadera apadera. Pankhani ya 32-byte chunks, awa ndi zolembera zolembedwa ndi Y. Ichi ndichifukwa chake mumawona YMM() ntchito mu code. Ndikadakhala ndikugwiritsa ntchito AVX-512 yokhala ndi 64-bit chunks, choyambirira chikanakhala Z.
Kupanga kwachiwiri ndikuti ndidaganiza zogwiritsa ntchito kukhathamiritsa kotchedwa loop unrolling, kutanthauza kuchita ma opareshoni asanu ndi atatu pamanja ndisanadumphire koyambirira kwa loop. Kukhathamiritsa uku kumachepetsa kuchuluka kwa nthambi zomwe zili mu code, ndipo zimachepetsedwa ndi chiwerengero cha zolembera zaulere zomwe zilipo.
Nanga bwanji za magwiridwe antchito? Iye ndi wokongola! Tinapeza liwiro la pafupifupi kasanu ndi kawiri poyerekeza ndi njira yabwino kwambiri ya Go. Zochititsa chidwi, chabwino?
Koma ngakhale kukhazikitsa kumeneku kutha kufulumizitsidwa pogwiritsa ntchito AVX-512, prefetching kapena JIT (yokha-mu-nthawi yophatikizira) ya wokonza mafunso. Koma uwu ndi mutu wa lipoti lapadera.
Mavuto ndi ma index a bitmap
Tsopano popeza tawona kale kukhazikitsa kosavuta kwa index ya bitmap mu Go ndi yopindulitsa kwambiri muchilankhulo chosonkhana, tiyeni tikambirane chifukwa chake ma index a bitmap sagwiritsidwa ntchito kawirikawiri.
Mapepala akale amatchula mavuto atatu ndi ma index a bitmap, koma mapepala atsopano ndipo ndikutsutsa kuti salinso oyenera. Sitidzamira mozama mu lililonse la mavutowa, koma kuyang'ana iwo mwachiphamaso.
Vuto la high cardinality
Chifukwa chake, timauzidwa kuti ma index a bitmap ndi oyenera minda yomwe ili ndi makhadi otsika, ndiye kuti, omwe ali ndi zikhalidwe zochepa (mwachitsanzo, jenda kapena mtundu wamaso), ndipo chifukwa chake ndikuti chiwonetsero chanthawi zonse cha magawo oterowo (chimodzi). pang'onopang'ono pa mtengo) pa nkhani ya high cardinality, idzatenga malo ochulukirapo ndipo, kuwonjezera apo, ma index a bitmap adzakhala osadzaza (kawirikawiri).
Nthawi zina tingagwiritse ntchito zizindikiro zosiyana, monga momwe timagwiritsira ntchito kuimira manambala. Koma kunali kubwera kwa compression algorithms komwe kunasintha chilichonse. Pazaka makumi angapo zapitazi, asayansi ndi ofufuza apeza njira zambiri zoponderezera ma bitmaps. Ubwino wawo waukulu ndikuti palibe chifukwa chochepetsera ma bitmaps kuti agwire ntchito pang'ono - titha kuchita maopaleshoni ang'onoang'ono mwachindunji pamapu othinikizidwa.
Posachedwapa, njira zosakanizidwa zayamba kuwonekera, monga ma bitmaps olira. Panthawi imodzimodziyo amagwiritsa ntchito zowonetsera zitatu zosiyana za bitmaps - bitmaps okha, masanjidwe ndi zomwe zimatchedwa bit runs - ndikuwongolera pakati pawo kuti apititse patsogolo ntchito ndikuchepetsa kukumbukira kukumbukira.
Mutha kupeza ma bitmaps akubangula pamapulogalamu odziwika kwambiri. Pali kale kuchuluka kwakukulu kokhazikitsidwa kwa zilankhulo zosiyanasiyana zamapulogalamu, kuphatikiza kupitilira katatu kwa Go.
Njira ina yomwe ingatithandize kuthana ndi makhadi akuluakulu imatchedwa binning. Tangoganizani muli ndi gawo loyimira kutalika kwa munthu. Kutalika ndi nambala yoyandama, koma anthufe sitimaganiza choncho. Kwa ife palibe kusiyana pakati pa kutalika kwa 185,2 cm ndi 185,3 cm.
Zikuoneka kuti tikhoza kugawa mfundo zofanana m'magulu mkati mwa 1 cm.
Ndipo ngati tikudziwanso kuti anthu ochepa kwambiri ndi amfupi kuposa 50 cm ndi aatali kuposa 250 cm, ndiye kuti tikhoza kutembenuza munda wokhala ndi kardinali wopanda malire kukhala munda wokhala ndi makadinali pafupifupi 200.
Inde, ngati kuli kofunikira, tikhoza kuchita zosefera zina pambuyo pake.
Vuto la Bandwidth Yapamwamba
Vuto lotsatira ndi ma index a bitmap ndikuti kukonzanso kumatha kukhala okwera mtengo kwambiri.
Madatabase akuyenera kukonzanso data pomwe mafunso ena mazana ambiri akufufuza. Timafunika maloko kuti tipewe zovuta zofikira nthawi imodzi kapena zovuta zina zogawana. Ndipo pomwe pali loko imodzi yayikulu, pali vuto - mikangano yotseka, loko loko kumakhala kotsekereza.
Vutoli litha kuthetsedwa kapena kuzunguliridwa pogwiritsa ntchito sharding kapena kugwiritsa ntchito indexed indexed.
Sharding ndi chinthu chosavuta komanso chodziwika bwino. Mutha kugawa index ya bitmap monga momwe mungachitire ndi data ina iliyonse. M'malo mwa loko imodzi yaikulu, mudzapeza mulu wa maloko ang'onoang'ono ndipo motero kuchotsa mikangano ya loko.
Njira yachiwiri yothetsera vutoli ndi kugwiritsa ntchito indexed indexed. Mutha kukhala ndi kope limodzi la index yomwe mumagwiritsa ntchito pofufuza kapena kuwerenga, ndi imodzi yomwe mungagwiritse ntchito polemba kapena kukonza. Ndipo kamodzi mu nthawi inayake (mwachitsanzo, kamodzi pa 100 ms kapena 500 ms) mumawabwereza ndikusinthanitsa. Zachidziwikire, njira iyi imagwira ntchito pokhapokha ngati pulogalamu yanu imatha kuthana ndi index yosaka yomwe yatsala pang'ono.
Njira ziwirizi zitha kugwiritsidwa ntchito nthawi imodzi: mutha kukhala ndi cholozera chosinthika.
Mafunso ovuta kwambiri
Vuto lomaliza ndi ma index a bitmap ndikuti timauzidwa kuti sizoyenera kuyankha mafunso ovuta kwambiri, monga mafunso anthawi yayitali.
Zowonadi, ngati mungaganizire, magwiridwe antchito ngati NDI, KAPENA, ndi zina zambiri sizoyenera kufunsa mafunso ngati "Ndiwonetseni mahotela okhala ndi zipinda kuyambira pa madola 200 mpaka 300 usiku uliwonse."
Yankho lopanda nzeru komanso lopanda nzeru kwambiri lingakhale kutenga zotsatira za mtengo uliwonse wa dola ndikuziphatikiza ndi kachitidwe kanzeru OR.
Njira yabwinoko pang'ono ingakhale kugwiritsa ntchito magulu. Mwachitsanzo, m'magulu a madola 50. Izi zitha kufulumizitsa ntchito yathu nthawi 50.
Koma vutoli limathetsedwanso mosavuta pogwiritsa ntchito malingaliro omwe adapangidwa makamaka pa pempho lamtunduwu. M'mapepala asayansi amatchedwa range-encoded bitmaps.
Pachithunzichi, sitimangoyika pang'ono mtengo wina (mwachitsanzo, 200), koma ikani mtengo uwu ndi chirichonse pamwamba. 200 ndi pamwamba. Zomwezo za 300: 300 ndi pamwamba. Ndi zina zotero.
Pogwiritsa ntchito choyimira ichi, titha kuyankha funso lamtunduwu podutsa mlozera kawiri kokha. Choyamba, tipeza mndandanda wamahotela omwe chipindacho chimawononga ndalama zochepa kapena $300, ndiyeno tidzachotsamo omwe mtengo wake ndi wocheperako kapena $199. Okonzeka.
Mudzadabwitsidwa, koma ngakhale ma geoqueries amatha kugwiritsa ntchito ma index a bitmap. Chinyengo ndikugwiritsa ntchito chithunzithunzi cha geometric chomwe chikuzungulira mgwirizano wanu ndi chithunzi cha geometric. Mwachitsanzo, S2 kuchokera ku Google. Chiwerengerocho chiyenera kukhala chotheka kuimira mu mawonekedwe a mizere itatu kapena kuposerapo yomwe ingathe kuwerengedwa. Mwanjira iyi titha kutembenuza geoquery yathu kukhala mafunso angapo "pampata" (m'mizere yolembedwayi).
Okonzeka zothetsera
Ndikukhulupirira kuti ndimakukondani pang'ono ndipo tsopano muli ndi chida china chothandiza muzosungira zanu. Ngati mungafunike kuchita chonga ichi, mudzadziwa njira yoyenera kuyang'ana.
Komabe, si aliyense amene ali ndi nthawi, kuleza mtima, kapena zida zopangira ma index a bitmap kuyambira poyambira. Makamaka apamwamba kwambiri, pogwiritsa ntchito SIMD, mwachitsanzo.
Mwamwayi, pali mayankho angapo okonzeka okuthandizani.
Ma bitmaps akubangula
Choyamba, pali laibulale yomweyi yobangula ya bitmaps yomwe ndinanena kale. Lili ndi zotengera zonse zofunika ndi ntchito pang'ono zomwe mungafunike kuti mupange cholozera chokwanira cha bitmap.
Tsoka ilo, pakadali pano, palibe machitidwe a Go omwe amagwiritsa ntchito SIMD, zomwe zikutanthauza kuti kukhazikitsidwa kwa Go sikugwira ntchito kwambiri kuposa kukhazikitsa kwa C, mwachitsanzo.
Pilosa
Chinthu china chomwe chingakuthandizeni ndi Pilosa DBMS, yomwe, kwenikweni, imakhala ndi ma index a bitmap okha. Iyi ndi njira yatsopano, koma ikugonjetsa mitima mwachangu kwambiri.
Pilosa imagwiritsa ntchito ma bitmaps obangula mkati ndipo imakupatsani mwayi wowagwiritsa ntchito, imathandizira ndikulongosola zonse zomwe ndidazifotokoza pamwambapa: kupanga magulu, ma bitmaps osungidwa, lingaliro lamunda, ndi zina zambiri.
Tiyeni tiwone mwachangu chitsanzo chogwiritsa ntchito Pilosa kuyankha funso lomwe mukulidziwa kale.
Chitsanzocho nβchofanana kwambiri ndi chimene munachiwona kale. Timapanga kasitomala ku seva ya Pilosa, kupanga cholozera ndi magawo ofunikira, kenako mudzaze minda yathu ndi zidziwitso zachisawawa ndi zotheka ndipo, pomaliza, perekani funso lodziwika bwino.
Pambuyo pake, timagwiritsa ntchito OSATI pamunda "wokwera mtengo", kenaka dutsani zotsatira (kapena NDI izo) ndi gawo la "terrace" ndi gawo la "kusungitsa". Ndipo potsiriza, timapeza zotsatira zomaliza.
Ndikukhulupirira kuti m'tsogolomu mtundu watsopano wa index udzawonekeranso mu DBMSs monga MySQL ndi PostgreSQL - bitmap indexes.
Pomaliza
Ngati simunagone, zikomo. Ndinayenera kukhudza mwachidule mitu yambiri chifukwa cha nthawi yochepa, koma ndikuyembekeza kuti nkhaniyi inali yothandiza komanso mwina yolimbikitsa.
Ma index a Bitmap ndi abwino kudziwa, ngakhale simukuwafuna pakali pano. Aloleni akhale chida china mubokosi lanu lazida.
Tayang'ana njira zingapo zogwirira ntchito za Go ndi zinthu zomwe Go compiler sazigwira bwinobe. Koma izi ndizothandiza kwambiri kuti aliyense wopanga mapulogalamu a Go adziwe.
Ndizo zonse zomwe ndimafuna kukuuzani. Zikomo!
Source: www.habr.com