Molo, Habr! Igama lam nguMaxim Vasiliev, ndisebenza njengomhlalutyi kunye nomphathi weprojekthi kwi-FINCH. Namhlanje ndingathanda ukukuxelela ukuba, sisebenzisa i-ElasticSearch, sikwazile ukusetyenzwa ngemibuzo ezizigidi ezili-15 kwimizuzu emi-6 kwaye songeze umthwalo wemihla ngemihla kwindawo yomnye wabathengi bethu. Ngelishwa, kuya kufuneka senze ngaphandle kwamagama, ekubeni sine-NDA, sinethemba lokuba umxholo wenqaku awuyi kubandezeleka kule nto. Masihambe.
Indlela iprojekthi isebenza ngayo
Kwi-backend yethu senza iinkonzo eziqinisekisa ukusebenza kwewebhusayithi zabathengi bethu kunye nesicelo seselula. Ubume ngokubanzi bunokubonwa kumzobo:
Kwinkqubo yomsebenzi, siqhuba inani elikhulu leentengiselwano: ukuthengwa, iintlawulo, ukuthengiselana ngeebhalansi zabasebenzisi, apho sigcina iilogi ezininzi, kunye nokungenisa nokuthumela le datha kwiinkqubo zangaphandle.
Kukho neenkqubo ezibuyela umva xa sifumana idatha kumxhasi kwaye siyidlulisela kumsebenzisi. Ukongeza, kukho neenkqubo zokusebenza ngeentlawulo kunye neenkqubo zebhonasi.
Imvelaphi emfutshane
Ekuqaleni, sasebenzisa i-PostgreSQL njengendawo yethu yokugcina idatha. Iinzuzo zayo zisemgangathweni kwi-DBMS: ubukho bokuthengiselana, ulwimi oluphuhlisiwe lokubuyisela idatha, uluhlu olubanzi lwezixhobo zokudibanisa; kudityaniswe nokusebenza kakuhle kwanelisa iimfuno zethu ixesha elide ngokwaneleyo.
Sigcine ngokupheleleyo yonke idatha kwiPostgres: ukusuka kwintengiselwano ukuya kwiindaba. Kodwa inani labasebenzisi likhule, kwaye kunye nenani lezicelo.
Ukuqonda, inani lonyaka leeseshoni kwi-2017 kwi-desktop site yodwa yayiyi-131 yezigidi. Kwi-2018 - izigidi ze-125. Kwi-2019 kwakhona izigidi ze-130. Yongeza enye i-100-200 yezigidi ukusuka kwi-mobile version yesayithi kunye nesicelo seselula, kwaye wena uya kufumana inani elikhulu lezicelo.
Njengoko iprojekthi ikhula, i-Postgres ayikwazanga ukujamelana nomthwalo; asikwazanga ukuqhubeka - inani elikhulu lemibuzo eyahlukeneyo yavela, apho asikwazanga ukwenza inani elaneleyo lezalathisi.
Siye saqaphela ukuba kukho imfuneko yezinye izitolo zedatha eziza kuhlangabezana neemfuno zethu kwaye zithathe umthwalo kwi-PostgreSQL. I-Elasticsearch kunye ne-MongoDB zijongwe njengokhetho olunokwenzeka. Aba bokugqibela baphulukene nezi ngongoma zilandelayo:
- Isantya sesalathisi esicothayo njengoko umthamo wedatha kwizalathisi ukhula. Nge-Elastic, isantya asixhomekekanga kubungakanani bedatha.
- Akukho phendlo lokubhaliweyo okupheleleyo
Ke sizikhethele i-Elastic kwaye salungiselela inguqu.
Tshintshela kwi-Elastic
1. Saqala utshintsho ngenkonzo yokukhangela indawo yokuthengisa. Umxhasi wethu unamanqaku angama-70 entengiso, kwaye kwangaxeshanye iintlobo ezininzi zokukhangela ziyafuneka kwiwebhusayithi nakwisicelo:
- Uphendlo lombhalo ngegama lendawo
- I-Geosearch ngaphakathi kwerediyasi enikiweyo ukusuka kwindawo ethile. Umzekelo, ukuba umsebenzisi ufuna ukubona ukuba zeziphi iindawo zokuthengisa ezikufutshane nekhaya lakhe.
- Khangela ngesikwere esinikiweyo - umsebenzisi uchaza isikwere kwimephu, kwaye zonke iingongoma kule radius ziboniswa kuye.
- Phendla ngezihluzo ezongezelelweyo. Amanqaku entengiso ahluka enye kwenye ngokwee-assortment
Ukuthetha ngombutho, kwi-Postgres sinomthombo wedatha kuzo zombini imephu kunye neendaba, kwaye kwi-Elastic senza i-Snapshots ukusuka kwidatha yokuqala. Inyani kukuba ekuqaleni iPostgres ayikwazanga ukumelana nokukhangela zonke iindlela. Akunjalo nje ukuba zininzi izalathisi, zinokuphinda zidlulelane, ngoko ke umcwangcisi wePostgres walahleka kwaye akazange aqonde ukuba yeyiphi isalathiso emayisetyenziswe.
2. Okulandelayo kumgca icandelo leendaba. Upapasho luvela kwindawo yonke imihla ukuze umsebenzisi angalahleki ekuhambeni kolwazi, idatha kufuneka ihlelwe ngaphambi kokukhutshwa. Nantsi into yokukhangela: kwisayithi unokukhangela ngokuhambelana nokubhaliweyo, kwaye kwangaxeshanye uqhagamshele izihluzi ezongezelelweyo, kuba zenziwe nge-Elastic.
3. Emva koko siye sahambisa inkqubo yentengiselwano. Abasebenzisi banokuthenga imveliso ethile kwisiza kwaye bathathe inxaxheba kutsalo lwamabhaso. Emva kokuthengwa okunjalo, siqhuba inani elikhulu ledatha, ngakumbi ngeempelaveki kunye neeholide. Ukuthelekisa, ukuba ngeentsuku eziqhelekileyo inani lokuthenga liyindawo ejikeleze i-1,5-2 yezigidi, ngoko ngeeholide inani linokufikelela kwi-53 yezigidi.
Ngexesha elifanayo, idatha kufuneka iqhutywe kwinani elincinci lexesha-abasebenzisi abathandi ukulinda iintsuku ezininzi ngeziphumo. Akukho ndlela yokufezekisa ixesha elibekiweyo nge-Postgres - besihlala sifumana iibhloko, kwaye ngelixa siqhuba zonke izicelo, abasebenzisi abakwazanga ukujonga ukuba bawafumene amabhaso okanye hayi. Oku akukho mnandi kakhulu kwishishini, ke ngoko siye sayisa inkqubo kwi-Elasticsearch.
Ixesha lokuhlaziya
Ngoku uhlaziyo lucwangcisiwe lusekwe kumsitho, ngokwemiqathango elandelayo:
- Amanqaku okuthengisa. Ngokukhawuleza xa sifumana idatha kumthombo wangaphandle, siqalisa ngokukhawuleza ukuhlaziywa.
- Iindaba. Nje ukuba naziphi na iindaba zihlelwe kwisiza, zithunyelwa ngokuzenzekelayo kwi-Elastic.
Apha kwakhona kufanelekile ukukhankanya izibonelelo ze-Elastic. Kwi-Postgres, xa uthumela isicelo, kufuneka ulinde de iqhube ngokunyanisekileyo zonke iirekhodi. Ungathumela iirekhodi eziliwaka ezili-10 kwi-Elastic kwaye uqale ukusebenza ngokukhawuleza, ngaphandle kokulinda ukuba iirekhodi zihanjiswe kuzo zonke iiShards. Ewe kunjalo, enye iShard okanye iReplica ayinakuyibona idatha kwangoko, kodwa kungekudala yonke into iya kufumaneka.
Iindlela zokudibanisa
Kukho iindlela ezi-2 zokudibanisa ne-Elastic:
- Ngokusebenzisa umxhasi womthonyama nge-TCP. Umqhubi womthonyama uyafa ngokuthe ngcembe: ayisaxhaswa, kwaye i-syntax yayo ayisebenzi kakhulu. Ke ngoko, asiyisebenzisi kwaye sizame ukuyiyeka ngokupheleleyo.
- Nge-HTTP interface apho ungasebenzisa zombini izicelo ze-JSON kunye ne-Lucene syntax. Eyokugqibela yinjini yokubhaliweyo esetyenziswa yi-Elastic. Kolu khetho, sifumana ukukwazi ukwenza iBatch ngezicelo ze-JSON ngaphezulu kwe-HTTP. Olu lukhetho esizama ukulusebenzisa.
Ngombulelo kwi-interface ye-HTTP, sinokusebenzisa iilayibrari ezibonelela ngokufezekiswa kwe-asynchronous yomxhasi we-HTTP. Sinokusebenzisa i-Batch kunye ne-asynchronous API, ekugqibeleni inika ukusebenza okuphezulu, okuye kwanceda kakhulu ngexesha leentsuku zokunyuswa okukhulu (ngaphezulu koku ngezantsi)
Amanye amanani othelekiso:
- Ukugcina abasebenzisi abafumene amabhaso kwiPostgres kwimisonto engama-20 ngaphandle kwamaqela: iirekhodi ezingama-460713 kwimizuzwana engama-42
- Umxhasi we-Elastic + osebenzayo wemisonto eli-10 + ibhetshi yezinto ezili-1000: iirekhodi ezingama-596749 kwimizuzwana eyi-11
- I-Elastic + umxhasi osebenzayo wemisonto eli-10 + ibhetshi yezinto ezili-1000: Iirekhodi ze-23801684 kwimizuzu emi-4
Ngoku sibhale umphathi wesicelo se-HTTP owakha i-JSON njengeBatch / non-Batch kwaye uyithumela ngayo nayiphi na iklayenti ye-HTTP, kungakhathaliseki ukuba ilayibrari. Unokukhetha ukuthumela izicelo ngolungelelwaniso okanye ngokungalungelelanisiyo.
Kolunye udibaniso sisasebenzisa umxhasi wezothutho osemthethweni, kodwa lo ngumbandela nje wohlengahlengiso olukhawulezileyo. Kule meko, umxhasi wayo, owakhiwe ngesiseko se-Spring WebClient, isetyenziselwa ukucubungula.
Ukunyuswa okukhulu
Kanye ngonyaka, iprojekthi ibamba ukukhuthazwa okukhulu kubasebenzisi - oku kuPhakamisa okufanayo, kuba ngeli xesha sisebenza ngamashumi ezigidi zabasebenzisi ngaxeshanye.
Ngokuqhelekileyo, imithwalo ephezulu iyenzeka ngexesha leeholide, kodwa olu khuthazo lukwinqanaba elahluke ngokupheleleyo. Kunyaka ongaphambi kokugqibela, ngomhla wokunyuselwa, sathengisa iiyunithi ezingama-27 zeempahla. Idatha ithathe ngaphezu kwesiqingatha seyure ukusetyenzwa, nto leyo ebangele ukuphazamiseka kubasebenzisi. Abasebenzisi bafumene amabhaso okuthatha inxaxheba, kodwa kwacaca ukuba inkqubo ifuna ukukhawuleziswa.
Ekuqaleni kuka-2019, sagqiba kwelokuba i-ElasticSearch iyafuneka. Unyaka wonke, siququzelele ukusetyenzwa kwedatha efunyenweyo kwi-Elastic kunye nemveliso yayo kwi-API yesicelo seselula kunye newebhusayithi. Ngenxa yoko, kunyaka olandelayo ngexesha lokunyuswa siqhubekile 15 amangenelo kwimizuzu emi-131.
Ekubeni sinabantu abaninzi abafuna ukuthenga imveliso kwaye bathathe inxaxheba kwi-draw draw kwi-promotions yethu, lo ngumlinganiselo wexeshana. Ngoku sithumela ulwazi lwangoku kwi-Elastic, kodwa kwixesha elizayo siceba ukudlulisela ulwazi olugciniweyo kwiinyanga ezidlulileyo kwi-Postgres njengendawo yokugcina esisigxina. Ukuze ungavali isalathisi se-Elastic, esinemida yayo.
Isiphelo/Izigqibo
Okwangoku, sidlulisele zonke iinkonzo ebesizifuna kwi-Elastic kwaye siphumle okwangoku. Ngoku sakha isalathisi kwi-Elastic phezulu kwendawo yokugcina eqhubekayo kwi-Postgres, ethatha umthwalo womsebenzisi.
Kwixesha elizayo, siceba ukudlulisela iinkonzo ukuba siyaqonda ukuba isicelo sedatha siya sihluke kakhulu kwaye sikhangelwa inani elingenamkhawulo lamakholomu. Oku akusengomsebenzi wePostgres.
Ukuba sifuna ukukhangela okubhaliweyo okugcweleyo ekusebenzeni okanye ukuba sineendlela ezininzi ezahlukeneyo zokukhangela, ngoko siyazi ukuba oku kufuneka kuguqulelwe kwi-Elastic.
βββ
Enkosi ngokufunda. Ukuba inkampani yakho ikwasebenzisa i-ElasticSearch kwaye inamatyala akho okuphumeza, nceda usixelele. Kuya kuba mnandi ukwazi ukuba abanye benza njani :)
umthombo: www.habr.com