Lesi yingxenye yesibili yochungechunge lwama-athikili mayelana nezinhlelo zokuhlaziya ().
Namuhla akusekho ukungabaza ukuthi ukucubungula idatha ngokucophelela nokuchazwa kwemiphumela kungasiza cishe noma yiluphi uhlobo lwebhizinisi. Mayelana nalokhu, izinhlelo zokuhlaziya ziya ngokuya zilayishwa ngamapharamitha, futhi inani lezimbangi nemicimbi yabasebenzisi kuzinhlelo zokusebenza liyakhula.
Ngenxa yalokhu, izinkampani zinikeza abahlaziyi bazo ulwazi oluthe xaxa noluluhlaza ukuze bahlaziye futhi baphenduke izinqumo eziphusile. Ukubaluleka kwesistimu ye-analytics yenkampani akufanele kuthathwe kancane, futhi uhlelo ngokwalo kufanele luthembeke futhi luzinze.
Abahlaziyi beklayenti
Izibalo zekhasimende yisevisi inkampani exhuma kuwebhusayithi yayo noma uhlelo lokusebenza nge-SDK esemthethweni, ihlanganise nekhodibase yayo bese ikhetha izibangeli zemicimbi. Kunobubi obusobala kule ndlela: yonke idatha eqoqiwe ingase ingacutshungulwa ngendlela ongathanda ngayo ngenxa yemikhawulo yanoma iyiphi isevisi oyikhethayo. Isibonelo, ohlelweni olulodwa ngeke kube lula ukwenza imisebenzi ye-MapReduce, kwenye ngeke ukwazi ukusebenzisa imodeli yakho. Okunye okungalungile kuzoba inkokhiso ejwayelekile (ehlaba umxhwele) yezinsizakalo.
Kunezixazululo eziningi ze-analytics zamakhasimende emakethe, kodwa ngokushesha noma kamuva abahlaziyi babhekene neqiniso lokuthi ayikho insizakalo yendawo yonke efanelekile kuwo wonke umsebenzi (ngenkathi amanani azo zonke lezi zinsizakalo ekhuphuka ngaso sonke isikhathi). Esimeni esinjalo, izinkampani zivame ukunquma ukwakha uhlelo lwazo lokuhlaziya ngazo zonke izilungiselelo ezidingekayo namakhono.
Abahlaziyi beseva
Izibalo zohlangothi lweseva yisevisi engafakwa ngaphakathi kwenkampani kumaseva ayo futhi (imvamisa) ngemizamo yayo. Kulo modeli, yonke imicimbi yomsebenzisi igcinwa kumaseva angaphakathi, okuvumela abathuthukisi ukuthi bazame i-database ehlukene yesitoreji futhi bakhethe i-architecture elula kakhulu. Futhi noma ngabe usafuna ukusebenzisa izibalo zeklayenti lenkampani yangaphandle kweminye imisebenzi, kusazokwenzeka.
Izibalo zohlangothi lweseva zingasetshenziswa ngezindlela ezimbili. Okokuqala: khetha izinsiza zomthombo ovulekile, zisebenzise emishinini yakho futhi uthuthukise ingqondo yebhizinisi.
ΠΠ»ΡΡΡ
ΠΠΈΠ½ΡΡΡ
Ungenza ngokwezifiso noma yini oyifunayo
Lokhu kuvame ukuba nzima kakhulu futhi kudinga onjiniyela abahlukene
Okwesibili: thatha izinsiza ze-SaaS (i-Amazon, i-Google, i-Azure) esikhundleni sokuzithumela ngokwakho. Sizokhuluma nge-SaaS ngokuningiliziwe engxenyeni yesithathu.
ΠΠ»ΡΡΡ
ΠΠΈΠ½ΡΡΡ
Ingase ishibhile kumavolumu aphakathi, kodwa ngokukhula okukhulu isazobiza kakhulu
Ngeke kwenzeke ukulawula wonke amapharamitha
Ukuphatha kudluliselwa ngokuphelele emahlombe omhlinzeki wesevisi
Akwaziwa njalo ukuthi yini engaphakathi kwesevisi (kungase kungadingeki)
Uziqoqa kanjani izibalo zeseva
Uma sifuna ukuqhela ekusebenziseni izibalo zamaklayenti futhi sizakhele okwethu, okokuqala kudingeka sicabange ngesakhiwo sohlelo olusha. Ngezansi ngizokutshela isinyathelo ngesinyathelo ukuthi yini okudingeka uyicabangele, kungani isinyathelo ngasinye sidingeka nokuthi yimaphi amathuluzi ongawasebenzisa.
1. Ukuthola idatha
Njengoba nje endabeni yokuhlaziya amakhasimende, okokuqala, abahlaziyi benkampani bakhetha izinhlobo zemicimbi abafuna ukuyifunda esikhathini esizayo futhi baziqoqe ohlwini. Ngokuvamile, lezi zenzakalo zenzeka ngokulandelana okuthile, okubizwa ngokuthi "iphethini yomcimbi."
Okulandelayo, cabanga ukuthi uhlelo lokusebenza lweselula (iwebhusayithi) lunabasebenzisi abavamile (amadivayisi) namaseva amaningi. Ukuze udlulise ngokuphephile imicimbi ukusuka kumadivayisi ukuya eziphakelini, kudingeka isendlalelo esimaphakathi. Kuye ngesakhiwo, kungase kube nolayini bemicimbi eminingi eyahlukene.
Ingabe , esetshenziswa njengomugqa wokuqoqa imicimbi.
Ngokusho ngo-2014, umdali we-Apache Kafka wanquma ukuqamba isofthiwe ngo-Franz Kafka ngoba "iwuhlelo olulungiselelwe ukubhala" futhi ngenxa yokuthi wayeyithanda imisebenzi ka-Kafka. -
Esibonelweni sethu, kukhona abakhiqizi bedatha abaningi nabathengi bedatha (amadivayisi namaseva), futhi i-Kafka isiza ukuxhumanisa komunye nomunye. Abathengi bazochazwa ngokuningiliziwe ezinyathelweni ezilandelayo, lapho bazoba yizihloko eziyinhloko. Manje sizocabangela abakhiqizi bedatha kuphela (imicimbi).
I-Kafka ihlanganisa imiqondo yomugqa nokuhlukanisa; kungcono ukufunda kabanzi mayelana nalokhu kwenye indawo (isibonelo, ku- ). Ngaphandle kokungena emininingwaneni, ake sicabange ukuthi uhlelo lokusebenza lweselula lwethulwa ama-OS amabili ahlukene. Bese inguqulo ngayinye idala ukusakazwa kwayo komcimbi okuhlukile. Abakhiqizi bathumela imicimbi e-Kafka, iqoshwa kulayini ofanele.
(isithombe )
Ngesikhathi esifanayo, i-Kafka ikuvumela ukuthi ufunde ngeziqephu futhi ucubungule uchungechunge lwemicimbi ngamaqoqo amancane. I-Kafka iyithuluzi elilula kakhulu elilinganisa kahle nezidingo ezikhulayo (isibonelo, ngokubekwa kwendawo kwemicimbi).
Ngokuvamile i-shard eyodwa yanele, kodwa izinto ziba nzima nakakhulu lapho kulinganiswa (njengoba zihlala zenza). Mhlawumbe akekho ozofuna ukusebenzisa ishadi elilodwa kuphela elibonakalayo ekukhiqizeni, njengoba ukwakheka kufanele kube nokubekezelela amaphutha. Ngaphezu kweKafka, kunesinye isixazululo esaziwayo - RabbitMQ. Asizange siyisebenzise ekukhiqizeni njengomugqa wokuhlaziya imicimbi (uma unolwazi olunjalo, sitshele ngakho kumazwana!). Nokho, sasebenzisa i-AWS Kinesis.
Ngaphambi kokudlulela esinyathelweni esilandelayo, sidinga ukusho esinye isendlalelo esengeziwe sesistimu - isitoreji selogi eluhlaza. Lesi akusona isendlalelo esidingekayo, kodwa sizosiza uma kukhona okungahambanga kahle futhi ulayini womcimbi e-Kafka usethwa kabusha. Ukugcina izingodo eziluhlaza akudingi isixazululo esiyinkimbinkimbi nesibizayo; ungavele uzibhale ndawana thize ngendlela efanele (ngisho naku-hard drive).
2. Icubungula ukusakazwa komcimbi
Ngemva kokuba sesilungiselele zonke izenzakalo futhi sizibeke emigqeni efanele, siqhubekela esinyathelweni sokucubungula. Lapha ngizokutshela mayelana nezinketho ezimbili ezivame kakhulu zokucubungula.
Inketho yokuqala ukunika amandla Ukusakazwa kwe-Spark ohlelweni lwe-Apache. Yonke imikhiqizo ye-Apache iphila ku-HDFS, isistimu yefayela evikelekile enemibhalo yamafayela. I-Spark Streaming iyithuluzi elisebenziseka kalula eliphatha idatha yokusakaza-bukhoma nezikali kahle. Nokho, kungase kube nzima ukulondoloza.
Enye inketho ukwakha isibambi sakho somcimbi. Ukuze wenze lokhu, udinga, isibonelo, ukubhala uhlelo lokusebenza lwe-Python, ulwakhe ku-Docker futhi ubhalisele kulayini we-Kafka. Lapho izibangeli zifika kuzibambi ze-docker, ukucubungula kuzoqala. Ngale ndlela, udinga ukugcina izinhlelo zokusebenza zisebenza ngaso sonke isikhathi.
Ake sicabange ukuthi sikhethe enye yezinketho ezichazwe ngenhla futhi siqhubekele ekucubunguleni ngokwayo. Abacubungula kufanele baqale ngokubheka ukufaneleka kwedatha, ukuhlunga udoti kanye nemicimbi βephukileβ. Ukuqinisekisa sivame ukusebenzisa . Ngemva kwalokhu, ungenza imephu yedatha: idatha evela emithonjeni ehlukene ijwayeleke futhi ifane ukuze yengezwe kuthebula elivamile.
3. Isizindalwazi
Isinyathelo sesithathu ukugcina imicimbi ejwayelekile. Lapho sisebenza ngesistimu yokuhlaziya esenziwe ngomumo, kuzodingeka sifinyelele kubo kaningi, ngakho-ke kubalulekile ukukhetha i-database elula.
Uma idatha ingena kahle esikimini esilungisiwe, ungakhetha noma enye i-database yekholomu. Ngale ndlela ama-aggregations azosebenza ngokushesha okukhulu. Okubi ukuthi uhlelo luqiniswe ngokuqinile ngakho-ke ngeke kwenzeke ukwengeza izinto ezingenangqondo ngaphandle kokuguqulwa (isibonelo, uma kwenzeka umcimbi ongajwayelekile). Kodwa ungabala ngokushesha okukhulu.
Ngedatha engahlelekile, ungathatha i-NoSQL, isibonelo, . Isebenza ku-HDFS, iphindaphinda kahle, ungaphakamisa izimo eziningi, futhi iyawabekezelela amaphutha.
Ungakwazi futhi ukuphakamisa okuthile okulula, isibonelo, . Iyanensa impela futhi ingeyamavolumu amancane. Kodwa okuhlanganisayo ukuthi kulula kakhulu futhi ngakho-ke kufanelekile ukuqala.
4. Ukuhlanganiswa
Ngemva kokulondoloza ngokucophelela yonke imicimbi, sifuna ukuqoqa lonke ulwazi olubalulekile kuqeqebana elifikile futhi sibuyekeze imininingwane. Emhlabeni jikelele, sifuna ukuthola amadeshibhodi namamethrikhi abalulekile. Isibonelo, qoqa iphrofayela yomsebenzisi ezenzakalweni futhi ulinganise ukuziphatha ngandlela thize. Imicimbi iyahlanganiswa, iqoqwe, bese ilondolozwa futhi (emathebula abasebenzisi). Ngesikhathi esifanayo, ungakha isistimu ukuze ukwazi futhi ukuxhuma isihlungi kumxhumanisi we-aggregator: qoqa abasebenzisi kuphela ohlotsheni oluthile lomcimbi.
Ngemva kwalokho, uma othile eqenjini edinga kuphela izibalo eziphezulu, amasistimu okuhlaziya angaphandle angaxhunywa. Ungathatha i-Mixpanel futhi. kodwa njengoba kubiza kakhulu, akuyona yonke imicimbi yabasebenzisi ethunyelwa lapho, kodwa kuphela okudingekayo. Ukuze senze lokhu, sidinga ukudala umxhumanisi ozodlulisela imicimbi ethile engavuthiwe noma okuthile thina ngokwethu esikuhlanganise ngaphambili kumasistimu angaphandle, ama-API noma izinkundla zokukhangisa.
5. Ingaphambili
Udinga ukuxhuma i-frontend ohlelweni oludaliwe. Isibonelo esihle isevisi , iyi-GUI yedathabheyisi esiza ukwakha amadeshibhodi. Indlela ukusebenzelana okusebenza ngayo:
- Umsebenzisi wenza umbuzo we-SQL.
- Ephendula uthola isibonakaliso.
- Idala 'ukubonakala okusha' kwayo futhi ithola igrafu enhle ongazigcinela yona.
Okubonwayo kusevisi kuzibuyekeza ngokuzenzakalela, ungenza ngendlela oyifisayo futhi ulandelele ukuqapha kwakho. I-Redash imahhala uma izibambele ngokwakho, kodwa njenge-SaaS izobiza u-$50 ngenyanga.
isiphetho
Ngemva kokuqeda zonke izinyathelo ezingenhla, uzodala izibalo zeseva yakho. Sicela uqaphele ukuthi lokhu akulula njengokuxhuma nje izibalo zekhasimende, ngoba yonke into idinga ukulungiselelwa ngokwakho. Ngakho-ke, ngaphambi kokudala isistimu yakho, kufanelekile ukuqhathanisa isidingo sohlelo olubucayi lwe-analytics nezinsiza ozimisele ukuyabela kulo.
Uma wenze izibalo futhi uthole ukuthi izindleko ziphezulu kakhulu, engxenyeni elandelayo ngizokhuluma ngendlela yokwenza inguqulo eshibhile yezibalo ze-server-side.
Siyabonga ngokufunda! Ngizojabula ukubuza imibuzo kumazwana.
Source: www.habr.com