Namuhla kuphela abantu abangenakho ukuxhumana kwangaphandle nezwe langaphandle abangakaze bezwe ngedatha enkulu. Ku-HabrΓ©, isihloko sezibalo ze-Big Data nezihloko ezihlobene sidumile. Kodwa kwabangewona ochwepheshe abangathanda ukuzinikela ocwaningweni lwe-Big Data, akucaci ngaso sonke isikhathi ukuthi le ndawo inamaphi amathemba, lapho ukuhlaziya kwe-Big Data kungasetshenziswa khona nokuthi umhlaziyi omuhle angathembela kuphi. Ake sizame ukukuqonda.
Inani lolwazi olukhiqizwa abantu liyakhula minyaka yonke. Ngo-2020, inani ledatha egciniwe lizokhuphuka libe ngu-40-44 zettabytes (1 ZB ~ 1 billion GB). Ngo-2025 - kufika cishe kuma-zettabytes angu-400. Ngokufanelekile, ukuphatha idatha ehlelekile nengahlelekile kusetshenziswa ubuchwepheshe besimanje yindawo ebaluleke kakhulu. Kokubili izinkampani ngazinye nawo wonke amazwe anentshisekelo kudatha enkulu.
Ngendlela, kwakuphakathi nengxoxo ye-boom yolwazi nezindlela zokucubungula idatha ekhiqizwa abantu lapho igama elithi Idatha Elikhulu lavela khona. Kukholakala ukuthi yahlongozwa okokuqala ngo-2008 ngumhleli wephephabhuku iNature, uClifford Lynch.
Kusukela lapho, imakethe ye-Big Data ibilokhu ikhula minyaka yonke ngamaphesenti angamashumi amaningana. Futhi lo mkhuba, ngokusho kochwepheshe, uzoqhubeka. Ngakho, ngokwezilinganiso zenkampani ngo-2021, imakethe yomhlaba wonke enkulu yokuhlaziya idatha izokhuphuka ifike ku-$67,2 billion.
Kungani sidinga izibalo ezinkulu zedatha?
Ikuvumela ukuthi ubone ulwazi olubaluleke kakhulu kumasethi edatha ahlelekile noma angahlelekile. Ngenxa yalokhu, ibhizinisi lingakwazi, ngokwesibonelo, ukuhlonza amathrendi, libikezele ukusebenza kokukhiqiza futhi lenze izindleko zalo ngokugcwele. Kuyacaca ukuthi ukuze kwehliswe izindleko, izinkampani zikulungele ukusebenzisa izixazululo zakamuva.
Ubuchwepheshe nezindlela zokuhlaziya ezisetshenziselwa ukuhlaziya Idatha Enkulu:
- Ukumbiwa Kwedatha;
- i-crowdsourcing;
- ukuxuba nokuhlanganiswa kwedatha;
- ukufunda ngomshini;
- amanethiwekhi emizwa yokwenziwa;
- ukuqashelwa kwephethini;
- izibalo zokubikezela;
- ukulingisa imodeli;
- ukuhlaziywa kwendawo;
- ukuhlaziya izibalo;
- ukubonwa kwedatha yokuhlaziya.
Izibalo Zedatha Enkulu emhlabeni
Ukuhlaziya idatha enkulu manje sekusetshenziswa izinkampani ezingaphezu kuka-50% emhlabeni jikelele. Naphezu kokuthi ngo-2015 lesi sibalo sasingu-17%. I-Big Data isetshenziswa kakhulu izinkampani ezisebenza emikhakheni yezokuxhumana nezinsizakalo zezezimali. Bese kuba nezinkampani ezisebenza ngobuchwepheshe bezempilo. Ukusetshenziswa okuncane kokuhlaziywa Kwedatha Enkulu ezinkampanini zemfundo: ezimweni eziningi, abameleli balo mkhakha bamemezele inhloso yabo yokusebenzisa ubuchwepheshe esikhathini esizayo esiseduze.
E-United States, izibalo ze-Big Data zisetshenziswa kakhulu: ngaphezu kuka-55% wezinkampani ezivela emikhakheni eyahlukene zisebenza ngalobu buchwepheshe. E-Europe nase-Asia, isidingo sokuhlaziya idatha enkulu asikho ngaphansi kakhulu - cishe ngama-53%.
Kuthiwani ngeRussia?
Ngokusho kwabahlaziyi be-IDC,. Ukukhula kwemakethe yezixazululo ezinjalo eCentral naseMpumalanga Yurophu kusebenza kakhulu, lesi sibalo sikhuphuka ngo-11% njalo ngonyaka. Ngo-2022, izofinyelela ku-$ 5,4 billion ngokwemibandela yobuningi.
Ngezindlela eziningi, lokhu kuthuthukiswa okusheshayo kwemakethe kungenxa yokukhula kwale ndawo eRussia. Ngo-2018, imali engenayo evela ekuthengisweni kwezixazululo ezifanele e-Russian Federation yaba ngu-40% wesamba sokutshalwa kwezimali kubuchwepheshe bokucubungula idatha enkulu esifundeni sonke.
E-Russian Federation, izinkampani ezivela emabhange kanye nemikhakha yomphakathi, imboni yezokuxhumana kanye nezimboni zisebenzisa kakhulu ekucubunguleni Idatha Enkulu.
Wenzani i-Big Data Analyst futhi uhola malini eRussia?
Umhlaziyi wedatha omkhulu unomthwalo wemfanelo wokuhlola inani elikhulu lolwazi, kokubili olunesakhiwo esincane nesingakhiwe. Ezinhlanganweni zamabhange lokhu ukuthengiselana, opharetha - izingcingo kanye nethrafikhi, ekuthengiseni - ukuvakasha kwamakhasimende nokuthenga. Njengoba kushiwo ngenhla, ukuhlaziywa kwe-Big Data kusivumela ukuthi sithole ukuxhumana phakathi kwezinto ezihlukahlukene "kumlando wolwazi olungahluziwe", isibonelo, inqubo yokukhiqiza noma ukusabela kwamakhemikhali. Ngokusekelwe kudatha yokuhlaziya, izindlela ezintsha nezisombululo zithuthukiswa ezindaweni ezihlukahlukene - kusukela ekukhiqizeni kuye kwezokwelapha.
Amakhono adingekayo kumhlaziyi Wedatha Enkulu:
- Ikhono lokuqonda ngokushesha izici endaweni lapho ukuhlaziya okwenziwa khona, kanye nokucwilisa izici zendawo oyifunayo. Lokhu kungaba ukudayisa, imboni kawoyela negesi, imithi, njll.
- Ulwazi lwezindlela zokuhlaziywa kwedatha yezibalo, ukwakhiwa kwamamodeli ezibalo (amanethiwekhi e-neural, amanethiwekhi e-Bayesia, ukuhlanganisa, ukuhlehla, isici, ukuhlukahluka kanye nokuhlaziywa kokuxhumana, njll.).
- Ukwazi ukukhipha idatha emithonjeni ehlukene, uyiguqule ukuze ihlaziywe, futhi uyilayishe kusizindalwazi sokuhlaziya.
- Unolwazi ku-SQL.
- Ulwazi lwesiNgisi ezingeni elanele ukufunda kalula imibhalo yezobuchwepheshe.
- Ulwazi lwePython (okungenani okuyisisekelo), i-Bash (kunzima kakhulu ukwenza ngaphandle kwayo ngesikhathi somsebenzi), futhi kuyafiseleka ukwazi izisekelo ze-Java ne-Scala (ezidingekayo ukuze kusetshenziswe ngokugcwele i-Spark, enye ye izinhlaka ezidume kakhulu zokusebenza ngedatha enkulu).
- Ikhono lokusebenza ne-Hadoop.
Awu, uzuza malini umhlaziyi we-Big Data?
Ochwepheshe Bedatha Enkulu manje bayashoda; isidingo sidlula ukunikezwa. Lokhu kungenxa yokuthi ibhizinisi liya ngokuya liqondana: ukuthuthukiswa kudinga ubuchwepheshe obusha, futhi ukuthuthukiswa kobuchwepheshe kudinga ochwepheshe.
Ngakho-ke, i-Data Scientist kanye ne-Data Analytics e-USA ngokusho kwesikhungo sokuqasha i-Glassdoor. Isilinganiso somholo walaba chwepheshe eMelika siqala ku-$100 XNUMX ngonyaka.
E-Russia, ochwepheshe bokufunda ngomshini bathola ama-ruble ayizinkulungwane eziyi-130 kuye kwezingama-300 ngenyanga, abahlaziyi bedatha abakhulu - kusuka kuma-ruble ayizinkulungwane ezingama-73 kuye kwangama-200 ngenyanga. Konke kuncike kwisipiliyoni kanye neziqu. Kunjalo-ke zikhona izikhala ezihola kancane, kanti ezinye zinezikhundla eziphezulu. Isidingo esiphezulu sabahlaziyi bedatha abakhulu eMoscow naseSt. I-Moscow, okungamangazi, ibala cishe i-50% yezikhala ezisebenzayo (ngokusho kwe-hh.ru). Isidingo esincane kakhulu siseMinsk naseKyiv. Kubalulekile ukuqaphela ukuthi ezinye izikhala zinikeza amahora aguquguqukayo nomsebenzi okude. Kodwa ngokuvamile, izinkampani zidinga ochwepheshe abasebenza ehhovisi.
Ngokuhamba kwesikhathi, singalindela ukwanda kwesidingo sabahlaziyi Bedatha Enkulu nabamele izici ezikhethekile ezihlobene. Njengoba kushiwo ngenhla, ukushoda kwabasebenzi emkhakheni wezobuchwepheshe akukhanseliwe. Kodwa-ke, ukuze ube umhlaziyi we-Big Data, udinga ukufunda futhi usebenze, uthuthukise amakhono abhalwe ngenhla kanye nalawo angeziwe. Elinye lamathuba okuqala indlela yomhlaziyi weBig Data bese uzama isandla sakho ekusebenzeni ngedatha enkulu.
Source: www.habr.com