Ngokutsho kwe-Statista, ngo-2025 ubukhulu bemakethe yedatha enkulu buya kukhula ukuya kwi-175 zettabytes xa kuthelekiswa ne-41 ngo-2019 (). Ukufumana umsebenzi kule ntsimi, kufuneka uqonde indlela yokusebenza kunye nedatha enkulu egcinwe efini. I-Cloud4Y iqulunqe uluhlu lwezifundo ze-12 ezihlawulweyo kunye nezasimahla zobunjineli eziya kwandisa ulwazi lwakho kwintsimi kwaye inokuba sisiqalo esihle kwindlela yakho eya kwiziqinisekiso zamafu.
I ngcaciso
Yintoni injineli yedatha? Lo ngumntu onoxanduva lokudala nokugcina ulwakhiwo lwedatha kwiprojekthi yeNzululwazi yeDatha. Uxanduva lunokubandakanya ukuqinisekisa ukuhamba kakuhle kwedatha phakathi komncedisi kunye nesicelo, ukudibanisa isofthiwe entsha yokulawula idatha, ukuphucula iinkqubo zedatha ezisisiseko, kunye nokudala imibhobho yedatha.
Kukho inani elikhulu lobuchwepheshe kunye nezixhobo ekufuneka injineli yedatha isebenze ukuze isebenze nge-computing yefu, iindawo zokugcina idatha, i-ETL (ukukhutshwa, ukuguqulwa, ukulayishwa), njl. Ngaphezu koko, inani lezakhono ezifunekayo likhula lonke ixesha; ngoko unonjineli wedatha kufuneka azalise ulwazi lwakhe rhoqo. Uluhlu lwethu lubandakanya iikhosi zabaqalayo kunye neengcali ezinamava. Khetha okukufaneleyo.
1. Isiqinisekiso seNanodegree yobuNjineli beDatha ()
Uya kufunda indlela yokuyila iimodeli zedatha, udale iindawo zokugcina idatha kunye namachibi edatha, ngokuzenzekelayo imibhobho yedatha kunye nokusebenza kunye noluhlu lweedatha. Ekupheleni kwenkqubo, uya kuvavanya izakhono zakho ezintsha ngokugqiba iprojekthi yeCapstone.
ubude bexesha: Iinyanga ezi-5, iiyure ezi-5 ngeveki
ULwimi: IsiNgesi
ixabiso: $ 1695
kwinqanaba: okokuqala
2. Yiba yiSatifikethi seNjineli yeDatha ()
Bafundisa kwizinto ezisisiseko. Unokuqhubela phambili inyathelo ngenyathelo, usebenzisa iintetho kunye neeprojekthi zezandla ukuze usebenze kwizakhono zakho. Ekupheleni koqeqesho, uya kuba ulungele ukusebenza kunye neML kunye nedatha enkulu. Kucetyiswa ukuba wazi iPython ubuncinci kwinqanaba elincinci.
ubude bexesha: Iinyanga ezi-8, iiyure ezi-10 ngeveki
ULwimi: IsiNgesi
ixabisoπ
kwinqanaba: okokuqala
3. Yiba yiNjineli yeDatha: Ukulawula iiNgcebiso ()
Uya kuphuhlisa izakhono zobunjineli bedatha kunye nezakhono ze-DevOps, ufunde ukwenza izicelo zeDatha enkulu, wenze imibhobho yedatha, uqhube izicelo ngexesha langempela usebenzisa iHazelcast kunye nedathabheyisi..
ubude bexesha: Kuxhomekeke kuwe
ULwimi: IsiNgesi
ixabiso: inyanga yokuqala - simahla
kwinqanaba: okokuqala
4. Iikhosi zobuNjineli beDatha ()
Nalu uluhlu lweenkqubo ezikwazisa ngobunjineli bedatha kwaye zifundise indlela yokuphuhlisa izisombululo zohlalutyo. Iikhosi zohlulwe ngokweendidi ngokusekelwe kwinqanaba lobunzima, ngoko ke unokukhetha enye ngokwenqanaba lamava akho. Ngethuba loqeqesho uya kufunda ukusebenzisa i-Spark, i-Hadoop, i-Azure kunye nokulawula idatha yenkampani.
ubude bexesha: Kuxhomekeke kuwe
ULwimi: IsiNgesi
ixabiso: kuxhomekeke kwikhosi ekhethiweyo
kwinqanaba: umqali, uphakathi, phambili
5. INjineli yeDatha ()
Le khosi ifanelekile ukuba uyithathe ukuba unamava ngePython kwaye ufuna ukwenza nzulu ulwazi lwakho kunye nokwakha ikhondo lomsebenzi njengenzululwazi yedatha. Uya kufunda indlela yokwakha imibhobho yedatha usebenzisa iPython kunye nepandas, ukulayisha iiseti ezinkulu zedatha kwi-Postgres database emva kokucoca, ukuguqula nokuqinisekisa.
ubude bexesha: Kuxhomekeke kuwe
ULwimi: IsiNgesi
ixabiso: kuxhomekeke kwifomu yobhaliso
kwinqanaba: umqali, ophakathi
6. Ubunjineli beDatha ngeLifu likaGoogle ()
Le khosi iya kukunceda ufumane izakhono ozidingayo ukuze wakhe umsebenzi kwidatha enkulu. Umzekelo, ukusebenza neBigQuery, Spark. Uya kufumana ulwazi oludingayo ukuze ulungiselele isiqinisekiso seNjineli yeLifu likaGoogle elaziwayo kwiLifu.
ubude bexesha: Iinyanga ezingama-4
ULwimi: IsiNgesi
ixabiso: simahla okwangoku
kwinqanaba: umqali, ophakathi
7. Ubunjineli beDatha, iDatha enkulu kwi-Google Cloud Platform ()
Ikhosi enika umdla ebonelela ngolwazi olusebenzayo lweenkqubo zokucwangcisa idatha kwi-GCP. Ngexesha leklasi, uya kufunda indlela yokuyila iinkqubo ngaphambi kokuqala inkqubo yophuhliso. Ukongeza, uya kuphinda uhlalutye idatha ecwangcisiweyo kunye nengacwangciswanga, usebenzise i-auto-scaling, kwaye usebenzise ubuchule be-ML ukukhupha ulwazi.
ubude bexesha: Iinyanga ezingama-3
ULwimi: IsiNgesi
ixabiso: simahla okwangoku
kwinqanaba: umqali, ophakathi
8. UC San Diego: Ubungcali beDatha enkulu ()
Ikhosi isekelwe ekusebenziseni i-Hadoop kunye ne-Spark framework kunye nokusebenzisa obu buchule bedatha enkulu kwinkqubo ye-ML. Uya kufunda iziseko zokusebenzisa iHadoop ngeMaphuNciphisa, iSpark, iHagu, kunye neHive. Funda indlela yokwakha imifuziselo eqikelelwayo kwaye usebenzise uhlalutyo lwegrafu ukumisela iingxaki. Nceda uqaphele ukuba le khosi ayifuni nawaphi na amava okuprograma.
ubude bexesha: Iinyanga ezisi-8 iiyure ezili-10 ngeveki
ULwimi: IsiNgesi
ixabiso: simahla okwangoku
kwinqanaba: okokuqala
9. Ukulawula iDatha enkulu ngeApache Spark kunye nePython ()
Uya kufunda indlela yokusebenzisa ubume bomjelo kunye neefreyimu zedatha kwi-Spark3, kwaye ufumane ukuqonda malunga nendlela yokusebenzisa inkonzo ye-Amazon Elastic MapReduce ukuze usebenze kunye neqela lakho leHadoop. Funda ukuchonga iingxaki kuhlalutyo olukhulu lwedatha kwaye uqonde ukuba iilayibrari zeGraphX ββzisebenza njani ngohlalutyo lwenethiwekhi kunye nendlela onokuyisebenzisa ngayo iMLlib.
ubude bexesha: Kuxhomekeke kuwe
ULwimi: IsiNgesi
ixabiso: ukusuka kwi-ruble ye-800 ukuya kwi-$ 149,99 (kuxhomekeke kwinhlanhla yakho)
kwinqanaba: umqali, ophakathi
10. Inkqubo yePG kubuNjineli beDatha enkulu ()
Le khosi iya kukunika ukuqonda malunga nendlela i-Aadhaar esebenza ngayo, indlela i-Facebook ilungisa ngayo ukutya kweendaba, kunye nendlela ubuNjineli beDatha obunokusetyenziswa ngayo ngokubanzi. Izihloko eziphambili ziya kuba yinkqubo yedatha (kubandakanywa nokulungiswa kwexesha langempela), i-MapReduce, kunye nohlalutyo olukhulu lwedatha.
ubude bexesha: iinyanga ezi-11
ULwimi: IsiNgesi
ixabiso: malunga ne-3000 yeedola
kwinqanaba: okokuqala
11. INzululwazi yeDatha yobuNzululwazi ()
Uya kufunda ukwenza inkqubo kwiPython, ufunde izikhokelo zoqeqesho lwe-neural network Tensorflow kunye neKeras. Inkosi yeMongoDB, iPostgreSQL, iSQLite3 yogcino-lwazi, funda ukusebenzisana nePandas, iNumPy kunye namathala eencwadi eMatpotlib.
ubude bexesha: Iiyure ezingama-300 zoqeqesho
ULwimi: IsiRashiya
ixabiso: Iinyanga ezintandathu zokuqala zikhululekile, emva koko i-ruble ye-3900 ngenyanga
kwinqanaba: okokuqala
12. Injineli yedatha 7.0 ()
Uya kufumana uphononongo olunzulu lweKafka, HDFS, ClickHouse, Spark, Airflow, lambda architecture kunye nekappa architecture. Uya kufunda indlela yokudibanisa izixhobo komnye nomnye, ukwenza imibhobho, ukufumana isisombululo esisisiseko. Ukufundisisa, ulwazi oluncinci lwePython 3 luyafuneka.
ubude bexesha: Izifundo ezingama-21, iiveki ezisi-7
ULwimi: IsiRashiya
ixabiso: ukusuka kwi-60 ukuya kwi-ruble ye-000
kwinqanaba: okokuqala
Ukuba ufuna ukongeza enye ikhosi elungileyo kuluhlu, ungazikhupha kwizimvo okanye kwi-PM. Siza kuhlaziya isithuba.
Yintoni enye onokuyifunda kwiblogi?
β
β
β
β
β
Bhalisela yethu -ijelo ukuze ungaphoswa linqaku elilandelayo. Asibhali ngaphezu kwesibini ngeveki kwaye kuphela kwishishini. Siyakukhumbuza kwakhona ukuba ngoMeyi 21 ngo-15: 00 (ixesha laseMoscow) siya kubamba kumxholo othi "Ukhuseleko lolwazi lweshishini xa usebenza ukude." Ukuba ufuna ukuqonda indlela yokukhusela ulwazi olubuthathaka kunye noshishino xa abasebenzi besebenza ekhaya, bhalisa!
umthombo: www.habr.com