Uhlelo lokulinganisa lokuthola unjiniyela weDatha yobungcweti

Kule minyaka eyisishiyagalombili edlule bengisebenza njengomphathi wephrojekthi (angibhali ikhodi emsebenzini), okuthinta kabi i-backend yami yezobuchwepheshe. Nginqume ukuvala igebe lami lezobuchwepheshe ngithole umsebenzi wobunjiniyela beDatha. Ikhono eliwumgogodla Lonjiniyela Wedatha yikhono lokuklama, ukwakha, nokugcina izindawo zokugcina idatha.

Ngenze uhlelo lokuqeqesha, ngicabanga ukuthi luzoba usizo hhayi kimina kuphela. Uhlelo lugxile ezifundweni zokuzifundela. Okubalulekile kunikezwa izifundo zamahhala ngesiRashiya.

Izigaba:

• Ama-algorithms nezakhiwo zedatha. Isigaba esibalulekile. Yifunde futhi konke okunye kuzosebenza. Kubalulekile ukuthola izandla zakho kukhodi futhi usebenzise izakhiwo eziyisisekelo nama-algorithms.
• Izingosi zolwazi kanye nezinqolobane zedatha, iBusiness Intelligence. Sisuka kuma-algorithms siye kwisitoreji nokucubungula idatha.
• I-Hadoop nedatha Enkulu. Uma i-database ingafakiwe ku-hard drive, noma lapho idatha idinga ukuhlaziywa, kodwa i-Excel ayikwazi ukuyilayisha, idatha enkulu iqala. Ngokubona kwami, kubalulekile ukuqhubekela kulesi sigaba kuphela ngemva kocwaningo olunzulu lwalezi ezimbili ezedlule.

Ama-algorithms nezakhiwo zedatha

Kuhlelo lwami, ngifake ukufunda iPython, ngiphinda izisekelo zezibalo kanye ne-algorithmization.

• Ukuhlela kuPython
• I-Python: Okuyisisekelo kanye nezicelo
• I-algebra yomugqa
• I-Likbez kwizibalo ezihlukene
• Ama-algorithms: ithiyori kanye nokwenza. Izindlela
• Ama-algorithms: ithiyori kanye nokwenza. Izakhiwo Zedatha

Izingosi zolwazi kanye nezinqolobane zedatha, iBusiness Intelligence

• Incwadi: UMartin Kleppman - Izinhlelo zokusebenza ezilayishwe kakhulu. Ukuhlela, ukukala, ukusekela. Incwadi ichaza ukuthi amamodeli edatha ahlukene asebenza kanjani, ukusetshenziswa kwawo ngaphakathi, imikhawulo kanye nokukhetha kuye ngomsebenzi.
• Isingeniso kusizindalwazi
• Gxumela ku-DBMS
• Isingeniso kusizindalwazi esingahlobene

Izihloko ezihlobene nokwakhiwa kwezindawo zokugcina idatha, i-ETL, amakhyubhu e-OLAP ancike kakhulu kumathuluzi, ngakho-ke angizinikezi izixhumanisi zezifundo ezikulo mbhalo. Kutuswa ukufunda izinhlelo ezinjalo lapho usebenza kuphrojekthi ethile enkampanini ethile. Ukuze ujwayelane ne-ETL, ungazama Ikhalenda noma Ukungena komoya.

Ngokubona kwami, kubalulekile ukufunda indlela yesimanje ye-Data Vault design isixhumanisi 1, isixhumanisi 2. Futhi indlela engcono kakhulu yokuyifunda ukuyithatha futhi uyisebenzise ngesibonelo esilula. Kunezibonelo ezimbalwa zokusebenzisa i-Data Vault ku-GitHub isixhumanisi. I-Modern Data Warehouse Book: Ukumodela kwe-Agile Data Warehouse nge-Data Vault ka-Hans Hultgren.

Ukuze ujwayelane namathuluzi eBusiness Intelligence kubasebenzisi bokugcina, ungasebenzisa umklami wamahhala wemibiko, amadeshibhodi, izindawo zokugcina idatha ezincane ze-Power BI Desktop. Izinto zokufunda: isixhumanisi 1, isixhumanisi 2.

I-Hadoop nedatha Enkulu

• Udinga ukuqala ngokusetshenziswa okuzimele kwe-MapReduce ngaphandle kwamalabhulali ezinkampani zangaphandle. Lokhu kuzovumela ukuqondwa okungcono kokusetshenziswa kwe-multithreaded esikhathini esizayo. Kuchazwe isibonelo esihle kakhulu kuPython lapha.
• I-Hadoop. Uhlelo lokucubungula amanani amakhulu edatha.
• Isingeniso ku-Big Data Engineering

isiphetho

Akukona konke okufundayo okungasetshenziswa emsebenzini. Ngakho-ke, udinga iphrojekthi yokuthweswa iziqu lapho uzozama khona ukusebenzisa ulwazi olusha.

Azikho izihloko ezihlobene nokuhlaziywa kwedatha kanye Nokufunda Ngomshini ohlelweni. lokhu kusebenza kakhulu emsebenzini we-Data Scientist. Azikho futhi izihloko ezihlobene namafu we-AWS, i-Azure. lawa matimu ancike kakhulu ekukhethweni kweplatifomu.

Imibuzo eya emphakathini:
Lanele kangakanani uhlelo lwami lokulinganisa? Yini okufanele uyisuse noma wengeze?
Iyiphi iphrojekthi ongayincoma njengethisisi?

Source: www.habr.com