Dongosolo lokhazikika lopezera ukadaulo wa Data

Kwa zaka zisanu ndi zitatu zapitazi ndakhala ndikugwira ntchito monga woyang'anira polojekiti (sindilemba code kuntchito), zomwe mwachibadwa zimasokoneza teknoloji yanga yakumbuyo. Ndinaganiza zotseka kusiyana kwanga paukadaulo ndikupeza ukadaulo wa Data engineer. Luso lofunika kwambiri la Data Engineer ndi luso lopanga, kumanga, ndi kusamalira malo osungiramo deta.

Ndinapanga dongosolo la maphunziro, ndikuganiza kuti lidzakhala lothandiza osati kwa ine ndekha. Ndondomekoyi ikuyang'ana pa maphunziro odzipangira okha. Chofunika kwambiri chimaperekedwa ku maphunziro aulere mu Russian.

Magawo:

• Ma algorithms ndi mapangidwe a data. Gawo lofunikira. Phunzirani ndipo zina zonse ziyenda bwino. Ndikofunikira kuyika manja anu pa code ndikugwiritsa ntchito zoyambira ndi ma aligorivimu.
• Ma database ndi malo osungiramo data, Business Intelligence. Tikuchoka ku ma aligorivimu kupita kusungirako ndi kukonza deta.
• Hadoop ndi Big Data. Pamene database sichikuphatikizidwa pa hard drive, kapena pamene deta ikufunika kufufuzidwa, koma Excel sangathenso kuwayika, deta yaikulu imayamba. Malingaliro anga, ndikofunikira kupitilira gawo ili pokhapokha mutaphunzira mozama ziwiri zam'mbuyomu.

Ma algorithms ndi mapangidwe a data

Mu dongosolo langa, ndinaphatikizapo kuphunzira Python, kubwereza zoyambira masamu ndi algorithmization.

• Kupanga mapulogalamu mu Python
• Python: Zoyambira ndi Ntchito
• Linear algebra
• Likbez pa masamu a discrete
• Algorithms: chiphunzitso ndi machitidwe. Njira
• Algorithms: chiphunzitso ndi machitidwe. Kapangidwe ka Data

Ma database ndi malo osungiramo data, Business Intelligence

• Bukhu: Martin Kleppman - Mapulogalamu odzaza kwambiri. Kupanga, makulitsidwe, thandizo. Bukuli likufotokoza momwe mitundu yosiyanasiyana ya deta imagwirira ntchito, kukhazikitsidwa kwawo kuchokera mkati, zoperewera ndi zosankha malinga ndi ntchitoyo.
• Chiyambi cha database
• Lowani mu DBMS
• Chidziwitso chazosunga zosagwirizana

Mitu yokhudzana ndi kumanga malo osungiramo data, ETL, OLAP cubes imadalira kwambiri zida, kotero sindikupereka maulalo ku maphunziro omwe ali mu chikalatachi. Ndikoyenera kuphunzira machitidwe otere pogwira ntchito inayake mu kampani inayake. Kuti mudziwe bwino ndi ETL, mutha kuyesa Tendala kapena Mayendedwe ampweya.

M'malingaliro anga, ndikofunikira kuphunzira njira zamakono za Data Vault ulalo 1, ulalo 2. Ndipo njira yabwino yophunzirira ndikuyitenga ndikuyigwiritsa ntchito ndi chitsanzo chosavuta. Pali zitsanzo zingapo zogwiritsira ntchito Data Vault pa GitHub ссылка. The Modern Data Warehouse Book: Modelling the Agile Data Warehouse with Data Vault lolemba Hans Hultgren.

Kuti mudziwe zida za Business Intelligence kwa ogwiritsa ntchito omaliza, mutha kugwiritsa ntchito wopanga malipoti aulere, ma dashboards, malo osungiramo data a Power BI Desktop. Zida zophunzitsira: ulalo 1, ulalo 2.

Hadoop ndi Big Data

• Muyenera kuyamba ndi kukhazikitsa paokha kwa MapReduce popanda malaibulale ena. Izi zidzalola kumvetsetsa bwino za kukhazikitsidwa kwa multithreaded mtsogolomo. Chitsanzo chabwino kwambiri cha Python chikufotokozedwa apa.
• Hadoop. Dongosolo lokonzekera kuchuluka kwa data.
• Chiyambi cha Big Data Engineering

Pomaliza

Sikuti zonse zomwe mumaphunzira zitha kugwiritsidwa ntchito kuntchito. Chifukwa chake, muyenera pulojekiti yomaliza maphunziro yomwe mudzayesere kugwiritsa ntchito chidziwitso chatsopano.

Palibe mitu yokhudzana ndi kusanthula kwa data ndi Kuphunzira Kwamakina mu dongosololi. izi zikugwiranso ntchito kwambiri pantchito ya Data Scientist. Palibenso mitu yokhudzana ndi mitambo ya AWS, Azure. mitu imeneyi zimadalira kwambiri kusankha nsanja.

Mafunso kwa anthu ammudzi:
Kodi dongosolo langa losanja likukwana bwanji? Chotsani kapena kuwonjezera chiyani?
Ndi pulojekiti yanji yomwe mungapangire ngati lingaliro?

Source: www.habr.com