{"id":34083,"date":"2019-10-31T21:56:19","date_gmt":"2019-10-31T18:56:19","guid":{"rendered":"https:\/\/prohoster.info\/blog\/kto-takie-data-inzhenery-i-kak-imi-stanovyatsya\/"},"modified":"2019-10-31T21:56:19","modified_gmt":"2019-10-31T18:56:19","slug":"kto-takie-data-inzhenery-i-kak-imi-stanovyatsya","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/kto-takie-data-inzhenery-i-kak-imi-stanovyatsya","title":{"rendered":"Kes on andmeinsenerid ja kuidas neist saavad?","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>Tere taas! Artikli pealkiri r\u00e4\u00e4gib enda eest. Enne kursuse algust <noindex><a rel=\"nofollow\" href=\"https:\/\/otus.pw\/a4Uj\/\">\u201eAndmeinsener\u201c<\/a><\/noindex> pakkume v\u00e4lja selgitada, kes on andmeinsenerid. Artiklis on palju kasulikke linke. Head lugemist.<\/p>\n<p><img decoding=\"async\" alt=\"Kes on andmeinsenerid ja kuidas neist saavad?\" src=\"\/wp-content\/uploads\/2019\/05\/5f831df35ab1292adafca20f937ed22d.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<i>Lihtne juhend selle kohta, kuidas haarata Data Engineering'ist ning mitte lasta end sellega liialt kaasa viia.<\/i><\/p>\n<p>Tundub, et t\u00e4nap\u00e4eval tahab iga\u00fcks saada andmeanal\u00fc\u00fctikuks (Data Scientist). Aga kuidas on lood andmeinseneri (Data Engineering) ametiga? <i>Sisuliselt on see mingisugune h\u00fcbriid andmeanal\u00fc\u00fctiku ja andmeanal\u00fc\u00fctiku vahel; andmeinsener vastutab tavaliselt t\u00f6\u00f6protsesside, t\u00f6\u00f6tlemiskonveierite ja ETL-protsesside haldamise eest<\/i>. Nende funktsioonide t\u00e4htsuse t\u00f5ttu on see praegu veel \u00fcks populaarne ametigrupp, mis aktiivselt kasvab.<\/p>\n<p>Kohene tasu ja suur n\u00f5udlus on vaid osa sellest, mis teeb t\u00f6\u00f6 \u00e4\u00e4rmiselt atraktiivseks! Kui soovite liituda kangelaste ridadega, pole kunagi hilja \u00f5ppima hakata. Selles postituses olen kogunud kogu vajaliku teabe, et aidata teil teha esimesed sammud.<\/p>\n<p>Nii et alustame!<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p><b>Mis on Data Engineering?<\/b><\/p>\n<p>Ahnelt \u00f6eldes pole paremat selgitust kui see:<\/p>\n<blockquote><p><i>\u201eTeadlane v\u00f5ib avastada uue t\u00e4hte, kuid ei suuda seda luua. Tal on vaja insenerilt seda paluda.\u201d<\/i><\/p>\n<p>\u2013Gordon Lindsay Glegg<\/p><\/blockquote>\n<p>Seega on andmeinseneri roll piisavalt oluline.<\/p>\n<p>Nimest j\u00e4reldub, et andmeinseneria seondub andmetele, nimelt nende edastamisele, s\u00e4ilitamisele ja t\u00f6\u00f6tlemisele. Seega on inseneride peamine \u00fclesanne tagada usaldusv\u00e4\u00e4rne infrastruktuur andmete jaoks. Kui vaatame AI vajaduste hierarhiat, siis andmeinseneeria h\u00f5lmab esimesi 2\u20133 etappi: <b>andmete kogumine, edastamine ja s\u00e4ilitamine, andmete ettevalmistamine<\/b>.<\/p>\n<p><img decoding=\"async\" alt=\"Kes on andmeinsenerid ja kuidas neist saavad?\" src=\"\/wp-content\/uploads\/2019\/05\/377481f2f52a4ed771508a450678afd0.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<b>Millega tegeleb andmeinsener?<\/b><\/p>\n<p>Suure andmemahu saabumisega on vastutusalad dramaatiliselt muutunud. Kui varem kirjutasid need spetsialistid suuri SQL-p\u00e4ringuid ja edastasid andmeid selliste t\u00f6\u00f6riistadega nagu Informatica ETL, Pentaho ETL, Talend, siis n\u00fc\u00fcd on n\u00f5uded andmeinseneridele t\u00f5usnud.<\/p>\n<p>Enamikul ettev\u00f5tetel, kus on avatud andmeinseneri ametikoht, on j\u00e4rgmised n\u00f5udmised:<\/p>\n<ul>\n<li>Suurep\u00e4rane SQL- ja Python'i teadmine.<\/li>\n<li>Kogemus pilveteenustega, eelk\u00f5ige Amazon Web Services'iga.<\/li>\n<li>Soovitav on Java\/Scala teadmine.<\/li>\n<li>Hea arusaam SQL ja NoSQL andmebaasidest (andmemudelite koostamine, andmete salvestamine).<\/li>\n<\/ul>\n<p>Pange t\u00e4hele, et see on vaid k\u00f5ige olulisem. Selle loendi p\u00f5hjal v\u00f5ib j\u00e4reldada, et andmeinsenerid on tarkvaraarenduse ja tagasis\u00fcsteemi spetsialistid.<br \/>\nN\u00e4iteks, kui ettev\u00f5te hakkab genereerima suurt hulka andmeid erinevatest allikatest, on teie \u00fclesanne andmeinsenerina korraldada teabe kogumine, t\u00f6\u00f6tlemine ja salvestamine.<\/p>\n<p>K\u00e4esolevas olukorras kasutatavad t\u00f6\u00f6riistad v\u00f5ivad erineda, k\u00f5ik s\u00f5ltub nende andmete mahust, sissetuleku kiirusest ja mitmekesisusest. Enamik ettev\u00f5tteid ei puutu kokku suure hulga andmetega, seet\u00f5ttu v\u00f5ib keskse salvestusruumina, mida nimetatakse andmehoidmiseks, kasutada SQL andmebaasi (PostgreSQL, MySQL jne) koos v\u00e4ikese hulga skriptidega, mis suunavad andmed salvestusse.<\/p>\n<p>IT-hiiglased, nagu Google, Amazon, Facebook v\u00f5i Dropbox, esitavad k\u00f5rgemaid n\u00f5udmisi: teadmised Pythonist, Java'st v\u00f5i Scala'st.<\/p>\n<ul>\n<li>Kogemus suurte andmete t\u00f6\u00f6tlemisel: Hadoop, Spark, Kafka.<\/li>\n<li>Teadmised algoritmidest ja andmestruktuuridest.<\/li>\n<li>Tundmine jaotus\u00fcsteemide p\u00f5hialustest.<\/li>\n<li>Kogemus andmete visualiseerimise t\u00f6\u00f6riistade, nagu Tableau v\u00f5i ElasticSearch, osas on suur pluss.<\/li>\n<\/ul>\n<p>K\u00e4ib selge suundumus suurte andmete poole, eriti nende t\u00f6\u00f6tlemisel suurte koormuste korral. Nendel ettev\u00f5tetel on s\u00fcsteemi rikke suhtes k\u00f5rgemad n\u00f5udmised.<\/p>\n<p><b>Andmeinsenerid vs. andmeadlased<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Kes on andmeinsenerid ja kuidas neist saavad?\" src=\"\/wp-content\/uploads\/2019\/05\/f01dd66863fcc0a89ab0b763d208a863.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Olgu, see oli lihtne ja naljakas v\u00f5rdlus (pole isiklik), kuid tegelikult on k\u00f5ik palju keerulisem.<\/i><\/p>\n<p>Esiteks peate teadma, et andmeadlase ja andmeinseneri rollide ja oskuste piiritlemises on piisavalt segadust. Te v\u00f5ite kergesti olla rikutud, millised oskused on vajalikud edukale andmeinsenerile. Loomulikult on teatud oskused, mis kattuvad m\u00f5lema rolliga. Kuid samas on ka terve hulk diametraalselt vastupidiseid oskusi.<\/p>\n<p>Andmeadlus on t\u00f5sine asi, kuid liikume maailma suunas funktsionaalse andmeadluse poole, kus praktikandid suudavad teha oma anal\u00fc\u00fcsi. Andmep\u00f5histe torude ja integreeritud andmestruktuuride kasutamiseks on vaja andmeinsenersid, mitte teadlasi.<\/p>\n<p><b>Kas andmete insener on rohkem n\u00f5utud kui andmete teadlane?<\/b><\/p>\n<blockquote><p><i> \u2014 Jah, sest enne kui saad porgandikoogi k\u00fcpsetada, pead k\u00f5igepealt porgandid koguma, puhastama ja varuma!<\/i><\/p><\/blockquote>\n<p>Andmete insener m\u00f5istab programmeerimist paremini kui \u00fckski andmete teadlane, kuid statistika osas on k\u00f5ik t\u00e4pselt vastupidine.<\/p>\n<p>Aga siin on andmete inseneri eelis:<\/p>\n<p>ilma temata\/oneta protot\u00fc\u00fcbi mudeli v\u00e4\u00e4rtus, mis enamasti koosneb kohutava kvaliteediga koodifragmendist Python failis, mille on andmete teadlane kuidagi loonud ja mis v\u00e4hemalt mingit tulemust annab, kaldub nulli.<\/p>\n<p>Ilma andmete insenerita ei saa see kood kunagi projektiks ning ei \u00fchtegi \u00e4riprobleemi ei suudeta t\u00f5husalt lahendada. Andmete insener p\u00fc\u00fcab sellest k\u00f5ik toote muuta.<\/p>\n<p><b>P\u00f5hiteadmised, mida andmete insener peab teadma<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Kes on andmeinsenerid ja kuidas neist saavad?\" src=\"\/wp-content\/uploads\/2019\/05\/f724004e5a30a0c021c4377d66985d16.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKuna, kui see t\u00f6\u00f6 \u00e4ratab teises valguse ja te olete t\u00e4is entusiasm \u2014 te suudate \u00f5ppida seda, te suudate omandada k\u00f5ik vajalikud oskused ja saada t\u00f5eliseks rock-staariks andmeinsenerite alal. Ja jah, te saate seda teha isegi ilma programmeerimisoskusteta v\u00f5i muude tehniliste teadmisteta. See on keeruline, kuid v\u00f5imalik!<\/p>\n<p><b>Millised on esimesed sammud?<\/b><\/p>\n<blockquote><p><i>Te peaksite omama \u00fcldist ettekujutust, mis on mis.<\/i><\/p><\/blockquote>\n<p>Esiteks, andmeinseneriteadus kuulub infotehnoloogia valdkonda. T\u00e4psemalt \u00f6eldes \u2014 te peaksite m\u00f5istma t\u00f5husaid algoritme ja andmestruktuure. Teiseks, kuna andmeinsenerid t\u00f6\u00f6tavad andmetega, on vajalik m\u00f5ista andmebaaside t\u00f6\u00f6p\u00f5him\u00f5tteid ja nende aluseks olevaid struktuure.<\/p>\n<p>N\u00e4iteks tavalised B-puu SQL andmebaasid p\u00f5hinevad B-puu andmestruktuuril, samuti t\u00e4nap\u00e4evastes jaotatud ladustamisplatsides LSM-puul ja muudel hash-tabelite modifikatsioonidel.<\/p>\n<p><i>* Need sammud p\u00f5hinevad suurep\u00e4rasel artiklil <noindex><a rel=\"nofollow\" href=\"https:\/\/medium.com\/@adil_45422\">Adil Hashtamovilt<\/a><\/noindex>. Nii et kui te r\u00e4\u00e4gite vene keelt, toetage seda autorit ja lugege <noindex><a rel=\"nofollow\" href=\"https:\/\/khashtamov.com\/ru\/data-engineer\/\">tema postitust<\/a><\/noindex>.<\/i><\/p>\n<p><b>1. Algoritmid ja andmestruktuurid<\/b><\/p>\n<p>\u00d5ige andmestruktuuri kasutamine v\u00f5ib oluliselt parandada algoritmi j\u00f5udlust. Idee j\u00e4rgi peaksime k\u00f5ik \u00f5ppima andmestruktuure ja algoritme meie koolides, kuid see harva kunagi kajastub. Igatahes pole kunagi liiga hilja tutvuda.<br \/>\nNii et siin on minu lemmik tasuta kursused, et \u00f5ppida andmestruktuure ja algoritme:<\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.udemy.com\/introduction-to-data-structures\/?ranMID=39197&amp;ranEAID=JVFxdTr9V80&amp;ranSiteID=JVFxdTr9V80-fcVI3mDcwPC4Q5dpzDABSg&amp;LSNPUBID=JVFxdTr9V80\">Lihtsatest keerulisteni: Andmestruktuurid (Udemy)<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.coursera.org\/learn\/algorithms-part1?ranMID=40328&amp;ranEAID=JVFxdTr9V80&amp;ranSiteID=JVFxdTr9V80-Y2SOzvEQ5UCqGZ8SQ8yPFA&amp;siteID=JVFxdTr9V80-Y2SOzvEQ5UCqGZ8SQ8yPFA&amp;utm_content=10&amp;utm_medium=partners&amp;utm_source=linkshare&amp;utm_campaign=JVFxdTr9V80\">Algoritmid, Osa I (Coursera)<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.coursera.org\/learn\/algorithms-part2?ranMID=40328&amp;ranEAID=JVFxdTr9V80&amp;ranSiteID=JVFxdTr9V80-sXWvgPKRyzYEBJ33lVooUQ&amp;siteID=JVFxdTr9V80-sXWvgPKRyzYEBJ33lVooUQ&amp;utm_content=10&amp;utm_medium=partners&amp;utm_source=linkshare&amp;utm_campaign=JVFxdTr9V80\">Algoritmid, Osa II (Coursera)<\/a><\/noindex>\n <\/li>\n<\/ul>\n<p>Ja \u00e4rge unustage klassikalist teost algoritmidest Thomas Cormenilt \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/www.amazon.com\/gp\/product\/0262033844\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=0262033844&amp;linkCode=as2&amp;tag=adilkhash-20&amp;linkId=74742875db503b1a899ca35159749067\">Algoritmide sissejuhatus<\/a><\/noindex>. See on ideaalne k\u00e4siraamat, kui peate oma m\u00e4lu v\u00e4rskendama.<\/p>\n<ul>\n<li>Oma oskuste parandamiseks kasutage <noindex><a rel=\"nofollow\" href=\"https:\/\/leetcode.com\/problemset\/algorithms\/\">Leetcode<\/a><\/noindex>.<\/li>\n<\/ul>\n<p>Samuti v\u00f5ite s\u00fcveneda andmebaaside maailma Carnegie Melloni \u00dclikooli suurep\u00e4raste videote kaudu Youtube'is:<\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/playlist?list=PLSE8ODhjZXjYutVzTeAds8xUt1rcmyT7x\">Sissejuhatus andmebaasis\u00fcsteemidesse<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/playlist?list=PLSE8ODhjZXja7K1hjZ01UTVDnGQdx5v5U\">Edasij\u00f5udnud andmebaasis\u00fcsteemid<\/a><\/noindex>.<\/li>\n<\/ul>\n<p><b>2. SQL-i \u00f5ppimine<\/b><\/p>\n<p>Kogu meie elu on andmed. Ja et neid andmeid andmebaasist v\u00e4lja tuua, tuleb teil nendega \u00fchel keelel \"r\u00e4\u00e4kida\".<\/p>\n<p>SQL (Structured Query Language \u2014 struktureeritud p\u00e4ringute keel) on andmete valdkonnas suhtlemise keel. \u00dcksk\u00f5ik, mida keegi \u00fctleb, on SQL elanud, elab ja elab veel v\u00e4ga pikka aega.<\/p>\n<p>Kui olete pikka aega arendustegevuses olnud, olete t\u00f5en\u00e4oliselt m\u00e4rganud, et kuulujutud SQL-i kiirest surmast ilmuvad perioodiliselt. Keel loodi 70. aastate alguses ning on siiani v\u00e4ga populaarne anal\u00fc\u00fctikute, arendajate ja lihtsalt entusiastide seas.<br \/>\nIlma SQL-i teadmata andmete inseneri valdkonnas ei ole midagi teha, kuna peate paratamatult looma p\u00e4ringuid andmete ekstraheerimiseks. K\u00f5ik kaasaegsed suurandmete ladustamised toetavad SQL-i:<\/p>\n<ul>\n<li>Amazon Redshift<\/li>\n<li>HP Vertica<\/li>\n<li>Oracle<\/li>\n<li>SQL Server<\/li>\n<\/ul>\n<p>\u2026 ja palju teisi.<\/p>\n<p>Suurte andmekoguste anal\u00fc\u00fcsimiseks jaotatud s\u00fcsteemides, nagu HDFS, on v\u00e4lja m\u00f5eldud SQL-mehhanismid: Apache Hive, Impala jms. N\u00e4hte, see ei kavatse kuhugi minna.<\/p>\n<p>Kuidas SQL-i \u00f5ppida? Lihtsalt tee seda praktikas.<\/p>\n<p>Selle jaoks soovitaksin tutvuda suurep\u00e4rase tasuta \u00f5pikuga, mille autor on <noindex><a rel=\"nofollow\" href=\"https:\/\/mode.com\/sql-tutorial\/introduction-to-sql\/\">Mode Analytics<\/a><\/noindex>.<\/p>\n<ol>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.datacamp.com\/courses\/intermediate-sql\/\">Keskmine SQL-i tase<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.datacamp.com\/courses\/joining-data-in-postgresql\">Andmete \u00fchendamine SQL-is<\/a><\/noindex><\/li>\n<\/ol>\n<p>Nende kursuste erip\u00e4ra on interaktiivne keskkond, kus saate SQL-p\u00e4ringuid otse brauseris kirjutada ja k\u00e4itada. Ressurss <noindex><a rel=\"nofollow\" href=\"https:\/\/modern-sql.com\/\">Modern SQL<\/a><\/noindex> on kindlasti kasulik. Ja saate neid teadmisi rakendada <noindex><a rel=\"nofollow\" href=\"https:\/\/leetcode.com\/problemset\/database\/\">Leetcodi \u00fclesannetes<\/a><\/noindex> andmebaaside osas.<\/p>\n<p><b>3. Programmeerimine Pythonis ja Java\/Scala<\/b><\/p>\n<p>Miks tasub \u00f5ppida programmeerimiskeelt Python, olen juba kirjutanud artiklis <noindex><a rel=\"nofollow\" href=\"https:\/\/medium.com\/datadriveninvestor\/python-vs-r-choosing-the-best-tool-for-ai-ml-data-science-7e0c2295e243\">Python vs R. Parima t\u00f6\u00f6riista valik AI, ML ja andmeteaduse jaoks<\/a><\/noindex>. Mis puutub Java ja Scala, siis enamus suurte andmemahtude salvestamise ja t\u00f6\u00f6tlemise t\u00f6\u00f6riistu on kirjutatud just nende keeltega. N\u00e4iteks:<\/p>\n<ul>\n<li>Apache Kafka (Scala)<\/li>\n<li>Hadoop, HDFS (Java)<\/li>\n<li>Apache Spark (Scala)<\/li>\n<li>Apache Cassandra (Java)<\/li>\n<li>HBase (Java)<\/li>\n<li>Apache Hive (Java)<\/li>\n<\/ul>\n<p>Nende t\u00f6\u00f6riistade toimimise m\u00f5istmiseks peate teadma keeli, milles need on kirjutatud. Scala funktsionaalne l\u00e4henemine v\u00f5imaldab t\u00f5husalt lahendada andmete paralleelt\u00f6\u00f6tluse \u00fclesandeid. Paraku ei saa Python kiirusest ja paralleelsest t\u00f6\u00f6tlemisest r\u00e4\u00e4kida. \u00dcldiselt m\u00f5jutab mitme keele ja programmeerimisparadigma tundmine positiivselt probleemide lahendamise l\u00e4henemisviisi laiust.<\/p>\n<p>Scala keele s\u00fcvitsi minekuks v\u00f5ite lugeda <noindex><a rel=\"nofollow\" href=\"https:\/\/www.amazon.com\/gp\/product\/0981531687\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=0981531687&amp;linkCode=as2&amp;tag=adilkhash-20&amp;linkId=69314619f1e546e3921eb97c72cf4850\">Scala programmeerimine<\/a><\/noindex> keele autorilt. Samuti avaldas Twitter hea sissejuhatava juhendi \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/twitter.github.io\/scala_school\/\">Scala Kool<\/a><\/noindex>.<\/p>\n<p>Mis puutub Pythonisse, ma arvan, <noindex><a rel=\"nofollow\" href=\"https:\/\/www.amazon.com\/gp\/product\/1491946008\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=1491946008&amp;linkCode=as2&amp;tag=adilkhash-20&amp;linkId=8a663e966770c24874e323133cc7a005\">Fluent Python<\/a><\/noindex> on parim keskastme raamat.<\/p>\n<p><b>4. Suurte andmete t\u00f6\u00f6riistad<\/b><\/p>\n<p>Siin on nimekiri k\u00f5ige populaarsematest t\u00f6\u00f6riistadest suures andme maailmas:<\/p>\n<ul>\n<li>Apache Spark<\/li>\n<li>Apache Kafka<\/li>\n<li>Apache Hadoop (HDFS, HBase, Hive)<\/li>\n<li>Apache Cassandra<\/li>\n<\/ul>\n<p>Rohkem teavet suurte andmeplokkide ehitamise kohta leiate sellest h\u00e4mmastavast <noindex><a rel=\"nofollow\" href=\"http:\/\/xyz.insightdataengineering.com\/blog\/pipeline_map\/\">interaktiivsest keskkonnast<\/a><\/noindex>. K\u00f5ige populaarsemad t\u00f6\u00f6riistad on Spark ja Kafka. Neid tasub kindlasti \u00f5ppida, eelistatavalt m\u00f5ista, kuidas need seestpoolt t\u00f6\u00f6tavad. Jay Kreps (Kafka kaasautor) avaldas 2013. aastal monumentaalse teose <noindex><a rel=\"nofollow\" href=\"https:\/\/engineering.linkedin.com\/distributed-systems\/log-what-every-software-engineer-should-know-about-real-time-datas-unifying\">The Log: mida iga tarkvaraarendaja peab teadma andmete reaalajas \u00fchendamise abstraktsioonist<\/a><\/noindex>, muide, selle talmudi p\u00f5hijooni kasutati Apache Kafka loomisel.<\/p>\n<ul>\n<li>Hadoopi sissejuhatuseks v\u00f5ib olla <noindex><a rel=\"nofollow\" href=\"https:\/\/data-flair.training\/blogs\/hadoop-tutorials-home\/\">Hadoopi t\u00e4ielik juhend (tasuta)<\/a><\/noindex>.<\/li>\n<li>Minu jaoks on k\u00f5ige t\u00e4ielikum juhend Apache Sparkile \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/www.amazon.com\/gp\/product\/1491912219\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=1491912219&amp;linkCode=as2&amp;tag=adilkhash-20&amp;linkId=5b206424060cfc80a33d13b05c0bfee2\">Spark: t\u00e4ielik juhend<\/a><\/noindex>.<\/li>\n<\/ul>\n<p><b>5. Pilveteenused<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Kes on andmeinsenerid ja kuidas neist saavad?\" src=\"\/wp-content\/uploads\/2019\/05\/cc435bc9f9b91ec87d1cf7a591cf84a2.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n\u00dcksk\u00f5ik millise pilveplatvormi tundmine on andmeinseneri ametikoha taotlejatele esitatud p\u00f5hiv\u00f5imetuste loendis. T\u00f6\u00f6andjad eelistavad Amazon Web Services'i, teisel kohal on Google'i pilveplatvorm ja kolmandat kohta jagab Microsoft Azure.<\/p>\n<p>Te peaks h\u00e4sti orienteeruma Amazon EC2, AWS Lambda, Amazon S3, DynamoDB.<\/p>\n<p><b>6. Jagatud s\u00fcsteemid<\/b><\/p>\n<p>Suure andmemahu t\u00f6\u00f6tlemine eeldab s\u00f5ltumatute arvutiklasterite olemasolu, mille vaheline suhtlus toimub v\u00f5rgus. Mida suurem on klaster, seda t\u00f5en\u00e4olisem on tema liikmete rikke. Et saada andmeekspert, peate s\u00fcvenema jagatud s\u00fcsteemide probleemidesse ja olemasolevatesse lahendustesse. See valdkond on vana ja keeruline.<\/p>\n<p>Andrew Tanenbaum on selle valdkonna pioneer. Neile, kes ei karda teooriat, soovitan tema raamatut <noindex><a rel=\"nofollow\" href=\"https:\/\/www.amazon.com\/gp\/product\/1543057381\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=1543057381&amp;linkCode=as2&amp;tag=adilkhash-20&amp;linkId=721aedeb23c313bc46a92c134c5baafa\">\u201eJagatud s\u00fcsteemid\u201c<\/a><\/noindex>, alustajatele v\u00f5ib see tunduda keeruline, kuid see aitab teil t\u00f5eliselt oma oskusi lihvida.<\/p>\n<p>Olen arvamusel <noindex><a rel=\"nofollow\" href=\"https:\/\/www.amazon.com\/gp\/product\/1449373321\/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=1449373321&amp;linkCode=as2&amp;tag=adilkhash-20&amp;linkId=e7e0e096aa5761066245eb90965ac849\">et \u201eAndmekasutusega rakenduste projekteerimine\u201c autoriks on Martin Kleppmann<\/a><\/noindex> k\u00f5ige parem sissejuhatav raamat. \u00dcksk\u00f5ik, kas Martinil on suurep\u00e4rane <noindex><a rel=\"nofollow\" href=\"https:\/\/martin.kleppmann.com\/\">blogi<\/a><\/noindex>. Tema t\u00f6\u00f6 aitab struktureerida teadmisi kaasaegse andmete salvestamise ja t\u00f6\u00f6tlemise infrastruktuuri ehitamise kohta.<br \/>\nNeile, kes armastavad videoid vaadata, on YouTube'is kursus <noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=w8KFPWkK0bI&amp;list=PLawkBQ15NDEkDJ5IyLIJUTZ1rRM9YQq6N&amp;index=2&amp;t=0s\">Jaotatud arvutis\u00fcsteemid<\/a><\/noindex>.<\/p>\n<p><b>7. Andmevood<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Kes on andmeinsenerid ja kuidas neist saavad?\" src=\"\/wp-content\/uploads\/2019\/05\/cf1c8e2baad0cc79c0b8c3707741353a.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAndmevood on see, ilma milleta ei saa te andmeinsenerina elada.<\/p>\n<p>Suure osa ajast loob andmeinsener nn andmevooge, see t\u00e4hendab andmete edastamise protsessi loomist \u00fchest kohast teise. Need v\u00f5ivad olla kasutuslood, mis suunavad v\u00e4listeenuse API-le v\u00f5i teevad SQL-p\u00e4ringu, t\u00e4iendavad andmeid ja paigutavad need kesksetesse salvestuskohtadesse (andmehoidlad) v\u00f5i struktureerimata andmete ladustamisse (andmej\u00e4rved).<\/p>\n<p><b>Kokkuv\u00f5tteks: andmeinseneri peamine kontrollnimekiri<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Kes on andmeinsenerid ja kuidas neist saavad?\" src=\"\/wp-content\/uploads\/2019\/05\/de0eca662b2246e63901afb89c9f255f.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKokkuv\u00f5tteks on vajalik hea arusaam j\u00e4rgmistest:<\/p>\n<ul>\n<li>Teabe s\u00fcsteemid;<\/li>\n<li>Tarkvaraarendus (Agile, DevOps, disainitehnika, SOA);<\/li>\n<li>Jaotatud s\u00fcsteemid ja paralleelne programmeerimine;<\/li>\n<li>Andmebaaside alused \u2014 planeerimine, projekteerimine, haldamine ja vigade parandus;<\/li>\n<li>Katsedegedeerimine \u2014 A\/B-testid kontseptsioonide t\u00f5estamiseks, s\u00fcsteemide usaldusv\u00e4\u00e4rsuse ja j\u00f5udluse m\u00e4\u00e4ramiseks ning usaldusv\u00e4\u00e4rsete lahenduste kiireks k\u00e4itamiseks.<\/li>\n<\/ul>\n<p>Need on vaid m\u00f5ned n\u00f5uded, et saada andmeinseneriks, seega tutvuge andmes\u00fcsteemide, infos\u00fcsteemide, pideva tarnimise\/deployimise\/integratsiooni, programmeerimiskeelte ja teiste IT-teemadega (mitte k\u00f5ikides ainetes).<\/p>\n<p><b>Ja l\u00f5puks, viimane, kuid \u00fclimalt oluline asi, mille ma soovin \u00f6elda.<\/b><\/p>\n<blockquote><p><i><b>Teekond andmeinseneriks saamisel ei ole nii lihtne, nagu see esialgu tunduda v\u00f5ib. See ei anna andestust, frustreerib ja peate olema valmis selleks. M\u00f5ned hetked sellel teekonnal v\u00f5ivad teid k\u00f5ikidest loobuma sundida. Aga see on t\u00f5eline t\u00f6\u00f6 ja \u00f5ppimisprotsess.<\/b><\/i><\/p><\/blockquote>\n<p>\u00c4rge lihtsalt kaunistage seda algusest peale. Kogu teekonna m\u00f5te on \u00f5ppida v\u00f5imalikult palju ja olla valmis uute v\u00e4ljakutsetega silmitsi seisma.<br \/>\nSiin on suurep\u00e4rane pilt, millega ma kokku puutusin, mis illustreerib seda punkti h\u00e4sti:<\/p>\n<p><img decoding=\"async\" alt=\"Kes on andmeinsenerid ja kuidas neist saavad?\" src=\"\/wp-content\/uploads\/2019\/05\/9cc8db9011e5f54717d7e9f31f915523.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nJa, \u00e4rge unustage v\u00e4ltida l\u00e4bip\u00f5lemist ning puhkama. See on samuti v\u00e4ga oluline. Edu!<\/p>\n<p>Kuidas teile artikkel meeldis, s\u00f5brad? Olete oodatud <noindex><a rel=\"nofollow\" href=\"https:\/\/otus.pw\/RCuj\/\">tasuta veebinarile<\/a><\/noindex>, mis toimub juba t\u00e4na kell 20.00. Veebiseminari raames arutame, kuidas luua t\u00f5hus ja skaleeritav andmete t\u00f6\u00f6tlemise s\u00fcsteem v\u00e4ikesele ettev\u00f5ttele v\u00f5i idufirmale minimaalsete kuludega. Praktika osana tutvume Google Cloudi andmete t\u00f6\u00f6tlemise t\u00f6\u00f6riistadega. N\u00e4gemiseni!<br \/>\n<br \/>Allikas: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/452670\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0418 \u0441\u043d\u043e\u0432\u0430 \u0437\u0434\u0440\u0430\u0432\u0441\u0442\u0432\u0443\u0439\u0442\u0435! \u0417\u0430\u0433\u043e\u043b\u043e\u0432\u043e\u043a \u0441\u0442\u0430\u0442\u044c\u0438 \u0433\u043e\u0432\u043e\u0440\u0438\u0442 \u0441\u0430\u043c \u043e \u0441\u0435\u0431\u0435. \u0412 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c \u0440\u0430\u0437\u043e\u0431\u0440\u0430\u0442\u044c\u0441\u044f \u0432 \u0442\u043e\u043c, \u043a\u0442\u043e \u0436\u0435 \u0442\u0430\u043a\u0438\u0435 \u0434\u0430\u0442\u0430-\u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b. \u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u043e\u0447\u0435\u043d\u044c \u043c\u043d\u043e\u0433\u043e \u043f\u043e\u043b\u0435\u0437\u043d\u044b\u0445 \u0441\u0441\u044b\u043b\u043e\u043a. \u041f\u0440\u0438\u044f\u0442\u043d\u043e\u0433\u043e \u043f\u0440\u043e\u0447\u0442\u0435\u043d\u0438\u044f. \u041f\u0440\u043e\u0441\u0442\u043e\u0435 \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0441\u0442\u0432\u043e \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u043f\u043e\u0439\u043c\u0430\u0442\u044c \u0432\u043e\u043b\u043d\u0443 Data Engineering \u0438 \u043d\u0435 \u0434\u0430\u0442\u044c \u0435\u0439 \u0437\u0430\u0442\u044f\u043d\u0443\u0442\u044c \u0432\u0430\u0441 \u0432 \u043f\u0443\u0447\u0438\u043d\u0443. \u0421\u043a\u043b\u0430\u0434\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u0432\u043f\u0435\u0447\u0430\u0442\u043b\u0435\u043d\u0438\u0435, \u0447\u0442\u043e \u0432 \u043d\u0430\u0448\u0438 \u0434\u043d\u0438 \u043a\u0430\u0436\u0434\u044b\u0439 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":25729,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-34083","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 4.9.10 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0418 \u0441\u043d\u043e\u0432\u0430 \u0437\u0434\u0440\u0430\u0432\u0441\u0442\u0432\u0443\u0439\u0442\u0435! \u0417\u0430\u0433\u043e\u043b\u043e\u0432\u043e\u043a \u0441\u0442\u0430\u0442\u044c\u0438 \u0433\u043e\u0432\u043e\u0440\u0438\u0442 \u0441\u0430\u043c \u043e \u0441\u0435\u0431\u0435. \u0412 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c \u0440\u0430\u0437\u043e\u0431\u0440\u0430\u0442\u044c\u0441\u044f \u0432 \u0442\u043e\u043c, \u043a\u0442\u043e \u0436\u0435 \u0442\u0430\u043a\u0438\u0435 \u0434\u0430\u0442\u0430-\u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b. \u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u043e\u0447\u0435\u043d\u044c \u043c\u043d\u043e\u0433\u043e \u043f\u043e\u043b\u0435\u0437\u043d\u044b\u0445 \u0441\u0441\u044b\u043b\u043e\u043a. \u041f\u0440\u0438\u044f\u0442\u043d\u043e\u0433\u043e \u043f\u0440\u043e\u0447\u0442\u0435\u043d\u0438\u044f. \u041f\u0440\u043e\u0441\u0442\u043e\u0435 \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0441\u0442\u0432\u043e \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u043f\u043e\u0439\u043c\u0430\u0442\u044c \u0432\u043e\u043b\u043d\u0443 Data Engineering \u0438 \u043d\u0435 \u0434\u0430\u0442\u044c \u0435\u0439 \u0437\u0430\u0442\u044f\u043d\u0443\u0442\u044c \u0432\u0430\u0441 \u0432 \u043f\u0443\u0447\u0438\u043d\u0443. \u0421\u043a\u043b\u0430\u0434\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u0432\u043f\u0435\u0447\u0430\u0442\u043b\u0435\u043d\u0438\u0435, \u0447\u0442\u043e \u0432 \u043d\u0430\u0448\u0438 \u0434\u043d\u0438 \u043a\u0430\u0436\u0434\u044b\u0439\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/kto-takie-data-inzhenery-i-kak-imi-stanovyatsya\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 4.9.10\" \/>\n\t\t<meta property=\"og:locale\" content=\"et_EE\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041a\u0442\u043e \u0442\u0430\u043a\u0438\u0435 \u0434\u0430\u0442\u0430-\u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b, \u0438 \u043a\u0430\u043a \u0438\u043c\u0438 \u0441\u0442\u0430\u043d\u043e\u0432\u044f\u0442\u0441\u044f? | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0418 \u0441\u043d\u043e\u0432\u0430 \u0437\u0434\u0440\u0430\u0432\u0441\u0442\u0432\u0443\u0439\u0442\u0435! \u0417\u0430\u0433\u043e\u043b\u043e\u0432\u043e\u043a \u0441\u0442\u0430\u0442\u044c\u0438 \u0433\u043e\u0432\u043e\u0440\u0438\u0442 \u0441\u0430\u043c \u043e \u0441\u0435\u0431\u0435. \u0412 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c \u0440\u0430\u0437\u043e\u0431\u0440\u0430\u0442\u044c\u0441\u044f \u0432 \u0442\u043e\u043c, \u043a\u0442\u043e \u0436\u0435 \u0442\u0430\u043a\u0438\u0435 \u0434\u0430\u0442\u0430-\u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b. \u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u043e\u0447\u0435\u043d\u044c \u043c\u043d\u043e\u0433\u043e \u043f\u043e\u043b\u0435\u0437\u043d\u044b\u0445 \u0441\u0441\u044b\u043b\u043e\u043a. \u041f\u0440\u0438\u044f\u0442\u043d\u043e\u0433\u043e \u043f\u0440\u043e\u0447\u0442\u0435\u043d\u0438\u044f. \u041f\u0440\u043e\u0441\u0442\u043e\u0435 \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0441\u0442\u0432\u043e \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u043f\u043e\u0439\u043c\u0430\u0442\u044c \u0432\u043e\u043b\u043d\u0443 Data Engineering \u0438 \u043d\u0435 \u0434\u0430\u0442\u044c \u0435\u0439 \u0437\u0430\u0442\u044f\u043d\u0443\u0442\u044c \u0432\u0430\u0441 \u0432 \u043f\u0443\u0447\u0438\u043d\u0443. \u0421\u043a\u043b\u0430\u0434\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u0432\u043f\u0435\u0447\u0430\u0442\u043b\u0435\u043d\u0438\u0435, \u0447\u0442\u043e \u0432 \u043d\u0430\u0448\u0438 \u0434\u043d\u0438 \u043a\u0430\u0436\u0434\u044b\u0439\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/kto-takie-data-inzhenery-i-kak-imi-stanovyatsya\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T18:56:19+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T18:56:19+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Kes on andmeinsenerid ja kuidas neist saadakse? | ProHoster","description":"Tere j\u00e4lle! Artikli pealkiri r\u00e4\u00e4gib enda eest. Kursuse \"Data Engineer\" alguse eel pakume v\u00e4lja, et uurime, kes on andmeinsenerid. Artiklis on palju kasulikke linke. Head lugemist. Lihtne juhend, kuidas tabada Data Engineering'i lainet ja mitte lasta sellel end s\u00fcgavikku kaasa viia. Tundub, et t\u00e4nap\u00e4eval on iga\u00fchel","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/kto-takie-data-inzhenery-i-kak-imi-stanovyatsya","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041a\u0442\u043e \u0442\u0430\u043a\u0438\u0435 \u0434\u0430\u0442\u0430-\u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b, \u0438 \u043a\u0430\u043a \u0438\u043c\u0438 \u0441\u0442\u0430\u043d\u043e\u0432\u044f\u0442\u0441\u044f? | ProHoster","og:description":"\u0418 \u0441\u043d\u043e\u0432\u0430 \u0437\u0434\u0440\u0430\u0432\u0441\u0442\u0432\u0443\u0439\u0442\u0435! \u0417\u0430\u0433\u043e\u043b\u043e\u0432\u043e\u043a \u0441\u0442\u0430\u0442\u044c\u0438 \u0433\u043e\u0432\u043e\u0440\u0438\u0442 \u0441\u0430\u043c \u043e \u0441\u0435\u0431\u0435. \u0412 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u043c \u0440\u0430\u0437\u043e\u0431\u0440\u0430\u0442\u044c\u0441\u044f \u0432 \u0442\u043e\u043c, \u043a\u0442\u043e \u0436\u0435 \u0442\u0430\u043a\u0438\u0435 \u0434\u0430\u0442\u0430-\u0438\u043d\u0436\u0435\u043d\u0435\u0440\u044b. \u0412 \u0441\u0442\u0430\u0442\u044c\u0435 \u043e\u0447\u0435\u043d\u044c \u043c\u043d\u043e\u0433\u043e \u043f\u043e\u043b\u0435\u0437\u043d\u044b\u0445 \u0441\u0441\u044b\u043b\u043e\u043a. \u041f\u0440\u0438\u044f\u0442\u043d\u043e\u0433\u043e \u043f\u0440\u043e\u0447\u0442\u0435\u043d\u0438\u044f. \u041f\u0440\u043e\u0441\u0442\u043e\u0435 \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0441\u0442\u0432\u043e \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u043f\u043e\u0439\u043c\u0430\u0442\u044c \u0432\u043e\u043b\u043d\u0443 Data Engineering \u0438 \u043d\u0435 \u0434\u0430\u0442\u044c \u0435\u0439 \u0437\u0430\u0442\u044f\u043d\u0443\u0442\u044c \u0432\u0430\u0441 \u0432 \u043f\u0443\u0447\u0438\u043d\u0443. \u0421\u043a\u043b\u0430\u0434\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u0432\u043f\u0435\u0447\u0430\u0442\u043b\u0435\u043d\u0438\u0435, \u0447\u0442\u043e \u0432 \u043d\u0430\u0448\u0438 \u0434\u043d\u0438 \u043a\u0430\u0436\u0434\u044b\u0439","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/kto-takie-data-inzhenery-i-kak-imi-stanovyatsya","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:56:19+00:00","article:modified_time":"2019-10-31T18:56:19+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"34083","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 17:49:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 02:28:35","updated":"2026-01-21 17:49:20"},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/34083","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=34083"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/34083\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media\/25729"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=34083"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=34083"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=34083"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}