{"id":39275,"date":"2019-10-31T22:29:10","date_gmt":"2019-10-31T19:29:10","guid":{"rendered":"https:\/\/prohoster.info\/blog\/plan-prokachki-dlya-polucheniya-professii-data-engineer\/"},"modified":"2019-10-31T22:29:10","modified_gmt":"2019-10-31T19:29:10","slug":"plan-prokachki-dlya-polucheniya-professii-data-engineer","status":"publish","type":"post","link":"https:\/\/prohoster.info\/no\/blog\/administrirovanie\/plan-prokachki-dlya-polucheniya-professii-data-engineer","title":{"rendered":"Plan for leveling up to become a Data Engineer","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>For the past eight years, I have been working as a project manager (not writing code at work), which has naturally negatively affected my tech background. I decided to bridge my technological gap and pursue the profession of Data Engineer. The main skill of a Data Engineer is the ability to design, build, and maintain data warehouses.<\/p>\n<p>I created a study plan that I believe will be useful not only for me. The plan is aimed at self-study of courses. Priority is given to free courses in Russian.<\/p>\n<p>Sections: <\/p>\n<ul>\n<li><b>Algorithms and Data Structures. <\/b>A key section. Master this and everything else will follow. It's important to get hands-on experience writing code and using basic data structures and algorithms.<\/li>\n<li><b>Databases and Data Warehouses, Business Intelligence.<\/b> From algorithms, we transition to data storage and processing.<\/li>\n<li><b>Hadoop and Big Data. <\/b>When the database doesn\u2019t fit on a hard drive, or when data needs to be analyzed but Excel can\u2019t load it, big data begins. In my opinion, you should only move to this section after thoroughly studying the previous two.<\/li>\n<\/ul>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Algorithms and Data Structures<\/h2>\n<p>\nI included studying Python, reviewing the fundamentals of mathematics and algorithmization in my plan.<\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/67\">Programming in Python<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/512\">Python: basics and application<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/2461\">Linear Algebra<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/91\/\">Basics of Discrete Mathematics <\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/217\/\">Algorithms: Theory and Practice. Methods<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/1547\">Algorithms: Theory and Practice. Data Structures<\/a><\/noindex><\/li>\n<\/ul>\n<p><\/p>\n<h2>Databases and Data Warehouses, Business Intelligence<\/h2>\n<p><\/p>\n<ul>\n<li>Book: Martin Kleppmann \u2014 Designing Data-Intensive Applications. Programming, Scaling, Supporting. This book describes how different data models work, their internal implementation, limitations, and choices based on the task. <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/551\">Introduction to Databases <\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/3203\">Diving into DBMS<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/2586\">Introduction to NoSQL Databases <\/a><\/noindex><\/li>\n<\/ul>\n<p>\nTopics related to building data warehouses, ETL, and OLAP cubes strongly depend on the tools, so I do not provide links to courses in this document. It makes sense to study such systems while working on a specific project in a specific company. For an introduction to ETL, you can try <noindex><a rel=\"nofollow\" href=\"https:\/\/www.talend.com\/\">Talend<\/a><\/noindex> eller <noindex><a rel=\"nofollow\" href=\"https:\/\/airflow.apache.org\/\">Airflow<\/a><\/noindex>.<\/p>\n<p>In my opinion, it is important to study the modern methodology of data warehouse design, Data Vault. <noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/348188\/\">link 1<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"http:\/\/www.dwh-club.com\/ru\/dwh-bi-articles\/vse-o-data-vault.html\">link 2<\/a><\/noindex>. Den beste m\u00e5ten \u00e5 l\u00e6re det p\u00e5 er \u00e5 ta det i bruk gjennom et enkelt eksempel. GitHub har flere eksempler p\u00e5 implementering av Data Vault. <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/semashkinvg\/DataVault\">lenke<\/a><\/noindex>. Moderne bok om datalager: Modeling the Agile Data Warehouse with Data Vault av Hans Hultgren.<\/p>\n<p>For \u00e5 bli kjent med Business Intelligence-verkt\u00f8yene for sluttbrukere kan man bruke den gratis rapportbyggeren, dashbordet, og mini datalageret Power BI Desktop. L\u00e6ringsmateriell: <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.microsoft.com\/ru-ru\/learn\/paths\/create-use-analytics-reports-power-bi\/\">link 1<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.microsoft.com\/ru-ru\/learn\/paths\/consume-data-with-power-bi\/\">link 2<\/a><\/noindex>.<\/p>\n<h2>Hadoop og Big Data<\/h2>\n<p><\/p>\n<ul>\n<li>Man m\u00e5 begynne med \u00e5 implementere MapReduce selv uten tredjepartsbiblioteker. Dette vil gj\u00f8re det lettere \u00e5 forst\u00e5 flertr\u00e5det implementering senere. Et utmerket eksempel p\u00e5 Python er beskrevet <noindex><a rel=\"nofollow\" href=\"http:\/\/michaelnielsen.org\/blog\/write-your-first-mapreduce-program-in-20-minutes\/\">here<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/150\">Hadoop. Et system for behandling av store datamengder.<\/a><\/noindex> <\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/openedu.ru\/course\/misis\/ABD\/\">Introduksjon til big data engineering <\/a><\/noindex><\/li>\n<\/ul>\n<p><\/p>\n<h2>Konklusjon<\/h2>\n<p>\nIkke alt man l\u00e6rer kan brukes i arbeidet. Derfor er det n\u00f8dvendig med et diplomprosjekt der man f\u00e5r pr\u00f8ve ut nye kunnskaper. <\/p>\n<p>Planen inneholder ikke emner relatert til dataanalyse og maskinl\u00e6ring, da dette mer h\u00f8rer til yrket Data Scientist. Det er heller ikke emner knyttet til skyene AWS, Azure, da disse temaene er sterkt avhengige av plattformvalget.<\/p>\n<p>Sp\u00f8rsm\u00e5l til fellesskapet:<br \/>\nHvor rimelig er min oppgraderingsplan? Hva b\u00f8r fjernes eller legges til?<br \/>\nHvilket prosjekt vil du anbefale som et diplomarbeid?<br \/>\n<br \/>Kilde: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/472622\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u0435 \u0432\u043e\u0441\u0435\u043c\u044c \u043b\u0435\u0442 \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u044e \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u0435\u043c \u043f\u0440\u043e\u0435\u043a\u0442\u043e\u0432 (\u043d\u0435 \u043f\u0438\u0448\u0443 \u043a\u043e\u0434 \u043d\u0430 \u0440\u0430\u0431\u043e\u0442\u0435), \u0447\u0442\u043e \u0435\u0441\u0442\u0435\u0441\u0442\u0432\u0435\u043d\u043d\u043e \u043d\u0435\u0433\u0430\u0442\u0438\u0432\u043d\u043e \u0432\u043b\u0438\u044f\u0435\u0442 \u043d\u0430 \u043c\u043e\u0439 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u0431\u0435\u043a\u044d\u043d\u0434. \u042f \u0440\u0435\u0448\u0438\u043b \u0441\u043e\u043a\u0440\u0430\u0442\u0438\u0442\u044c \u0441\u0432\u043e\u0451 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u043e\u0435 \u043e\u0442\u0441\u0442\u0430\u0432\u0430\u043d\u0438\u0435 \u0438 \u043f\u043e\u043b\u0443\u0447\u0438\u0442\u044c \u043f\u0440\u043e\u0444\u0435\u0441\u0441\u0438\u044e Data engineer. \u041e\u0441\u043d\u043e\u0432\u043d\u043e\u0439 \u043d\u0430\u0432\u044b\u043a Data engineer \u2014 \u0441\u043f\u043e\u0441\u043e\u0431\u043d\u043e\u0441\u0442\u044c \u0440\u0430\u0437\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c, \u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438 \u043f\u043e\u0434\u0434\u0435\u0440\u0436\u0438\u0432\u0430\u0442\u044c \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0430 \u0434\u0430\u043d\u043d\u044b\u0445. \u0421\u043e\u0441\u0442\u0430\u0432\u0438\u043b \u043f\u043b\u0430\u043d \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f, \u0434\u0443\u043c\u0430\u044e \u043e\u043d \u0431\u0443\u0434\u0435\u0442 \u043f\u043e\u043b\u0435\u0437\u0435\u043d \u043d\u0435 \u0442\u043e\u043b\u044c\u043a\u043e \u0434\u043b\u044f \u043c\u0435\u043d\u044f. \u041f\u043b\u0430\u043d [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-39275","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.5 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u0435 \u0432\u043e\u0441\u0435\u043c\u044c \u043b\u0435\u0442 \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u044e \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u0435\u043c \u043f\u0440\u043e\u0435\u043a\u0442\u043e\u0432 (\u043d\u0435 \u043f\u0438\u0448\u0443 \u043a\u043e\u0434 \u043d\u0430 \u0440\u0430\u0431\u043e\u0442\u0435), \u0447\u0442\u043e \u0435\u0441\u0442\u0435\u0441\u0442\u0432\u0435\u043d\u043d\u043e \u043d\u0435\u0433\u0430\u0442\u0438\u0432\u043d\u043e \u0432\u043b\u0438\u044f\u0435\u0442 \u043d\u0430 \u043c\u043e\u0439 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u0431\u0435\u043a\u044d\u043d\u0434.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/no\/blog\/administrirovanie\/plan-prokachki-dlya-polucheniya-professii-data-engineer\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.5\" \/>\n\t\t<meta property=\"og:locale\" content=\"nn_NO\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041f\u043b\u0430\u043d \u043f\u0440\u043e\u043a\u0430\u0447\u043a\u0438 \u0434\u043b\u044f \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u0438\u044f \u043f\u0440\u043e\u0444\u0435\u0441\u0441\u0438\u0438 Data engineer | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u0435 \u0432\u043e\u0441\u0435\u043c\u044c \u043b\u0435\u0442 \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u044e \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u0435\u043c \u043f\u0440\u043e\u0435\u043a\u0442\u043e\u0432 (\u043d\u0435 \u043f\u0438\u0448\u0443 \u043a\u043e\u0434 \u043d\u0430 \u0440\u0430\u0431\u043e\u0442\u0435), \u0447\u0442\u043e \u0435\u0441\u0442\u0435\u0441\u0442\u0432\u0435\u043d\u043d\u043e \u043d\u0435\u0433\u0430\u0442\u0438\u0432\u043d\u043e \u0432\u043b\u0438\u044f\u0435\u0442 \u043d\u0430 \u043c\u043e\u0439 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u0431\u0435\u043a\u044d\u043d\u0434.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/no\/blog\/administrirovanie\/plan-prokachki-dlya-polucheniya-professii-data-engineer\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T19:29:10+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T19:29:10+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Oppgraderingsplan for \u00e5 oppn\u00e5 yrket Data engineer | ProHoster","description":"De siste \u00e5tte \u00e5rene har jeg jobbet som prosjektleder (jeg skriver ikke kode p\u00e5 jobb), noe som naturligvis p\u00e5virker min teknologiske bakgrunn negativt.","canonical_url":"https:\/\/prohoster.info\/no\/blog\/administrirovanie\/plan-prokachki-dlya-polucheniya-professii-data-engineer","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"nn_NO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u043b\u0430\u043d \u043f\u0440\u043e\u043a\u0430\u0447\u043a\u0438 \u0434\u043b\u044f \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u0438\u044f \u043f\u0440\u043e\u0444\u0435\u0441\u0441\u0438\u0438 Data engineer | ProHoster","og:description":"\u041f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u0435 \u0432\u043e\u0441\u0435\u043c\u044c \u043b\u0435\u0442 \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u044e \u0440\u0443\u043a\u043e\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u0435\u043c \u043f\u0440\u043e\u0435\u043a\u0442\u043e\u0432 (\u043d\u0435 \u043f\u0438\u0448\u0443 \u043a\u043e\u0434 \u043d\u0430 \u0440\u0430\u0431\u043e\u0442\u0435), \u0447\u0442\u043e \u0435\u0441\u0442\u0435\u0441\u0442\u0432\u0435\u043d\u043d\u043e \u043d\u0435\u0433\u0430\u0442\u0438\u0432\u043d\u043e \u0432\u043b\u0438\u044f\u0435\u0442 \u043d\u0430 \u043c\u043e\u0439 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u0431\u0435\u043a\u044d\u043d\u0434.","og:url":"https:\/\/prohoster.info\/no\/blog\/administrirovanie\/plan-prokachki-dlya-polucheniya-professii-data-engineer","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:29:10+00:00","article:modified_time":"2019-10-31T19:29:10+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"39275","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 01:32:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 23:11:43","updated":"2026-01-24 01:32:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/posts\/39275","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/comments?post=39275"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/posts\/39275\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/media?parent=39275"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/categories?post=39275"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/no\/wp-json\/wp\/v2\/tags?post=39275"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}