{"id":33123,"date":"2019-10-31T21:50:52","date_gmt":"2019-10-31T18:50:52","guid":{"rendered":"https:\/\/prohoster.info\/blog\/terraformer-infrastructure-to-code\/"},"modified":"2019-10-31T21:50:52","modified_gmt":"2019-10-31T18:50:52","slug":"terraformer-infrastructure-to-code","status":"publish","type":"post","link":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/terraformer-infrastructure-to-code","title":{"rendered":"Terraformer \u2014 Infrastruktuur Koodina","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Terraformer
\nTahaksin r\u00e4\u00e4kida uuest CLI t\u00f6\u00f6riistast, mille ma kirjutasin, et lahendada \u00fcks vana probleem. <\/p>\n

Probleem<\/h3>\n

\nTerraform on juba pikka aega olnud standard DevOps \/ Cloud \/ IT kogukonnas. See on v\u00e4ga mugav ja kasulik infrastruktuuri koodina haldamiseks. Terraformis on palju eeliseid, aga ka s\u00f5lmi, teravaid noazi ja augusid.
\nTerraformiga on v\u00e4ga mugav luua uusi asju ja seej\u00e4rel neid hallata, muuta v\u00f5i kustutada. Aga mida teha nendele, kellel on suur infrastruktuur pilves, mis ei ole loodud Terraformiga? Kogu pilve \u00fclekirjutamine ja uuesti loomine on kuidagi kallis ja ohtlik.
\nMa olen sellise probleemiga kokku puutunud kahel t\u00f6\u00f6kohal, k\u00f5ige lihtsam n\u00e4ide on see, kui tahad, et k\u00f5ik oleks versioonihaldes Terraformi failide kujul, aga sul on 250+ baketit ja kirjutada need k\u00e4sitsi Terraformiks on kuidagi liiga palju.
\nJah probleem<\/a><\/noindex> juba alates 2014. aastast Terraformis, mis suleti 2016. aastal lootuses, et tuleb import.<\/p>\n

\u00dches\u00f5naga, k\u00f5ik on nagu pildil, ainult paremale vasakule
\n<\/a><\/noindex><\/p>\n

Hoiatus: Autor elab pool elu v\u00e4ljaspool Venemaad ja kirjutab vene keeles harva. Olge ettevaatlikud \u00f5igekirja vigade osas.<\/p>\n

Lahendused<\/h3>\n

\n1. On olemas valmis ja vana lahendus AWS jaoks terraforming<\/a><\/noindex>. Kui proovisin selle abil saada minu 250+ baketit, sain aru, et seal on k\u00f5ik v\u00e4ga halvasti. AWS on juba ammu lisanud palju uusi valikuid, mille kohta terraforming ei tea ning see kasutab ruby keelt. mallid n\u00e4ivad vaesed<\/a><\/noindex>. P\u00e4rast kahte \u00f5htut saatsin Pull request<\/a><\/noindex> , et lisada sinna rohkem v\u00f5imalusi ja sain aru, et selline lahendus ei sobi \u00fcldse.
\nKuidas terraforming t\u00f6\u00f6tab: see v\u00f5tab AWS SDK-st andmed ja genereerib tf ja tfstate mallide kaudu.
\nSiin on kolm probleemi:
\n1. Seal on alati viivitus uuendustes
\n2. tf failid on m\u00f5nikord rikutud
\n3. tfstate kogutakse eraldi tf-st ja ei \u00fchti alati
\n\u00dcldiselt on keeruline saada tulemust, kus `terraform plan` \u00fctleb, et muudatust pole.<\/p>\n

2. `terraform import` \u2014 sisseehitatud k\u00e4sk Terraformis. Kuidas see t\u00f6\u00f6tab?
\nSa kirjutad t\u00fchja TF faili ressursi nime ja kuvandi jaoks, siis k\u00e4ivitad `terraform import` ja edastad ressursi ID. Terraform p\u00f6\u00f6rdub teenusepakkuja poole, saab andmed ja loob tfstate faili.
\nSiin on kolm probleemi:
\n1. Saame ainult tfstate faili, aga tf on t\u00fchi, selle peab k\u00e4sitsi kirjutama v\u00f5i konverteerima tfstate-st
\n2. Suudab t\u00f6\u00f6tada ainult \u00fche ressursiga korraga ega toeta k\u00f5iki ressursse. Ja mida ma pean j\u00e4lle tegema 250+ baketiga?
\n3. Pead teadma ressursi ID-d \u2014 see t\u00e4hendab, et tuleb selle \u00fcmber kirjutada kood, mis hangib ressurside nimekirja
\n\u00dcldiselt on tulemus osaline ja ei sobi h\u00e4sti skaleerimiseks.<\/p>\n

Minu lahendus
\n<\/h3>\n

N\u00f5uded:
\n1. V\u00f5ime luua tf ja tfstate faile ressurssidest. N\u00e4iteks alla laadida k\u00f5ik bucketid\/security group\/koormuse tasakaalustajad ja et `terraform plan` n\u00e4itaks, et muudatusi pole.
\n2. V\u00e4hemalt 2 pilve GCP + AWS.
\n3. Globaalne lahendus, mida on lihtne igal korral uuendada, mitte iga ressursi peale 3 p\u00e4eva raisata.
\n4. Teha avatud l\u00e4htekoodiga \u2013 probleem, millega k\u00f5ik kokku puutuvad.<\/p>\n

Go keel \u2013 sellep\u00e4rast mulle see meeldib, ja sellel on HCL failide loomise teek, mida kasutatakse terraformis + palju koodi terraformis, mis v\u00f5ib olla kasulik.<\/p>\n

Tee<\/h3>\n

\nEsimene katse
\nAlustasin lihtsa variandiga. \u00dchendused pilve kaudu SDK abil vajaliku ressursi hankimiseks ja selle konverteerimine terraformi v\u00e4ljadeks. Katse suri kohe security group'iga, kuna mulle ei meeldinud 1.5 p\u00e4eva kulutada ainult security group'i konverteerimiseks (ressursse on palju). Aeglane ja siis v\u00f5ivad v\u00e4ljad muutuda\/atteraakon \u00f5lmida.<\/p>\n

Teine katse
\nP\u00f5hineb kirjeldatud ideel. siit<\/a><\/noindex>. Lihtsalt v\u00f5tta ja konverteerida tfstate tf-ks. K\u00f5ik andmed on seal olemas ja v\u00e4ljad on samad. Kuidas saada t\u00e4ielik tfstate paljude ressursside jaoks?? Siin tuli appi k\u00e4sk `terraform refresh`. Terraform v\u00f5tab k\u00f5ik ressursid tfstate's ja toob ID j\u00e4rgi need andmed ning kirjutab k\u00f5ik tfstate'sse. Seega tuleb luua t\u00fchi tfstate, millel on ainult nimed ja ID, k\u00e4ivitada `terraform refresh`, siis saame t\u00e4ielikud tfstate'id. Hurraa!
\nN\u00fc\u00fcd hakkame tegelema rekursiivse konverteri kirjutamisega tfstate'ist tf-ks. Neile, kes pole kunagi tfstate'i lugenud, on see JSON, aga erilisel kujul.
\nSiin on selle oluline osa attributes<\/p>\n

 \"attributes\": {\n                            \"id\": \"default\\\/backend-logging-load-deployment\",\n                            \"metadata.#\": \"1\",\n                            \"metadata.0.annotations.%\": \"0\",\n                            \"metadata.0.generate_name\": \"\",\n                            \"metadata.0.generation\": \"24\",\n                            \"metadata.0.labels.%\": \"1\",\n                            \"metadata.0.labels.app\": \"backend-logging\",\n                            \"metadata.0.name\": \"backend-logging-load-deployment\",\n                            \"metadata.0.namespace\": \"default\",\n                            \"metadata.0.resource_version\": \"109317427\",\n                            \"metadata.0.self_link\": \"\\\/apis\\\/apps\\\/v1\\\/namespaces\\\/default\\\/deployments\\\/backend-logging-load-deployment\",\n                            \"metadata.0.uid\": \"300ecda1-4138-11e9-9d5d-42010a8400b5\",\n                            \"spec.#\": \"1\",\n                            \"spec.0.min_ready_seconds\": \"0\",\n                            \"spec.0.paused\": \"false\",\n                            \"spec.0.progress_deadline_seconds\": \"600\",\n                            \"spec.0.replicas\": \"1\",\n                            \"spec.0.revision_history_limit\": \"10\",\n                            \"spec.0.selector.#\": \"1\",\n<\/code><\/pre>\n
\nSiit leiate:
\n1. id \u2014 string
\n2. metadata \u2014 massiiv suurusega 1 ja selles on objekt, mille v\u00e4ljad on allpool kirjeldatud
\n3. spec \u2014 hash, mille suurus on 1 ja sellel on key,value
\n\u00dches\u00f5naga tobe formaat, k\u00f5ik v\u00f5ib minna s\u00fcgavamale ka mitme tasandi v\u00f5rra.<\/p>\n
                   \"spec.#\": \"1\",\n                            \"spec.0.min_ready_seconds\": \"0\",\n                            \"spec.0.paused\": \"false\",\n                            \"spec.0.progress_deadline_seconds\": \"600\",\n                            \"spec.0.replicas\": \"1\",\n                            \"spec.0.revision_history_limit\": \"10\",\n                            \"spec.0.selector.#\": \"1\",\n                            \"spec.0.selector.0.match_expressions.#\": \"0\",\n                            \"spec.0.selector.0.match_labels.%\": \"1\",\n                            \"spec.0.selector.0.match_labels.app\": \"backend-logging-load\",\n                            \"spec.0.strategy.#\": \"0\",\n                            \"spec.0.template.#\": \"1\",\n                            \"spec.0.template.0.metadata.#\": \"1\",\n                            \"spec.0.template.0.metadata.0.annotations.%\": \"0\",\n                            \"spec.0.template.0.metadata.0.generate_name\": \"\",\n                            \"spec.0.template.0.metadata.0.generation\": \"0\",\n                            \"spec.0.template.0.metadata.0.labels.%\": \"1\",\n                            \"spec.0.template.0.metadata.0.labels.app\": \"backend-logging-load\",\n                            \"spec.0.template.0.metadata.0.name\": \"\",\n                            \"spec.0.template.0.metadata.0.namespace\": \"\",\n                            \"spec.0.template.0.metadata.0.resource_version\": \"\",\n                            \"spec.0.template.0.metadata.0.self_link\": \"\",\n                            \"spec.0.template.0.metadata.0.uid\": \"\",\n                            \"spec.0.template.0.spec.#\": \"1\",\n                            \"spec.0.template.0.spec.0.active_deadline_seconds\": \"0\",\n                            \"spec.0.template.0.spec.0.container.#\": \"1\",\n                            \"spec.0.template.0.spec.0.container.0.args.#\": \"3\",<\/code><\/pre>\n
\n\u00dcldiselt, kes tahab programmeerimise \u00fclesannet vestlusintervjuuks, siis paluge lihtsalt kirjutada parser selle jaoks \ud83d\ude42
\nP\u00e4rast pikki katseid kirjutada parserit ilma vigadeta leidsin osa sellest terraformi koodist, mis on k\u00f5ige olulisem osa. Ja k\u00f5ik t\u00f6\u00f6tab nagu peab.<\/p>\n
Katse kolm
\nTerraform provider on binaarfailid, milles on kood k\u00f5ikide ressursside ja loogikaga, et t\u00f6\u00f6tada pilvete API-dega. Igal pilvel on oma provider ja terraform kutsub neid v\u00e4lja l\u00e4bi oma RPC protokolli kahe protsessi vahel.
\nN\u00fc\u00fcd otsustasin p\u00f6\u00f6rduda otse terraformi providerite poole RPC kutsete kaudu. See toimis kenasti ja andis v\u00f5imaluse asendada terraformi providerid uuemate vastu, saamata uut funktsionaalsust koodi muutmata. Selgus, et mitte k\u00f5ik v\u00e4ljad tfstate-s ei pea olema tf-s, aga kuidas seda teada saada? Ainult k\u00fcsides providerilt selle kohta. Siis algas veel \u00fcks rekursiivne h\u00f5beke regulaarsete v\u00e4ljendite koostamiseks, otsides v\u00e4lju tfstate seest igal tasemel s\u00fcgavusse.<\/p>\n
L\u00f5puks sai kasulik CLI t\u00f6\u00f6riist, millel on \u00fchine infrastruktuur k\u00f5igi terraformi providerite jaoks ja uus on lihtne lisada. Samuti v\u00f5tab ressursside lisamine v\u00e4he koodi. Pluss igasuguseid mugavusi, nagu ressursside vaheline seos. Loomulikult oli palju erinevaid probleeme, mida k\u00f5iki ei saa kirja panna.
\nNimetasin looma Terrafomeriks. <\/p>\n
Finaal<\/h3>\n
\nKasutasime Terrafomeri abil 500-700 tuhat rida tf + tfstate koodi kahe pilvega. Suutsime v\u00f5tta p\u00e4randlahendusi ja hakata neist k\u00e4itama ainult terraformiga, nagu parimates ideedes infrastructure as code. Lihtsalt maagia, kui v\u00f5tad suure pilve ja saad selle k\u00e4suga terraformifailidesse. Edasi grep\/replace\/git ja nii edasi.<\/p>\n
Kalkuleerisin ja korrastasin, sain load. Avaldasin GitHubis k\u00f5igile neljap\u00e4eval (02.05.19). github.com\/GoogleCloudPlatform\/terraformer<\/a><\/noindex>
\nOlen juba saanud 600 t\u00e4hte, 2 pull requesti openstacki ja kubernete toe lisamiseks. H\u00e4id arvustusi. \u00dcldiselt on projekt inimestele kasulik.
\nSoovitan k\u00f5igile, kes soovivad alustada Terraformiga t\u00f6\u00f6tamist ja mitte k\u00f5ike selleks \u00fcmber kirjutada. 
\nOlen r\u00f5\u00f5mus pull requestide, probleemide, t\u00e4htede \u00fcle.<\/p>\n
Demo
\n\"Terraformer<\/noindex>
\n
Allikas: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
\u0425\u043e\u0442\u0435\u043b \u0431\u044b \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u0430\u0442\u044c \u043f\u0440\u043e \u043d\u043e\u0432\u044b\u0439 CLI tool \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u044f \u043d\u0430\u043f\u0438\u0441\u0430\u043b \u0434\u043b\u044f \u0440\u0435\u0448\u0435\u043d\u0438\u044f \u043e\u0434\u043d\u043e\u0439 \u0441\u0442\u0430\u0440\u043e\u0439 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u044b. \u041f\u0440\u043e\u0431\u043b\u0435\u043c\u0430 Terraform \u0443\u0436\u0435 \u0434\u0430\u0432\u043d\u043e \u0441\u0442\u0430\u043b \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043e\u043c \u0432 Devops\/Cloud\/IT \u0441\u043e\u043e\u0431\u0449\u0435\u0441\u0442\u0432\u0435. \u0412\u0435\u0449\u044c \u043e\u0447\u0435\u043d\u044c \u0443\u0434\u043e\u0431\u043d\u0430\u044f \u0438 \u043f\u043e\u043b\u0435\u0437\u043d\u0430\u044f \u0447\u0442\u043e\u0431 \u0437\u0430\u043d\u0438\u043c\u0430\u0442\u0441\u044f infrastructure as code. \u0415\u0441\u0442\u044c \u043c\u043d\u043e\u0433\u043e \u043f\u0440\u0435\u043b\u0435\u0441\u0442\u0435\u0439 \u0432 Terraform \u0430 \u0442\u0430\u043a \u0436\u0435 \u043c\u043d\u043e\u0433\u043e \u0432\u0438\u043b\u043e\u043a, \u043e\u0441\u0442\u0440\u044b\u0445 \u043d\u043e\u0436\u0435\u0439 \u0438 \u0433\u0440\u0430\u0431\u043b\u0435\u0439. \u0421 Terraform \u043e\u0447\u0435\u043d\u044c \u0443\u0434\u043e\u0431\u043d\u043e \u0434\u0435\u043b\u0430\u0442\u044c \u043d\u043e\u0432\u044b\u0435 \u0432\u0435\u0449\u0438 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":24869,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-33123","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Terraformer \u2014 Infrastruktuur Koodiks | ProHoster","description":"Tahaksin r\u00e4\u00e4kida uuest CLI t\u00f6\u00f6riistast, mille ma kirjutasin.","canonical_url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/terraformer-infrastructure-to-code","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"et_EE","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47Terraformer \u2014 Infrastructure To Code | ProHoster","og:description":"\u0425\u043e\u0442\u0435\u043b \u0431\u044b \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u0430\u0442\u044c \u043f\u0440\u043e \u043d\u043e\u0432\u044b\u0439 CLI tool \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u044f \u043d\u0430\u043f\u0438\u0441\u0430\u043b.","og:url":"https:\/\/prohoster.info\/et\/blog\/administrirovanie\/terraformer-infrastructure-to-code","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:50:52+00:00","article:modified_time":"2019-10-31T18:50:52+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"33123","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 14:02:48","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 02:46:23","updated":"2026-01-21 14:02:48","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/33123","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/comments?post=33123"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/posts\/33123\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media\/24869"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/media?parent=33123"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/categories?post=33123"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/et\/wp-json\/wp\/v2\/tags?post=33123"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}