{"id":33123,"date":"2019-10-31T21:50:52","date_gmt":"2019-10-31T18:50:52","guid":{"rendered":"https:\/\/prohoster.info\/blog\/terraformer-infrastructure-to-code\/"},"modified":"2019-10-31T21:50:52","modified_gmt":"2019-10-31T18:50:52","slug":"terraformer-infrastructure-to-code","status":"publish","type":"post","link":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/terraformer-infrastructure-to-code","title":{"rendered":"Terraformer \u2014 Infrastruktura kao Kod","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Terraformer
\n\u017delim vam ispri\u010dati o novom CLI alatu koji sam napisao za rje\u0161avanje jednog starog problema. <\/p>\n

Problem<\/h3>\n

\nTerraform je ve\u0107 dugo standard u DevOps\/Cloud\/IT zajednici. To je vrlo prakti\u010dna i korisna stvar za rad s infrastrukturom kao kodom. Ima mnogo prednosti, ali i niz pote\u0161ko\u0107a i izazova.
\nS Terraformom je lako stvoriti nove resurse i kasnije njima upravljati, mijenjati ili brisati. No, \u0161to u\u010diniti ako imate veliku infrastrukturu u oblaku koja nije napravljena putem Terraforma? Prepisivanje i ponovna izgradnja cijelog oblaka je preskupo i nesigurno.
\nS tim sam se susreo u dvije tvrtke, najjednostavniji primjer je kada \u017eeli\u0161 da sve bude u Gitu u obliku Terraform datoteka, a ima\u0161 vi\u0161e od 250 kolekcija, pa pisati ih ru\u010dno za Terraform je prili\u010dno naporno.
\nPostoji problem<\/a><\/noindex> jo\u0161 od 2014. u Terraformu koji su zatvorili 2016. s nadom da \u0107e postojati mogu\u0107nost uvoza.<\/p>\n

Uglavnom, sve je kao na slici, samo s desna na lijevo
\n<\/a><\/noindex><\/p>\n

Upozorenje: Autor pola \u017eivota \u017eivi izvan Rusije i rijetko pi\u0161e na ruskom. Opasnost od gre\u0161aka u pravopisu.<\/p>\n

Rje\u0161enja<\/h3>\n

\n1. Postoje gotova i starija rje\u0161enja za AWS terraforming<\/a><\/noindex>. Kada sam poku\u0161ao preko njega dobiti moje 250+ kolekcija, shvatio sam da je to sve lo\u0161e. AWS je odavno dodao mnogo novih opcija, a terraforming o njima ne zna i zapravo koristi Ruby. Templati izgledaju skromno<\/a><\/noindex>. Nakon 2 ve\u010deri, poslao sam Pull request<\/a><\/noindex> da dodam vi\u0161e mogu\u0107nosti, ali shvatio sam da to rje\u0161enje uop\u0107e nije prikladno.
\nKako radi terraforming? Uzima podatke iz AWS SDK-a i generira tf i tfstate kroz templati.
\nOvdje su 3 problema:
\n1. Uvijek \u0107e biti ka\u0161njenja u a\u017euriranjima.
\n2. tf datoteke ponekad ispadnu o\u0161te\u0107ene.
\n3. tfstate se prikuplja odvojeno od tf i ne uvijek odgovara.
\nOstvariti rezultat gdje `terraform plan` ka\u017ee da nema promjena je zapravo te\u0161ko.<\/p>\n

2. `terraform import` \u2014 ugra\u0111ena naredba u terraformu. Kako to radi?
\nPi\u0161ete praznu TF datoteku s imenom i tipom resursa, zatim pokre\u0107ete `terraform import` i predajete ID resursa. Terraform se obra\u0107a provajderu, dobiva podatke i stvara tfstate datoteku.
\nOvdje su 3 problema:
\n1. Dobivamo samo tfstate datoteku, a tf je prazan, treba ga ru\u010dno napisati ili konvertirati iz tfstate.
\n2. Mo\u017ee raditi samo s jednim resursom svaki put i ne podr\u017eava sve resurse. \u0160to da radim opet s 250+ kolekcija?
\n3. Trebamo znati ID resursa \u2014 dakle, trebamo to obaviti u kodu koji dohva\u0107a popis resursa.
\nUglavnom, rezultat je djelomi\u010dan i ne skalira se dobro.<\/p>\n

Moja rje\u0161enja
\n<\/h3>\n

Zahtjevi:
\n1. Mogu\u0107nost stvaranja tf i tfstate datoteka na temelju resursa. Na primjer, preuzmite sve kante \/ sigurnosne grupe \/ ravnote\u017ee optere\u0107enja i da `terraform plan` vrati da nema promjena
\n2. Dva oblaka GCP + AWS
\n3. Globalno rje\u0161enje koje se lako a\u017eurira svaki put i ne tro\u0161i vrijeme na svaki resurs od tri dana rada
\n4. Napraviti open source \u2014 problem koji svi imaju<\/p>\n

Jezik Go \u2014 zato ga volim i postoji biblioteku za stvaranje HCL datoteka koja se koristi u terraformu + puno koda u terraformu koji mo\u017ee biti koristan<\/p>\n

Put<\/h3>\n

\nPoku\u0161aj prvi
\nZapo\u010deo sam s jednostavnom verzijom. Pozivi u oblak preko SDK za potrebni resurs i konvertiranje u polja za terraform. Poku\u0161aj je odmah propao na sigurnosnoj grupi jer mi nije odgovaralo 1,5 dana konvertirati samo sigurnosnu grupu (a resursa ima mnogo). Dugotrajno, a onda polja mogu izmijeniti\/dodavati<\/p>\n

Poku\u0161aj drugi
\nTemeljeno na ideji opisanoj ovdje<\/a><\/noindex>. Jednostavno uzeti i konvertirati tfstate u tf. Svi podaci su tamo i polja su ista. Kako dobiti puni tfstate za vi\u0161e resursa?? Ovdje je pomogla komanda `terraform refresh`. terraform uzima sve resurse u tfstate i po ID-u izvla\u010di podatke i zapisuje sve u tfstate. Dakle, stvoriti prazan tfstate samo s imenima i ID-ovima, pokrenuti `terraform refresh` i dobijemo potpuni tfstate. Ura!
\nSada \u0107emo se baviti rekursivnom pornografijom pisanja konvertera za tfstate u tf. Za one koji nikada nisu \u010ditali tfstate, to je JSON, ali posebna vrsta.
\nEvo njegova va\u017ena dijela attributes<\/p>\n

 \"attributes\": {\n                            \"id\": \"default\/backend-logging-load-deployment\",\n                            \"metadata.#\": \"1\",\n                            \"metadata.0.annotations.%\": \"0\",\n                            \"metadata.0.generate_name\": \"\",\n                            \"metadata.0.generation\": \"24\",\n                            \"metadata.0.labels.%\": \"1\",\n                            \"metadata.0.labels.app\": \"backend-logging\",\n                            \"metadata.0.name\": \"backend-logging-load-deployment\",\n                            \"metadata.0.namespace\": \"default\",\n                            \"metadata.0.resource_version\": \"109317427\",\n                            \"metadata.0.self_link\": \"\\\/apis\\\/apps\\\/v1\\\/namespaces\\\/default\\\/deployments\\\/backend-logging-load-deployment\",\n                            \"metadata.0.uid\": \"300ecda1-4138-11e9-9d5d-42010a8400b5\",\n                            \"spec.#\": \"1\",\n                            \"spec.0.min_ready_seconds\": \"0\",\n                            \"spec.0.paused\": \"false\",\n                            \"spec.0.progress_deadline_seconds\": \"600\",\n                            \"spec.0.replicas\": \"1\",\n                            \"spec.0.revision_history_limit\": \"10\",\n                            \"spec.0.selector.#\": \"1\",\n<\/code><\/pre>\n
\nOvdje se nalaze:
\n1. id \u2014 string
\n2. metadata \u2014 niz veli\u010dine 1 u kojem je objekt s poljima koja su opisano ni\u017ee
\n3. spec \u2014 hash veli\u010dine 1 u kojem je key,value
\nKratki zabavni format, sve mo\u017ee i\u0107i u dubinu na nekoliko razina.<\/p>\n
                   \"spec.#\": \"1\",\n                            \"spec.0.min_ready_seconds\": \"0\",\n                            \"spec.0.paused\": \"false\",\n                            \"spec.0.progress_deadline_seconds\": \"600\",\n                            \"spec.0.replicas\": \"1\",\n                            \"spec.0.revision_history_limit\": \"10\",\n                            \"spec.0.selector.#\": \"1\",\n                            \"spec.0.selector.0.match_expressions.#\": \"0\",\n                            \"spec.0.selector.0.match_labels.%\": \"1\",\n                            \"spec.0.selector.0.match_labels.app\": \"backend-logging-load\",\n                            \"spec.0.strategy.#\": \"0\",\n                            \"spec.0.template.#\": \"1\",\n                            \"spec.0.template.0.metadata.#\": \"1\",\n                            \"spec.0.template.0.metadata.0.annotations.%\": \"0\",\n                            \"spec.0.template.0.metadata.0.generate_name\": \"\",\n                            \"spec.0.template.0.metadata.0.generation\": \"0\",\n                            \"spec.0.template.0.metadata.0.labels.%\": \"1\",\n                            \"spec.0.template.0.metadata.0.labels.app\": \"backend-logging-load\",\n                            \"spec.0.template.0.metadata.0.name\": \"\",\n                            \"spec.0.template.0.metadata.0.namespace\": \"\",\n                            \"spec.0.template.0.metadata.0.resource_version\": \"\",\n                            \"spec.0.template.0.metadata.0.self_link\": \"\",\n                            \"spec.0.template.0.metadata.0.uid\": \"\",\n                            \"spec.0.template.0.spec.#\": \"1\",\n                            \"spec.0.template.0.spec.0.active_deadline_seconds\": \"0\",\n                            \"spec.0.template.0.spec.0.container.#\": \"1\",\n                            \"spec.0.template.0.spec.0.container.0.args.#\": \"3\",<\/code><\/pre>\n
\nOp\u0107enito, tko \u017eeli zadatak za programiranje za intervju, neka jednostavno zatra\u017ei da napi\u0161u parser za to \ud83d\ude42.
\nNakon dugih poku\u0161aja da napi\u0161em parser bez gre\u0161aka, na\u0161ao sam dio koda u terrafomu, a zapravo najva\u017eniji dio. I sve je, \u010dini se, radilo normalno.<\/p>\n
Poku\u0161aj tri.
\nTerraform provider je binarna datoteka koja sadr\u017ei kod sa svim resursima i logikom za rad s API-jem oblaka. Svaki oblak ima svoj provider, a sam terraform ih poziva putem svog RPC protokola izme\u0111u dva procesa.
\nSada sam odlu\u010dio obra\u0107ati se izravno terraform providerima putem RPC poziva. Tako je to ispalo lijepo i omogu\u0107ilo mi da mijenjam terraform providere za novije verzije i dobivam nove mogu\u0107nosti bez promjene koda. Ispostavilo se da ne sva polja u tfstate moraju biti u tf-u, a kako to saznati? Samo pitaju\u0107i providera o tome. Zatim je zapo\u010dela jo\u0161 jedna rekurzivna pornografija po sastavljanju regularnih izraza s mukama pronala\u017eenja polja unutar tfstate na svim razinama.<\/p>\n
Na kraju smo dobili koristan CLI alat koji ima zajedni\u010dku infrastrukturu za sve Terraform provajdere i lako je dodati nove. Tako\u0111e, dodavanje resursa zahteva malo koda. Plus, tu su razne prednosti kao \u0161to su veze izme\u0111u resursa. Naravno, bilo je mnogo razli\u010ditih problema koje je te\u0161ko sve opisati.
\nZvao sam ga Terrafomer. <\/p>\n
Finale<\/h3>\n
\nUz pomo\u0107 Terrafomera generisali smo 500-700 hiljada linija koda tf + tfstate iz dva oblaka. Uspeli smo da uzmemo legacijske stvari i zapo\u010dnemo ih menjati isklju\u010divo putem Terraforma, kao u najboljim idejama infrastrukture kao koda. Jednostavno magija kada uzmete ogroman oblak i dobijete ga uz pomo\u0107 komande u obliku radnih Terraform datoteka. A dalje grep\/replace\/git i tako dalje.<\/p>\n
Sve sam izra\u010dunao i sredio, dobio dozvole. Izdano na GitHub-u za sve u \u010detvrtak (02.05.19). github.com\/GoogleCloudPlatform\/terraformer<\/a><\/noindex>
\nVe\u0107 je dobio 600 zvezdica, 2 pull request-a za dodavanje podr\u0161ke za OpenStack i Kubernetes. Dobre recenzije. Uop\u0161te, projekat je koristan za ljude.
\nPreporu\u010dujem svima koji \u017eele da po\u010dnu raditi sa Terraform-om i ne prepisuju sve za to. 
\nBi\u0107u rad na pull request-ima, pitanjima, zvezdama.<\/p>\n
Demo
\n\"Terraformer<\/noindex>
\n
Izvor: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
\u0425\u043e\u0442\u0435\u043b \u0431\u044b \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u0430\u0442\u044c \u043f\u0440\u043e \u043d\u043e\u0432\u044b\u0439 CLI tool \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u044f \u043d\u0430\u043f\u0438\u0441\u0430\u043b \u0434\u043b\u044f \u0440\u0435\u0448\u0435\u043d\u0438\u044f \u043e\u0434\u043d\u043e\u0439 \u0441\u0442\u0430\u0440\u043e\u0439 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u044b. \u041f\u0440\u043e\u0431\u043b\u0435\u043c\u0430 Terraform \u0443\u0436\u0435 \u0434\u0430\u0432\u043d\u043e \u0441\u0442\u0430\u043b \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043e\u043c \u0432 Devops\/Cloud\/IT \u0441\u043e\u043e\u0431\u0449\u0435\u0441\u0442\u0432\u0435. \u0412\u0435\u0449\u044c \u043e\u0447\u0435\u043d\u044c \u0443\u0434\u043e\u0431\u043d\u0430\u044f \u0438 \u043f\u043e\u043b\u0435\u0437\u043d\u0430\u044f \u0447\u0442\u043e\u0431 \u0437\u0430\u043d\u0438\u043c\u0430\u0442\u0441\u044f infrastructure as code. \u0415\u0441\u0442\u044c \u043c\u043d\u043e\u0433\u043e \u043f\u0440\u0435\u043b\u0435\u0441\u0442\u0435\u0439 \u0432 Terraform \u0430 \u0442\u0430\u043a \u0436\u0435 \u043c\u043d\u043e\u0433\u043e \u0432\u0438\u043b\u043e\u043a, \u043e\u0441\u0442\u0440\u044b\u0445 \u043d\u043e\u0436\u0435\u0439 \u0438 \u0433\u0440\u0430\u0431\u043b\u0435\u0439. \u0421 Terraform \u043e\u0447\u0435\u043d\u044c \u0443\u0434\u043e\u0431\u043d\u043e \u0434\u0435\u043b\u0430\u0442\u044c \u043d\u043e\u0432\u044b\u0435 \u0432\u0435\u0449\u0438 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":24869,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-33123","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Terraformer \u2014 Infrastructure To Code | ProHoster","description":"\u017deleo bih da pri\u010dam o novom CLI alatu koji sam napisao.","canonical_url":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/terraformer-infrastructure-to-code","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"hr_HR","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47Terraformer \u2014 Infrastructure To Code | ProHoster","og:description":"\u0425\u043e\u0442\u0435\u043b \u0431\u044b \u0440\u0430\u0441\u0441\u043a\u0430\u0437\u0430\u0442\u044c \u043f\u0440\u043e \u043d\u043e\u0432\u044b\u0439 CLI tool \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u044f \u043d\u0430\u043f\u0438\u0441\u0430\u043b.","og:url":"https:\/\/prohoster.info\/hr\/blog\/administrirovanie\/terraformer-infrastructure-to-code","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:50:52+00:00","article:modified_time":"2019-10-31T18:50:52+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"33123","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 14:02:48","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 02:46:23","updated":"2026-01-21 14:02:48","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts\/33123","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/comments?post=33123"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/posts\/33123\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/media\/24869"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/media?parent=33123"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/categories?post=33123"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/hr\/wp-json\/wp\/v2\/tags?post=33123"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}