I-ProHoster > Π‘Π»ΠΎΠ³ > Ukuphatha > Samukela imicimbi engu-10 ku-Yandex.Cloud. Ingxenye 000

Samukela imicimbi engu-10 ku-Yandex.Cloud. Ingxenye 000

Sanibonani nonke, bangane!

* Lesi sihloko sisekelwe ku-REBRAIN & Yandex.Cloud evulekile workshop, uma ukhetha ukubuka ividiyo, ungayithola kulesi sixhumanisi - https://youtu.be/cZLezUm0ekE

Sisanda kuba nethuba lokuzama i-Yandex.Cloud bukhoma. Njengoba besifuna ukuphenya isikhathi eside futhi kanzima, sawushiya ngokushesha umqondo wokwethula ibhulogi ye-Wordpress elula enesisekelo samafu - bekuyisicefe kakhulu. Ngemuva komcabango othile, sinqume ukusebenzisa into efanayo nesakhiwo sesevisi yokukhiqiza ukuze samukele futhi sihlaziye imicimbi eduze nemodi yesikhathi sangempela.

Nginesiqiniseko esiphelele sokuthi iningi lamabhizinisi aku-inthanethi (hhayi kuphela) aqoqa inqwaba yolwazi mayelana nabasebenzisi bawo nezenzo zabo. Okungenani, lokhu kuyadingeka ukuze wenze izinqumo ezithile - isibonelo, uma uphatha igeyimu eku-inthanethi, ungabheka izibalo lapho abasebenzisi bavame ukubhajwa khona futhi basuse ithoyizi lakho. Noma kungani abasebenzisi beshiya isayithi lakho ngaphandle kokuthenga lutho (sawubona, Yandex.Metrica).

Ngakho-ke, indaba yethu: indlela esibhale ngayo isicelo ku-golang, ihlolwe i-kafka vs rabbitmq vs yqs, sabhala ukusakazwa kwedatha kuqoqo le-Clickhouse futhi sabona idatha sisebenzisa i-yandex datalens. Ngokwemvelo, konke lokhu kwakunongwe ngezinto ezithokozisa ingqalasizinda ngendlela ye-docker, i-terraform, i-gitlab ci futhi, yiqiniso, i-prometheus. Asambe!

Ngingathanda ukubhuka ngokushesha ukuthi ngeke sikwazi ukumisa yonke into ngesikhathi esisodwa - kulokhu sizodinga izindatshana ezimbalwa ochungechungeni. Okuncane mayelana nesakhiwo:

Ingxenye 1 (uyayifunda). Sizonquma ngokucaciswa nokwakhiwa kwesixazululo, futhi sibhale isicelo ku-golang.
Ingxenye 2. Sikhulula uhlelo lwethu lokusebenza ekukhiqizeni, silwenze lukhule futhi sihlole umthwalo.
Ingxenye 3. Ake sizame ukuthola ukuthi kungani sidinga ukugcina imilayezo ku-buffer hhayi kumafayela, futhi siqhathanise isevisi yomugqa we-kafka, rabbitmq kanye ne-yandex.
Ingxenye yesi-4 Sizosebenzisa iqoqo le-Clickhouse, sibhale isevisi yokusakaza-bukhoma ukuze sidlulisele idatha kusuka kusigcinalwazi lapho, futhi simise ukubonwa kuma-datalens.
Ingxenye 5 Masilethe yonke ingqalasizinda esimweni esifanele - setha i-ci/cd usebenzisa i-gitlab ci, xhuma ukuqapha nokutholwa kwesevisi usebenzisa i-prometheus ne-consul.

TK

Okokuqala, ake sakhe imigomo yereferensi - lokho ngempela esifuna ukukuthola njengomphumela.

  1. Sifuna ukuba nesiphetho njenge-events.kis.im (i-kis.im isizinda sokuhlola esizosisebenzisa kuwo wonke ama-athikili), okufanele samukele imicimbi kusetshenziswa i-HTTPS.
  2. Imicimbi iyi-json elula efana nokuthi: {β€œevent”: β€œview”, β€œos”: β€œlinux”, β€œbrowser”: β€œchrome”}. Esigabeni sokugcina sizokwengeza izinkambu ezengeziwe, kodwa lokhu ngeke kudlale indima enkulu. Uma ufisa, ungashintshela ku-protobuf.
  3. Isevisi kumele ikwazi ukucubungula imicimbi eyi-10 ngomzuzwana.
  4. Kufanele kwenzeke ukukala ngokuvundlile ngokungeza izimo ezintsha esixazululweni sethu. Futhi kuzoba kuhle uma singahambisa ingxenye engaphambili ezindaweni ezihlukene ukuze sinciphise ukubambezeleka kwezicelo zeklayenti.
  5. Ukubekezelela amaphutha. Isixazululo kufanele sizinze ngokwanele futhi sikwazi ukusinda ekwindla kwanoma yiziphi izingxenye (kuze kube yinombolo ethile, kunjalo).

bokwakha

Ngokuvamile, kulolu hlobo lomsebenzi, izakhiwo zakudala sekuyisikhathi eside zasungulwa ezivumela ukukala okuphumelelayo. Umfanekiso ubonisa isibonelo sesixazululo sethu.

Samukela imicimbi engu-10 ku-Yandex.Cloud. Ingxenye 000

Ngakho esinakho:

1. Kwesokunxele kukhona amadivaysi ethu akhiqiza imicimbi ehlukahlukene, kungaba abadlali abagcwalisa ileveli yokudlala ku-smartphone noma abakha i-oda esitolo se-inthanethi ngesiphequluli esivamile. Umcimbi, njengoba kucacisiwe ekucacisweni, yi-json elula ethunyelwa endaweni yethu yokugcina - events.kis.im.

2. Amaseva amabili okuqala angamabhalansi alula, imisebenzi yawo eyinhloko yile:

  • Yiba khona njalo. Ukuze wenze lokhu, ungasebenzisa, ngokwesibonelo, i- keepalived, ezoshintsha i-IP ebonakalayo phakathi kwamanodi uma kwenzeka kuba nezinkinga.
  • Susa i-TLS. Yebo, sizoyinqamula i-TLS kuzo. Okokuqala, ukuze isisombululo sethu sihambisane nokucaciswa kobuchwepheshe, futhi okwesibili, ukuze sikhulule umthwalo wokusungula uxhumano olubethelwe kusuka kumaseva ethu angemuva.
  • Linganisa izicelo ezingenayo kumaseva atholakala ngemuva. Igama eliyisihluthulelo lapha liyafinyeleleka. Ngokusekelwe kulokhu, sifinyelela ekuqondeni ukuthi izilinganisi zokulayisha kufanele zikwazi ukuqapha amaseva ethu ngezinhlelo zokusebenza futhi ziyeke ukulinganisa ithrafikhi kumanodi ahlulekile.

3. Ngemuva kwamabhalansi, sinamaseva ohlelo lokusebenza asebenzisa uhlelo olulula. Kufanele ikwazi ukwamukela izicelo ezingenayo nge-HTTP, iqinisekise i-json ethunyelwe futhi ifake idatha ku-buffer.

4. Umdwebo ubonisa i-kafka njenge-buffer, nakuba, yiqiniso, ezinye izinsizakalo ezifanayo zingasetshenziswa kuleli zinga. Sizoqhathanisa i-Kafka, i-rabbitmq kanye ne-yqs esihlokweni sesithathu.

5. Iphuzu elingaphambili lezakhiwo zethu yiClickhouse - isizindalwazi sekholomu esikuvumela ukuthi ugcine futhi ucubungule inani elikhulu ledatha. Kuleli zinga, sidinga ukudlulisa idatha isuka kusigcinalwazi iye ohlelweni lwesitoreji ngokwayo (okuningi kulokhu esihlokweni sesi-4).

Lo mklamo usivumela ukuthi silinganise isendlalelo ngasinye ngokuzimela ngokuvundlile. Amaseva angemuva awakwazi ukubhekana nakho - ake sengeze enye into - ngemuva kwakho konke, ayizinhlelo zokusebenza ezingenasici, ngakho-ke, lokhu kungenziwa ngokuzenzakalelayo. Ibhafa yesitayela se-Kafka ayisebenziβ€”ake sengeze amaseva engeziwe futhi sidlulisele ezinye izingxenye zesihloko sethu kuzo. I-Clickhouse ayikwazi ukuyiphatha - akunakwenzeka :) Eqinisweni, sizophinde sixhume amaseva futhi sihlukanise idatha.

Kodwa-ke, uma ufuna ukusebenzisa ingxenye yokuzikhethela yezicaciso zethu zobuchwepheshe kanye nezikali kuma-geolocations ahlukene, akukho lutho olulula:

Samukela imicimbi engu-10 ku-Yandex.Cloud. Ingxenye 000

Ku-geolocation ngayinye sisebenzisa isilinganisi somthwalo esinesicelo kanye ne-kafka. Ngokuvamile, amaseva ohlelo lokusebenza angu-2, ama-kafka nodes angu-3 kanye ne-balancer yefu, isibonelo, i-cloudflare, anele, okuzohlola ukutholakala kwamanodi ohlelo lokusebenza kanye nezicelo zebhalansi nge-geolocation esekelwe ekhelini le-IP lomthombo weklayenti. Ngakho-ke, idatha ethunyelwe iklayenti laseMelika izofika kumaseva aseMelika. Futhi idatha evela e-Afrika inge-Afrika.

Khona-ke yonke into ilula kakhulu - sisebenzisa ithuluzi lesibuko kusuka kusethi ye-Kafka futhi sikopishe yonke idatha kusuka kuzo zonke izindawo kuya esikhungweni sethu sedatha esimaphakathi esiseRussia. Ngaphakathi, sihlaziya idatha futhi siyiqopha ku-Clickhouse ukuze ibonwe ngokulandelayo.

Ngakho-ke, silungise izakhiwo - ake siqale ukuzamazama i-Yandex.Cloud!

Ukubhala isicelo

Ngaphambi Kwefu, kusafanele ube nesineke esincane futhi ubhale isevisi elula ukucubungula imicimbi engenayo. Sizosebenzisa i-golang ngoba izibonakalise kahle kakhulu njengolimi lokubhala izinhlelo zokusebenza zenethiwekhi.

Ngemva kokuchitha ihora (mhlawumbe amahora ambalwa), sithola okuthile okufana nalokhu: https://github.com/RebrainMe/yandex-cloud-events/blob/master/app/main.go.

Yimaphi amaphuzu abalulekile engingathanda ukuwaphawula lapha:

1. Lapho uqala uhlelo lokusebenza, ungacacisa amafulegi amabili. Omunye unesibopho sechweba lapho sizolalela khona izicelo ezingenayo ze-http (-addr). Eyesibili ingeyekheli leseva ye-kafka lapho sizoqopha khona imicimbi yethu (-kafka):

addr     = flag.String("addr", ":8080", "TCP address to listen to")
kafka    = flag.String("kafka", "127.0.0.1:9092", "Kafka endpoints”)

2. Isicelo sisebenzisa umtapo wezincwadi we-sarama ([] github.com/Shopify/sarama) ukuthumela imilayezo kuqoqo le-kafka. Ngokushesha setha izilungiselelo ezihloselwe isivinini esikhulu sokucubungula:

config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForLocal
config.Producer.Compression = sarama.CompressionSnappy
config.Producer.Return.Successes = true

3. Isicelo sethu futhi sineklayenti le-prometheus elakhelwe ngaphakathi, eliqoqa amamethrikhi ahlukahlukene, afana nalawa:

  • inani lezicelo esicelweni sethu;
  • inani lamaphutha lapho wenza isicelo (akunakwenzeka ukufunda isicelo sokuthunyelwe, i-json ephukile, akunakwenzeka ukubhalela i-Kafka);
  • isikhathi sokucubungula isicelo esisodwa esivela kuklayenti, okuhlanganisa nesikhathi sokubhala umlayezo ku-Kafka.

4. Iziphetho ezintathu isicelo sethu esizicubungulayo:

  • /isimo - vele ubuyele kulungile ukukhombisa ukuthi siyaphila. Yize ungangeza amasheke athile, njengokutholakala kweqoqo le-Kafka.
  • /amamethrikhi - ngokuya ngale url, iklayenti le-prometheus lizobuyisela amamethrikhi eliwaqoqile.
  • /okuthunyelwe kuyisiphetho esikhulu lapho izicelo ze-POST ezine-json ngaphakathi zizothunyelwa khona. Uhlelo lwethu lokusebenza luhlola i-json ukuthi iyasebenza futhi uma yonke into ilungile, ibhala idatha kuqoqo le-Kafka.

Ngizokwenza ukubhuka ukuthi ikhodi ayiphelele - ingaqedwa (futhi kufanele!) iqedwe. Isibonelo, ungayeka ukusebenzisa inethi/http eyakhelwe ngaphakathi bese ushintshela ku-fasthttp esheshayo. Noma ungathola isikhathi sokucubungula nezinsiza ze-cpu ngokuhambisa ukuhlola kokufaneleka kwe-json kusigaba sakamuva - lapho idatha idluliswa isuka kubhafa iye kuqoqo le-clickhouse.

Ngaphezu kohlangothi lwentuthuko yenkinga, ngokushesha sacabanga ngengqalasizinda yethu yesikhathi esizayo futhi sanquma ukusebenzisa isicelo sethu nge-docker. I-Dockerfile yokugcina yokwakha uhlelo lokusebenza https://github.com/RebrainMe/yandex-cloud-events/blob/master/app/Dockerfile. Ngokuvamile, kulula kakhulu, iphuzu elilodwa kuphela engingathanda ukulinaka umhlangano we-multistage, osivumela ukuthi sinciphise isithombe sokugcina sesitsha sethu.

Izinyathelo zokuqala efwini

Okokuqala, bhalisa ku ifu.yandex.ru. Ngemva kokugcwalisa zonke izinkambu ezidingekayo, sizodalwa i-akhawunti futhi sinikezwe isibonelelo semali ethile, engasetshenziswa ukuhlola izinsizakalo zamafu. Uma ufuna ukuphinda zonke izinyathelo ezivela esihlokweni sethu, lesi sibonelelo kufanele sikwanele.

Ngemuva kokubhaliswa, ifu elihlukile kanye nenkomba ezenzakalelayo izodalelwa wena, lapho ungaqala khona ukudala izinsiza zamafu. Ngokuvamile, ku-Yandex.Cloud, ubudlelwano bezinsiza bubukeka kanje:

Samukela imicimbi engu-10 ku-Yandex.Cloud. Ingxenye 000

Ungakha amafu amaningana ku-akhawunti eyodwa. Futhi ngaphakathi kwefu, yenza izinkomba ezihlukene zamaphrojekthi enkampani ahlukene. Ungafunda kabanzi ngalokhu kumadokhumenti - https://cloud.yandex.ru/docs/resource-manager/concepts/resources-hierarchy. Ngendlela, ngizovame ukubhekisela kuyo ngezansi embhalweni. Lapho ngimisa yonke ingqalasizinda kusukela ekuqaleni, imibhalo yangisiza izikhathi ezingaphezu kwesisodwa, ngakho ngikweluleka ukuthi uyifunde.

Ukuphatha ifu, ungasebenzisa kokubili isixhumi esibonakalayo sewebhu kanye nesisetshenziswa sekhonsoli - yc. Ukufaka kwenziwa ngomyalo owodwa (we-Linux ne-Mac Os):

curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bash

Uma uchwepheshe wakho wezokuphepha wangaphakathi ethukuthele mayelana nokusebenzisa imibhalo evela ku-inthanethi, khona-ke, okokuqala, ungakwazi ukuvula iskripthi futhi usifunde, futhi okwesibili, sisisebenzisa ngaphansi komsebenzisi wethu - ngaphandle kwamalungelo ezimpande.

Uma ufuna ukufaka iklayenti le-Windows, ungasebenzisa imiyalelo lapha bese ukhipha yc initukuze ukwenze ngendlela oyifisayo ngokugcwele:

vozerov@mba:~ $ yc init
Welcome! This command will take you through the configuration process.
Please go to https://oauth.yandex.ru/authorize?response_type=token&client_id= in order to obtain OAuth token.

Please enter OAuth token:
Please select cloud to use:
 [1] cloud-b1gv67ihgfu3bp (id = b1gv67ihgfu3bpt24o0q)
 [2] fevlake-cloud (id = b1g6bvup3toribomnh30)
Please enter your numeric choice: 2
Your current cloud has been set to 'fevlake-cloud' (id = b1g6bvup3toribomnh30).
Please choose folder to use:
 [1] default (id = b1g5r6h11knotfr8vjp7)
 [2] Create a new folder
Please enter your numeric choice: 1
Your current folder has been set to 'default' (id = b1g5r6h11knotfr8vjp7).
Do you want to configure a default Compute zone? [Y/n]
Which zone do you want to use as a profile default?
 [1] ru-central1-a
 [2] ru-central1-b
 [3] ru-central1-c
 [4] Don't set default zone
Please enter your numeric choice: 1
Your profile default Compute zone has been set to 'ru-central1-a'.
vozerov@mba:~ $

Empeleni, inqubo ilula - okokuqala udinga ukuthola ithokheni yesifungo ukuphatha ifu, khetha ifu nefolda ozoyisebenzisa.

Uma unama-akhawunti ambalwa noma amafolda ngaphakathi kwefu elifanayo, ungakha amaphrofayili engeziwe ngezilungiselelo ezihlukene usebenzisa iphrofayili ye-yc config bese ushintsha phakathi kwawo.

Ngaphandle kwezindlela ezingenhla, ithimba le-Yandex.Cloud libhale okuhle kakhulu i-plugin ye-terraform yokuphatha izinsiza zamafu. Ngokwami, ngilungiselele inqolobane ye-git, lapho ngachaza khona zonke izinsiza ezizokwakhiwa njengengxenye yesihloko - https://github.com/rebrainme/yandex-cloud-events/. Sinentshisekelo egatsheni eliyinhloko, masilihlanganise endaweni:


vozerov@mba:~ $ git clone https://github.com/rebrainme/yandex-cloud-events/ events
Cloning into 'events'...
remote: Enumerating objects: 100, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 100 (delta 37), reused 89 (delta 26), pack-reused 0
Receiving objects: 100% (100/100), 25.65 KiB | 168.00 KiB/s, done.
Resolving deltas: 100% (37/37), done.
vozerov@mba:~ $ cd events/terraform/

Zonke izinto eziguquguqukayo eziyinhloko ezisetshenziswa ku-terraform zibhalwe kufayela elikhulu.tf. Ukuze uqalise, dala ifayela langasese.auto.tfvars kufolda ye-terraform enokuqukethwe okulandelayo:

# Yandex Cloud Oauth token
yc_token = ""
# Yandex Cloud ID
yc_cloud_id = ""
# Yandex Cloud folder ID
yc_folder_id = ""
# Default Yandex Cloud Region
yc_region = "ru-central1-a"
# Cloudflare email
cf_email = ""
# Cloudflare token
cf_token = ""
# Cloudflare zone id
cf_zone_id = ""

Zonke izinto eziguquguqukayo zingathathwa ohlwini lwe-yc config, njengoba sesivele siyilungisile insiza ye-console. Ngikweluleka ukuthi wengeze ngokushesha i-private.auto.tfvars ku-.gitignore, ukuze ungashicileli ngephutha idatha eyimfihlo.

Kwangasese.auto.tfvars siphinde sacacisa idatha evela ku-Cloudflare - ukudala amarekhodi e-DNS kanye nommeleli wesizinda esikhulu semicimbi.kis.im kumaseva ethu. Uma ungafuni ukusebenzisa i-cloudflare, susa ukuqaliswa komhlinzeki we-cloudflare ku-main.tf kanye nefayela le-dns.tf, elinesibopho sokudala amarekhodi e-dns adingekayo.

Emsebenzini wethu sizohlanganisa zonke izindlela ezintathu - i-web interface, insiza ye-console, kanye ne-terraform.

Amanethiwekhi abonakalayo

Ukukhuluma iqiniso, ungeqa lesi sinyathelo, njengoba uma udala ifu elisha, uzovele ube nenethiwekhi ehlukile futhi kudalwe ama-subnet angu-3 - eyodwa yendawo ngayinye etholakalayo. Kodwa sisangathanda ukwenza inethiwekhi ehlukile yephrojekthi yethu enekheli layo. Umdwebo ojwayelekile wokuthi inethiwekhi isebenza kanjani ku-Yandex.Cloud iboniswa esithombeni esingezansi (ithathwe ngokwethembeka https://cloud.yandex.ru/docs/vpc/concepts/)

Samukela imicimbi engu-10 ku-Yandex.Cloud. Ingxenye 000

Ngakho, udala inethiwekhi evamile lapho izinsiza zingaxhumana zodwa. Kundawo ngayinye etholakalayo, i-subnet iyakhiwa enekheli layo futhi ixhunywe kunethiwekhi evamile. Ngenxa yalokho, zonke izinsiza zamafu ezikuyo zingakwazi ukuxhumana, ngisho noma zisezindaweni ezihlukene zokutholakala. Izinsiza ezixhunywe kumanethiwekhi amafu ahlukene zingabonana kuphela ngamakheli angaphandle. Ngendlela, lo mlingo usebenza kanjani ngaphakathi, yachazwa kahle kuHabre.

Ukudalwa kwenethiwekhi kuchazwe kufayela le-network.tf elisuka endaweni yokugcina. Lapho sakha inethiwekhi yangasese evamile eyodwa futhi sixhuma ama-subnet amathathu kuyo ezindaweni ezitholakalayo ezihlukene - kwangaphakathi-a (172.16.1.0/24), kwangaphakathi-b (172.16.2.0/24), kwangaphakathi-c (172.16.3.0/24) ).

Qalisa i-terraform futhi udale amanethiwekhi:

vozerov@mba:~/events/terraform (master) $ terraform init
... skipped ..

vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_vpc_subnet.internal-a -target yandex_vpc_subnet.internal-b -target yandex_vpc_subnet.internal-c

... skipped ...

Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

yandex_vpc_network.internal: Creating...
yandex_vpc_network.internal: Creation complete after 3s [id=enp2g2rhile7gbqlbrkr]
yandex_vpc_subnet.internal-a: Creating...
yandex_vpc_subnet.internal-b: Creating...
yandex_vpc_subnet.internal-c: Creating...
yandex_vpc_subnet.internal-a: Creation complete after 6s [id=e9b1dad6mgoj2v4funog]
yandex_vpc_subnet.internal-b: Creation complete after 7s [id=e2liv5i4amu52p64ac9p]
yandex_vpc_subnet.internal-c: Still creating... [10s elapsed]
yandex_vpc_subnet.internal-c: Creation complete after 10s [id=b0c2qhsj2vranoc9vhcq]

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Kuhle! Sidale inethiwekhi yethu futhi manje sesilungele ukudala amasevisi ethu angaphakathi.

Ukudala imishini ebonakalayo

Ukuhlola uhlelo lokusebenza, sizodinga kuphela ukudala imishini emibili ebonakalayo - sizodinga eyokuqala ukwakha nokusebenzisa uhlelo lokusebenza, okwesibili ukusebenzisa i-kafka, esizoyisebenzisa ukugcina imilayezo engenayo. Futhi sizodala omunye umshini lapho sizomisa khona i-prometheus ukuqapha uhlelo lokusebenza.

Imishini ebonakalayo izolungiswa kusetshenziswa i-ansible, ngakho-ke ngaphambi kokuqala i-terraform, qiniseka ukuthi unenguqulo yakamuva ye-ansible. Futhi faka izindima ezidingekayo nge-galaxy enengqondo:

vozerov@mba:~/events/terraform (master) $ cd ../ansible/
vozerov@mba:~/events/ansible (master) $ ansible-galaxy install -r requirements.yml
- cloudalchemy-prometheus (master) is already installed, skipping.
- cloudalchemy-grafana (master) is already installed, skipping.
- sansible.kafka (master) is already installed, skipping.
- sansible.zookeeper (master) is already installed, skipping.
- geerlingguy.docker (master) is already installed, skipping.
vozerov@mba:~/events/ansible (master) $

Ngaphakathi kwefolda efanelekile kunesibonelo sefayela lokumisa elithi .ansible.cfg engilisebenzisayo. Kungase kube usizo.

Ngaphambi kokudala imishini ebonakalayo, qiniseka ukuthi une-ssh-ejenti esebenzayo kanye nokhiye we-ssh wengeziwe, ngaphandle kwalokho i-terraform ngeke ikwazi ukuxhuma emishinini edaliwe. Impela, ngihlangabezane nesiphazamisi ku-os x: https://github.com/ansible/ansible/issues/32499#issuecomment-341578864. Ukuze uvimbele lokhu ukuthi kungenzeki futhi, engeza okuhlukile okuncane ku-env ngaphambi kokwethula i-Terraform:

vozerov@mba:~/events/terraform (master) $ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

Kufolda ene-terraform sakha izinsiza ezidingekayo:

vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_compute_instance.build -target yandex_compute_instance.monitoring -target yandex_compute_instance.kafka
yandex_vpc_network.internal: Refreshing state... [id=enp2g2rhile7gbqlbrkr]
data.yandex_compute_image.ubuntu_image: Refreshing state...
yandex_vpc_subnet.internal-a: Refreshing state... [id=e9b1dad6mgoj2v4funog]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

... skipped ...

Plan: 3 to add, 0 to change, 0 to destroy.

... skipped ...

Uma konke kuphelile ngempumelelo (futhi kufanele kube), sizoba nemishini emithathu ebonakalayo:

  1. ukwakha - umshini wokuhlola nokwakha isicelo. I-Docker ifakwe ngokuzenzakalelayo ngabakwa-Ansible.
  2. ukuqapha - umshini wokuqapha - i-prometheus & grafana efakwe kuyo. Indinganiso yokungena / iphasiwedi: admin / admin
  3. I-kafka ingumshini omncane ofakwe i-kafka, otholakala ku-port 9092.

Masiqinisekise ukuthi zonke zisendaweni:

vozerov@mba:~/events (master) $ yc compute instance list
+----------------------+------------+---------------+---------+---------------+-------------+
|          ID          |    NAME    |    ZONE ID    | STATUS  |  EXTERNAL IP  | INTERNAL IP |
+----------------------+------------+---------------+---------+---------------+-------------+
| fhm081u8bkbqf1pa5kgj | monitoring | ru-central1-a | RUNNING | 84.201.159.71 | 172.16.1.35 |
| fhmf37k03oobgu9jmd7p | kafka      | ru-central1-a | RUNNING | 84.201.173.41 | 172.16.1.31 |
| fhmt9pl1i8sf7ga6flgp | build      | ru-central1-a | RUNNING | 84.201.132.3  | 172.16.1.26 |
+----------------------+------------+---------------+---------+---------------+-------------+

Izinsiza sezikhona, futhi ukusuka lapha singathola amakheli azo e-IP. Kukho konke okulandelayo ngizosebenzisa amakheli e-IP ukuxhuma nge-ssh futhi ngivivinye uhlelo lokusebenza. Uma une-akhawunti ye-cloudflare exhunywe ku-terraform, zizwe ukhululekile ukusebenzisa amagama e-DNS asanda kwakhiwa.
Ngendlela, lapho udala umshini we-virtual, i-IP yangaphakathi negama le-DNS langaphakathi linikezwa, ukuze ukwazi ukufinyelela amaseva ngaphakathi kwenethiwekhi ngamagama:

ubuntu@build:~$ ping kafka.ru-central1.internal
PING kafka.ru-central1.internal (172.16.1.31) 56(84) bytes of data.
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=1 ttl=63 time=1.23 ms
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=2 ttl=63 time=0.625 ms
^C
--- kafka.ru-central1.internal ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.625/0.931/1.238/0.308 ms

Lokhu kuzoba usizo kithi ukukhombisa uhlelo lokusebenza indawo yokugcina nge-kafk.

Ukuhlanganisa isicelo

Kuhle, kukhona amaseva, kukhona uhlelo lokusebenza - okusele nje ukulihlanganisa futhi lishicilelwe. Ekwakhiweni sizosebenzisa i-docker evamile yokwakha, kodwa njengesitoreji sesithombe sizosebenzisa isevisi evela ku-Yandex - ukubhaliswa kwesitsha. Kodwa izinto zokuqala kuqala.

Sikopisha uhlelo lokusebenza emshinini wokwakha, ngena nge-ssh bese sihlanganisa isithombe:

vozerov@mba:~/events/terraform (master) $ cd ..
vozerov@mba:~/events (master) $ rsync -av app/ [email protected]:app/

... skipped ...

sent 3849 bytes  received 70 bytes  7838.00 bytes/sec
total size is 3644  speedup is 0.93

vozerov@mba:~/events (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cd app
ubuntu@build:~/app$ sudo docker build -t app .
Sending build context to Docker daemon  6.144kB
Step 1/9 : FROM golang:latest AS build
... skipped ...

Successfully built 9760afd8ef65
Successfully tagged app:latest

Ingxenye yempi yenziwe - manje singabheka ukusebenza kohlelo lwethu ngokulwethula futhi silithumele ku-kafka:

ubuntu@build:~/app$ sudo docker run --name app -d -p 8080:8080 app /app/app -kafka=kafka.ru-central1.internal:9092</code>

Π‘ локальной машинки ΠΌΠΎΠΆΠ½ΠΎ ΠΎΡ‚ΠΏΡ€Π°Π²ΠΈΡ‚ΡŒ тСстовый event ΠΈ ΠΏΠΎΡΠΌΠΎΡ‚Ρ€Π΅Ρ‚ΡŒ Π½Π° ΠΎΡ‚Π²Π΅Ρ‚:

<code>vozerov@mba:~/events (master) $ curl -D - -s -X POST -d '{"key1":"data1"}' http://84.201.132.3:8080/post
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 13 Apr 2020 13:53:54 GMT
Content-Length: 41

{"status":"ok","partition":0,"Offset":0}
vozerov@mba:~/events (master) $

Uhlelo lokusebenza luphendule ngempumelelo yokurekhoda futhi lubonisa i-id yokuhlukanisa kanye ne-offset lapho umlayezo ufakwe khona. Osekusele ukuthi ukwenze ukudala ukubhalisa ku-Yandex.Cloud bese ulayisha isithombe sethu lapho (ukwenza lokhu usebenzisa imigqa emithathu kuchazwe kufayela le-registry.tf). Dala isitoreji:

vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_container_registry.events

... skipped ...

Plan: 1 to add, 0 to change, 0 to destroy.

... skipped ...

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Kunezindlela ezimbalwa zokuqinisekisa ekubhalisweni kweziqukathi - usebenzisa ithokheni yesifungo, ithokheni ye-iam, noma ukhiye we-akhawunti yesevisi. Imininingwane eyengeziwe mayelana nalezi zindlela ingatholakala kumadokhumenti. https://cloud.yandex.ru/docs/container-registry/operations/authentication. Sizosebenzisa ukhiye we-akhawunti yesevisi, ngakho-ke sidala i-akhawunti:

vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_iam_service_account.docker -target yandex_resourcemanager_folder_iam_binding.puller -target yandex_resourcemanager_folder_iam_binding.pusher

... skipped ...

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Manje okusele ukwenza ukhiye wayo:

vozerov@mba:~/events/terraform (master) $ yc iam key create --service-account-name docker -o key.json
id: ajej8a06kdfbehbrh91p
service_account_id: ajep6d38k895srp9osij
created_at: "2020-04-13T14:00:30Z"
key_algorithm: RSA_2048

Sithola ulwazi mayelana ne-id yesitoreji sethu, sidlulise ukhiye bese singena ngemvume:

vozerov@mba:~/events/terraform (master) $ scp key.json [email protected]:
key.json                                                                                                                    100% 2392   215.1KB/s   00:00

vozerov@mba:~/events/terraform (master) $ ssh 84.201.132.3 -l ubuntu

ubuntu@build:~$ cat key.json | sudo docker login --username json_key --password-stdin cr.yandex
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
ubuntu@build:~$

Ukuze ulayishe isithombe ebhukwini, sidinga i-ID yesiqukathi, siyithatha ku-yc utility:

vozerov@mba:~ $ yc container registry get events
id: crpdgj6c9umdhgaqjfmm
folder_id:
name: events
status: ACTIVE
created_at: "2020-04-13T13:56:41.914Z"

Ngemuva kwalokho, simaka isithombe sethu ngegama elisha bese siyalayisha:

ubuntu@build:~$ sudo docker tag app cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
ubuntu@build:~$ sudo docker push cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
The push refers to repository [cr.yandex/crpdgj6c9umdhgaqjfmm/events]
8c286e154c6e: Pushed
477c318b05cb: Pushed
beee9f30bc1f: Pushed
v1: digest: sha256:1dd5aaa9dbdde2f60d833be0bed1c352724be3ea3158bcac3cdee41d47c5e380 size: 946

Singaqinisekisa ukuthi isithombe silayishwe ngempumelelo:

vozerov@mba:~/events/terraform (master) $ yc container repository list
+----------------------+-----------------------------+
|          ID          |            NAME             |
+----------------------+-----------------------------+
| crpe8mqtrgmuq07accvn | crpdgj6c9umdhgaqjfmm/events |
+----------------------+-----------------------------+

Ngendlela, uma ufaka insiza ye-yc emshinini we-Linux, ungasebenzisa umyalo 

yc container registry configure-docker

ukulungisa i-docker.

isiphetho

Senze umsebenzi onzima omningi futhi ngenxa yalokho:

  1. Siqhamuke nesakhiwo senkonzo yethu yesikhathi esizayo.
  2. Sibhale isicelo nge-golang esebenzisa ingqondo yebhizinisi lethu.
  3. Sayiqoqa futhi sayithela endaweni yokubhalisa yeziqukathi ezizimele.

Engxenyeni elandelayo, sizodlulela ezintweni ezithakazelisayo - sizokhulula uhlelo lwethu lokusebenza ekukhiqizeni futhi ekugcineni sethule umthwalo kulo. Ungashintshi!

Lokhu okuqukethwe kukurekhodiwe ngevidiyo yendawo yokusebenzela evulekile REBRAIN & Yandex.Cloud: Samukela izicelo eziyi-10 ngomzuzwana ku-Yandex Cloud - https://youtu.be/cZLezUm0ekE

Uma ungathanda ukuhambela imicimbi enjalo ku-inthanethi futhi ubuze imibuzo ngesikhathi sangempela, xhuma ku isiteshi i-DevOps ngu-REBRAIN.

Sithanda ukubonga ngokukhethekile ku-Yandex.Cloud ngethuba lokusingatha umcimbi onjalo. Xhumana nabo - https://cloud.yandex.ru/prices

Uma udinga ukuya efwini noma unemibuzo mayelana nengqalasizinda yakho, zizwe ukhululekile ukushiya isicelo.

PS Sinokuhlolwa kwamahhala okungu-2 ngenyanga, mhlawumbe iphrojekthi yakho izoba enye yazo.

Source: www.habr.com

Engeza amazwana