Kaabo gbogbo eniyan, awọn ọrẹ!
Nkan yii da lori idanileko ṣiṣi REBRAIN & Yandex.Cloud, ti o ba fẹ lati wo fidio naa, o le rii ni ọna asopọ yii -
Laipẹ a ni aye lati gbiyanju Yandex.Cloud laaye. Niwọn igba ti a fẹ lati ṣe iwadii gigun ati lile, a kọ lẹsẹkẹsẹ imọran ti ifilọlẹ bulọọgi Wordpress ti o rọrun pẹlu ipilẹ awọsanma - o jẹ alaidun pupọ. Lẹhin ironu diẹ, a pinnu lati ran nkan ti o jọra si faaji iṣẹ iṣelọpọ fun gbigba ati itupalẹ awọn iṣẹlẹ ni ipo akoko gidi nitosi.
Mo ni idaniloju daadaa pe opo julọ ti awọn iṣowo ori ayelujara (kii ṣe nikan) bakan gba oke alaye nipa awọn olumulo wọn ati awọn iṣe wọn. Ni o kere ju, eyi jẹ pataki fun ṣiṣe awọn ipinnu kan - fun apẹẹrẹ, ti o ba ṣakoso ere ori ayelujara, o le wo awọn iṣiro ni eyiti awọn olumulo ipele ti nigbagbogbo di ati paarẹ ohun-iṣere rẹ. Tabi idi ti awọn olumulo fi kuro ni aaye rẹ laisi rira ohunkohun (hello, Yandex.Metrica).
Nitorinaa, itan wa: bawo ni a ṣe kọ ohun elo kan ni golang, idanwo kafka vs rabbitmq vs yqs, kọ ṣiṣanwọle data ni iṣupọ Clickhouse kan ati wiwo data naa nipa lilo yandex datalens. Nipa ti, gbogbo eyi jẹ akoko pẹlu awọn igbadun amayederun ni irisi docker, terraform, gitlab ci ati, dajudaju, prometheus. Jeka lo!
Emi yoo fẹ lati ṣe ifiṣura lẹsẹkẹsẹ pe a kii yoo ni anfani lati tunto ohun gbogbo ni ijoko kan - fun eyi a yoo nilo ọpọlọpọ awọn nkan ninu jara. Diẹ diẹ nipa eto naa:
Apakan 1 (o n ka a). A yoo pinnu lori awọn pato ati faaji ti ojutu, ati tun kọ ohun elo ni golang.
Apa keji. A tu ohun elo wa sinu iṣelọpọ, jẹ ki o jẹ iwọn ati idanwo fifuye naa.
Apa 3. Jẹ ká gbiyanju lati ro ero idi ti a nilo lati fi awọn ifiranṣẹ ni a saarin ati ki o ko ni awọn faili, ki o si tun afiwe kafka, rabbitmq ati yandex ti isinyi iṣẹ.
Apa 4 A yoo ran iṣupọ Clickhouse ṣiṣẹ, kọ iṣẹ ṣiṣanwọle kan lati gbe data lati inu ifipamọ nibẹ, ati ṣeto iworan ni datalens.
Apa 5 Jẹ ki a mu gbogbo awọn amayederun wa sinu apẹrẹ to dara - ṣeto ci/cd nipa lilo gitlab ci, so ibojuwo ati wiwa iṣẹ nipa lilo prometheus ati consul.
ТЗ
Ni akọkọ, jẹ ki a ṣe agbekalẹ awọn ofin itọkasi - kini gangan a fẹ lati gba bi abajade.
- A fẹ lati ni aaye ipari bi events.kis.im (kis.im ni aaye idanwo ti a yoo lo jakejado gbogbo awọn nkan), eyiti o yẹ ki o gba awọn iṣẹlẹ ni lilo HTTPS.
- Awọn iṣẹlẹ jẹ json rọrun bi: {“iṣẹlẹ”: “view”, “os”: “linux”, “browser”: “chrome”}. Ni ipele ikẹhin a yoo ṣafikun awọn aaye diẹ diẹ sii, ṣugbọn eyi kii yoo ṣe ipa nla. Ti o ba fẹ, o le yipada si protobuf.
- Iṣẹ naa gbọdọ ni anfani lati ṣe ilana awọn iṣẹlẹ 10 fun iṣẹju kan.
- O yẹ ki o ṣee ṣe lati ṣe iwọn ni ita nipa fifi awọn iṣẹlẹ tuntun kun si ojutu wa. Ati pe yoo dara ti a ba le gbe apakan iwaju si oriṣiriṣi awọn agbegbe geolocation lati dinku lairi fun awọn ibeere alabara.
- Ifarada aṣiṣe. Ojutu naa gbọdọ jẹ iduroṣinṣin to ati ni anfani lati ye ninu isubu ti eyikeyi awọn ẹya (to nọmba kan, dajudaju).
faaji
Ni gbogbogbo, fun iru iṣẹ-ṣiṣe yii, awọn ile ayaworan ile-aye ti pẹ ni a ti ṣẹda ti o gba laaye igbelowọn daradara. Nọmba naa fihan apẹẹrẹ ti ojutu wa.
Nitorina kini a ni:
1. Ni apa osi ni awọn ẹrọ wa ti o ṣe agbekalẹ awọn iṣẹlẹ lọpọlọpọ, jẹ awọn oṣere ti o pari ipele kan ninu ohun-iṣere kan lori foonuiyara tabi ṣiṣẹda aṣẹ ni ile itaja ori ayelujara nipasẹ ẹrọ aṣawakiri deede. Iṣẹlẹ kan, gẹgẹbi pato ninu sipesifikesonu, jẹ json ti o rọrun ti o firanṣẹ si aaye ipari wa - events.kis.im.
2. Awọn olupin akọkọ meji jẹ awọn iwọntunwọnsi ti o rọrun, awọn iṣẹ-ṣiṣe akọkọ wọn ni:
- Wa nigbagbogbo. Lati ṣe eyi, o le lo, fun apẹẹrẹ, keepalive, eyi ti yoo yi awọn foju IP laarin awọn apa ni irú ti isoro.
- Fi opin si TLS. Bẹẹni, a yoo fopin si TLS lori wọn. Ni akọkọ, ki ojutu wa ni ibamu pẹlu awọn alaye imọ-ẹrọ, ati keji, lati le yọkuro ẹru ti iṣeto asopọ ti paroko lati ọdọ awọn olupin ẹhin wa.
- Ṣe iwọntunwọnsi awọn ibeere ti nwọle si awọn olupin ẹhin ti o wa. Ọrọ bọtini nibi ni wiwọle. Da lori eyi, a wa si oye pe awọn iwọntunwọnsi fifuye gbọdọ ni anfani lati ṣe atẹle awọn olupin wa pẹlu awọn ohun elo ati dawọ iwọntunwọnsi ijabọ si awọn apa ikuna.
3. Lẹhin awọn iwọntunwọnsi, a ni awọn olupin ohun elo nṣiṣẹ ohun elo ti o rọrun. O yẹ ki o ni anfani lati gba awọn ibeere ti nwọle nipasẹ HTTP, fọwọsi json ti a firanṣẹ ati fi data naa sinu ifipamọ kan.
4. Aworan naa fihan kafka bi ifipamọ, botilẹjẹpe, dajudaju, awọn iṣẹ miiran ti o jọra le ṣee lo ni ipele yii. A yoo ṣe afiwe Kafka, rabbitmq ati yqs ni nkan kẹta.
5. Awọn penultimate ojuami ti wa faaji ni Clickhouse - a columnar database ti o faye gba o lati fipamọ ati ilana kan tobi iye ti data. Ni ipele yii, a nilo lati gbe data lati ifipamọ si eto ipamọ funrararẹ (diẹ sii lori eyi ni nkan 4).
Apẹrẹ yii gba wa laaye lati ṣe iwọn ipele kọọkan ni ominira ni ita. Awọn olupin afẹyinti ko le koju - jẹ ki a ṣafikun ohun kan diẹ sii - lẹhinna, wọn jẹ awọn ohun elo ti ko ni ipinlẹ, ati nitorinaa, eyi le ṣee ṣe paapaa laifọwọyi. Ifipamọ ara Kafka ko ṣiṣẹ-jẹ ki a ṣafikun awọn olupin diẹ sii ki a gbe diẹ ninu awọn ipin ti koko-ọrọ wa si wọn. Clickhouse ko le mu - ko ṣee ṣe :) Ni otitọ, a yoo tun so awọn olupin pọ ati pin data naa.
Nipa ọna, ti o ba fẹ ṣe imuse apakan iyan ti awọn alaye imọ-ẹrọ wa ati iwọn ni awọn agbegbe oriṣiriṣi, lẹhinna ko si ohun ti o rọrun:
Ni kọọkan geolocation a ran a fifuye iwontunwonsi pẹlu ohun elo ati ki o kafka. Ni gbogbogbo, awọn olupin ohun elo 2, awọn nodes kafka 3 ati iwọntunwọnsi awọsanma, fun apẹẹrẹ, cloudflare, ti to, eyi ti yoo ṣayẹwo wiwa awọn apa ohun elo ati awọn ibeere iwọntunwọnsi nipasẹ geolocation ti o da lori adiresi IP orisun alabara. Nitorinaa, data ti o firanṣẹ nipasẹ alabara Amẹrika kan yoo de lori awọn olupin Amẹrika. Ati data lati Afirika wa ni Afirika.
Lẹhinna ohun gbogbo rọrun pupọ - a lo ohun elo digi lati ṣeto Kafka ati daakọ gbogbo data lati gbogbo awọn ipo si ile-iṣẹ data aarin wa ti o wa ni Russia. Ni inu, a ṣe itupalẹ data naa ati gbasilẹ ni Clickhouse fun iworan atẹle.
Nitorinaa, a ti ṣe lẹsẹsẹ awọn faaji - jẹ ki a bẹrẹ gbigbọn Yandex.Cloud!
Kikọ ohun elo
Ṣaaju ki Awọsanma naa, o tun ni lati jẹ alaisan diẹ ki o kọ iṣẹ ti o rọrun lati ṣe ilana awọn iṣẹlẹ ti nwọle. A yoo lo golang nitori pe o ti fi ara rẹ han daradara bi ede fun kikọ awọn ohun elo nẹtiwọọki.
Lẹhin lilo wakati kan (boya awọn wakati meji), a gba nkan bii eyi: .
Kini awọn koko akọkọ ti Emi yoo fẹ lati ṣe akiyesi nibi:
1. Nigbati o ba bẹrẹ ohun elo, o le pato awọn asia meji. Ọkan jẹ iduro fun ibudo lori eyiti a yoo tẹtisi awọn ibeere http ti nwọle (-addr). Ekeji jẹ fun adirẹsi olupin kafka nibiti a yoo ṣe igbasilẹ awọn iṣẹlẹ wa (-kafka):
addr = flag.String("addr", ":8080", "TCP address to listen to")
kafka = flag.String("kafka", "127.0.0.1:9092", "Kafka endpoints”)2. Ohun elo naa nlo ile-ikawe sarama () lati fi awọn ifiranṣẹ ranṣẹ si iṣupọ kafka. A ṣeto awọn eto lẹsẹkẹsẹ ti a pinnu ni iyara sisẹ ti o pọju:
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForLocal
config.Producer.Compression = sarama.CompressionSnappy
config.Producer.Return.Successes = true3. Ohun elo wa tun ni alabara prometheus ti a ṣe sinu, eyiti o gba ọpọlọpọ awọn metiriki, bii:
- nọmba awọn ibeere si ohun elo wa;
- nọmba awọn aṣiṣe nigba ṣiṣe ibeere naa (ko ṣee ṣe lati ka ibeere ifiweranṣẹ, json baje, ko ṣee ṣe lati kọ si Kafka);
- akoko processing fun ibeere kan lati ọdọ alabara, pẹlu akoko fun kikọ ifiranṣẹ si Kafka.
4. Awọn aaye ipari mẹta ti awọn ilana elo wa:
- / ipo - nìkan pada ok lati fihan pe a wa laaye. Botilẹjẹpe o le ṣafikun diẹ ninu awọn sọwedowo, gẹgẹbi wiwa ti iṣupọ Kafka.
- / metiriki - ni ibamu si url yii, alabara prometheus yoo da awọn metiriki ti o ti gba pada.
- / ifiweranṣẹ jẹ aaye ipari akọkọ nibiti awọn ibeere POST pẹlu json inu yoo firanṣẹ. Ohun elo wa n ṣayẹwo json fun iwulo ati pe ti ohun gbogbo ba dara, o kọ data naa si iṣupọ Kafka.
Emi yoo ṣe ifiṣura pe koodu naa ko pe - o le (ati pe o yẹ!) Ti pari. Fun apẹẹrẹ, o le da lilo net/http ti a ṣe sinu rẹ duro ki o yipada si iyara http. Tabi o le jèrè akoko sisẹ ati awọn orisun cpu nipa gbigbe ayẹwo iwifun json lọ si ipele nigbamii - nigbati a ba gbe data naa lati inu ifipamọ si iṣupọ ile-tẹ.
Ni afikun si ẹgbẹ idagbasoke ti ọran naa, a ronu lẹsẹkẹsẹ nipa awọn amayederun ọjọ iwaju ati pinnu lati fi ohun elo wa ranṣẹ nipasẹ docker. Dockerfile ikẹhin fun kikọ ohun elo naa jẹ . Ni gbogbogbo, o rọrun pupọ, aaye kan ṣoṣo ti Emi yoo fẹ lati fiyesi si ni apejọ multistage, eyiti o jẹ ki a dinku aworan ikẹhin ti eiyan wa.
Awọn igbesẹ akọkọ ninu awọsanma
Ni akọkọ, forukọsilẹ . Lẹhin ti o kun gbogbo awọn aaye ti a beere, a yoo ṣẹda akọọlẹ kan ati fifun ni ẹbun fun iye owo kan, eyiti o le ṣee lo lati ṣe idanwo awọn iṣẹ awọsanma. Ti o ba fẹ tun gbogbo awọn igbesẹ lati nkan wa, ẹbun yii yẹ ki o to fun ọ.
Lẹhin iforukọsilẹ, awọsanma lọtọ ati itọsọna aiyipada yoo ṣẹda fun ọ, ninu eyiti o le bẹrẹ ṣiṣẹda awọn orisun awọsanma. Ni gbogbogbo, ni Yandex.Cloud, ibatan ti awọn orisun dabi eyi:
O le ṣẹda awọn awọsanma pupọ fun akọọlẹ kan. Ati inu awọsanma, ṣe awọn ilana oriṣiriṣi fun awọn iṣẹ akanṣe ile-iṣẹ oriṣiriṣi. O le ka diẹ sii nipa eyi ninu iwe-ipamọ - . Nipa ọna, Emi yoo nigbagbogbo tọka si ni isalẹ ninu ọrọ naa. Nigbati mo ṣeto gbogbo awọn amayederun lati ibere, iwe naa ṣe iranlọwọ fun mi diẹ sii ju ẹẹkan lọ, nitorinaa Mo gba ọ ni imọran lati kawe rẹ.
Lati ṣakoso awọsanma, o le lo mejeeji ni wiwo wẹẹbu ati ohun elo console - yc. Fifi sori ẹrọ ni a ṣe pẹlu aṣẹ kan (fun Linux ati Mac OS):
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bashTi o ba jẹ pe alamọja aabo inu rẹ n binu nipa ṣiṣe awọn iwe afọwọkọ lati Intanẹẹti, lẹhinna, ni akọkọ, o le ṣii iwe afọwọkọ naa ki o ka, ati ni keji, a ṣiṣẹ labẹ olumulo wa - laisi awọn ẹtọ gbongbo.
Ti o ba fẹ fi sori ẹrọ alabara kan fun Windows, o le lo awọn ilana naa ati lẹhinna ṣiṣẹ yc initlati ṣatunṣe rẹ ni kikun:
vozerov@mba:~ $ yc init
Welcome! This command will take you through the configuration process.
Please go to https://oauth.yandex.ru/authorize?response_type=token&client_id= in order to obtain OAuth token.
Please enter OAuth token:
Please select cloud to use:
[1] cloud-b1gv67ihgfu3bp (id = b1gv67ihgfu3bpt24o0q)
[2] fevlake-cloud (id = b1g6bvup3toribomnh30)
Please enter your numeric choice: 2
Your current cloud has been set to 'fevlake-cloud' (id = b1g6bvup3toribomnh30).
Please choose folder to use:
[1] default (id = b1g5r6h11knotfr8vjp7)
[2] Create a new folder
Please enter your numeric choice: 1
Your current folder has been set to 'default' (id = b1g5r6h11knotfr8vjp7).
Do you want to configure a default Compute zone? [Y/n]
Which zone do you want to use as a profile default?
[1] ru-central1-a
[2] ru-central1-b
[3] ru-central1-c
[4] Don't set default zone
Please enter your numeric choice: 1
Your profile default Compute zone has been set to 'ru-central1-a'.
vozerov@mba:~ $Ni opo, ilana naa rọrun - akọkọ o nilo lati gba ami-ami oauth lati ṣakoso awọsanma, yan awọsanma ati folda ti iwọ yoo lo.
Ti o ba ni awọn akọọlẹ pupọ tabi awọn folda laarin awọsanma kanna, o le ṣẹda awọn profaili afikun pẹlu awọn eto lọtọ nipasẹ profaili atunto yc ṣẹda ati yipada laarin wọn.
Ni afikun si awọn ọna ti o wa loke, ẹgbẹ Yandex.Cloud kowe ti o dara julọ fun iṣakoso awọn orisun awọsanma. Fun apakan mi, Mo pese ibi ipamọ git kan, nibiti Mo ṣe apejuwe gbogbo awọn orisun ti yoo ṣẹda bi apakan ti nkan naa - . A nifẹ si ẹka titunto si, jẹ ki a ṣe oniye rẹ ni agbegbe:
vozerov@mba:~ $ git clone https://github.com/rebrainme/yandex-cloud-events/ events
Cloning into 'events'...
remote: Enumerating objects: 100, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 100 (delta 37), reused 89 (delta 26), pack-reused 0
Receiving objects: 100% (100/100), 25.65 KiB | 168.00 KiB/s, done.
Resolving deltas: 100% (37/37), done.
vozerov@mba:~ $ cd events/terraform/Gbogbo awọn oniyipada akọkọ ti o lo ni terraform ni a kọ sinu faili main.tf. Lati bẹrẹ, ṣẹda ikọkọ.auto.tfvars faili ninu folda terraform pẹlu akoonu atẹle:
# Yandex Cloud Oauth token
yc_token = ""
# Yandex Cloud ID
yc_cloud_id = ""
# Yandex Cloud folder ID
yc_folder_id = ""
# Default Yandex Cloud Region
yc_region = "ru-central1-a"
# Cloudflare email
cf_email = ""
# Cloudflare token
cf_token = ""
# Cloudflare zone id
cf_zone_id = ""Gbogbo awọn oniyipada le ṣee mu lati inu atokọ atunto yc, nitori a ti tunto ohun elo console tẹlẹ. Mo gba ọ ni imọran lati ṣafikun ikọkọ.auto.tfvars lẹsẹkẹsẹ si .gitignore, ki o maṣe ṣe atẹjade data ikọkọ lairotẹlẹ.
Ni private.auto.tfvars a tun pato data lati Cloudflare - lati ṣẹda awọn igbasilẹ DNS ati aṣoju awọn iṣẹlẹ agbegbe akọkọ.kis.im si awọn olupin wa. Ti o ko ba fẹ lati lo Cloudflare, lẹhinna yọ ipilẹṣẹ ti olupese awọsanmaflare ni main.tf ati faili dns.tf, eyiti o jẹ iduro fun ṣiṣẹda awọn igbasilẹ dns pataki.
Ninu iṣẹ wa a yoo darapọ gbogbo awọn ọna mẹta - wiwo wẹẹbu, ohun elo console, ati terraform.
Awọn nẹtiwọki foju
Lati sọ ootọ, o le foju igbesẹ yii, nitori nigbati o ba ṣẹda awọsanma tuntun, iwọ yoo ni nẹtiwọọki lọtọ laifọwọyi ati awọn subnets 3 ti a ṣẹda - ọkan fun agbegbe wiwa kọọkan. Sugbon a yoo tun fẹ lati ṣe kan lọtọ nẹtiwọki fun ise agbese wa pẹlu awọn oniwe-ara adirẹsi. Aworan atọka gbogbogbo ti bii nẹtiwọọki naa ṣe n ṣiṣẹ ni Yandex.Cloud ni a fihan ni nọmba ti o wa ni isalẹ (ti a mu nitootọ lati )
Nitorinaa, o ṣẹda nẹtiwọọki ti o wọpọ laarin eyiti awọn orisun le ṣe ibasọrọ pẹlu ara wọn. Fun agbegbe wiwa kọọkan, subnet kan ti ṣẹda pẹlu adirẹsi tirẹ ati sopọ si nẹtiwọọki gbogbogbo. Bi abajade, gbogbo awọn orisun awọsanma ninu rẹ le ṣe ibaraẹnisọrọ, paapaa ti wọn ba wa ni awọn agbegbe wiwa ti o yatọ. Awọn orisun ti a ti sopọ si awọn nẹtiwọọki awọsanma oriṣiriṣi le rii ara wọn nikan nipasẹ awọn adirẹsi ita. Nipa ọna, bawo ni idan yii ṣe n ṣiṣẹ ninu, .
Ṣiṣẹda nẹtiwọki jẹ apejuwe ninu faili network.tf lati ibi ipamọ. Nibẹ ni a ṣẹda ọkan ti o wọpọ inu nẹtiwọọki ikọkọ ti o wọpọ ati so awọn subnet mẹta si rẹ ni awọn agbegbe wiwa ti o yatọ - ti abẹnu-a (172.16.1.0/24), ti abẹnu-b (172.16.2.0/24), ti abẹnu-c (172.16.3.0/24). ).
Bẹrẹ terraform ki o ṣẹda awọn nẹtiwọki:
vozerov@mba:~/events/terraform (master) $ terraform init
... skipped ..
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_vpc_subnet.internal-a -target yandex_vpc_subnet.internal-b -target yandex_vpc_subnet.internal-c
... skipped ...
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
yandex_vpc_network.internal: Creating...
yandex_vpc_network.internal: Creation complete after 3s [id=enp2g2rhile7gbqlbrkr]
yandex_vpc_subnet.internal-a: Creating...
yandex_vpc_subnet.internal-b: Creating...
yandex_vpc_subnet.internal-c: Creating...
yandex_vpc_subnet.internal-a: Creation complete after 6s [id=e9b1dad6mgoj2v4funog]
yandex_vpc_subnet.internal-b: Creation complete after 7s [id=e2liv5i4amu52p64ac9p]
yandex_vpc_subnet.internal-c: Still creating... [10s elapsed]
yandex_vpc_subnet.internal-c: Creation complete after 10s [id=b0c2qhsj2vranoc9vhcq]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.Nla! A ti ṣẹda nẹtiwọki wa ati pe o ti ṣetan lati ṣẹda awọn iṣẹ inu wa.
Ṣiṣẹda foju ero
Lati ṣe idanwo ohun elo, a yoo nilo lati ṣẹda awọn ẹrọ foju meji nikan - a yoo nilo akọkọ lati kọ ati ṣiṣẹ ohun elo, keji lati ṣiṣẹ kafka, eyiti a yoo lo lati tọju awọn ifiranṣẹ ti nwọle. Ati pe a yoo ṣẹda ẹrọ miiran nibiti a yoo tunto prometheus lati ṣe atẹle ohun elo naa.
Awọn ẹrọ foju naa yoo tunto ni lilo ohun ti o ṣeeṣe, nitorinaa ṣaaju ki o to bẹrẹ terraform, rii daju pe o ni ọkan ninu awọn ẹya tuntun ti o ṣeeṣe. Ati fi awọn ipa pataki sori ẹrọ pẹlu galaxy ti o ṣeeṣe:
vozerov@mba:~/events/terraform (master) $ cd ../ansible/
vozerov@mba:~/events/ansible (master) $ ansible-galaxy install -r requirements.yml
- cloudalchemy-prometheus (master) is already installed, skipping.
- cloudalchemy-grafana (master) is already installed, skipping.
- sansible.kafka (master) is already installed, skipping.
- sansible.zookeeper (master) is already installed, skipping.
- geerlingguy.docker (master) is already installed, skipping.
vozerov@mba:~/events/ansible (master) $Ninu folda ti o ṣee ṣe apẹẹrẹ wa .ansible.cfg iṣeto ni faili ti Mo lo. O le wa ni ọwọ.
Ṣaaju ṣiṣẹda awọn ẹrọ foju, rii daju pe o ni ṣiṣiṣẹ aṣoju ssh ati bọtini ssh ti a ṣafikun, bibẹẹkọ terraform kii yoo ni anfani lati sopọ si awọn ẹrọ ti a ṣẹda. Emi, dajudaju, pade kokoro kan ninu os x: . Lati ṣe idiwọ eyi lati ṣẹlẹ lẹẹkansi, ṣafikun oniyipada kekere kan si env ṣaaju ifilọlẹ Terraform:
vozerov@mba:~/events/terraform (master) $ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YESNinu folda pẹlu terraform a ṣẹda awọn orisun pataki:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_compute_instance.build -target yandex_compute_instance.monitoring -target yandex_compute_instance.kafka
yandex_vpc_network.internal: Refreshing state... [id=enp2g2rhile7gbqlbrkr]
data.yandex_compute_image.ubuntu_image: Refreshing state...
yandex_vpc_subnet.internal-a: Refreshing state... [id=e9b1dad6mgoj2v4funog]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
... skipped ...
Plan: 3 to add, 0 to change, 0 to destroy.
... skipped ...Ti ohun gbogbo ba pari ni aṣeyọri (ati pe o yẹ ki o jẹ), lẹhinna a yoo ni awọn ẹrọ foju mẹta:
- kọ - ẹrọ kan fun idanwo ati kọ ohun elo kan. Docker ti fi sori ẹrọ laifọwọyi nipasẹ Ansible.
- ibojuwo - ẹrọ ibojuwo - prometheus & grafana ti a fi sori rẹ. Wọle / boṣewa ọrọ igbaniwọle: abojuto / abojuto
- kafka jẹ ẹrọ kekere kan pẹlu kafka ti fi sori ẹrọ, wiwọle lori ibudo 9092.
Jẹ ki a rii daju pe gbogbo wọn wa ni aaye:
vozerov@mba:~/events (master) $ yc compute instance list
+----------------------+------------+---------------+---------+---------------+-------------+
| ID | NAME | ZONE ID | STATUS | EXTERNAL IP | INTERNAL IP |
+----------------------+------------+---------------+---------+---------------+-------------+
| fhm081u8bkbqf1pa5kgj | monitoring | ru-central1-a | RUNNING | 84.201.159.71 | 172.16.1.35 |
| fhmf37k03oobgu9jmd7p | kafka | ru-central1-a | RUNNING | 84.201.173.41 | 172.16.1.31 |
| fhmt9pl1i8sf7ga6flgp | build | ru-central1-a | RUNNING | 84.201.132.3 | 172.16.1.26 |
+----------------------+------------+---------------+---------+---------------+-------------+
Awọn orisun wa ni aye, ati lati ibi a le gba awọn adirẹsi IP wọn. Ni gbogbo ohun ti o tẹle Emi yoo lo awọn adirẹsi IP lati sopọ nipasẹ ssh ati idanwo ohun elo naa. Ti o ba ni akọọlẹ Cloudflare ti o sopọ si terraform, lero ọfẹ lati lo awọn orukọ DNS tuntun ti a ṣẹda.
Nipa ọna, nigba ṣiṣẹda ẹrọ foju, IP inu ati orukọ DNS inu ni a fun, nitorinaa o le wọle si awọn olupin laarin nẹtiwọọki nipasẹ orukọ:
ubuntu@build:~$ ping kafka.ru-central1.internal
PING kafka.ru-central1.internal (172.16.1.31) 56(84) bytes of data.
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=1 ttl=63 time=1.23 ms
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=2 ttl=63 time=0.625 ms
^C
--- kafka.ru-central1.internal ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.625/0.931/1.238/0.308 msEyi yoo wulo fun wa lati tọka si ohun elo ipari ipari pẹlu kafk.
Nto ohun elo
Nla, awọn olupin wa, ohun elo kan wa - gbogbo ohun ti o ku ni lati pejọ ati gbejade. Fun kikọ a yoo lo iṣẹ docker deede, ṣugbọn bi ibi ipamọ aworan a yoo lo iṣẹ kan lati Yandex - iforukọsilẹ eiyan. Sugbon akọkọ ohun akọkọ.
A daakọ ohun elo naa si ẹrọ kikọ, wọle nipasẹ ssh ati pejọ aworan naa:
vozerov@mba:~/events/terraform (master) $ cd ..
vozerov@mba:~/events (master) $ rsync -av app/ [email protected]:app/
... skipped ...
sent 3849 bytes received 70 bytes 7838.00 bytes/sec
total size is 3644 speedup is 0.93
vozerov@mba:~/events (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cd app
ubuntu@build:~/app$ sudo docker build -t app .
Sending build context to Docker daemon 6.144kB
Step 1/9 : FROM golang:latest AS build
... skipped ...
Successfully built 9760afd8ef65
Successfully tagged app:latestIdaji ogun naa ti pari - ni bayi a le ṣayẹwo iṣẹ ṣiṣe ti ohun elo wa nipa ifilọlẹ ati fifiranṣẹ si kafka:
ubuntu@build:~/app$ sudo docker run --name app -d -p 8080:8080 app /app/app -kafka=kafka.ru-central1.internal:9092</code>
С локальной машинки можно отправить тестовый event и посмотреть на ответ:
<code>vozerov@mba:~/events (master) $ curl -D - -s -X POST -d '{"key1":"data1"}' http://84.201.132.3:8080/post
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 13 Apr 2020 13:53:54 GMT
Content-Length: 41
{"status":"ok","partition":0,"Offset":0}
vozerov@mba:~/events (master) $Ohun elo naa dahun pẹlu aṣeyọri ti gbigbasilẹ ati afihan id ti ipin ati aiṣedeede ninu eyiti ifiranṣẹ naa wa. Gbogbo ohun ti o kù lati ṣe ni ṣẹda iforukọsilẹ ni Yandex.Cloud ati gbe aworan wa sibẹ (bii o ṣe le ṣe eyi nipa lilo awọn ila mẹta ni a ṣe apejuwe ninu faili registry.tf). Ṣẹda ibi ipamọ kan:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_container_registry.events
... skipped ...
Plan: 1 to add, 0 to change, 0 to destroy.
... skipped ...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Awọn ọna pupọ lo wa lati jẹri ni iforukọsilẹ eiyan - lilo ami iwẹ, ami ami iam, tabi bọtini akọọlẹ iṣẹ kan. Awọn alaye diẹ sii nipa awọn ọna wọnyi ni a le rii ninu iwe. . A yoo lo bọtini akọọlẹ iṣẹ, nitorinaa a ṣẹda akọọlẹ kan:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_iam_service_account.docker -target yandex_resourcemanager_folder_iam_binding.puller -target yandex_resourcemanager_folder_iam_binding.pusher
... skipped ...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.Bayi gbogbo ohun ti o ku ni lati ṣe bọtini fun rẹ:
vozerov@mba:~/events/terraform (master) $ yc iam key create --service-account-name docker -o key.json
id: ajej8a06kdfbehbrh91p
service_account_id: ajep6d38k895srp9osij
created_at: "2020-04-13T14:00:30Z"
key_algorithm: RSA_2048A gba alaye nipa id ti ibi ipamọ wa, gbe bọtini ati buwolu wọle:
vozerov@mba:~/events/terraform (master) $ scp key.json [email protected]:
key.json 100% 2392 215.1KB/s 00:00
vozerov@mba:~/events/terraform (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cat key.json | sudo docker login --username json_key --password-stdin cr.yandex
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
ubuntu@build:~$Lati gbe aworan naa sori iforukọsilẹ, a nilo ID iforukọsilẹ eiyan, a gba lati inu ohun elo yc:
vozerov@mba:~ $ yc container registry get events
id: crpdgj6c9umdhgaqjfmm
folder_id:
name: events
status: ACTIVE
created_at: "2020-04-13T13:56:41.914Z"Lẹhin iyẹn, a fi aami aami si aworan wa pẹlu orukọ tuntun ati gbejade:
ubuntu@build:~$ sudo docker tag app cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
ubuntu@build:~$ sudo docker push cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
The push refers to repository [cr.yandex/crpdgj6c9umdhgaqjfmm/events]
8c286e154c6e: Pushed
477c318b05cb: Pushed
beee9f30bc1f: Pushed
v1: digest: sha256:1dd5aaa9dbdde2f60d833be0bed1c352724be3ea3158bcac3cdee41d47c5e380 size: 946A le rii daju pe aworan ti kojọpọ ni aṣeyọri:
vozerov@mba:~/events/terraform (master) $ yc container repository list
+----------------------+-----------------------------+
| ID | NAME |
+----------------------+-----------------------------+
| crpe8mqtrgmuq07accvn | crpdgj6c9umdhgaqjfmm/events |
+----------------------+-----------------------------+Nipa ọna, ti o ba fi ohun elo yc sori ẹrọ Linux kan, o le lo aṣẹ naa
yc container registry configure-dockerlati tunto docker.
ipari
A ti ṣe ọpọlọpọ iṣẹ lile ati bi abajade:
- A wá soke pẹlu awọn faaji ti wa ojo iwaju iṣẹ.
- A kowe ohun elo kan ni golang ti o ṣe ilana ọgbọn iṣowo wa.
- A kó o si dà o sinu kan ikọkọ gba eiyan iforukọsilẹ.
Ni apakan atẹle, a yoo lọ si nkan ti o nifẹ - a yoo tu ohun elo wa silẹ sinu iṣelọpọ ati nikẹhin ṣe ifilọlẹ ẹru lori rẹ. Maṣe yipada!
Ohun elo yii wa ninu gbigbasilẹ fidio ti idanileko ṣiṣi REBRAIN & Yandex.Cloud: A gba awọn ibeere 10 fun iṣẹju kan lori Yandex Cloud -
Ti o ba nifẹ si wiwa si iru awọn iṣẹlẹ lori ayelujara ati bibeere awọn ibeere ni akoko gidi, sopọ si.
A yoo fẹ lati sọ ọpẹ pataki kan si Yandex.Cloud fun aye lati gbalejo iru iṣẹlẹ kan. Ọna asopọ si wọn -
Ti o ba nilo lati lọ si awọsanma tabi ni awọn ibeere nipa awọn amayederun rẹ, .
PS A ni awọn iṣayẹwo ọfẹ 2 fun oṣu kan, boya iṣẹ akanṣe rẹ yoo jẹ ọkan ninu wọn.
orisun: www.habr.com