Waad salaaman tihiin dhammaan, asxaabta!
* Maqaalkani wuxuu ku saleysan yahay aqoon-is-weydaarsiga furan ee REBRAIN & Yandex.Cloud, haddii aad doorbideyso inaad daawato fiidiyowga, waxaad ka heli kartaa xiriirkan -
Waxaan dhawaan helnay fursad aan isku dayno Yandex.Cloud si toos ah. Tan iyo markii aan rabnay inaan baarno waqti dheer oo adag, waxaan isla markiiba ka tagnay fikradda ah inaan bilowno blog WordPress fudud oo leh saldhig daruur - waxay ahayd mid caajis ah. Feker ka dib, waxaan go'aansanay in aan geyno wax la mid ah qaab dhismeedka adeegga wax soo saarka si loo helo loona falanqeeyo dhacdooyinka qaabka waqtiga dhabta ah ee dhow.
Waxaan si buuxda u hubaa in badi intarneedka (oo aan ahayn) meheradaha si uun u ururiyaan xog badan oo ku saabsan isticmaalkooda iyo falalkooda. Ugu yaraan, tani waxay lagama maarmaan u tahay samaynta go'aannada qaarkood - tusaale ahaan, haddii aad maamusho ciyaarta internetka, waxaad eegi kartaa tirakoobka heerka isticmaalayaashu inta badan ku xayiran yihiin oo tirtiraan alaabtaada. Ama sababta isticmaalayaashu uga tagaan goobtaada iyagoon wax iibsanayn (hello, Yandex.Metrica).
Marka, sheekadeena: sida aan ugu qornay arji gudaha golang, tijaabiyey kafka vs rabbitmq vs yqs, ku qor xogta qulqulka gudaha kooxda Clickhouse oo aan sawirnay xogta iyadoo la adeegsanayo yandex datalens. Dabiici ahaan, waxaas oo dhan waxaa lagu raaxaystay kaabayaasha kaabayaasha qaab docker, terraform, gitlab ci iyo, dabcan, prometheus. Aan tagno!
Waxaan jeclaan lahaa inaan isla markiiba sameeyo boos celin ah inaanan awoodi doonin inaan wax walba ku habeyno hal fadhi - tan waxaan u baahan doonaa maqaallo dhowr ah oo taxane ah. Wax yar oo ku saabsan qaabka:
Qaybta 1 (waad akhrinaysaa). Waxaan go'aansan doonaa sifooyinka iyo qaab-dhismeedka xalka, sidoo kale waxaan ku qori doonaa codsi golang.
Qaybta 2. Waxaan u sii deynaa codsigeena wax soo saar, ka dhig mid la miisaami karo oo tijaabi culeyska.
Qaybta 3. Aan isku dayno inaan ogaano sababta aan ugu baahanahay inaan ku kaydino fariimaha kaydka ee aan lagu kaydin faylasha, sidoo kale aan is barbar dhigno adeega kafka, rabbitmq iyo yandex.
Qaybta 4 Waxaan geyn doonaa kooxda Clickhouse, qori doona adeegga qulqulka si aan xogta uga soo wareejinno kaydka halkaas, oo aan u dejino muuqaal-muuqaalka datalens.
Qaybta 5 Aan u keenno kaabayaasha oo dhan qaab ku habboon - samee ci/cd addoo isticmaalaya gitlab ci, ku xidhid la socodka iyo helitaanka adeega anagoo adeegsanayna prometheus iyo qunsulka.
Π’Π
Marka hore, aan dejino shuruudaha tixraaca - waxa dhabta ah ee aan rabno inaan helno natiijada.
- Waxaan rabnaa inaan yeelano barta ugu dambeysa sida events.kis.im (kis.im waa goobta tijaabada ah ee aan u isticmaali doono maqaalada oo dhan), kaas oo ah inuu helo munaasabado isticmaalaya HTTPS.
- Dhacdooyinka waa json fudud sida: {"dhacdo":"view", "os": "linux", "browser": "chrome"}. Marxaladda ugu dambeysa waxaan ku dari doonaa wax yar oo beero ah, laakiin tani ma ciyaari doonto door weyn. Haddii aad rabto, waxaad u wareegi kartaa protobuf.
- Adeeggu waa inuu awood u leeyahay inuu socodsiiyo 10 dhacdo ilbiriqsikii.
- Waa in ay suurtowdo in si siman loo cabbiro iyada oo si fudud loogu dari doono xaalado cusub xalkeenna. Way fiicnaan doontaa haddii aan u wareejin karno qaybta hore ee meelo kala duwan si loo yareeyo daahitaanka codsiyada macmiilka.
- Dulqaadka qaladka. Xalku waa inuu ahaado mid deggan oo awood u leh inuu ka badbaado dhicitaanka qaybo kasta (ilaa tiro go'an, dabcan).
naqshadaha
Guud ahaan, hawsha noocan ah, naqshadaha qadiimiga ah ayaa muddo dheer la allifay kuwaas oo u oggolaanaya miisaan wax ku ool ah. Sawirku wuxuu muujinayaa tusaale xalkeena.
Haddaba maxaan haysanaa:
1. Dhinaca bidix waxaa ku jira qalabkeena kuwaas oo soo saara dhacdooyin kala duwan, ha ahaato ciyaartoyda dhamaystiraya heerka alaabta lagu ciyaaro ee casriga ah ama abuurista dalabka dukaanka internetka iyada oo loo marayo browserka caadiga ah. Dhacdo, sida lagu qeexay qeexitaanka, waa json fudud oo loo soo diro dhammaadkayaga - events.kis.im.
2.Labada server ee hore waa kuwa fudud oo dheeli tiran, hawlahooda ugu muhiimsan waa:
- Si joogto ah u ahaado Si tan loo sameeyo, waxaad isticmaali kartaa, tusaale ahaan, keepalive, kaas oo u bedeli doona IP-ga dalwaddii inta u dhaxaysa noodhadhka haddii ay jiraan dhibaatooyin.
- Jooji TLS. Haa, waan ku joojin doonaa TLS iyaga. Marka hore, si xalkeenu uu u hoggaansamo qeexitaannada farsamada, iyo marka labaad, si loo yareeyo culeyska sameynta xiriir qarsoodi ah oo ka yimaada server-yadayada dambe.
- Isku dheelitir codsiyada soo galaya server-yada dambe ee la heli karo. Ereyga muhiimka ah ee halkan waa la heli karaa. Iyada oo ku saleysan tan, waxaan nimid fahamka in miisaan-bixiyeyaasha culeyska ay tahay inay awoodaan inay la socdaan server-yadayada codsiyada oo ay joojiyaan isu-dheellitirka taraafikada qanjidhada aan dhicin.
3. Dheelayaasha ka dib, waxaan haynaa adeegayaal codsi ah oo ku shaqeeya codsi cadaalad ah. Waa inay awood u yeelataa inay aqbasho codsiyada soo gelaya HTTP, xaqiijiso json la soo diray oo ay xogta geliso kayd.
4. Jaantusku waxa uu muujinayaa kafka sida bakhaar, inkastoo, dabcan, adeegyo kale oo la mid ah ayaa loo isticmaali karaa heerkan. Waxaan is barbar dhigi doonaa Kafka, bakayle iyo yqs maqaalka saddexaad.
5. Meesha ugu sarraysa ee qaab-dhismeedkeenu waa Clickhouse - xog-ururin tiirar ah oo kuu oggolaanaysa inaad kaydiso oo aad socodsiiso xog aad u badan. Heerkan, waxaan u baahanahay inaan ka wareejino xogta kaydiyaha ilaa nidaamka kaydinta laftiisa (wax badan oo ku saabsan qodobka 4).
Naqshadeyntani waxay noo ogolaaneysaa inaan cabbirno lakab kasta si madax bannaan oo siman. Adeegayaasha dhabarka ah ma xamili karaan - aynu ku darno hal shay oo kale - ka dib oo dhan, waa codsiyo aan waddan lahayn, sidaa darteed, tan si toos ah ayaa loo samayn karaa. Bakhaarka-qaabka Kafka ma shaqeeyo-aan ku darno adeegayaal badan oo aan u wareejino qaybo ka mid ah mawduucayada iyaga. Clickhouse ma xamili karo - macquul maaha :) Dhab ahaantii, waxaan sidoo kale isku xiri doonaa server-yada oo aan burburin doonaa xogta.
Jid ahaan, haddii aad rabto inaad hirgeliso qaybta ikhtiyaariga ah ee qeexitaannada farsamada iyo cabbirka geolocations kala duwan, markaa ma jiraan wax ka fudud:
Goob kasta waxaan geynnaa culeyska culeyska oo leh codsi iyo kafka. Guud ahaan, adeegayaasha codsiyada 2, 3 kafka qanjidhada iyo dheellitirka daruuraha, tusaale ahaan, Cloudflare, ayaa ku filan, kuwaas oo hubin doona helitaanka qanjidhada codsiyada iyo codsiyada dheelitirka geolocation ee ku salaysan isha macmiilka ee cinwaanka IP. Haddaba, xogta uu soo diray macmiil Maraykan ah ayaa ku soo degi doonta server-yada Maraykanka. Xogta Afrikana waa Afrikaan.
Markaa wax walba waa sahlan yihiin - waxaan isticmaalnaa qalabka muraayadda ee Kafka set oo nuqul ka samee dhammaan xogta laga helo goobaha oo dhan ilaa xarunteena xogta dhexe ee ku taal Ruushka. Gudaha, waxaanu kala saarnay xogta oo aanu ku duubnay Clickhouse si loo arko soo socota.
Markaa, waxaan kala soocnay qaab-dhismeedka - aan bilowno ruxitaanka Yandex.Cloud!
Qorista codsi
Cloud ka hor, wali waa inaad xoogaa dulqaadataa oo aad qortaa adeeg fudud oo aad ku socodsiiso dhacdooyinka soo socda. Waxaan isticmaali doonaa golang sababtoo ah waxay si fiican isu caddeysay inay tahay luqad qorista codsiyada shabakadaha.
Ka dib markaan qaadano saacad (laga yaabee dhowr saacadood), waxaan helnaa wax sidan oo kale ah: .
Maxay yihiin qodobbada ugu muhiimsan ee aan jeclaan lahaa inaan halkan ku xuso:
1. Markaad bilowdo codsiga, waxaad cayimi kartaa laba calan. Mid ayaa mas'uul ka ah dekedda aan ku dhageysan doono codsiyada http ee soo socda (-addr). Midda labaad waa ciwaanka kafka ee aan ku duubi doono dhacdooyinkayaga (-kafka):
addr = flag.String("addr", ":8080", "TCP address to listen to")
kafka = flag.String("kafka", "127.0.0.1:9092", "Kafka endpointsβ)2. Codsigu wuxuu isticmaalaa maktabadda sarama () inuu fariimo u diro kooxda kafka. Waxaan isla markiiba dejinay dejinta loogu talagalay xawaaraha socodsiinta ugu sarreeya:
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForLocal
config.Producer.Compression = sarama.CompressionSnappy
config.Producer.Return.Successes = true3. Codsigayagu waxa kale oo uu leeyahay macmiilka prometheus-ku-dhisan, kaas oo ururiya qiyaaso kala duwan, sida:
- tirada codsiyada codsigayaga;
- tirada khaladaadka marka la fulinayo codsiga (aan suurtagal ahayn in la akhriyo codsiga boostada, json jabay, aan suurtagal ahayn in loo qoro Kafka);
- wakhtiga habaynta hal codsi oo ka yimi macmiilka, oo ay ku jirto wakhtiga fariinta loo qorayo Kafka.
4. Saddexda qodob ee uu codsigayagu ka shaqeeyo:
- /status - si fudud u soo celi ok si aad u muujiso inaan nool nahay. Inkasta oo aad ku dari karto jeegaga qaar, sida helitaanka kooxda Kafka.
- / metrics - marka loo eego url-kan, macmiilka prometheus ayaa soo celin doona cabbirada uu ururiyay.
- Boostada waa barta ugu weyn ee codsiyada POST ee json gudaha lagu soo diri doono. Codsigayagu waxa uu hubinayaa saxsanaanta json iyo haddii wax walba ay sax yihiin, waxa ay u qortaa xogta kooxda Kafka.
Waxaan samayn doonaa boos celin ah in koodku aanu ahayn mid qumman - waa la dhamaystiri karaa (oo waa in!) Tusaale ahaan, waxaad joojin kartaa isticmaalka net/http oo ku dhex jira oo u beddelo xawaaraha degdegga ahhttp. Ama waxaad heli kartaa wakhti habbaynta iyo agabka cpu adiga oo u raraynaya json hubinta ansaxnimada marxalad dambe - marka xogta laga wareejiyo kaydka loona wareejiyo kooxda riixa.
Marka laga soo tago dhinaca horumarinta ee arrinta, waxaanu isla markiiba ka fikirnay kaabayaashayaga mustaqbalka waxaanan go'aansanay inaan ku dirno codsigeena iyada oo loo marayo docker. Dockerfile-ka ugu dambeeya ee lagu dhisayo arjiga waa . Guud ahaan, waa wax fudud, qodobka kaliya ee aan jeclaan lahaa in aan fiiro gaar ah u yeesho waa kulanka multistage, kaas oo noo ogolaanaya inaan hoos u dhigno sawirka kama dambaysta ah ee weelkayaga.
Tallaabooyinka ugu horreeya ee daruuraha
Marka hore iska diiwaan geli . Ka dib marka la buuxiyo dhammaan goobaha lagama maarmaanka ah, waxaa nala abuuri doonaa xisaab waxaana nala siin doonaa deeq lacag gaar ah, taas oo loo isticmaali karo in lagu tijaabiyo adeegyada daruuraha. Haddii aad rabto inaad ku celiso dhammaan tillaabooyinka maqaalkeena, deeqdani waa inay kugu filnaataa.
Diiwaangelinta ka dib, daruur gaar ah iyo tusaha caadiga ah ayaa laguu abuuri doonaa, kaas oo aad ka bilaabi karto abuurista ilaha daruuraha. Guud ahaan, Yandex.Cloud, xiriirka kheyraadku wuxuu u eg yahay sidan:
Waxaad u abuuri kartaa dhowr daruur hal akoon. Iyo gudaha daruuraha, samee hagayaal kala duwan oo loogu talagalay mashruucyo shirkadeed oo kala duwan. Waxaad wax badan oo arrintan ku saabsan ka akhriyi kartaa dukumeentiyada - . By habka, waxaan inta badan tixraaci doonaa hoos ee qoraalka. Markii aan aasaasey dhammaan kaabayaasha meel eber, dukumeentiyadu waxay iga caawiyeen in ka badan hal mar, markaa waxaan kugula talinayaa inaad barato.
Si aad u maamusho daruuraha, waxaad isticmaali kartaa labada interneedka webka iyo utility console - yc. Rakibaadda waxaa lagu sameeyaa hal amar (loogu talagalay Linux iyo Mac Os):
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bashHaddii khabiirkaaga amniga gudaha uu ka xanaaqo socodsiinta qoraallada internetka, markaa, marka hore, waad furi kartaa qoraalka oo akhri, marka labaadna, waxaan ku maamulnaa isticmaaleheena - iyada oo aan lahayn xuquuqaha xididka.
Haddii aad rabto inaad ku rakibto macmiilka Windows, waxaad isticmaali kartaa tilmaamaha ka dibna fuliya yc initin si buuxda loo habeeyo:
vozerov@mba:~ $ yc init
Welcome! This command will take you through the configuration process.
Please go to https://oauth.yandex.ru/authorize?response_type=token&client_id= in order to obtain OAuth token.
Please enter OAuth token:
Please select cloud to use:
[1] cloud-b1gv67ihgfu3bp (id = b1gv67ihgfu3bpt24o0q)
[2] fevlake-cloud (id = b1g6bvup3toribomnh30)
Please enter your numeric choice: 2
Your current cloud has been set to 'fevlake-cloud' (id = b1g6bvup3toribomnh30).
Please choose folder to use:
[1] default (id = b1g5r6h11knotfr8vjp7)
[2] Create a new folder
Please enter your numeric choice: 1
Your current folder has been set to 'default' (id = b1g5r6h11knotfr8vjp7).
Do you want to configure a default Compute zone? [Y/n]
Which zone do you want to use as a profile default?
[1] ru-central1-a
[2] ru-central1-b
[3] ru-central1-c
[4] Don't set default zone
Please enter your numeric choice: 1
Your profile default Compute zone has been set to 'ru-central1-a'.
vozerov@mba:~ $Mabda 'ahaan, nidaamku waa sahlan yahay - marka hore waxaad u baahan tahay inaad hesho calaamadda dhaarta si aad u maamusho daruuraha, dooro daruuraha iyo faylka aad isticmaali doonto.
Haddii aad leedahay dhowr akoon ama fayl gudaha daruur isku mid ah, waxaad samayn kartaa profiles dheeraad ah oo leh goobo gooni ah adoo isticmaalaya yc config profile samee oo u kala beddel.
Marka lagu daro hababka kor ku xusan, kooxda Yandex.Cloud ayaa qoray mid aad u wanaagsan si loo maareeyo kheyraadka daruuraha. Dhankayga, waxaan diyaariyey git repository, halkaas oo aan ku tilmaamay dhammaan agabyada loo abuuri doono qayb ka mid ah maqaalka - . Waxaan xiisayneynaa laanta master-ka, aan ku soo koobno ββgudaha:
vozerov@mba:~ $ git clone https://github.com/rebrainme/yandex-cloud-events/ events
Cloning into 'events'...
remote: Enumerating objects: 100, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 100 (delta 37), reused 89 (delta 26), pack-reused 0
Receiving objects: 100% (100/100), 25.65 KiB | 168.00 KiB/s, done.
Resolving deltas: 100% (37/37), done.
vozerov@mba:~ $ cd events/terraform/Dhammaan doorsoomayaasha ugu muhiimsan ee lagu isticmaalo terraform waxay ku qoran yihiin faylka main.tf. Si aad u bilowdo, ku samee fayl private.auto.tfvars galka terraformka oo wata waxa soo socda:
# Yandex Cloud Oauth token
yc_token = ""
# Yandex Cloud ID
yc_cloud_id = ""
# Yandex Cloud folder ID
yc_folder_id = ""
# Default Yandex Cloud Region
yc_region = "ru-central1-a"
# Cloudflare email
cf_email = ""
# Cloudflare token
cf_token = ""
# Cloudflare zone id
cf_zone_id = ""Dhammaan doorsoomayaasha waxaa laga soo saari karaa liiska qaabeynta yc, maadaama aan horey u habeynay utility console-ka. Waxaan kugula talinayaa inaad si degdeg ah ugu darto private.auto.tfvars .gitignore, si aanad si lama filaan ah u daabacin xogta gaarka ah.
Private.auto.tfvars waxaan sidoo kale qeexnay xogta Cloudflare - si aan u abuurno diiwaannada DNS iyo wakiilka dhacdooyinka domain ee ugu muhiimsan.kis.im adeegayaashayada. Haddii aadan rabin inaad isticmaasho Cloudflare, ka dib ka saar bilawga bixiyaha Cloudflare ee main.tf iyo faylka dns.tf, kaas oo mas'uul ka ah abuurista diiwaannada DNS ee lagama maarmaanka ah.
Shaqadeena waxaan isku dari doonaa dhammaan seddexda qaab - interface-ka shabakadda, utility console, iyo terraform.
Shabakado muuqaal ah
Si daacad ah, waxaad ka boodi kartaa tallaabadan, maadaama markaad abuurto daruur cusub, waxaad si toos ah u yeelan doontaa shabakad gaar ah iyo 3 subnets ayaa la sameeyay - mid ka mid ah aag kasta oo la heli karo. Laakiin waxaan wali jeclaan lahayn inaan u samayno shabakad gaar ah mashruucayaga oo ciwaan u gaar ah leh. Jaantuska guud ee sida shabakadu ugu shaqeyso Yandex.Cloud ayaa lagu muujiyay sawirka hoose (si daacad ah ayaa laga soo qaatay )
Markaa, waxaad abuureysaa shabakad guud oo kheyraadku midba midka kale kula xiriiri karo. Aag kasta oo la heli karo, subnet ayaa loo sameeyay ciwaan u gaar ah oo ku xidhan shabakada guud. Natiijo ahaan, dhammaan ilaha daruuriga ah ee ku jira waa ay wada xiriiri karaan, xitaa haddii ay ku sugan yihiin meelo kala duwan oo la heli karo. Khayraadka ku xidhan shabakadaha daruuriga ah waxay isku arki karaan kaliya ciwaannada dibadda. By the way, sidee sixirkani u shaqeeyaa gudaha, .
Abuuritaanka shabakada waxa lagu sifeeyay faylka network.tf ee kaydka Halkaas waxaan ku abuureynaa hal shabakad gaar ah oo gudaha ah oo aan ku xirno saddex shabakadood oo hoose oo kala duwan oo la heli karo - gudaha-a (172.16.1.0/24), gudaha-b (172.16.2.0/24), gudaha-c (172.16.3.0/24). ).
Bilow terraform oo abuur shabakado:
vozerov@mba:~/events/terraform (master) $ terraform init
... skipped ..
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_vpc_subnet.internal-a -target yandex_vpc_subnet.internal-b -target yandex_vpc_subnet.internal-c
... skipped ...
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
yandex_vpc_network.internal: Creating...
yandex_vpc_network.internal: Creation complete after 3s [id=enp2g2rhile7gbqlbrkr]
yandex_vpc_subnet.internal-a: Creating...
yandex_vpc_subnet.internal-b: Creating...
yandex_vpc_subnet.internal-c: Creating...
yandex_vpc_subnet.internal-a: Creation complete after 6s [id=e9b1dad6mgoj2v4funog]
yandex_vpc_subnet.internal-b: Creation complete after 7s [id=e2liv5i4amu52p64ac9p]
yandex_vpc_subnet.internal-c: Still creating... [10s elapsed]
yandex_vpc_subnet.internal-c: Creation complete after 10s [id=b0c2qhsj2vranoc9vhcq]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.Wayn! Waxaan abuurnay shabakadeena oo hadda waxaan diyaar u nahay inaan abuurno adeegyadayada gudaha.
Abuuritaanka mashiinnada casriga ah
Si loo tijaabiyo arjiga, waxaan u baahan doonaa oo kaliya inaan abuurno laba mashiin oo farsamaysan - waxaan u baahan doonaa kan ugu horreeya si aan u dhisno oo u socodsiino codsiga, kan labaad si uu u socodsiiyo kafka, kaas oo aan u isticmaali doono si loo kaydiyo fariimaha soo socda. Oo waxaan abuuri doonaa mishiin kale halkaas oo aan ku habayn doono prometheus si ay ula socdaan codsiga.
Mashiinnada casriga ah waxaa loo habeyn doonaa iyadoo la adeegsanayo wax macquul ah, markaa ka hor intaadan bilaabin terraform, hubi inaad haysato mid ka mid ah noocyadii ugu dambeeyay ee suurtogalka ah. Oo ku rakib doorarka lagama maarmaanka u ah galaxyada la awoodi karo:
vozerov@mba:~/events/terraform (master) $ cd ../ansible/
vozerov@mba:~/events/ansible (master) $ ansible-galaxy install -r requirements.yml
- cloudalchemy-prometheus (master) is already installed, skipping.
- cloudalchemy-grafana (master) is already installed, skipping.
- sansible.kafka (master) is already installed, skipping.
- sansible.zookeeper (master) is already installed, skipping.
- geerlingguy.docker (master) is already installed, skipping.
vozerov@mba:~/events/ansible (master) $Gudaha gal-galka macquulka ah waxaa ku jira tusaale .ansible.cfg qaabeynta faylka oo aan isticmaalo. Waxaa laga yaabaa inay ku anfacdo.
Kahor intaadan abuurin mishiinada farsamada gacanta, hubi inaad haysatid wakiilka ssh oo ordaya iyo furaha ssh lagu daray, haddii kale terraform ma awoodi doono inuu ku xidhmo mishiinnada la abuuray. Aniga, dabcan, waxaan kala kulmay bug os x: . Si looga hortago in tani mar kale dhacdo, ku dar doorsoome yar env ka hor inta aanad bilaabin Terraform:
vozerov@mba:~/events/terraform (master) $ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YESGalka leh terraform waxaan ku abuureynaa agabyada lagama maarmaanka ah:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_compute_instance.build -target yandex_compute_instance.monitoring -target yandex_compute_instance.kafka
yandex_vpc_network.internal: Refreshing state... [id=enp2g2rhile7gbqlbrkr]
data.yandex_compute_image.ubuntu_image: Refreshing state...
yandex_vpc_subnet.internal-a: Refreshing state... [id=e9b1dad6mgoj2v4funog]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
... skipped ...
Plan: 3 to add, 0 to change, 0 to destroy.
... skipped ...Haddii wax waliba ay si guul leh ku dhamaadeen (oo waa inay ahaataa), markaa waxaan lahaan doonaa seddex mashiinada farsamada:
- dhis - mishiin lagu tijaabiyo laguna dhiso codsi. Docker waxaa si toos ah u rakibay macquul
- la socodka - mashiinka la socodka - prometheus & grafana oo lagu rakibay. Login / password standard: admin / admin
- kafka waa mishiin yar oo kafka lagu rakibay, laga heli karo dekedda 9092.
Aan hubino inay dhamaantood meesha yaaleen:
vozerov@mba:~/events (master) $ yc compute instance list
+----------------------+------------+---------------+---------+---------------+-------------+
| ID | NAME | ZONE ID | STATUS | EXTERNAL IP | INTERNAL IP |
+----------------------+------------+---------------+---------+---------------+-------------+
| fhm081u8bkbqf1pa5kgj | monitoring | ru-central1-a | RUNNING | 84.201.159.71 | 172.16.1.35 |
| fhmf37k03oobgu9jmd7p | kafka | ru-central1-a | RUNNING | 84.201.173.41 | 172.16.1.31 |
| fhmt9pl1i8sf7ga6flgp | build | ru-central1-a | RUNNING | 84.201.132.3 | 172.16.1.26 |
+----------------------+------------+---------------+---------+---------------+-------------+
Khayraadka ayaa jira, halkanna waxaan ka heli karnaa cinwaanadooda IP-ga. Inta soo socota waxaan isticmaali doonaa ciwaanada IP si aan ugu xidho ssh oo aan u tijaabiyo codsiga. Haddii aad leedahay koonto Cloudflare ah oo ku xidhan terraform, waxaad xor u tahay inaad isticmaasho magacyada DNS cusub ee la sameeyay.
Jid ahaan, marka la abuurayo mashiinka farsamada, IP gudaha iyo magaca DNS gudaha ayaa la bixiyaa, si aad u geli karto server-yada shabakadda magaca:
ubuntu@build:~$ ping kafka.ru-central1.internal
PING kafka.ru-central1.internal (172.16.1.31) 56(84) bytes of data.
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=1 ttl=63 time=1.23 ms
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=2 ttl=63 time=0.625 ms
^C
--- kafka.ru-central1.internal ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.625/0.931/1.238/0.308 msTani waxay noo noqon doontaa faa'iido inaan ku muujino codsiga barta dhamaadka ee kafk.
Ururinta codsiga
Way fiican tahay, waxaa jira server, waxaa jira codsi - waxa hadhay oo dhan waa in la ururiyo oo la daabaco. Dhismaha waxaan u isticmaali doonaa dhismaha caadiga ah ee docker, laakiin sida kaydinta sawirka waxaan isticmaali doonaa adeegga Yandex - diiwaanka weelka. Laakiin marka hore wax walba.
Waxaan ku koobiyeynaa codsiga mashiinka wax lagu dhisayo, gal ssh oo soo aruuri sawirka:
vozerov@mba:~/events/terraform (master) $ cd ..
vozerov@mba:~/events (master) $ rsync -av app/ [email protected]:app/
... skipped ...
sent 3849 bytes received 70 bytes 7838.00 bytes/sec
total size is 3644 speedup is 0.93
vozerov@mba:~/events (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cd app
ubuntu@build:~/app$ sudo docker build -t app .
Sending build context to Docker daemon 6.144kB
Step 1/9 : FROM golang:latest AS build
... skipped ...
Successfully built 9760afd8ef65
Successfully tagged app:latestDagaalka badhkii waa la dhammeeyey - hadda waxaan hubin karnaa shaqeynta codsigeyga annagoo u dirnayna kafka:
ubuntu@build:~/app$ sudo docker run --name app -d -p 8080:8080 app /app/app -kafka=kafka.ru-central1.internal:9092</code>
Π‘ Π»ΠΎΠΊΠ°Π»ΡΠ½ΠΎΠΉ ΠΌΠ°ΡΠΈΠ½ΠΊΠΈ ΠΌΠΎΠΆΠ½ΠΎ ΠΎΡΠΏΡΠ°Π²ΠΈΡΡ ΡΠ΅ΡΡΠΎΠ²ΡΠΉ event ΠΈ ΠΏΠΎΡΠΌΠΎΡΡΠ΅ΡΡ Π½Π° ΠΎΡΠ²Π΅Ρ:
<code>vozerov@mba:~/events (master) $ curl -D - -s -X POST -d '{"key1":"data1"}' http://84.201.132.3:8080/post
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 13 Apr 2020 13:53:54 GMT
Content-Length: 41
{"status":"ok","partition":0,"Offset":0}
vozerov@mba:~/events (master) $Codsigu waxa uu kaga jawaabay guusha duubista oo tilmaamaysa id ee qaybinta iyo ka-goynta farriinta lagu soo daray. Waxa kaliya ee hadhay in la sameeyo waa in la sameeyo diiwaan gudaha Yandex.Cloud oo ku dheji sawirkayaga halkaas (sida loo sameeyo tan iyadoo la adeegsanayo saddex xariiq ayaa lagu qeexay faylka registry.tf). Abuur kayd:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_container_registry.events
... skipped ...
Plan: 1 to add, 0 to change, 0 to destroy.
... skipped ...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Waxaa jira dhowr siyaabood oo lagu xaqiijin karo diiwaanka weelka - iyadoo la adeegsanayo calaamadda dhaarta, iam token, ama furaha akoonka adeegga. Faahfaahin dheeraad ah oo ku saabsan hababkan ayaa laga heli karaa dukumeentiyada. . Waxaan isticmaali doonaa furaha koontada adeegga, si aan u abuurno akoon:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_iam_service_account.docker -target yandex_resourcemanager_folder_iam_binding.puller -target yandex_resourcemanager_folder_iam_binding.pusher
... skipped ...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.Hadda waxa hadhay oo dhan waa inaad furaha u samayso:
vozerov@mba:~/events/terraform (master) $ yc iam key create --service-account-name docker -o key.json
id: ajej8a06kdfbehbrh91p
service_account_id: ajep6d38k895srp9osij
created_at: "2020-04-13T14:00:30Z"
key_algorithm: RSA_2048Waxaan helnaa macluumaadka ku saabsan aqoonsiga kaydintayada, wareejinta furaha oo gal:
vozerov@mba:~/events/terraform (master) $ scp key.json [email protected]:
key.json 100% 2392 215.1KB/s 00:00
vozerov@mba:~/events/terraform (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cat key.json | sudo docker login --username json_key --password-stdin cr.yandex
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
ubuntu@build:~$Si aad sawirka ugu dhejiso diiwaanka, waxaan u baahanahay aqoonsiga diiwaanka weelka, waxaan ka soo qaadanaa yc utility:
vozerov@mba:~ $ yc container registry get events
id: crpdgj6c9umdhgaqjfmm
folder_id:
name: events
status: ACTIVE
created_at: "2020-04-13T13:56:41.914Z"Intaa ka dib, waxaan ku dhejineynaa sawirkayaga magac cusub oo soo gal:
ubuntu@build:~$ sudo docker tag app cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
ubuntu@build:~$ sudo docker push cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
The push refers to repository [cr.yandex/crpdgj6c9umdhgaqjfmm/events]
8c286e154c6e: Pushed
477c318b05cb: Pushed
beee9f30bc1f: Pushed
v1: digest: sha256:1dd5aaa9dbdde2f60d833be0bed1c352724be3ea3158bcac3cdee41d47c5e380 size: 946Waxaan xaqiijin karnaa in sawirka si guul leh loo raray:
vozerov@mba:~/events/terraform (master) $ yc container repository list
+----------------------+-----------------------------+
| ID | NAME |
+----------------------+-----------------------------+
| crpe8mqtrgmuq07accvn | crpdgj6c9umdhgaqjfmm/events |
+----------------------+-----------------------------+Jid ahaan, haddii aad ku rakibto utility yc mashiinka Linux, waxaad isticmaali kartaa amarka
yc container registry configure-dockersi loo habeeyo docker.
gunaanad
Waxaan qabanay shaqo aad u adag, waxayna taasi keentay:
- Waxaan la nimid qaab dhismeedka adeegeena mustaqbalka.
- Waxaanu ku qornay arji gudaha golang kaasoo fulinaya caqli-galnimadayada ganacsi.
- Waanu soo ururinay oo ku shubnay kaydka haamaha gaarka ah.
Qaybta soo socota, waxaan u gudbi doonaa waxyaabaha xiisaha leh - waxaan u sii deyn doonaa codsigeena wax soo saarka ugu dambeyntiina waxaan bilaabi doonaa culeyska. Ha badalin!
Qalabkani wuxuu ku jiraa duubista fiidiyowga ee aqoon-is-weydaarsiga furan REBRAIN & Yandex.Cloud: Waxaan aqbalnaa 10 codsi ilbiriqsi kasta Yandex Cloud -
Haddii aad xiisaynayso inaad ka qaybgasho dhacdooyinkan oo kale onlayn oo aad su'aalo waydiiso wakhtiga dhabta ah, ku xidhnow.
Waxaan jeclaan lahayn inaan si gaar ah ugu mahadcelinno Yandex.Cloud fursadda lagu martigelinayo dhacdadan oo kale. Ku xidh iyaga -
Haddii aad u baahan tahay inaad u guurto daruuraha ama aad wax su'aalo ah ka qabto kaabayaashaaga, .
PS Waxaanu haynaa 2 xisaab xidh oo bilaash ah bishii, laga yaabee in mashruucaagu uu ka mid noqdo.
Source: www.habr.com