Sannu kowa da kowa, abokai!
* Wannan labarin ya dogara ne akan bude bitar REBRAIN & Yandex.Cloud, idan kun fi son kallon bidiyon, zaku iya samunsa ta wannan hanyar haɗin gwiwar -
Kwanan nan mun sami damar gwada Yandex.Cloud live. Tun da muna son yin bincike mai tsawo da wuya, nan da nan mun watsar da ra'ayin ƙaddamar da sauƙi na WordPress blog tare da tushen girgije - ya kasance mai ban sha'awa. Bayan wasu tunani, mun yanke shawarar tura wani abu mai kama da tsarin gine-ginen sabis na samarwa don karɓa da nazarin abubuwan da suka faru a cikin yanayin kusa.
Na tabbata cewa galibin kasuwancin kan layi (kuma ba kawai) kasuwancin ko ta yaya suke tattara tudun bayanai game da masu amfani da su da ayyukansu ba. Aƙalla, wannan ya zama dole don yanke wasu shawarwari - alal misali, idan kuna sarrafa wasan kan layi, zaku iya duba kididdigar da masu amfani da matakin suka fi makale su goge abin wasan ku. Ko me yasa masu amfani ke barin rukunin yanar gizon ku ba tare da siyan komai ba (sannu, Yandex.Metrica).
Don haka, labarinmu: yadda muka rubuta aikace-aikace a cikin golang, gwada kafka vs rabbitmq vs yqs, rubuta bayanan yawo a cikin gungu na Clickhouse kuma muka hango bayanan ta amfani da yandex datalens. A zahiri, duk wannan an haɗa shi da abubuwan jin daɗin abubuwan more rayuwa ta hanyar docker, terraform, gitlab ci kuma, ba shakka, prometheus. Mu tafi!
Ina so in yi ajiyar wuri nan da nan cewa ba za mu iya saita komai a cikin zama ɗaya ba - don wannan muna buƙatar labarai da yawa a cikin jerin. Kadan game da tsarin:
Part 1 (kana karantawa). Za mu yanke shawara game da ƙayyadaddun bayanai da gine-gine na bayani, kuma za mu rubuta aikace-aikace a cikin golang.
Kashi na 2. Muna sakin aikace-aikacen mu a cikin samarwa, sanya shi mai daidaitawa kuma muna gwada kaya.
Kashi na 3. Bari mu yi ƙoƙarin gano dalilin da yasa muke buƙatar adana saƙonni a cikin buffer ba a cikin fayiloli ba, sannan kuma kwatanta sabis na layin kafka, rabbitmq da yandex.
Kashi na 4 Za mu tura gungu na Clickhouse, rubuta sabis na yawo don canja wurin bayanai daga ma'ajin da ke wurin, da saita gani a cikin datalens.
Kashi na 5 Bari mu kawo dukkan abubuwan more rayuwa cikin sigar da ta dace - saita ci/cd ta amfani da gitlab ci, haɗa saka idanu da gano sabis ta amfani da prometheus da consul.
TK
Da farko, bari mu tsara sharuɗɗan tunani - menene ainihin abin da muke so mu samu a sakamakon haka.
- Muna son samun ƙarshen ƙarshen kamar events.kis.im (kis.im shine yankin gwajin da za mu yi amfani da shi a cikin duk labaran), wanda yakamata ya karɓi abubuwan da suka faru ta amfani da HTTPS.
- Abubuwan da suka faru json ne masu sauƙi kamar: {"event":"view", "os": "linux", "browser": "chrome"}. A mataki na ƙarshe za mu ƙara ƙarin filayen kaɗan, amma wannan ba zai taka muhimmiyar rawa ba. Idan kuna so, zaku iya canzawa zuwa protobuf.
- Dole ne sabis ɗin ya iya aiwatar da abubuwan 10 a cikin daƙiƙa guda.
- Yakamata a yi ma'auni a kwance ta hanyar ƙara sabbin lokuta a cikin maganinmu. Kuma zai yi kyau idan za mu iya motsa sashin gaba zuwa wurare daban-daban don rage jinkirin buƙatun abokin ciniki.
- Haƙuri na kuskure. Dole ne maganin ya kasance mai ƙarfi sosai kuma ya sami damar tsira daga faɗuwar kowane sassa (har zuwa wani adadi, ba shakka).
gine
Gabaɗaya, don irin wannan aikin, an daɗe ana ƙirƙira kayan gine-gine na gargajiya waɗanda ke ba da damar ƙima mai inganci. Adadin yana nuna misalin maganin mu.
Don haka abin da muke da shi:
1. A gefen hagu akwai na'urorinmu waɗanda ke haifar da al'amuran daban-daban, kasancewa 'yan wasa suna kammala matakin a cikin abin wasan yara akan wayar hannu ko ƙirƙirar oda a cikin kantin sayar da kan layi ta hanyar bincike na yau da kullun. Wani lamari, kamar yadda aka ƙayyade a cikin ƙayyadaddun bayanai, json ne mai sauƙi wanda aka aika zuwa ƙarshen ƙarshenmu - events.kis.im.
2. Sabar guda biyu na farko sune masu daidaitawa masu sauki, manyan ayyukansu sune:
- Kasance a koyaushe. Don yin wannan, zaku iya amfani da, alal misali, kiyayewa, wanda zai canza IP ɗin kama-da-wane tsakanin nodes idan akwai matsaloli.
- Kashe TLS. Ee, za mu ƙare TLS akan su. Da fari dai, domin maganin mu ya bi ƙayyadaddun fasaha, na biyu kuma, don sauke nauyin kafa haɗin da aka ɓoye daga sabar mu ta baya.
- Daidaita buƙatun masu shigowa zuwa samammun sabar baya. Mabuɗin kalmar anan ana iya samun dama. Dangane da wannan, mun zo ga fahimtar cewa ma'aunin nauyi dole ne su iya saka idanu akan sabar mu tare da aikace-aikace kuma su daina daidaita zirga-zirga zuwa nodes da suka gaza.
3. Bayan masu daidaitawa, muna da sabar aikace-aikacen da ke gudanar da aikace-aikacen mai sauƙi. Ya kamata ya iya karɓar buƙatun masu shigowa ta hanyar HTTP, inganta json da aka aiko kuma ya sanya bayanan cikin ma'ajin.
4. Hoton yana nuna kafka a matsayin buffer, kodayake, ba shakka, ana iya amfani da wasu ayyuka masu kama da wannan matakin. Za mu kwatanta Kafka, rabbitmq da yqs a kasida ta uku.
5. Babban mahimmin tsarin gine-ginen mu shine Clickhouse - rumbun adana bayanai na columnar wanda ke ba ku damar adanawa da aiwatar da adadi mai yawa na bayanai. A wannan matakin, muna buƙatar canja wurin bayanai daga buffer zuwa tsarin ajiya kanta (ƙari akan wannan a cikin labarin 4).
Wannan zane yana ba mu damar yin ma'auni kowane Layer da kansa a kwance. Sabis na baya ba zai iya jurewa ba - bari mu ƙara wani abu ɗaya - bayan haka, aikace-aikace ne marasa jiha, sabili da haka, ana iya yin hakan ko da ta atomatik. Salon salon Kafka baya aiki-bari mu ƙara ƙarin sabobin kuma mu canza musu wasu ɓangarori na batunmu. Clickhouse ba zai iya rike shi ba - ba zai yiwu ba :) A zahiri, za mu kuma haɗa sabar kuma mu share bayanan.
Af, idan kuna son aiwatar da ɓangaren zaɓi na ƙayyadaddun fasaha da sikelin mu a cikin wurare daban-daban, to babu wani abu mafi sauƙi:
A cikin kowane geolocation muna tura ma'aunin nauyi tare da aikace-aikacen da kafka. Gabaɗaya, sabobin aikace-aikacen 2, nodes kafka 3 da ma'aunin girgije, alal misali, Cloudflare, sun isa, wanda zai bincika kasancewar nodes ɗin aikace-aikacen da buƙatun daidaitawa ta hanyar geolocation dangane da adireshin IP na abokin ciniki. Don haka, bayanan da wani abokin ciniki na Amurka ya aiko zai sauka akan sabar Amurka. Kuma bayanai daga Afirka na cikin Afirka.
Sa'an nan komai yana da sauƙi - muna amfani da kayan aikin madubi daga saitin Kafka kuma muna kwafin duk bayanan daga duk wurare zuwa cibiyar bayanan mu ta tsakiya da ke cikin Rasha. A ciki, muna rarraba bayanan kuma muna yin rikodin su a cikin Clickhouse don hangen nesa na gaba.
Don haka, mun tsara tsarin gine-gine - bari mu fara girgiza Yandex.Cloud!
Rubuta aikace-aikace
Kafin Cloud, har yanzu kuna da ɗan haƙuri kuma ku rubuta sabis mai sauƙi mai sauƙi don aiwatar da abubuwan da ke shigowa. Za mu yi amfani da Golang saboda ya tabbatar da kansa sosai a matsayin harshe don rubuta aikace-aikacen cibiyar sadarwa.
Bayan shafe awa daya (watakila sa'o'i biyu), muna samun wani abu kamar haka: .
Menene manyan abubuwan da zan so in lura anan:
1. Lokacin fara aikace-aikacen, zaku iya ƙayyade tutoci guda biyu. Daya ne ke da alhakin tashar tashar da za mu saurari buƙatun http masu shigowa (-addr). Na biyu shine adireshin uwar garken kafka inda zamu rubuta abubuwan da suka faru (-kafka):
addr = flag.String("addr", ":8080", "TCP address to listen to")
kafka = flag.String("kafka", "127.0.0.1:9092", "Kafka endpoints”)2. Aikace-aikacen yana amfani da ɗakin karatu na sarama () don aika saƙonni zuwa gungu na kafka. Nan da nan muka saita saitunan da nufin iyakar sarrafawa:
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForLocal
config.Producer.Compression = sarama.CompressionSnappy
config.Producer.Return.Successes = true3. Aikace-aikacen mu kuma yana da ginannen abokin ciniki na prometheus, wanda ke tattara ma'auni daban-daban, kamar:
- adadin buƙatun zuwa aikace-aikacen mu;
- adadin kurakurai lokacin aiwatar da buƙatar (ba zai yiwu a karanta buƙatun post ba, karye json, ba zai yiwu a rubuta zuwa Kafka ba);
- lokacin aiki don buƙatu ɗaya daga abokin ciniki, gami da lokacin rubuta saƙo zuwa Kafka.
4. Abubuwa uku na ƙarshe waɗanda aikace-aikacenmu ke aiwatarwa:
- /status - kawai komawa ok don nuna cewa muna raye. Ko da yake za ka iya ƙara wasu cak, kamar samuwar gungu na Kafka.
- /metrics - bisa ga wannan url, abokin ciniki na prometheus zai dawo da ma'aunin da ya tattara.
- /post shine babban ƙarshen ƙarshen inda za a aika buƙatun POST tare da json a ciki. Aikace-aikacen mu yana bincika json don inganci kuma idan komai yayi daidai, yana rubuta bayanan zuwa gungu na Kafka.
Zan yi ajiyar cewa lambar ba ta cika ba - tana iya (kuma ya kamata!) Za a kammala. Misali, zaku iya dakatar da amfani da ginanniyar net/http sannan ku canza zuwa sauri http. Ko za ku iya samun lokacin sarrafawa da albarkatun cpu ta hanyar matsar da tabbacin ingancin json zuwa wani mataki na gaba - lokacin da aka canja wurin bayanai daga ma'ajin zuwa gunkin gidan dannawa.
Baya ga bangaren ci gaban al'amarin, nan da nan muka yi tunani game da ababen more rayuwa na gaba kuma muka yanke shawarar tura aikace-aikacenmu ta hanyar docker. Dockerfile na ƙarshe don gina aikace-aikacen shine . Gabaɗaya, abu ne mai sauƙi, kawai batun da zan so in kula da shi shine taron multistage, wanda ke ba mu damar rage hoton ƙarshe na akwati.
Matakan farko a cikin gajimare
Da farko, yi rajista a kan . Bayan cika dukkan filayen da ake bukata, za a ƙirƙiri asusun kuma a ba mu kyauta don wani adadin kuɗi, wanda za a iya amfani da shi don gwada ayyukan girgije. Idan kuna son maimaita duk matakai daga labarinmu, wannan tallafin ya kamata ya ishe ku.
Bayan rajista, za a ƙirƙiri wani gajimare dabam da tsohowar adireshi, wanda a ciki za ku iya fara ƙirƙirar albarkatun girgije. Gabaɗaya, a cikin Yandex.Cloud, dangantakar albarkatun tana kama da haka:
Kuna iya ƙirƙirar gajimare da yawa don asusu ɗaya. Kuma a cikin gajimare, yi kundayen adireshi daban-daban don ayyukan kamfanoni daban-daban. Kuna iya karanta ƙarin game da wannan a cikin takaddun - . Af, sau da yawa zan yi nuni da shi a ƙasa a cikin rubutu. Lokacin da na kafa dukkanin abubuwan more rayuwa daga karce, takaddun sun taimaka mini fiye da sau ɗaya, don haka ina ba ku shawarar ku yi nazarinsa.
Don sarrafa gajimare, zaku iya amfani da mahaɗin yanar gizo da kayan aikin wasan bidiyo - yc. Ana yin shigarwa tare da umarni ɗaya (na Linux da Mac Os):
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bashIdan ƙwararren tsaro na cikin gida yana fushi game da tafiyar da rubutun daga Intanet, to, da farko, za ku iya buɗe rubutun ku karanta shi, kuma na biyu, muna gudanar da shi a ƙarƙashin mai amfani - ba tare da haƙƙin tushen ba.
Idan kuna son shigar da abokin ciniki don Windows, zaku iya amfani da umarnin sannan kisa yc initdon daidaita shi gabaɗaya:
vozerov@mba:~ $ yc init
Welcome! This command will take you through the configuration process.
Please go to https://oauth.yandex.ru/authorize?response_type=token&client_id= in order to obtain OAuth token.
Please enter OAuth token:
Please select cloud to use:
[1] cloud-b1gv67ihgfu3bp (id = b1gv67ihgfu3bpt24o0q)
[2] fevlake-cloud (id = b1g6bvup3toribomnh30)
Please enter your numeric choice: 2
Your current cloud has been set to 'fevlake-cloud' (id = b1g6bvup3toribomnh30).
Please choose folder to use:
[1] default (id = b1g5r6h11knotfr8vjp7)
[2] Create a new folder
Please enter your numeric choice: 1
Your current folder has been set to 'default' (id = b1g5r6h11knotfr8vjp7).
Do you want to configure a default Compute zone? [Y/n]
Which zone do you want to use as a profile default?
[1] ru-central1-a
[2] ru-central1-b
[3] ru-central1-c
[4] Don't set default zone
Please enter your numeric choice: 1
Your profile default Compute zone has been set to 'ru-central1-a'.
vozerov@mba:~ $A ka'ida, tsari yana da sauƙi - da farko kuna buƙatar samun alamar rantsuwa don sarrafa girgije, zaɓi girgije da babban fayil ɗin da za ku yi amfani da su.
Idan kuna da asusu ko manyan fayiloli da yawa a cikin gajimare guda, zaku iya ƙirƙirar ƙarin bayanan martaba tare da saituna daban ta hanyar yc config profile ƙirƙira kuma canza tsakanin su.
Baya ga hanyoyin da ke sama, ƙungiyar Yandex.Cloud ta rubuta mai kyau sosai don sarrafa albarkatun girgije. A nawa bangare, na shirya wurin ajiyar git, inda na bayyana duk albarkatun da za a ƙirƙira a matsayin ɓangare na labarin - . Muna sha'awar reshen babban, bari mu rufe shi a gida:
vozerov@mba:~ $ git clone https://github.com/rebrainme/yandex-cloud-events/ events
Cloning into 'events'...
remote: Enumerating objects: 100, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 100 (delta 37), reused 89 (delta 26), pack-reused 0
Receiving objects: 100% (100/100), 25.65 KiB | 168.00 KiB/s, done.
Resolving deltas: 100% (37/37), done.
vozerov@mba:~ $ cd events/terraform/Duk manyan masu canji waɗanda ake amfani da su a cikin terraform an rubuta su a cikin babban fayil na main.tf. Don farawa, ƙirƙirar fayil na sirri.auto.tfvars a cikin babban fayil ɗin terraform tare da abun ciki mai zuwa:
# Yandex Cloud Oauth token
yc_token = ""
# Yandex Cloud ID
yc_cloud_id = ""
# Yandex Cloud folder ID
yc_folder_id = ""
# Default Yandex Cloud Region
yc_region = "ru-central1-a"
# Cloudflare email
cf_email = ""
# Cloudflare token
cf_token = ""
# Cloudflare zone id
cf_zone_id = ""Ana iya ɗaukar duk masu canji daga lissafin daidaitawar yc, tunda mun riga mun tsara kayan aikin wasan bidiyo. Ina ba ku shawara da ku hanzarta ƙara private.auto.tfvars zuwa .gitignore, don kada ku buga bayanan sirri da gangan.
A cikin private.auto.tfvars mun kuma ƙayyadaddun bayanai daga Cloudflare - don ƙirƙirar bayanan DNS da wakili na babban yankin events.kis.im zuwa sabar mu. Idan baku son amfani da Cloudflare, to, cire farawa na mai ba da sabis na Cloudflare a main.tf da fayil ɗin dns.tf, wanda ke da alhakin ƙirƙirar mahimman bayanan dns.
A cikin aikinmu za mu haɗu da duk hanyoyin uku - mu'amalar yanar gizo, kayan aikin wasan bidiyo, da terraform.
Rukunin cibiyoyin sadarwa
A gaskiya, zaku iya tsallake wannan matakin, tunda lokacin da kuka ƙirƙiri sabon gajimare, za ku sami hanyar sadarwa daban ta atomatik da ƙirƙira 3 subnets - ɗaya don kowane yankin samuwa. Amma har yanzu muna son yin hanyar sadarwa ta daban don aikinmu tare da adireshinta. Babban zane na yadda hanyar sadarwa ke aiki a Yandex.Cloud ana nuna shi a cikin adadi da ke ƙasa (da gaske an ɗauka daga )
Don haka, kun ƙirƙiri hanyar sadarwa gama gari wacce albarkatun za su iya sadarwa da juna a cikinta. Ga kowane yankin samuwa, ana ƙirƙira subnet tare da adireshinsa kuma an haɗa shi zuwa cibiyar sadarwa ta gaba ɗaya. A sakamakon haka, duk albarkatun girgije a cikinsa na iya sadarwa, koda kuwa suna cikin yankuna daban-daban. Abubuwan da aka haɗa zuwa cibiyoyin sadarwar girgije daban-daban suna iya ganin juna ta hanyar adiresoshin waje kawai. Af, yaya wannan sihiri ke aiki a ciki, .
An bayyana ƙirƙirar hanyar sadarwa a cikin fayil ɗin network.tf daga ma'ajiyar. A can za mu ƙirƙiri cibiyar sadarwa mai zaman kanta guda ɗaya ta ciki kuma muna haɗa subnets guda uku zuwa gare ta a wurare daban-daban na samuwa - ciki-a (172.16.1.0/24), na ciki-b (172.16.2.0/24), na ciki-c (172.16.3.0/24). ).
Fara terraform kuma ƙirƙirar cibiyoyin sadarwa:
vozerov@mba:~/events/terraform (master) $ terraform init
... skipped ..
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_vpc_subnet.internal-a -target yandex_vpc_subnet.internal-b -target yandex_vpc_subnet.internal-c
... skipped ...
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
yandex_vpc_network.internal: Creating...
yandex_vpc_network.internal: Creation complete after 3s [id=enp2g2rhile7gbqlbrkr]
yandex_vpc_subnet.internal-a: Creating...
yandex_vpc_subnet.internal-b: Creating...
yandex_vpc_subnet.internal-c: Creating...
yandex_vpc_subnet.internal-a: Creation complete after 6s [id=e9b1dad6mgoj2v4funog]
yandex_vpc_subnet.internal-b: Creation complete after 7s [id=e2liv5i4amu52p64ac9p]
yandex_vpc_subnet.internal-c: Still creating... [10s elapsed]
yandex_vpc_subnet.internal-c: Creation complete after 10s [id=b0c2qhsj2vranoc9vhcq]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.Mai girma! Mun ƙirƙiri hanyar sadarwar mu kuma yanzu muna shirye don ƙirƙirar ayyukanmu na ciki.
Ƙirƙirar injuna masu kama-da-wane
Don gwada aikace-aikacen, za mu buƙaci ƙirƙirar inji guda biyu kawai - za mu buƙaci na farko don ginawa da gudanar da aikace-aikacen, na biyu don sarrafa kafka, wanda za mu yi amfani da shi don adana saƙonni masu shigowa. Kuma za mu ƙirƙiri wata na'ura inda za mu saita prometheus don saka idanu akan aikace-aikacen.
Za a daidaita injinan kama-da-wane ta amfani da mai yiwuwa, don haka kafin fara terraform, tabbatar cewa kuna da ɗayan sabbin nau'ikan yuwuwar. Kuma shigar da rawar da suka dace tare da galaxy mai yiwuwa:
vozerov@mba:~/events/terraform (master) $ cd ../ansible/
vozerov@mba:~/events/ansible (master) $ ansible-galaxy install -r requirements.yml
- cloudalchemy-prometheus (master) is already installed, skipping.
- cloudalchemy-grafana (master) is already installed, skipping.
- sansible.kafka (master) is already installed, skipping.
- sansible.zookeeper (master) is already installed, skipping.
- geerlingguy.docker (master) is already installed, skipping.
vozerov@mba:~/events/ansible (master) $A cikin babban fayil mai yiwuwa akwai misali .ansible.cfg fayil na daidaitawa wanda nake amfani da shi. Zai iya zuwa da amfani.
Kafin ƙirƙirar injunan kama-da-wane, tabbatar cewa kuna da wakili na ssh yana gudana kuma an ƙara maɓallin ssh, in ba haka ba terraform ba zai iya haɗawa da injinan da aka ƙirƙira ba. Ni, ba shakka, na ci karo da bug a cikin os x: . Don hana hakan sake faruwa, ƙara ƙaramin canji zuwa env kafin ƙaddamar da Terraform:
vozerov@mba:~/events/terraform (master) $ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YESA cikin babban fayil tare da terraform muna ƙirƙirar albarkatun da suka dace:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_compute_instance.build -target yandex_compute_instance.monitoring -target yandex_compute_instance.kafka
yandex_vpc_network.internal: Refreshing state... [id=enp2g2rhile7gbqlbrkr]
data.yandex_compute_image.ubuntu_image: Refreshing state...
yandex_vpc_subnet.internal-a: Refreshing state... [id=e9b1dad6mgoj2v4funog]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
... skipped ...
Plan: 3 to add, 0 to change, 0 to destroy.
... skipped ...Idan komai ya ƙare cikin nasara (kuma yakamata ya kasance), to zamu sami injin kama-da-wane guda uku:
- gina - inji don gwaji da gina aikace-aikace. An shigar da Docker ta atomatik ta Mai yiwuwa.
- saka idanu - injin sa ido - prometheus & grafana da aka sanya akan shi. Login / kalmar sirri daidaitaccen: admin / admin
- kafka ƙaramin inji ne da aka shigar da kafka, ana iya samunsa akan tashar jiragen ruwa 9092.
Mu tabbatar duk suna nan a wurin:
vozerov@mba:~/events (master) $ yc compute instance list
+----------------------+------------+---------------+---------+---------------+-------------+
| ID | NAME | ZONE ID | STATUS | EXTERNAL IP | INTERNAL IP |
+----------------------+------------+---------------+---------+---------------+-------------+
| fhm081u8bkbqf1pa5kgj | monitoring | ru-central1-a | RUNNING | 84.201.159.71 | 172.16.1.35 |
| fhmf37k03oobgu9jmd7p | kafka | ru-central1-a | RUNNING | 84.201.173.41 | 172.16.1.31 |
| fhmt9pl1i8sf7ga6flgp | build | ru-central1-a | RUNNING | 84.201.132.3 | 172.16.1.26 |
+----------------------+------------+---------------+---------+---------------+-------------+
Abubuwan suna cikin wurin, kuma daga nan za mu iya samun adiresoshin IP ɗin su. Duk abin da ke biyo baya zan yi amfani da adiresoshin IP don haɗawa ta ssh kuma gwada aikace-aikacen. Idan kuna da asusun Cloudflare da aka haɗa da terraform, jin daɗin amfani da sabbin sunayen DNS da aka ƙirƙira.
Af, lokacin ƙirƙirar injin kama-da-wane, ana ba da IP na ciki da sunan DNS na ciki, don haka zaku iya samun damar sabar a cikin hanyar sadarwar ta suna:
ubuntu@build:~$ ping kafka.ru-central1.internal
PING kafka.ru-central1.internal (172.16.1.31) 56(84) bytes of data.
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=1 ttl=63 time=1.23 ms
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=2 ttl=63 time=0.625 ms
^C
--- kafka.ru-central1.internal ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.625/0.931/1.238/0.308 msWannan zai zama da amfani a gare mu mu nuna wa aikace-aikacen ƙarshen ƙarshen tare da kafk.
Haɗa aikace-aikacen
Babban, akwai sabobin, akwai aikace-aikace - abin da ya rage shi ne a haɗa shi a buga shi. Don ginin za mu yi amfani da ginin docker na yau da kullun, amma azaman ajiyar hoto za mu yi amfani da sabis daga Yandex - rajistar akwati. Amma abubuwa na farko.
Muna kwafin aikace-aikacen zuwa injin gini, shiga ta ssh kuma mu haɗa hoton:
vozerov@mba:~/events/terraform (master) $ cd ..
vozerov@mba:~/events (master) $ rsync -av app/ [email protected]:app/
... skipped ...
sent 3849 bytes received 70 bytes 7838.00 bytes/sec
total size is 3644 speedup is 0.93
vozerov@mba:~/events (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cd app
ubuntu@build:~/app$ sudo docker build -t app .
Sending build context to Docker daemon 6.144kB
Step 1/9 : FROM golang:latest AS build
... skipped ...
Successfully built 9760afd8ef65
Successfully tagged app:latestRabin yakin an yi - yanzu za mu iya duba ayyukan aikace-aikacen mu ta hanyar ƙaddamar da shi da aika shi zuwa kafka:
ubuntu@build:~/app$ sudo docker run --name app -d -p 8080:8080 app /app/app -kafka=kafka.ru-central1.internal:9092</code>
С локальной машинки можно отправить тестовый event и посмотреть на ответ:
<code>vozerov@mba:~/events (master) $ curl -D - -s -X POST -d '{"key1":"data1"}' http://84.201.132.3:8080/post
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 13 Apr 2020 13:53:54 GMT
Content-Length: 41
{"status":"ok","partition":0,"Offset":0}
vozerov@mba:~/events (master) $Aikace-aikacen ya amsa tare da nasarar yin rikodin tare da nuna id na bangare da kashewa wanda aka haɗa saƙon. Abin da kawai ya rage don yin shi ne ƙirƙirar rajista a cikin Yandex.Cloud kuma shigar da hotonmu a can (yadda ake yin wannan ta amfani da layi uku an bayyana shi a cikin fayil registry.tf). Ƙirƙiri ma'aji:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_container_registry.events
... skipped ...
Plan: 1 to add, 0 to change, 0 to destroy.
... skipped ...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Akwai hanyoyi da yawa don tantancewa a cikin rajistar ganga - ta amfani da alamar oauth, alamar iam, ko maɓallin asusun sabis. Ana iya samun ƙarin cikakkun bayanai game da waɗannan hanyoyin a cikin takaddun. . Za mu yi amfani da maɓallin asusun sabis, don haka mu ƙirƙiri asusu:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_iam_service_account.docker -target yandex_resourcemanager_folder_iam_binding.puller -target yandex_resourcemanager_folder_iam_binding.pusher
... skipped ...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.Yanzu abin da ya rage shi ne yin maɓalli don shi:
vozerov@mba:~/events/terraform (master) $ yc iam key create --service-account-name docker -o key.json
id: ajej8a06kdfbehbrh91p
service_account_id: ajep6d38k895srp9osij
created_at: "2020-04-13T14:00:30Z"
key_algorithm: RSA_2048Muna karɓar bayani game da id ɗin ajiyarmu, canja wurin maɓalli kuma mu shiga:
vozerov@mba:~/events/terraform (master) $ scp key.json [email protected]:
key.json 100% 2392 215.1KB/s 00:00
vozerov@mba:~/events/terraform (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cat key.json | sudo docker login --username json_key --password-stdin cr.yandex
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
ubuntu@build:~$Don loda hoton zuwa wurin yin rajista, muna buƙatar ID ɗin rajista na ganga, muna ɗaukar shi daga mai amfani yc:
vozerov@mba:~ $ yc container registry get events
id: crpdgj6c9umdhgaqjfmm
folder_id:
name: events
status: ACTIVE
created_at: "2020-04-13T13:56:41.914Z"Bayan haka, muna yiwa hotonmu alama da sabon suna kuma mu loda:
ubuntu@build:~$ sudo docker tag app cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
ubuntu@build:~$ sudo docker push cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
The push refers to repository [cr.yandex/crpdgj6c9umdhgaqjfmm/events]
8c286e154c6e: Pushed
477c318b05cb: Pushed
beee9f30bc1f: Pushed
v1: digest: sha256:1dd5aaa9dbdde2f60d833be0bed1c352724be3ea3158bcac3cdee41d47c5e380 size: 946Za mu iya tabbatar da cewa hoton ya yi nasara:
vozerov@mba:~/events/terraform (master) $ yc container repository list
+----------------------+-----------------------------+
| ID | NAME |
+----------------------+-----------------------------+
| crpe8mqtrgmuq07accvn | crpdgj6c9umdhgaqjfmm/events |
+----------------------+-----------------------------+Af, idan kun shigar da yc utility akan na'urar Linux, zaku iya amfani da umarnin
yc container registry configure-dockerdon saita docker.
ƙarshe
Mun yi aiki tuƙuru da yawa kuma a sakamakon haka:
- Mun fito da tsarin gine-ginen sabis ɗinmu na gaba.
- Mun rubuta aikace-aikace a cikin golang wanda ke aiwatar da dabarun kasuwancin mu.
- Mun tattara shi kuma muka zuba shi a cikin wurin rajista na sirri na sirri.
A kashi na gaba, za mu ci gaba zuwa abubuwa masu ban sha'awa - za mu saki aikace-aikacen mu don samarwa kuma a ƙarshe za mu ƙaddamar da kaya a kansa. Kar a canza!
Wannan kayan yana cikin rikodin bidiyo na bude bita REBRAIN & Yandex.Cloud: Muna karɓar buƙatun 10 a sakan daya akan Yandex Cloud -
Idan kuna sha'awar halartar irin waɗannan abubuwan akan layi da yin tambayoyi a ainihin lokacin, haɗa zuwa.
Muna so mu ce godiya ta musamman ga Yandex.Cloud don samun damar gudanar da irin wannan taron. Link zuwa gare su -
Idan kuna buƙatar matsawa zuwa gajimare ko kuna da tambayoyi game da kayan aikin ku, .
PS Muna da bincike na kyauta 2 kowane wata, watakila aikin ku zai kasance ɗaya daga cikinsu.
source: www.habr.com