Ndewo onye ọ bụla, enyi!
* Edemede a dabere na ụlọ ọrụ mepere emepe REBRAIN & Yandex.Cloud, ọ bụrụ na ịchọrọ ikiri vidiyo, ị nwere ike ịhụ ya na njikọ a -
N'oge na-adịbeghị anya, anyị nwere ohere ịnwale Yandex.Cloud ndụ. Ebe ọ bụ na anyị chọrọ inyocha ogologo oge siri ike, anyị hapụrụ ozugbo echiche nke ịmalite blọọgụ WordPress dị mfe na isi igwe ojii - ọ na-agwụ ike. Mgbe echiche ụfọdụ gasịrị, anyị kpebiri ibuga ihe yiri ụlọ ọrụ mmepụta ihe maka ịnata na nyocha ihe omume n'oge dị nso.
Eji m n'aka na ọnụ ọgụgụ ka ukwuu nke ụlọ ọrụ ịntanetị (ọ bụghị naanị) na-anakọta ugwu ozi gbasara ndị ọrụ ha na omume ha. Opekempe, nke a dị mkpa maka ịme mkpebi ụfọdụ - dịka ọmụmaatụ, ọ bụrụ na ị na-ejikwa egwuregwu ịntanetị, ị nwere ike lelee ọnụ ọgụgụ nke ndị ọrụ ọkwa na-araparakarị ma hichapụ ihe egwuregwu ụmụaka gị. Ma ọ bụ ihe kpatara ndị ọrụ ji ahapụ saịtị gị n'azụghị ihe ọ bụla (ndewo, Yandex.Metrica).
Yabụ, akụkọ anyị: otu anyị si dee ngwa na golang, nwalere kafka vs rabbitmq vs yqs, dee nkwanye data na ụyọkọ Clickhouse wee jiri anya nke uche hụ data site na iji yandex datalens. Dị ka o kwesịrị ịdị, ihe ndị a niile bụ ihe ụtọ akụrụngwa n'ụdị docker, terraform, gitlab ci na, n'ezie, prometheus. Ka a pụọ!
Ọ ga-amasị m ịme ndoputa ozugbo na anyị agaghị enwe ike ịhazi ihe niile n'otu oche - maka nke a, anyị ga-achọ ọtụtụ isiokwu na usoro. Obere maka nhazi ahụ:
Nkeji 1 (ị na-agụ ya). Anyị ga-ekpebi nkọwapụta na ụkpụrụ ụlọ nke ngwọta, ma deekwa ngwa na golang.
Akụkụ 2. Anyị na-ahapụ ngwa anyị n'ime mmepụta, mee ka ọ dị ike ma nwalee ibu ahụ.
Akụkụ nke 3. Ka anyị gbalịa chọpụta ihe kpatara na anyị kwesịrị iji chekwaa ozi na nchekwa ọ bụghị na faịlụ, ma tụnyere kafka, rabbitmq na yandex queue service.
Akụkụ 4 Anyị ga-ebuga ụyọkọ Clickhouse, dee ọrụ mgbasa ozi iji bufee data site na nchekwa ebe ahụ, wee guzobe visualization na datalens.
Akụkụ nke 5 Ka anyị webata akụrụngwa niile n'ụdị kwesịrị ekwesị - melite ci/cd site na iji gitlab ci, jikọọ nlekota na nchọpụta ọrụ site na iji prometheus na consul.
ТЗ
Nke mbụ, ka anyị mepụta usoro nrụtụ aka - ihe kpọmkwem anyị chọrọ inweta n'ihi ya.
- Anyị chọrọ inwe njedebe dị ka events.kis.im (kis.im bụ ngalaba nyocha nke anyị ga-eji n'ime akụkọ niile), nke kwesịrị ịnata mmemme site na iji HTTPS.
- Ihe omume bụ json dị mfe dị ka: {“omume”: “elele”, “os”: “linux”, “browser”: “chrome”}. N'oge ikpeazụ anyị ga-agbakwunye ntakịrị ubi, ma nke a agaghị arụ nnukwu ọrụ. Ọ bụrụ na-ịchọrọ, ị nwere ike ịgbanwe na protobuf.
- Ọrụ ahụ ga-enwerịrị ike ịhazi mmemme 10 kwa nkeji.
- Ọ ga-ekwe omume ịgbatị n'ụzọ kwụ ọtọ site n'ịgbakwunye ihe ọhụrụ na ngwọta anyị. Ọ ga-adị mma ma ọ bụrụ na anyị nwere ike ịkwaga akụkụ ihu na mpaghara geolocation dị iche iche iji belata latency maka arịrịọ ndị ahịa.
- Nkwenye mmejọ. Ngwọta ahụ ga-abụrịrị nke ọma ma nwee ike ịlanarị ọdịda nke akụkụ ọ bụla (ruo ọnụ ọgụgụ ụfọdụ, n'ezie).
ije
N'ozuzu, maka ụdị ọrụ a, ogologo oge emepụtawo ihe owuwu oge ochie nke na-enye ohere ịmepụta nke ọma. Ọnụ ọgụgụ ahụ na-egosi ihe atụ nke ngwọta anyị.
Yabụ ihe anyị nwere:
1. N'aka ekpe bụ ngwaọrụ anyị nke na-emepụta ihe omume dị iche iche, bụrụ ndị egwuregwu na-emecha ọkwa na ihe egwuregwu ụmụaka na smartphone ma ọ bụ ịmepụta usoro na ụlọ ahịa dị n'ịntanetị site na ihe nchọgharị mgbe niile. Ihe omume, dịka akọwapụtara na nkọwapụta, bụ json dị mfe ezigara na njedebe anyị - events.kis.im.
2. Sava abụọ mbụ bụ ndị na-edozi ihe dị mfe, ọrụ ha bụ isi bụ:
- Na-adị mgbe niile. Iji mee nke a, ịnwere ike iji, dịka ọmụmaatụ, keepalive, nke ga-agbanwe IP mebere n'etiti ọnụ ma ọ bụrụ na enwere nsogbu.
- Kwụsị TLS. Ee, anyị ga-akwụsị TLS na ha. Nke mbụ, ka ngwọta anyị dabara na nkọwapụta teknụzụ, na nke abụọ, iji wepụ ibu dị n'ịmepụta njikọ ezoro ezo site na sava anyị azụ.
- Hazie arịrịọ mbata na sava azụ azụ dị. Isi okwu ebe a nwere ike ịnweta. Dabere na nke a, anyị na-abịa na nghọta na ibu balancers ga-enwe ike nyochaa sava anyị na ngwa na-akwụsị itule okporo ụzọ ka ọdịda ọnụ.
3. Mgbe balancers, anyị nwere ngwa sava na-agba ọsọ a pụtara mfe ngwa. Ọ ga-enwe ike ịnakwere arịrịọ mbata site na HTTP, kwadoo json zitere wee tinye data n'ime nchekwa.
4. Eserese na-egosi kafka dị ka ihe nchekwa, ọ bụ ezie na, n'ezie, a pụrụ iji ọrụ ndị ọzọ yiri ya mee ihe na ọkwa a. Anyị ga-atụnyere Kafka, rabbitmq na yqs na edemede nke atọ.
5. Ebe penultimate nke ụlọ anyị bụ Clickhouse - nchekwa data columnar nke na-enye gị ohere ịchekwa ma hazie nnukwu data. N'ọkwa a, anyị kwesịrị ịnyefe data site na ihe nchekwa na usoro nchekwa n'onwe ya (karịa na nke a na isiokwu 4).
Nhazi a na-enye anyị ohere ịgbanye oyi akwa ọ bụla n'adabereghị na ya. Sava azụ azụ enweghị ike ịnagide - ka anyị gbakwunye otu ihe - ka emechara, ha bụ ngwa enweghị obodo, yabụ, enwere ike ime nke a ọbụlagodi na akpaghị aka. Ihe nchekwa ụdị Kafka anaghị arụ ọrụ-ka anyị tinyekwuo sava ma nyefee ha ụfọdụ akụkụ nke isiokwu anyị. Clickhouse enweghị ike ijikwa ya - ọ gaghị ekwe omume :) N'ezie, anyị ga-ejikọkwa sava ma kesaa data ahụ.
Site n'ụzọ, ọ bụrụ na ịchọrọ itinye akụkụ nhọrọ nke nkọwa teknụzụ anyị na ọnụ ọgụgụ dị iche iche na geolocations, mgbe ahụ ọ nweghị ihe dị mfe:
Na mpaghara geolocation ọ bụla anyị na-ebunye ihe nrụzi ibu na ngwa na kafka. N'ozuzu, ihe nkesa ngwa 2, 3 kafka nodes na igwe nlekota igwe ojii, dịka ọmụmaatụ, Cloudflare, zuru ezu, nke ga-enyocha nnweta ngwa ngwa na arịrịọ nguzozi site na geolocation dabere na adreesị IP nke onye ahịa. Ya mere, data onye ahịa America zitere ga-adaba na sava America. Na data sitere na Africa dị n'Africa.
Mgbe ahụ, ihe niile dị nnọọ mfe - anyị na-eji ngwá ọrụ mirror si na Kafka set na detuo niile data site na ebe niile na anyị Central data center dị na Russia. N'ime, anyị na-atụgharị data ahụ wee dekọọ ya na Clickhouse maka nleba anya na-esote.
Yabụ, anyị ahaziela ihe owuwu - ka anyị bido ịma jijiji Yandex.Cloud!
Na-ede ngwa
Tupu igwe ojii, ị ka ga-enwerịrị ndidi wee dee ọrụ dị mfe iji hazie mmemme mbata. Anyị ga-eji golang n'ihi na ọ gosipụtara onwe ya nke ọma dị ka asụsụ maka ide ngwa netwọkụ.
Mgbe anyị nọrọ otu awa (ma eleghị anya awa ole na ole), anyị ga-enweta ihe dịka nke a: .
Kedu isi ihe m ga-achọ ịhụ ebe a:
1. Mgbe ịmalite ngwa ahụ, ị nwere ike ịkọwa ọkọlọtọ abụọ. Otu na-ahụ maka ọdụ ụgbọ mmiri nke anyị ga-ege ntị na arịrịọ http na-abata (-addr). Nke abụọ bụ maka adreesị nkesa kafka ebe anyị ga-edekọ ihe omume anyị (-kafka):
addr = flag.String("addr", ":8080", "TCP address to listen to")
kafka = flag.String("kafka", "127.0.0.1:9092", "Kafka endpoints”)2. Ngwa na-eji sarama ọba akwụkwọ () iziga ozi na ụyọkọ kafka. Anyị na-edobe ntọala ozugbo maka ọsọ nhazi kachasị:
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForLocal
config.Producer.Compression = sarama.CompressionSnappy
config.Producer.Return.Successes = true3. Ngwa anyị nwekwara onye ahịa prometheus arụnyere, nke na-anakọta metrik dị iche iche, dịka:
- ọnụ ọgụgụ nke arịrịọ anyị ngwa;
- ọnụ ọgụgụ nke njehie mgbe ị na-eme arịrịọ ahụ (agaghị ekwe omume ịgụ arịrịọ post, gbajiri json, agaghị ekwe omume idegara Kafka);
- oge nhazi maka otu arịrịọ sitere n'aka onye ahịa, gụnyere oge maka ide ozi na Kafka.
4. Isi njedebe atọ nke ngwa anyị na-eme:
- / status - nanị laghachi ok iji gosi na anyị dị ndụ. Ọ bụ ezie na ị nwere ike ịgbakwunye ụfọdụ ndenye ego, dị ka nnweta nke ụyọkọ Kafka.
- /metrics - dị ka url a si dị, onye ahịa prometheus ga-eweghachite metrik ọ chịkọtara.
- / post bụ isi njedebe ebe a ga-eziga arịrịọ POST na json n'ime. Ngwa anyị na-enyocha json maka izi ezi ma ọ bụrụ na ihe niile dị mma, ọ na-ede data na ụyọkọ Kafka.
M ga-eme ndoputa na koodu ahụ ezughị oke - ọ nwere ike (ma kwesị!) Mechaa. Dịka ọmụmaatụ, ịnwere ike ịkwụsị iji net/http arụnyere arụnyere wee gbanwee gaa na ngwa ngwa ngwa http. Ma ọ bụ ị nwere ike nweta oge nhazi na akụrụngwa cpu site na ịkwaga json validity check na nke ọzọ - mgbe ebufe data site na ihe nchekwa gaa na ụyọkọ clickhouse.
Na mgbakwunye na akụkụ mmepe nke okwu ahụ, anyị chere ozugbo maka akụrụngwa anyị n'ọdịnihu wee kpebie ibuga ngwa anyị site na docker. Dockerfile ikpeazụ maka iwulite ngwa a bụ . N'ozuzu, ọ dị nnọọ mfe, naanị isi ihe m ga-achọ ịṅa ntị bụ mgbakọ multistage, nke na-enye anyị ohere ibelata ihe oyiyi ikpeazụ nke akpa anyị.
Nzọụkwụ mbụ n'ime igwe ojii
Mbụ niile, debanye aha na . Mgbe emejuchara mpaghara niile dị mkpa, a ga-emepụta akaụntụ ma nye anyị onyinye maka ego ụfọdụ, nke enwere ike iji nwalee ọrụ igwe ojii. Ọ bụrụ na ịchọrọ ịmegharị usoro niile sitere na edemede anyị, onyinye a kwesịrị ezuru gị.
Mgbe ndebanye aha, a ga-emepụta ígwé ojii dị iche iche na akwụkwọ ndekọ aha maka gị, nke ị nwere ike ịmalite ịmepụta akụrụngwa igwe ojii. N'ozuzu, na Yandex.Cloud, mmekọrịta nke akụrụngwa dị ka nke a:
Ị nwere ike ịmepụta ọtụtụ igwe ojii maka otu akaụntụ. Na n'ime igwe ojii, mee akwụkwọ ndekọ aha dị iche iche maka ọrụ ụlọ ọrụ dị iche iche. Ị nwere ike ịgụkwu gbasara nke a na akwụkwọ - . Site n'ụzọ, m ga-emekarị na-ezo aka na ya n'okpuru ederede. Mgbe m guzobere akụrụngwa niile site na ọkọ, akwụkwọ ahụ nyeere m aka karịa otu ugboro, yabụ m na-adụ gị ọdụ ka ị mụọ ya.
Iji jikwaa igwe ojii, ị nwere ike iji ma interface webụ yana akụrụngwa njikwa - yc. A na-eji otu iwu arụ ọrụ (maka Linux na Mac Os):
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bashỌ bụrụ na onye ọkachamara na-ahụ maka nchekwa n'ime gị na-ewe iwe banyere ịgba ọsọ scripts na Ịntanetị, mgbe ahụ, nke mbụ, ị nwere ike mepee edemede ahụ ma gụọ ya, na nke abụọ, anyị na-agba ya n'okpuru onye ọrụ anyị - na-enweghị ikike mgbọrọgwụ.
Ọ bụrụ na ịchọrọ ịwụnye onye ahịa maka Windows, ịnwere ike iji ntuziaka wee gbuo yc initka hazie ya nke ọma:
vozerov@mba:~ $ yc init
Welcome! This command will take you through the configuration process.
Please go to https://oauth.yandex.ru/authorize?response_type=token&client_id= in order to obtain OAuth token.
Please enter OAuth token:
Please select cloud to use:
[1] cloud-b1gv67ihgfu3bp (id = b1gv67ihgfu3bpt24o0q)
[2] fevlake-cloud (id = b1g6bvup3toribomnh30)
Please enter your numeric choice: 2
Your current cloud has been set to 'fevlake-cloud' (id = b1g6bvup3toribomnh30).
Please choose folder to use:
[1] default (id = b1g5r6h11knotfr8vjp7)
[2] Create a new folder
Please enter your numeric choice: 1
Your current folder has been set to 'default' (id = b1g5r6h11knotfr8vjp7).
Do you want to configure a default Compute zone? [Y/n]
Which zone do you want to use as a profile default?
[1] ru-central1-a
[2] ru-central1-b
[3] ru-central1-c
[4] Don't set default zone
Please enter your numeric choice: 1
Your profile default Compute zone has been set to 'ru-central1-a'.
vozerov@mba:~ $Na ụkpụrụ, usoro a dị mfe - nke mbụ ị ga-enweta akara ngosi iji jikwaa igwe ojii, họrọ igwe ojii na folda ị ga-eji.
Ọ bụrụ na ị nwere ọtụtụ akaụntụ ma ọ bụ nchekwa n'ime otu igwe ojii, ị nwere ike ịmepụta profaịlụ ndị ọzọ nwere ntọala dị iche iche site na profaịlụ yc config mepụta na gbanwee n'etiti ha.
Na mgbakwunye na ụzọ ndị dị n'elu, ndị otu Yandex.Cloud dere nke ọma maka ijikwa akụ igwe ojii. Maka akụkụ nke m, m kwadebere ebe nchekwa git, ebe m kọwara ihe niile a ga-emepụta dịka akụkụ nke akụkọ - . Anyị nwere mmasị na ngalaba nna ukwu, ka anyị mechie ya na mpaghara:
vozerov@mba:~ $ git clone https://github.com/rebrainme/yandex-cloud-events/ events
Cloning into 'events'...
remote: Enumerating objects: 100, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 100 (delta 37), reused 89 (delta 26), pack-reused 0
Receiving objects: 100% (100/100), 25.65 KiB | 168.00 KiB/s, done.
Resolving deltas: 100% (37/37), done.
vozerov@mba:~ $ cd events/terraform/Edere mgbanwe niile bụ isi nke a na-eji na terraform na faịlụ main.tf. Iji bido, mepụta faịlụ private.auto.tfvars na folda terraform nwere ọdịnaya ndị a:
# Yandex Cloud Oauth token
yc_token = ""
# Yandex Cloud ID
yc_cloud_id = ""
# Yandex Cloud folder ID
yc_folder_id = ""
# Default Yandex Cloud Region
yc_region = "ru-central1-a"
# Cloudflare email
cf_email = ""
# Cloudflare token
cf_token = ""
# Cloudflare zone id
cf_zone_id = ""Enwere ike iwere mgbanwe niile na listi yc config, ebe anyị ahazilarị ngwa njikwa. M na-adụ ọdụ ka ị gbakwunye private.auto.tfvars ozugbo na .gitignore, ka ị ghara ibipụta data nzuzo na mberede.
Na private.auto.tfvars anyị kọwapụtakwara data sitere na Cloudflare - ka imepụta ndekọ DNS na proxy isi ngalaba events.kis.im na sava anyị. Ọ bụrụ na ịchọghị iji cloudflare, wee wepụ mmalite nke onye na-eweta cloudflare na main.tf na faịlụ dns.tf, nke na-ahụ maka ịmepụta ihe ndekọ dns dị mkpa.
N'ime ọrụ anyị, anyị ga-ejikọta ụzọ atọ a - interface weebụ, ihe njikwa njikwa, na terraform.
Netwọk mebere
N'ikwu eziokwu, ị nwere ike ịgafe nzọụkwụ a, ebe ọ bụ na mgbe ịmepụtara igwe ojii ọhụrụ, ị ga-enwe netwọk dị iche iche na subnets 3 emepụtara - otu maka mpaghara ọ bụla dị. Mana anyị ka ga-achọ ịmepụta netwọkụ dị iche maka ọrụ anyị nwere adreesị nke ya. E gosipụtara eserese izugbe nke otu netwọkụ si arụ ọrụ na Yandex.Cloud na foto dị n'okpuru (nke sitere n'eziokwu were )
Ya mere, ị na-emepụta netwọk nkịtị n'ime nke akụrụngwa nwere ike ịkparịta ụka n'etiti onwe gị. Maka mpaghara ọ bụla enwere, a na-emepụta subnet nwere adreesị nke ya yana jikọọ na netwọk izugbe. N'ihi ya, ihe niile igwe ojii dị na ya nwere ike ịkparịta ụka, ọbụlagodi na ha nọ na mpaghara nnweta dị iche iche. Akụrụngwa ejikọrọ na netwọk igwe ojii dị iche iche nwere ike ịhụ ibe ha naanị site na adreesị mpụga. Site n'ụzọ, kedu ka anwansi a si arụ ọrụ n'ime, .
A kọwara imepụta netwọkụ na faịlụ network.tf sitere na ebe nchekwa. N'ebe ahụ, anyị na-emepụta otu netwọk nkeonwe nke dị n'ime ma jikọọ subnet atọ na ya na mpaghara dị iche iche - internal-a (172.16.1.0/24), internal-b (172.16.2.0/24), internal-c (172.16.3.0/24). ).
Bido terraform wee mepụta netwọk:
vozerov@mba:~/events/terraform (master) $ terraform init
... skipped ..
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_vpc_subnet.internal-a -target yandex_vpc_subnet.internal-b -target yandex_vpc_subnet.internal-c
... skipped ...
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
yandex_vpc_network.internal: Creating...
yandex_vpc_network.internal: Creation complete after 3s [id=enp2g2rhile7gbqlbrkr]
yandex_vpc_subnet.internal-a: Creating...
yandex_vpc_subnet.internal-b: Creating...
yandex_vpc_subnet.internal-c: Creating...
yandex_vpc_subnet.internal-a: Creation complete after 6s [id=e9b1dad6mgoj2v4funog]
yandex_vpc_subnet.internal-b: Creation complete after 7s [id=e2liv5i4amu52p64ac9p]
yandex_vpc_subnet.internal-c: Still creating... [10s elapsed]
yandex_vpc_subnet.internal-c: Creation complete after 10s [id=b0c2qhsj2vranoc9vhcq]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.Nnukwu! Anyị ekepụtala netwọkụ anyị ma dị njikere ugbu a imepụta ọrụ ime anyị.
Ịmepụta igwe mebere
Iji nwalee ngwa ahụ, anyị ga-achọ naanị ịmepụta igwe mebere abụọ - anyị ga-achọ nke mbụ iji wuo ma mee ngwa ahụ, nke abụọ na-agba kafka, nke anyị ga-eji chekwaa ozi na-abata. Anyị ga-emepụta igwe ọzọ ebe anyị ga-ahazi prometheus iji nyochaa ngwa ahụ.
A ga-ahazi igwe ndị mebere site na iji ike, yabụ tupu ịmalite terraform, jide n'aka na ị nwere otu n'ime ụdị kachasị ọhụrụ nke enwere ike. Ma tinye ọrụ dị mkpa na ụyọkọ kpakpando nwere ike:
vozerov@mba:~/events/terraform (master) $ cd ../ansible/
vozerov@mba:~/events/ansible (master) $ ansible-galaxy install -r requirements.yml
- cloudalchemy-prometheus (master) is already installed, skipping.
- cloudalchemy-grafana (master) is already installed, skipping.
- sansible.kafka (master) is already installed, skipping.
- sansible.zookeeper (master) is already installed, skipping.
- geerlingguy.docker (master) is already installed, skipping.
vozerov@mba:~/events/ansible (master) $N'ime folda enwere ike enwere ihe atụ .ansible.cfg nhazi faịlụ m na-eji. Ọ nwere ike ịba uru.
Tupu ịmepụta igwe mebere, gbaa mbọ hụ na ị nwere ssh-agent na-agba ọsọ yana igodo ssh agbakwunyere, ma ọ bụghị ya, terraform agaghị enwe ike jikọọ na igwe ndị emepụtara. M, n'ezie, hụrụ ahụhụ na os x: . Iji gbochie nke a ime ọzọ, tinye obere mgbanwe na env tupu ịmalite Terraform:
vozerov@mba:~/events/terraform (master) $ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YESN'ime folda nwere terraform anyị na-emepụta ihe ndị dị mkpa:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_compute_instance.build -target yandex_compute_instance.monitoring -target yandex_compute_instance.kafka
yandex_vpc_network.internal: Refreshing state... [id=enp2g2rhile7gbqlbrkr]
data.yandex_compute_image.ubuntu_image: Refreshing state...
yandex_vpc_subnet.internal-a: Refreshing state... [id=e9b1dad6mgoj2v4funog]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
... skipped ...
Plan: 3 to add, 0 to change, 0 to destroy.
... skipped ...Ọ bụrụ na ihe niile kwụsịrị nke ọma (ma ọ ga-abụ), mgbe ahụ anyị ga-enwe igwe mebere atọ:
- wuo - igwe maka nnwale na iwu ngwa. Enwere ike itinye Docker na akpaghị aka.
- nlekota - igwe nlekota - prometheus & grafana arụnyere na ya. Nbanye / paswọọdụ ọkọlọtọ: admin / admin
- kafka bụ obere igwe nwere kafka arụnyere, dị na ọdụ ụgbọ mmiri 9092.
Ka anyị hụ na ha niile nọ ebe:
vozerov@mba:~/events (master) $ yc compute instance list
+----------------------+------------+---------------+---------+---------------+-------------+
| ID | NAME | ZONE ID | STATUS | EXTERNAL IP | INTERNAL IP |
+----------------------+------------+---------------+---------+---------------+-------------+
| fhm081u8bkbqf1pa5kgj | monitoring | ru-central1-a | RUNNING | 84.201.159.71 | 172.16.1.35 |
| fhmf37k03oobgu9jmd7p | kafka | ru-central1-a | RUNNING | 84.201.173.41 | 172.16.1.31 |
| fhmt9pl1i8sf7ga6flgp | build | ru-central1-a | RUNNING | 84.201.132.3 | 172.16.1.26 |
+----------------------+------------+---------------+---------+---------------+-------------+
Ihe onwunwe dị na ebe a, anyị nwere ike nweta adreesị IP ha. N'ime ihe ndị a, m ga-eji adreesị IP jikọọ site na ssh wee nwalee ngwa ahụ. Ọ bụrụ na ị nwere akaụntụ cloudflare ejikọrọ na terraform, nweere onwe gị iji aha DNS emepụtara ọhụrụ.
Site n'ụzọ, mgbe ị na-eke igwe mebere, a na-enye IP dị n'ime na aha DNS dị n'ime, yabụ ị nwere ike ịnweta sava n'ime netwọkụ ahụ n'aha:
ubuntu@build:~$ ping kafka.ru-central1.internal
PING kafka.ru-central1.internal (172.16.1.31) 56(84) bytes of data.
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=1 ttl=63 time=1.23 ms
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=2 ttl=63 time=0.625 ms
^C
--- kafka.ru-central1.internal ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.625/0.931/1.238/0.308 msNke a ga-abara anyị uru igosi ngwa ahụ njedebe na kafk.
Ịchịkọta ngwa ahụ
Ọ dị mma, enwere sava, enwere ngwa - naanị ihe fọdụrụ bụ ịchịkọta ya ma bipụta ya. Maka ihe owuwu ahụ, anyị ga-eji ụlọ docker na-emebu, mana dịka nchekwa ihe oyiyi anyị ga-eji ọrụ sitere na Yandex - ndekọ ndekọ. Ma mbụ ihe mbụ.
Anyị na-eṅomi ngwa ahụ na igwe nrụpụta, banye na ssh wee kpokọta onyonyo a:
vozerov@mba:~/events/terraform (master) $ cd ..
vozerov@mba:~/events (master) $ rsync -av app/ [email protected]:app/
... skipped ...
sent 3849 bytes received 70 bytes 7838.00 bytes/sec
total size is 3644 speedup is 0.93
vozerov@mba:~/events (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cd app
ubuntu@build:~/app$ sudo docker build -t app .
Sending build context to Docker daemon 6.144kB
Step 1/9 : FROM golang:latest AS build
... skipped ...
Successfully built 9760afd8ef65
Successfully tagged app:latestEmere ọkara agha ahụ - ugbu a, anyị nwere ike lelee arụmọrụ nke ngwa anyị site na ịmalite ya na iziga ya na kafka:
ubuntu@build:~/app$ sudo docker run --name app -d -p 8080:8080 app /app/app -kafka=kafka.ru-central1.internal:9092</code>
С локальной машинки можно отправить тестовый event и посмотреть на ответ:
<code>vozerov@mba:~/events (master) $ curl -D - -s -X POST -d '{"key1":"data1"}' http://84.201.132.3:8080/post
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 13 Apr 2020 13:53:54 GMT
Content-Length: 41
{"status":"ok","partition":0,"Offset":0}
vozerov@mba:~/events (master) $Ngwa ahụ zara ya site na ịga nke ọma nke ndekọ ahụ yana egosi id nke nkebi na nkwụsị nke etinyere ozi ahụ. Naanị ihe fọdụrụ ime bụ imepụta ndekọ na Yandex.Cloud wee bulite onyonyo anyị ebe ahụ (otu esi eme nke a site na iji ahịrị atọ ka akọwara na faịlụ registry.tf). Mepụta nchekwa:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_container_registry.events
... skipped ...
Plan: 1 to add, 0 to change, 0 to destroy.
... skipped ...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Enwere ụzọ dị iche iche iji nyochaa n'ime ndekọ akpa - iji akara outh, akara iam, ma ọ bụ igodo akaụntụ ọrụ. Enwere ike ịchọta nkọwa ndị ọzọ gbasara ụzọ ndị a na akwụkwọ. . Anyị ga-eji igodo akaụntụ ọrụ, yabụ anyị mepụta akaụntụ:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_iam_service_account.docker -target yandex_resourcemanager_folder_iam_binding.puller -target yandex_resourcemanager_folder_iam_binding.pusher
... skipped ...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.Ugbu a naanị ihe fọdụrụ bụ ime igodo maka ya:
vozerov@mba:~/events/terraform (master) $ yc iam key create --service-account-name docker -o key.json
id: ajej8a06kdfbehbrh91p
service_account_id: ajep6d38k895srp9osij
created_at: "2020-04-13T14:00:30Z"
key_algorithm: RSA_2048Anyị na-enweta ozi gbasara id nke nchekwa anyị, bufee igodo wee banye:
vozerov@mba:~/events/terraform (master) $ scp key.json [email protected]:
key.json 100% 2392 215.1KB/s 00:00
vozerov@mba:~/events/terraform (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cat key.json | sudo docker login --username json_key --password-stdin cr.yandex
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
ubuntu@build:~$Iji bulite onyonyo a na ndekọ, anyị chọrọ NJ ndekọ ndekọ akpa, anyị na-ewere ya na ngwa yc:
vozerov@mba:~ $ yc container registry get events
id: crpdgj6c9umdhgaqjfmm
folder_id:
name: events
status: ACTIVE
created_at: "2020-04-13T13:56:41.914Z"Mgbe nke ahụ gasịrị, anyị na-eji aha ọhụrụ kpado onyonyo anyị wee bulite:
ubuntu@build:~$ sudo docker tag app cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
ubuntu@build:~$ sudo docker push cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
The push refers to repository [cr.yandex/crpdgj6c9umdhgaqjfmm/events]
8c286e154c6e: Pushed
477c318b05cb: Pushed
beee9f30bc1f: Pushed
v1: digest: sha256:1dd5aaa9dbdde2f60d833be0bed1c352724be3ea3158bcac3cdee41d47c5e380 size: 946Anyị nwere ike ịchọpụta na a kwajuru onyonyo a nke ọma:
vozerov@mba:~/events/terraform (master) $ yc container repository list
+----------------------+-----------------------------+
| ID | NAME |
+----------------------+-----------------------------+
| crpe8mqtrgmuq07accvn | crpdgj6c9umdhgaqjfmm/events |
+----------------------+-----------------------------+Site n'ụzọ, ọ bụrụ na ị wụnye yc utility na igwe Linux, ị nwere ike iji iwu ahụ
yc container registry configure-dockeriji hazie docker.
nkwubi
Anyị emeela ọtụtụ ọrụ siri ike na n'ihi ya:
- Anyị weputara ihe owuwu nke ọrụ anyị n'ọdịnihu.
- Anyị dere ngwa na golang nke na-emejuputa mgbagha azụmahịa anyị.
- Anyị nakọtara ya ma wụsa ya n'ime akwụkwọ ndekọ aha nkeonwe.
N'akụkụ nke ọzọ, anyị ga-aga n'ihu na ihe ndị na-adọrọ mmasị - anyị ga-ahapụ ngwa anyị n'ime mmepụta ma mesịa malite ibu na ya. Agbanwela!
Ihe a dị na ndekọ vidiyo nke ụlọ ọrụ oghe REBRAIN & Yandex.Cloud: Anyị na-anabata arịrịọ 10 kwa sekọnd na Yandex Cloud -
Ọ bụrụ na ị nwere mmasị ịga mmemme dị otú ahụ n'ịntanetị na ịjụ ajụjụ ozugbo, jikọọ na.
Ọ ga-amasị anyị ịkele Yandex.Cloud pụrụ iche maka ohere ịnabata mmemme dị otú ahụ. Njikọ na ha -
Ọ bụrụ na ịchọrọ ịkwaga igwe ojii ma ọ bụ nwee ajụjụ gbasara akụrụngwa gị, .
PS Anyị nwere nyocha abụọ n'efu kwa ọnwa, ikekwe ọrụ gị ga-abụ otu n'ime ha.
isi: www.habr.com