Kia ora katoa, e hoa ma!
* Ko tenei tuhinga kei runga i te REBRAIN & Yandex.Cloud awheawhe tuwhera, ki te pai koe ki te matakitaki i te ataata, ka kitea e koe i tenei hononga -
I whai waahi taatau ki te whakamatau i a Yandex.Cloud ora. I te mea e hiahia ana matou ki te tirotiro roa me te uaua, ka whakarerea e matou te whakaaro ki te whakarewa i tetahi blog Wordpress ngawari me te turanga kapua - he tino hoha. Whai muri i etahi whakaaro, i whakatau matou ki te tuku i tetahi mea e rite ana ki te hoahoanga ratonga whakangao mo te whiwhi me te tātari i nga huihuinga i roto i te waa tino tata.
E tino mohio ana ahau ko te nuinga o nga umanga ipurangi (a ehara i te mea anake) ka kohikohi i te maunga o nga korero mo o raatau kaiwhakamahi me a raatau mahi. I te iti rawa, he mea tika tenei mo te whakatau - hei tauira, ki te whakahaere koe i tetahi keemu ipurangi, ka taea e koe te titiro ki nga tatauranga kei hea nga kaiwhakamahi taumata e mau tonu ana me te muku i to taakaro. He aha ranei nga kaiwhakamahi i wehe atu ai i to pae me te kore e hoko i tetahi mea (kia ora, Yandex.Metrica).
Na, ko ta maatau korero: me pehea te tuhi tono i roto i te golang, i whakamatauhia te kafka vs rabbitmq vs yqs, i tuhi i te rerenga raraunga i roto i te kohinga Clickhouse me te tiro i nga raraunga ma te whakamahi i nga Yandex Datalens. Ko te tikanga, ko enei mea katoa i whakakakahuhia ki nga taonga whakapaipai i roto i te ahua o te docker, terraform, gitlab ci me, o te akoranga, prometheus. Haere tatou!
E hiahia ana ahau ki te whakarite i tetahi rahui e kore e taea e matou te whirihora i nga mea katoa i roto i te noho kotahi - mo tenei ka hiahia matou ki etahi tuhinga i roto i te raupapa. He iti mo te hanganga:
Wāhanga 1 (kei te panui koe). Ka whakatauhia e matou nga korero me te hoahoanga o te otinga, ka tuhia hoki he tono ki te golang.
Wāhanga 2. Ka tukuna e matou to maatau tono ki roto i nga mahi whakaputa, kia whakahiatohia me te whakamatautau i te kawenga.
Wāhanga 3. Me ngana tatou ki te whakaaro he aha te take me penapena nga karere ki roto i te parapara, kaua ki roto i nga konae, me te whakataurite hoki i te ratonga kafka, rabbitmq me yandex queue.
Wāhanga 4 Ka tohatohahia e matou he kahui Clickhouse, tuhia he ratonga rerema hei whakawhiti raraunga mai i te kaitarai ki reira, ka whakatuu whakakitenga i roto i nga raraunga.
Wāhanga 5 Me kawe te hanganga katoa ki te ahua tika - te whakarite ci/cd ma te whakamahi i te gitlab ci, honoa te aroturuki me te kitenga ratonga ma te whakamahi i te prometheus me te consul.
TK
Tuatahi, me whakatakoto e tatou nga tikanga tohutoro - he aha ta tatou e hiahia ana hei hua.
- Kei te pirangi matou ki te whai waahi mutunga penei i nga events.kis.im (ko kis.im te rohe whakamatautau ka whakamahia e matou puta noa i nga tuhinga katoa), me whiwhi huihuinga ma te whakamahi i te HTTPS.
- Ko nga huihuinga he json ngawari penei: {“event”: “view”, “os”: “linux”, “browser”: “chrome”}. I te waahi whakamutunga ka taapirihia e matou etahi atu mara, engari kaore tenei e whai waahi nui. Ki te hiahia koe, ka taea e koe te huri ki te protobuf.
- Me kaha te ratonga ki te tukatuka i nga kaupapa 10 ia hekona.
- Ka taea te awhatatia whakapae ma te taapiri noa i nga tauira hou ki ta maatau otinga. A he pai mena ka taea e taatau te neke te waahanga o mua ki nga waahi whenua rereke hei whakaiti i te waatea mo nga tono a nga kaihoko.
- Whakaaetanga he. Ko te otinga me tino pumau, ka taea te ora i te hinga o tetahi waahanga (tae atu ki etahi tau, o te akoranga).
hoahoanga
I te nuinga o te waa, mo tenei momo mahi, kua roa kua hangaia nga hoahoanga puāwaitanga e pai ai te whakatauine. Ko te ahua e whakaatu ana i tetahi tauira o ta maatau otinga.
Na he aha ta tatou:
1. Kei te taha maui ko a maatau taputapu e whakaputa ana i nga momo huihuinga, ahakoa ko nga kaitakaro e whakaoti ana i te taumata o te taakaro i runga i te waea atamai, i te hanga ota ranei i roto i te toa tuihono ma te tirotiro tirotiro. Ko tetahi huihuinga, kua tohua i roto i te waahanga, he json ngawari ka tukuna ki to maatau mutunga - events.kis.im.
2. Ko nga tūmau tuatahi e rua he taurite ngawari, ko o raatau mahi matua ko:
- Kia watea tonu. Ki te mahi i tenei, ka taea e koe te whakamahi, hei tauira, keepalived, ka huri i te IP mariko i waenga i nga node mena he raru.
- Whakamutua te TLS. Ae, ka whakamutua e matou te TLS ki runga i a raatau. Tuatahi, kia u to taatau otinga ki nga whakaritenga hangarau, tuarua, kia whakamama ake i te taumahatanga o te whakatuu hononga whakamunatia mai i a maatau tuunga tuara.
- Whakatauritehia nga tono taumai ki nga tuumau tuara e waatea ana. Ko te kupu matua i konei e waatea ana. I runga i tenei, ka mohio matou me kaha nga kaitaurite kawenga ki te aro turuki i o maatau kaitoro me nga tono me te aukati i te whakataurite i nga waka ki nga pona kua rahua.
3. I muri i nga taurite, kei a matou nga kaiwhakarato tono e whakahaere ana i tetahi tono tino ngawari. Ka taea e ia te whakaae ki nga tono taumai ma te HTTP, te whakamana i te json i tukuna me te tuku i nga raraunga ki roto i te papaa.
4. E whakaatu ana te hoahoa i te kafka hei parepare, ahakoa, ko te tikanga, ka taea etahi atu ratonga rite ki tenei taumata. Ka whakatauritehia e matou a Kafka, rabbitmq me yqs i te tuhinga tuatoru.
5. Ko te pito whakamutunga o to maatau hoahoanga ko te Clickhouse - he papaarangi paparangi e taea ai e koe te penapena me te tukatuka i te nui o nga raraunga. I tenei taumata, me whakawhiti i nga raraunga mai i te kaapiri ki te punaha rokiroki ano (he nui ake mo tenei i te tuhinga 4).
Ma tenei hoahoa ka taea e tatou te awhata i ia paparanga motuhake kia whakapae. Kaore e taea e nga kaitoro o muri - me tapiri tetahi mea - inaa, he tono kore whenua, na reira ka taea te mahi aunoa. Kare e mahi te kaitarai-ahua Kafka—taapirihia etahi atu tuumau ka whakawhiti etahi wahanga o ta tatou kaupapa ki a ratou. Kaore e taea e Clickhouse te whakahaere - kaore e taea :) Inaa, ka honoa ano e matou nga kaitoro me te tarai i nga raraunga.
Ma te ara, mena kei te hiahia koe ki te whakatinana i te waahanga waahanga o a maatau tohu hangarau me te tauine i roto i nga waahi whenua rereke, kaore he mea ngawari ake:
I ia geolocation ka tukuna e matou he taurite kawenga me te tono me te kafka. I te nuinga o te waa, e 2 nga kaitoro tono, e 3 nga kafka nodes me te taurite kapua, hei tauira, cloudflare, he nui, ka tirohia te waatea o nga pona tono me nga tono toenga ma te geolocation i runga i te wahitau IP puna a te kaihoko. Na, ko nga raraunga ka tukuna e tetahi kaihoko Amerika ka tau ki runga i nga kaitoro o Amerika. A ko nga raraunga mai i Awherika kei Awherika.
Na he tino ngawari nga mea katoa - ka whakamahia e matou te taputapu whakaata mai i te huinga Kafka me te kape i nga raraunga katoa mai i nga waahi katoa ki to maatau pokapū raraunga pokapū kei Russia. I roto, ka tohatohahia e matou nga raraunga ka tuhia ki Clickhouse mo te tirohanga a muri ake.
Na, kua whakatauhia e matou te hoahoanga - me timata te wiri i a Yandex.Cloud!
Te tuhi tono
I mua i te Kapua, me noho manawanui koe me te tuhi i tetahi ratonga tino ngawari ki te whakahaere i nga huihuinga taumai. Ka whakamahia e matou te golang na te mea kua tino kitea e ia hei reo mo te tuhi tono whatunga.
I muri i te whakapau i te haora (he rua pea nga haora), ka whiwhi tatou i tetahi mea penei: .
He aha nga kaupapa matua e hiahia ana ahau ki te tuhi i konei:
1. Ka timata te tono, ka taea e koe te tohu e rua nga haki. Ko tetahi te kawenga mo te tauranga e whakarongo ai matou ki nga tono http taumai (-addr). Ko te tuarua mo te wahitau tūmau kafka hei tuhi i a maatau kaupapa (-kafka):
addr = flag.String("addr", ":8080", "TCP address to listen to")
kafka = flag.String("kafka", "127.0.0.1:9092", "Kafka endpoints”)2. Ka whakamahia e te tono te whare pukapuka sarama () ki te tuku karere ki te kāhui kafka. Ka tautuhi tonu matou i nga tautuhinga e whai ana ki te tere tere tukatuka:
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForLocal
config.Producer.Compression = sarama.CompressionSnappy
config.Producer.Return.Successes = true3. Kei roto ano i ta maatau tono he kiritaki prometheus kua hangaia, e kohikohi ana i nga momo inenga, penei:
- te maha o nga tono ki ta maatau tono;
- te maha o nga hapa i te wa e mahia ana te tono (kaore e taea te panui i te tono panui, pakaru json, kaore e taea te tuhi ki a Kafka);
- te wa tukatuka mo te tono kotahi mai i te kiritaki, tae atu ki te wa mo te tuhi karere ki a Kafka.
4. E toru nga waahanga mutunga e mahi ana a maatau tono:
- /status - hoki mai pai ki te whakaatu kei te ora tatou. Ahakoa ka taea e koe te taapiri i etahi arowhai, penei i te waatea o te kahui Kafka.
- /metrics - e ai ki tenei url, ka whakahokia e te kiritaki prometheus nga inenga kua kohia e ia.
- Ko te /post te pito matua ka tukuna nga tono POST me te json o roto. Ka tirohia e ta maatau tono te json mo te whaimana a mena he pai nga mea katoa, ka tuhia e ia nga raraunga ki te roopu Kafka.
Ka mahia e au he rahui kaore i te tino tika te waehere - ka taea (me!) te whakaoti. Hei tauira, ka taea e koe te whakamutu i te whakamahi i te kupenga / http i hangaia ka huri ki te http tere tere. Ka taea ranei e koe te whiwhi i te wa tukatuka me nga rauemi cpu ma te neke i te tirotiro tika json ki tetahi waahanga i muri mai - ka whakawhitia nga raraunga mai i te papaa ki te roopu clickhouse.
I tua atu i te taha whanaketanga o te take, i whakaaro tonu matou mo o maatau hanganga a meake nei ka whakatau ki te tuku i a maatau tono ma te docker. Ko te Dockerfile whakamutunga mo te hanga i te tono ko . I te nuinga o te waa, he maamaa noa, ko te kaupapa anake e hiahia ana ahau ki te aro atu ko te huihuinga maha, e taea ai e tatou te whakaiti i te ahua whakamutunga o ta tatou ipu.
Nga hikoinga tuatahi i te kapua
Tuatahi, rēhita i runga . I muri i te whakakii i nga waahi katoa e tika ana, ka hangaia he kaute ka hoatu he putea mo etahi moni, ka taea te whakamahi hei whakamatautau i nga ratonga kapua. Mena kei te hiahia koe ki te whakahoki ano i nga hikoinga katoa mai i ta maatau tuhinga, me ranea tenei tahua mo koe.
I muri i te rehitatanga, ka hangaia he kapua motuhake me tetahi raarangi taunoa mo koe, ka taea e koe te timata ki te hanga rauemi kapua. I te nuinga o te waa, i Yandex.Cloud, he penei te ahua o te hononga o nga rauemi:
Ka taea e koe te hanga i etahi kapua mo te kaute kotahi. A i roto i te kapua, hangahia nga raarangi rereke mo nga kaupapa kamupene rereke. Ka taea e koe te panui atu mo tenei i roto i nga tuhinga - . Ma te ara, he maha nga wa ka tirohia e ahau i raro i te tuhinga. I taku whakaturanga i nga hanganga katoa mai i te wahanga, na nga tuhinga i awhina mai i ahau neke atu i te kotahi, no reira ka tohutohu ahau ki a koe kia ako koe.
Hei whakahaere i te kapua, ka taea e koe te whakamahi i te atanga tukutuku me te taputapu papatohu - yc. Ka mahia te whakaurunga ma te tono kotahi (mo Linux me Mac Os):
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bashMena kei te riri to tohunga mo te haumarutanga o roto mo te whakahaere tuhinga mai i te Ipurangi, katahi ka taea e koe te whakatuwhera i te tuhinga me te panui, tuarua, ka whakahaerehia e matou i raro i to maatau kaiwhakamahi - kaore he mana pakiaka.
Mena kei te hiahia koe ki te whakauru i tetahi kaihoko mo Windows, ka taea e koe te whakamahi i nga tohutohu katahi ka mahia yc initki te whakarite katoa:
vozerov@mba:~ $ yc init
Welcome! This command will take you through the configuration process.
Please go to https://oauth.yandex.ru/authorize?response_type=token&client_id= in order to obtain OAuth token.
Please enter OAuth token:
Please select cloud to use:
[1] cloud-b1gv67ihgfu3bp (id = b1gv67ihgfu3bpt24o0q)
[2] fevlake-cloud (id = b1g6bvup3toribomnh30)
Please enter your numeric choice: 2
Your current cloud has been set to 'fevlake-cloud' (id = b1g6bvup3toribomnh30).
Please choose folder to use:
[1] default (id = b1g5r6h11knotfr8vjp7)
[2] Create a new folder
Please enter your numeric choice: 1
Your current folder has been set to 'default' (id = b1g5r6h11knotfr8vjp7).
Do you want to configure a default Compute zone? [Y/n]
Which zone do you want to use as a profile default?
[1] ru-central1-a
[2] ru-central1-b
[3] ru-central1-c
[4] Don't set default zone
Please enter your numeric choice: 1
Your profile default Compute zone has been set to 'ru-central1-a'.
vozerov@mba:~ $Ko te tikanga, he ngawari te mahi - tuatahi me tiki he tohu oauth hei whakahaere i te kapua, tohua te kapua me te kōpaki ka whakamahia e koe.
Mēnā he maha ō pūkete, kōpaki rānei i roto i te kapua kotahi, ka taea e koe te hanga kōtaha taapiri me nga tautuhinga motuhake ma te yc config profile hanga me te huri i waenga i a raatau.
I tua atu i nga tikanga o runga ake nei, i tuhia e te roopu Yandex.Cloud he tino pai mo te whakahaere i nga rauemi kapua. Mo taku taha, i whakaritea e ahau he putunga git, i whakaahuahia e au nga rauemi katoa ka hangaia hei waahanga o te tuhinga - . Kei te pirangi matou ki te peka matua, me whakakao i te rohe:
vozerov@mba:~ $ git clone https://github.com/rebrainme/yandex-cloud-events/ events
Cloning into 'events'...
remote: Enumerating objects: 100, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 100 (delta 37), reused 89 (delta 26), pack-reused 0
Receiving objects: 100% (100/100), 25.65 KiB | 168.00 KiB/s, done.
Resolving deltas: 100% (37/37), done.
vozerov@mba:~ $ cd events/terraform/Ko nga taurangi matua katoa e whakamahia ana i te terraform ka tuhia ki te konae main.tf. Hei timata, hanga he kōnae private.auto.tfvars ki te kōpaki terraform me nga ihirangi e whai ake nei:
# Yandex Cloud Oauth token
yc_token = ""
# Yandex Cloud ID
yc_cloud_id = ""
# Yandex Cloud folder ID
yc_folder_id = ""
# Default Yandex Cloud Region
yc_region = "ru-central1-a"
# Cloudflare email
cf_email = ""
# Cloudflare token
cf_token = ""
# Cloudflare zone id
cf_zone_id = ""Ka taea te tango i nga taurangi katoa mai i te rarangi whirihora yc, i te mea kua whirihorahia e matou te taputapu papatohu. Ka tohutohu ahau ki a koe ki te taapiri tonu i te private.auto.tfvars ki te .gitignore, kia kore ai e whakaputa pohehe i nga raraunga tūmataiti.
I roto i te private.auto.tfvars i tohua ano e matou nga raraunga mai i Cloudflare - ki te hanga i nga rekoata DNS me te takawaenga i nga kaupapa matua.kis.im ki o maatau tūmau. Mena kaore koe e hiahia ki te whakamahi i te cloudflare, katahi ka tango i te timatanga o te kaiwhakarato cloudflare i main.tf me te konae dns.tf, kei a ia te kawenga mo te hanga i nga rekoata dns e tika ana.
I a maatau mahi ka whakakotahihia e matou nga tikanga e toru - te atanga tukutuku, te taputapu papatohu, me te terraform.
Whatunga mariko
Kia pono, ka taea e koe te peke i tenei taahiraa, na te mea ka waihanga koe i tetahi kapua hou, ka whiwhi aunoa koe i tetahi whatunga motuhake me te hanga kupengaroto e 3 - kotahi mo ia rohe waatea. Engari kei te pirangi tonu matou ki te hanga kupenga motuhake mo ta matou kaupapa me ona ake korero. Ko te hoahoa whanui mo te mahi a te whatunga i Yandex.Cloud e whakaatuhia ana i te ahua i raro nei (i tangohia tika mai i )
Na, ka hangaia e koe he whatunga noa e taea ai e nga rauemi te whakawhitiwhiti korero ki a raatau ano. Mo ia rohe waatea, ka hangaia he kupengaroto me ona ake korero, ka hono ki te whatunga whanui. Ko te mutunga, ko nga rauemi kapua katoa kei roto ka taea te whakawhitiwhiti korero, ahakoa kei roto i nga waahi rereke rereke. Ko nga rauemi e hono ana ki nga whatunga kapua rereke ka kite tetahi i tetahi ma nga wahitau o waho anake. Ma te ara, me pehea te mahi makutu o roto, .
Kei te whakaahuahia te hanga whatunga ki te konae network.tf mai i te putunga. I reira ka hangahia e matou tetahi whatunga motuhake noa o roto ka hono atu e toru nga kupenga-roto ki a ia i roto i nga waahi rereke rereke - roto-a (172.16.1.0/24), roto-b (172.16.2.0/24), roto-c (172.16.3.0/24 ).
Arawhiti te terraform me te hanga whatunga:
vozerov@mba:~/events/terraform (master) $ terraform init
... skipped ..
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_vpc_subnet.internal-a -target yandex_vpc_subnet.internal-b -target yandex_vpc_subnet.internal-c
... skipped ...
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
yandex_vpc_network.internal: Creating...
yandex_vpc_network.internal: Creation complete after 3s [id=enp2g2rhile7gbqlbrkr]
yandex_vpc_subnet.internal-a: Creating...
yandex_vpc_subnet.internal-b: Creating...
yandex_vpc_subnet.internal-c: Creating...
yandex_vpc_subnet.internal-a: Creation complete after 6s [id=e9b1dad6mgoj2v4funog]
yandex_vpc_subnet.internal-b: Creation complete after 7s [id=e2liv5i4amu52p64ac9p]
yandex_vpc_subnet.internal-c: Still creating... [10s elapsed]
yandex_vpc_subnet.internal-c: Creation complete after 10s [id=b0c2qhsj2vranoc9vhcq]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.Rawe! Kua hanga e matou ta matou whatunga, kua reri inaianei ki te hanga i a matou ratonga o roto.
Te hanga miihini mariko
Hei whakamatautau i te tono, me hanga noa e matou nga miihini mariko e rua - ka hiahia matou ki te tuatahi ki te hanga me te whakahaere i te tono, te tuarua ki te whakahaere i te kafka, ka whakamahia e matou ki te penapena i nga karere taumai. A ka hanga e matou tetahi atu miihini ka whirihorahia e matou te prometheus hei aroturuki i te tono.
Ka whirihorahia nga miihini mariko ma te whakamahi i te ansible, no reira i mua i te tiimata i te terraform, kia mohio kei a koe tetahi o nga putanga hou o te ansible. Me te whakauru i nga mahi e tika ana me te tupuni ansible:
vozerov@mba:~/events/terraform (master) $ cd ../ansible/
vozerov@mba:~/events/ansible (master) $ ansible-galaxy install -r requirements.yml
- cloudalchemy-prometheus (master) is already installed, skipping.
- cloudalchemy-grafana (master) is already installed, skipping.
- sansible.kafka (master) is already installed, skipping.
- sansible.zookeeper (master) is already installed, skipping.
- geerlingguy.docker (master) is already installed, skipping.
vozerov@mba:~/events/ansible (master) $Kei roto i te kōpaki ansible he tauira konae whirihoranga .ansible.cfg ka whakamahia e au. Ka whai hua pea.
I mua i te hanga mihini mariko, kia mohio kei a koe te ssh-agent e whakahaere ana me te taapiri ssh key, ki te kore ka kore e taea e te terraform te hono atu ki nga mihini i hangaia. Ae ra, i kite ahau i tetahi bug i roto i te os x: . Kia kore ai e puta ano, taapirihia he taurangi iti ki te env i mua i te whakarewatanga o Terraform:
vozerov@mba:~/events/terraform (master) $ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YESI roto i te kōpaki me te terraform ka hangaia e matou nga rauemi e tika ana:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_compute_instance.build -target yandex_compute_instance.monitoring -target yandex_compute_instance.kafka
yandex_vpc_network.internal: Refreshing state... [id=enp2g2rhile7gbqlbrkr]
data.yandex_compute_image.ubuntu_image: Refreshing state...
yandex_vpc_subnet.internal-a: Refreshing state... [id=e9b1dad6mgoj2v4funog]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
... skipped ...
Plan: 3 to add, 0 to change, 0 to destroy.
... skipped ...Mena i pai te mutunga o nga mea katoa (e tika ana), ka toru nga miihini mariko:
- hanga - he miihini mo te whakamatautau me te hanga tono. I whakauruhia aunoatia a Docker e Ansible.
- aroturuki - he mihini aroturuki - prometheus & grafana i whakauruhia ki runga. Paerewa takiuru / kupuhipa: kaiwhakahaere / kaiwhakahaere
- Ko te kafka he miihini iti me te kafka kua whakauruhia, ka uru ki te tauranga 9092.
Me whakarite kei te noho katoa ratou:
vozerov@mba:~/events (master) $ yc compute instance list
+----------------------+------------+---------------+---------+---------------+-------------+
| ID | NAME | ZONE ID | STATUS | EXTERNAL IP | INTERNAL IP |
+----------------------+------------+---------------+---------+---------------+-------------+
| fhm081u8bkbqf1pa5kgj | monitoring | ru-central1-a | RUNNING | 84.201.159.71 | 172.16.1.35 |
| fhmf37k03oobgu9jmd7p | kafka | ru-central1-a | RUNNING | 84.201.173.41 | 172.16.1.31 |
| fhmt9pl1i8sf7ga6flgp | build | ru-central1-a | RUNNING | 84.201.132.3 | 172.16.1.26 |
+----------------------+------------+---------------+---------+---------------+-------------+
Kei te waahi nga rauemi, a mai i konei ka taea e taatau te tiki i o raatau wahitau IP. I roto i nga mea e whai ake nei ka whakamahi ahau i nga wahitau IP hei hono ma te ssh me te whakamatautau i te tono. Mena kei a koe he putea cloudflare e hono ana ki te terraform, me pai ki te whakamahi i nga ingoa DNS hou i hangaia.
Ma te ara, i te wa e hanga ana he miihini mariko, ka tukuna he IP o roto me tetahi ingoa DNS o roto, na reira ka taea e koe te uru atu ki nga kaitoro i roto i te whatunga ma te ingoa:
ubuntu@build:~$ ping kafka.ru-central1.internal
PING kafka.ru-central1.internal (172.16.1.31) 56(84) bytes of data.
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=1 ttl=63 time=1.23 ms
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=2 ttl=63 time=0.625 ms
^C
--- kafka.ru-central1.internal ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.625/0.931/1.238/0.308 msKa whai hua tenei mo taatau ki te tohu ki te tono te pito mutunga me te kafk.
Te whakaemi i te tono
He rawe, he tūmau, he tono - ko te mea e toe ana ko te whakahiato me te whakaputa. Mo te hanga ka whakamahia e matou te hanga docker o mua, engari hei rokiroki whakaahua ka whakamahia e matou he ratonga mai i Yandex - rehita ipu. Engari ko nga mea tuatahi.
Ka kapehia e matou te tono ki te miihini hanga, takiuru ma te ssh ka kohia te ahua:
vozerov@mba:~/events/terraform (master) $ cd ..
vozerov@mba:~/events (master) $ rsync -av app/ [email protected]:app/
... skipped ...
sent 3849 bytes received 70 bytes 7838.00 bytes/sec
total size is 3644 speedup is 0.93
vozerov@mba:~/events (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cd app
ubuntu@build:~/app$ sudo docker build -t app .
Sending build context to Docker daemon 6.144kB
Step 1/9 : FROM golang:latest AS build
... skipped ...
Successfully built 9760afd8ef65
Successfully tagged app:latestKua oti te haurua o te whawhai - inaianei ka taea e taatau te tirotiro i te mahi o ta maatau tono ma te whakarewa me te tuku ki te kafka:
ubuntu@build:~/app$ sudo docker run --name app -d -p 8080:8080 app /app/app -kafka=kafka.ru-central1.internal:9092</code>
С локальной машинки можно отправить тестовый event и посмотреть на ответ:
<code>vozerov@mba:~/events (master) $ curl -D - -s -X POST -d '{"key1":"data1"}' http://84.201.132.3:8080/post
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 13 Apr 2020 13:53:54 GMT
Content-Length: 41
{"status":"ok","partition":0,"Offset":0}
vozerov@mba:~/events (master) $I whakautu te tono me te angitu o te rekoata me te tohu i te id o te wehewehenga me te taapiri i whakauruhia ai te karere. Ko nga mea katoa e toe ana ko te hanga i tetahi rehitatanga i Yandex.Cloud me te tuku i to maatau ahua ki reira (me pehea te mahi ma te whakamahi i nga rarangi e toru e whakaahuatia ana i roto i te registry.tf file). Waihangatia he rokiroki:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_container_registry.events
... skipped ...
Plan: 1 to add, 0 to change, 0 to destroy.
... skipped ...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.He maha nga huarahi hei whakamotuhēhē i roto i te rehita ipu - ma te whakamahi i te tohu oauth, i te tohu iam, i te taviri kaute ratonga ranei. Ko etahi atu korero mo enei tikanga ka kitea i roto i nga tuhinga. . Ka whakamahia e matou te taviri putea ratonga, na reira ka hangaia he kaute:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_iam_service_account.docker -target yandex_resourcemanager_folder_iam_binding.puller -target yandex_resourcemanager_folder_iam_binding.pusher
... skipped ...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.Ko nga mea katoa e toe ana ko te hanga i tetahi kii mo taua mea:
vozerov@mba:~/events/terraform (master) $ yc iam key create --service-account-name docker -o key.json
id: ajej8a06kdfbehbrh91p
service_account_id: ajep6d38k895srp9osij
created_at: "2020-04-13T14:00:30Z"
key_algorithm: RSA_2048Ka whiwhi korero matou mo te id o to matou rokiroki, whakawhiti i te ki me te takiuru:
vozerov@mba:~/events/terraform (master) $ scp key.json [email protected]:
key.json 100% 2392 215.1KB/s 00:00
vozerov@mba:~/events/terraform (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cat key.json | sudo docker login --username json_key --password-stdin cr.yandex
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
ubuntu@build:~$Hei tuku ake i te ahua ki te rehitatanga, ka hiahia matou ki te ID rehita ipu, tangohia mai i te whaipainga yc:
vozerov@mba:~ $ yc container registry get events
id: crpdgj6c9umdhgaqjfmm
folder_id:
name: events
status: ACTIVE
created_at: "2020-04-13T13:56:41.914Z"I muri i tera, ka tohuhia to maatau whakaahua me te ingoa hou ka tukuna:
ubuntu@build:~$ sudo docker tag app cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
ubuntu@build:~$ sudo docker push cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
The push refers to repository [cr.yandex/crpdgj6c9umdhgaqjfmm/events]
8c286e154c6e: Pushed
477c318b05cb: Pushed
beee9f30bc1f: Pushed
v1: digest: sha256:1dd5aaa9dbdde2f60d833be0bed1c352724be3ea3158bcac3cdee41d47c5e380 size: 946Ka taea e tatou te manatoko kua oti te utaina te atahanga:
vozerov@mba:~/events/terraform (master) $ yc container repository list
+----------------------+-----------------------------+
| ID | NAME |
+----------------------+-----------------------------+
| crpe8mqtrgmuq07accvn | crpdgj6c9umdhgaqjfmm/events |
+----------------------+-----------------------------+Ma te ara, mena ka whakauruhia e koe te taputapu yc i runga i te miihini Linux, ka taea e koe te whakamahi i te whakahau
yc container registry configure-dockerki te whirihora docker.
mutunga
He maha nga mahi i mahia e matou, a, ko te mutunga:
- I puta mai i a maatau te hoahoanga o ta maatau ratonga a meake nei.
- I tuhia e matou he tono i roto i te golang e whakatinana ana i o maatau whakaaro pakihi.
- I kohia e matou ka ringihia ki roto i te rehita ipu motuhake.
I te waahanga e whai ake nei, ka neke atu tatou ki nga mea whakamere - ka tukuna e tatou ta tatou tono ki te whakaputanga ka mutu ka whakarewahia te uta ki runga. Kaua e huri!
Ko tenei rauemi kei roto i te rekoata ataata o te awheawhe tuwhera REBRAIN & Yandex.Cloud: Ka whakaaetia e matou nga tono 10 mo ia hekona i runga i te Yandex Cloud -
Mena kei te pirangi koe ki te haere ki nga huihuinga penei i runga ipurangi me te patai i nga paatai i roto i te waa, hono atu ki.
Ka mihi nui matou ki a Yandex.Cloud mo te whai waahi ki te manaaki i taua huihuinga. Hononga ki a raatau -
Mena ka hiahia koe ki te neke ki te kapua, he patai ranei mo to hanganga, .
PS E 2 a matou arotake kore utu ia marama, tera pea ko to kaupapa tetahi o era.
Source: will.com