Lumelang bohle, metsoalle!
* Sengoliloeng sena se thehiloe ho REBRAIN & Yandex.Cloud workshop e bulehileng, haeba u rata ho shebella video, u ka e fumana sehokelong sena -
Haufinyane tjena re bile le monyetla oa ho leka Yandex.Cloud live. Kaha re ne re batla ho hlahloba nako e telele le ka thata, hang-hang re ile ra lahla khopolo ea ho qala blog e bonolo ea Wordpress e nang le leru la motheo - e ne e le monate haholo. Ka mor'a ho nahana, re ile ra etsa qeto ea ho sebelisa ntho e tšoanang le mohaho oa litšebeletso tsa tlhahiso bakeng sa ho amohela le ho hlahloba liketsahalo tse haufi le nako ea sebele.
Ke na le bonnete ba hore boholo ba likhoebo tsa marang-rang (eseng feela) li bokella lintlha tse ngata mabapi le basebelisi ba tsona le liketso tsa bona. Bonyane, sena sea hlokahala bakeng sa ho etsa liqeto tse itseng - ka mohlala, haeba u laola papali ea inthanete, u ka sheba lipalo-palo tseo basebelisi ba atisang ho khomarela le ho hlakola ntho ea hau ea ho bapala. Kapa ke hobane'ng ha basebelisi ba tlohela sebaka sa hau sa marang-rang ntle le ho reka letho (hello, Yandex.Metrica).
Kahoo, pale ea rona: kamoo re ngotseng kopo kateng ho golang, ho lekoa kafka vs rabbitmq vs yqs, ho ngola phallo ea data ka har'a sehlopha sa Clickhouse mme ra bona data ka pono e sebelisang yandex datalens. Ka tlhaho, sena sohle se ne se natefisitsoe ke menyaka ea meaho ka sebopeho sa docker, terraform, gitlab ci le, ehlile, prometheus. Ha re ee!
Ke kopa ho etsa pehelo hang-hang hore re ke ke ra khona ho hlophisa ntho e 'ngoe le e' ngoe ka nako e le 'ngoe - bakeng sa sena re tla hloka lihlooho tse' maloa letotong. Hanyenyane ka sebopeho:
Karolo ea 1 (u ntse u e bala). Re tla etsa qeto ka litlhaloso le meralo ea tharollo, hape re ngole kopo ka golang.
Karolo ea 2. Re lokolla kopo ea rona hore e be tlhahiso, e etsa hore e fokotsehe le ho leka mojaro.
Karolo ea 3. Ha re leke ho fumana hore na ke hobane'ng ha re hloka ho boloka melaetsa ka har'a buffer eseng lifaeleng, hape re bapise tšebeletso ea queue ea kafka, rabbitmq le yandex.
Karolo ea 4 Re tla sebelisa sehlopha sa Clickhouse, re ngole tšebeletso ea ho phallela ho fetisetsa data ho tswa ho buffer moo, le ho theha pono ho datalens.
Karolo ea 5 Ha re tliseng lits'ebetso tsohle ka sebopeho se nepahetseng - theha ci/cd u sebelisa gitlab ci, hokahanya tlhahlobo le ho sibolla litšebeletso ka prometheus le consul.
TK
Taba ea pele, a re qaleng ka ho etsa litšupiso - seo re batlang ho se fumana ka lebaka leo.
- Re batla ho ba le qetello e kang events.kis.im (kis.im ke sebaka sa teko seo re tla se sebelisa ho pholletsa le lihlooho tsohle), tse lokelang ho amohela liketsahalo ho sebelisa HTTPS.
- Liketsahalo ke json e bonolo joalo ka: {" ketsahalo": "view", "os": "linux", "browser": "chrome"}. Mothating oa ho qetela re tla eketsa masimo a mang, empa sena se ke ke sa bapala karolo e kholo. Haeba u lakatsa, u ka fetohela ho protobuf.
- Tšebeletso e tlameha ho khona ho sebetsana le liketsahalo tsa 10 ka motsotsoana.
- E lokela ho khonahala ho hola ho ea holimo ka ho kenyelletsa mehlala e mecha tharollong ea rona. 'Me ho tla ba monate ha re ka tsamaisa karolo e ka pele ho li-geolocation tse fapaneng ho fokotsa latency bakeng sa likopo tsa bareki.
- Ho mamella liphoso. Tharollo e tlameha ho ba e tsitsitseng ka ho lekaneng mme e khone ho pholoha ho oa ha likarolo leha e le life (ho fihlela palo e itseng, ho hlakile).
mehaho
Ka kakaretso, bakeng sa mofuta ona oa mosebetsi, meralo ea khale ea khale e qapiloe e lumellang ho lekanya hantle. Setšoantšo se bontša mohlala oa tharollo ea rona.
Kahoo seo re nang le sona:
1. Ka ho le letšehali ho na le lisebelisoa tsa rona tse hlahisang liketsahalo tse fapa-fapaneng, ebang ke libapali tse tlatsang boemo ba ho bapala ho smartphone kapa ho etsa odara lebenkeleng la inthanete ka sebatli se tloaelehileng. Ketsahalo, joalo ka ha ho hlalositsoe polelong ea mosebetsi, ke json e bonolo e rometsoeng pheletsong ea rona - events.kis.im.
2. Li-server tse peli tsa pele ke li-balancers tse bonolo, mesebetsi ea tsona ea mantlha ke:
- E-ba teng kamehla. Ho etsa sena, o ka sebelisa, ho etsa mohlala, keepalived, e leng ho tla switjha ho ka thoeng IP pakeng tsa nodes haeba ho na le mathata.
- Felisa TLS. E, re tla felisa TLS ho tsona. Taba ea pele, e le hore tharollo ea rona e lumellane le litlhaloso tsa tekheniki, 'me ea bobeli, e le ho imolla moroalo oa ho theha khokahano e patiloeng ho tsoa ho li-server tsa rona tsa backend.
- Leka-lekanya likopo tse kenang ho li-server tse fumanehang ka morao. Lentsoe la bohlokoa mona lea fumaneha. Ho latela sena, re utloisisa hore li-balancers tsa mojaro li tlameha ho khona ho beha leihlo li-server tsa rona ka lits'ebetso le ho emisa ho leka-lekanya sephethephethe ho li-node tse hlōlehileng.
3. Ka mor'a li-balancers, re na le li-server tsa kopo tse sebetsang ka mokhoa o bonolo. E lokela ho khona ho amohela likopo tse kenang ka HTTP, ho netefatsa json e rometsoeng le ho kenya data ka har'a buffer.
4. Setšoantšo se bontša kafka e le buffer, le hoja, ha e le hantle, litšebeletso tse ling tse tšoanang li ka sebelisoa boemong bona. Re tla bapisa Kafka, rabbitmq le yqs sehloohong sa boraro.
5. Ntlha ea pele ea meralo ea rona ke Clickhouse - database ea columnar e u lumellang ho boloka le ho sebetsana le palo e kholo ea data. Boemong bona, re hloka ho fetisetsa data ho tloha ho buffer ho ea tsamaisong ea polokelo ka boeona (ho feta ka sena sehloohong sa 4).
Moqapi ona o re lumella ho lekanya lera le leng le le leng ka boikemelo ka mokhoa o otlolohileng. Li-server tse ka morao ha li khone ho sebetsana le boemo - ha re kenyeng ntho e 'ngoe hape - ka mor'a moo, ke lits'ebetso tse se nang palo, ka hona, sena se ka etsoa le ka bohona. Sesebelisoa sa mokhoa oa Kafka ha se sebetse - ha re kenye li-server tse ling 'me re fetisetse likarolo tse ling tsa sehlooho sa rona ho tsona. Clickhouse ha e khone ho e sebetsana - ha e khonehe :) Ha e le hantle, re tla boela re hokahane le li-server le ho arola data.
Ha e le hantle, haeba u batla ho kenya ts'ebetsong karolo ea boikhethelo ea litlhaloso tsa rona tsa tekheniki le tekanyo ea libaka tse fapaneng tsa geolocation, ha ho letho le bonolo ho feta:
Sebakeng se seng le se seng sa geolocation re kenya sekala sa mojaro ka kopo le kafka. Ka kakaretso, li-server tse 2 tsa kopo, 3 kafka nodes le cloud balancer, mohlala, cloudflare, li lekane, tse tla hlahloba ho fumaneha ha li-node tsa kopo le likōpo tsa ho leka-lekana ka geolocation e thehiloeng ho aterese ea IP ea mohloli oa mofani. Kahoo, data e rometsoeng ke moreki oa Amerika e tla lula ho li-server tsa Amerika. Mme data e tsoang Afrika ke ea Afrika.
Joale ntho e 'ngoe le e' ngoe e bonolo haholo - re sebelisa sesebelisoa sa seipone ho tloha ho sete ea Kafka le ho kopitsa lintlha tsohle ho tloha libakeng tsohle ho ea setsing sa rona sa data se bohareng se Russia. Ka hare, re arola lintlha ebe re li rekota ho Clickhouse bakeng sa pono e latelang.
Kahoo, re hlophisitse meralo ea kaho - ha re qaleng ho sisinya Yandex.Cloud!
Ho ngola kopo
Pele ho Leru, o ntse o tlameha ho ba le mamello e nyane mme o ngole ts'ebeletso e bonolo ho sebetsana le liketsahalo tse tlang. Re tla sebelisa golang hobane e ipakile e le puo ea ho ngola likopo tsa marang-rang.
Kamora ho qeta hora (mohlomong lihora tse 'maloa), re fumana ntho e kang ena: .
Ke lintlha life tsa bohlokoa tseo ke ratang ho li ela hloko mona:
1. Ha o qala kopo, o ka bolela lifolakha tse peli. E mong ke eena ea ikarabellang bakeng sa koung eo re tla mamela likopo tse tlang tsa http (-addr). Ea bobeli ke ea aterese ea seva ea kafka moo re tla rekota liketsahalo tsa rona (-kafka):
addr = flag.String("addr", ":8080", "TCP address to listen to")
kafka = flag.String("kafka", "127.0.0.1:9092", "Kafka endpoints”)2. Sesebelisoa se sebelisa laeborari ea sarama () ho romela melaetsa ho sehlopha sa kafka. Hang-hang re seta li-setting tse reretsoeng lebelo le phahameng la ho sebetsa:
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForLocal
config.Producer.Compression = sarama.CompressionSnappy
config.Producer.Return.Successes = true3. Kopo ea rona e boetse e na le moreki ea hahelletsoeng ka har'a prometheus, ea bokellang metrics e fapaneng, joalo ka:
- palo ea likopo ho kopo ea rona;
- palo ea liphoso ha u etsa kopo (ho ke ke ha khoneha ho bala kopo ea poso, json e robehileng, ho ke ke ha khoneha ho ngolla Kafka);
- nako ea ho sebetsa bakeng sa kopo e le 'ngoe ho tsoa ho mofani, ho kenyelletsa le nako ea ho ngola molaetsa ho Kafka.
4. Lintlha tse tharo tseo kopo ea rona e li sebetsang:
- /maemo - kgutlela hantle ho bontsha hore re a phela. Leha o ka eketsa licheke, joalo ka boteng ba sehlopha sa Kafka.
- /metrics - ho latela url ena, moreki oa prometheus o tla khutlisa metrics eo a e bokeletseng.
- /post ke pheletso ea mantlha moo likopo tsa POST tse nang le json kahare li tla romelloa. Kopo ea rona e hlahloba json hore na e nepahetse 'me haeba ntho e' ngoe le e 'ngoe e lokile, e ngola lintlha ho sehlopha sa Kafka.
Ke tla etsa pehelo ea hore khoutu ha ea phethahala - e ka (mme e lokela ho phethoa). Mohlala, o ka emisa ho sebelisa net/http e hahelletsoeng 'me u fetohele ho lebelo le potlakilenghttp. Kapa u ka fumana nako ea ho sebetsa le lisebelisoa tsa cpu ka ho tsamaisa cheke ea netefatso ea json hamorao - ha data e fetisoa ho tloha ho buffer ho ea sehlopheng sa Clickhouse.
Ntle le lehlakore la nts'etsopele ea taba, hang-hang re ile ra nahana ka lisebelisoa tsa rona tsa kamoso mme ra etsa qeto ea ho tsamaisa kopo ea rona ka docker. Dockerfile ea ho qetela ea ho aha kopo ke . Ka kakaretso, e bonolo haholo, ntlha feela eo ke ratang ho e ela hloko ke kopano ea li-multistage, e re lumellang ho fokotsa setšoantšo sa ho qetela sa setshelo sa rona.
Mehato ea pele lerung
Pele ho tsohle, ngodisa ho . Ka mor'a ho tlatsa masimo ohle a hlokahalang, re tla bōptjoa akhaonto mme re fuoe thuso bakeng sa chelete e itseng, e ka sebelisetsoang ho hlahloba litšebeletso tsa leru. Haeba u batla ho pheta mehato eohle ea sengoloa sa rona, thuso ena e lokela ho lekana bakeng sa hau.
Kamora ho ngolisoa, ho tla etsoa leru le arohaneng le lethathamo la kamehla, leo ho lona u ka qalang ho theha lisebelisoa tsa maru. Ka kakaretso, ho Yandex.Cloud, kamano ea lisebelisoa e shebahala tjena:
U ka etsa maru a 'maloa bakeng sa akhaonto e le' ngoe. 'Me ka har'a leru, etsa li-directory tse fapaneng bakeng sa merero e fapaneng ea lik'hamphani. U ka bala haholoanyane ka sena litokomaneng - . Ka tsela, hangata ke tla bua ka eona ka tlase ho mongolo. Ha ke theha lisebelisoa tsohle ho tloha qalong, litokomane li ile tsa nthusa ho feta hang, kahoo ke u eletsa hore u ithute eona.
Ho laola leru, o ka sebelisa sebopeho sa marang-rang le sesebelisoa sa console - yc. Ho kenya ho etsoa ka taelo e le 'ngoe (bakeng sa Linux le Mac Os):
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bashHaeba setsebi sa hau sa ts'ireletso ea ka hare se ntse se halefela ka ho sebelisa mangolo a tsoang Inthaneteng, joale, pele, u ka bula script le ho e bala, 'me ea bobeli, re e tsamaisa tlas'a mosebedisi oa rona - ntle le litokelo tsa motso.
Haeba u batla ho kenya sesebelisoa bakeng sa Windows, u ka sebelisa litaelo ebe o phethahatsa yc initho e etsa ka botlalo:
vozerov@mba:~ $ yc init
Welcome! This command will take you through the configuration process.
Please go to https://oauth.yandex.ru/authorize?response_type=token&client_id= in order to obtain OAuth token.
Please enter OAuth token:
Please select cloud to use:
[1] cloud-b1gv67ihgfu3bp (id = b1gv67ihgfu3bpt24o0q)
[2] fevlake-cloud (id = b1g6bvup3toribomnh30)
Please enter your numeric choice: 2
Your current cloud has been set to 'fevlake-cloud' (id = b1g6bvup3toribomnh30).
Please choose folder to use:
[1] default (id = b1g5r6h11knotfr8vjp7)
[2] Create a new folder
Please enter your numeric choice: 1
Your current folder has been set to 'default' (id = b1g5r6h11knotfr8vjp7).
Do you want to configure a default Compute zone? [Y/n]
Which zone do you want to use as a profile default?
[1] ru-central1-a
[2] ru-central1-b
[3] ru-central1-c
[4] Don't set default zone
Please enter your numeric choice: 1
Your profile default Compute zone has been set to 'ru-central1-a'.
vozerov@mba:~ $Ha e le hantle, mokhoa ona o bonolo - pele o hloka ho fumana letšoao la kano ho laola leru, khetha leru le foldara eo u tla e sebelisa.
Haeba u na le li-account kapa lifoldara tse 'maloa ka har'a leru le le leng, u ka theha li-profiles tse ling ka litlhophiso tse arohaneng ka yc config profil mme o fetole lipakeng tsa tsona.
Ho phaella mekhoeng e ka holimo, sehlopha sa Yandex.Cloud se ngotse se setle haholo bakeng sa ho laola mehloli ea maru. Ka lehlakoreng la ka, ke lokiselitse polokelo ea git, moo ke hlalositseng lisebelisoa tsohle tse tla etsoa e le karolo ea sengoloa - . Re thahasella lekala le leholo, ha re e kopanye sebakeng sa heno:
vozerov@mba:~ $ git clone https://github.com/rebrainme/yandex-cloud-events/ events
Cloning into 'events'...
remote: Enumerating objects: 100, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 100 (delta 37), reused 89 (delta 26), pack-reused 0
Receiving objects: 100% (100/100), 25.65 KiB | 168.00 KiB/s, done.
Resolving deltas: 100% (37/37), done.
vozerov@mba:~ $ cd events/terraform/Liphetoho tsohle tse ka sehloohong tse sebelisoang ho terraform li ngotsoe faeleng ea main.tf. Ho qala, theha faele ea private.auto.tfvars ho foldara ea terraform ka litaba tse latelang:
# Yandex Cloud Oauth token
yc_token = ""
# Yandex Cloud ID
yc_cloud_id = ""
# Yandex Cloud folder ID
yc_folder_id = ""
# Default Yandex Cloud Region
yc_region = "ru-central1-a"
# Cloudflare email
cf_email = ""
# Cloudflare token
cf_token = ""
# Cloudflare zone id
cf_zone_id = ""Liphetoho tsohle li ka nkuoa lethathamong la yc config, kaha re se re hlophisitse sesebelisoa sa console. Ke u eletsa hore hang-hang u kenye "private.auto.tfvars" ho .gitignore, e le hore u se ke ua phatlalatsa boitsebiso ba lekunutu ka phoso.
Ka lekunutu.auto.tfvars re boetse re hlalositse lintlha tse tsoang ho Cloudflare - ho theha lirekoto tsa DNS le proxy domain main events.kis.im ho li-server tsa rona. Haeba u sa batle ho sebelisa cloudflare, joale tlosa ho qalisoa ha mofani oa cloudflare ho main.tf le faele ea dns.tf, e ikarabellang bakeng sa ho theha lirekoto tse hlokahalang tsa dns.
Mosebetsing oa rona re tla kopanya mekhoa eohle e meraro - sebopeho sa marang-rang, sesebelisoa sa console le terraform.
Marang-rang a Virtual
Ho bua 'nete, u ka tlola mohato ona, kaha ha u theha leru le lecha, u tla ba le marang-rang a arohaneng le li-subnet tse 3 tse entsoeng - e le' ngoe bakeng sa sebaka se seng le se seng sa ho fumaneha. Empa re ntse re ka rata ho etsa marang-rang a arohaneng bakeng sa morero oa rona ka liaterese tsa ona. Setšoantšo se akaretsang sa kamoo marang-rang a sebetsang kateng ho Yandex.Cloud se bontšoa setšoantšong se ka tlase (ka botšepehi se nkiloe ho )
Kahoo, o theha marang-rang a tloaelehileng ao mehloli ea lisebelisoa e ka buisanang ka eona. Bakeng sa sebaka se seng le se seng sa ho fumaneha, subnet e etsoa e nang le liaterese tsa eona 'me e hokahane le marang-rang a akaretsang. Ka lebaka leo, lisebelisoa tsohle tsa leru ho eona li ka buisana, le haeba li le libakeng tse fapaneng tsa ho fumaneha. Lisebelisoa tse hokahaneng le marang-rang a fapaneng a maru li ka bonana feela ka liaterese tsa kantle. Ka tsela, boselamose bona bo sebetsa joang kahare, .
Tlhahiso ea marang-rang e hlalositsoe faeleng ea network.tf ho tloha polokelong. Ha re le moo re theha marang-rang a le mong a tloaelehileng a poraefete ka hare mme re hokahanya li-subnets tse tharo ho eona libakeng tse fapaneng tsa ho fumaneha - ka hare-a (172.16.1.0/24), ka hare-b (172.16.2.0/24), ka hare-c (172.16.3.0/24) ).
Qala terraform mme u thehe marang-rang:
vozerov@mba:~/events/terraform (master) $ terraform init
... skipped ..
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_vpc_subnet.internal-a -target yandex_vpc_subnet.internal-b -target yandex_vpc_subnet.internal-c
... skipped ...
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
yandex_vpc_network.internal: Creating...
yandex_vpc_network.internal: Creation complete after 3s [id=enp2g2rhile7gbqlbrkr]
yandex_vpc_subnet.internal-a: Creating...
yandex_vpc_subnet.internal-b: Creating...
yandex_vpc_subnet.internal-c: Creating...
yandex_vpc_subnet.internal-a: Creation complete after 6s [id=e9b1dad6mgoj2v4funog]
yandex_vpc_subnet.internal-b: Creation complete after 7s [id=e2liv5i4amu52p64ac9p]
yandex_vpc_subnet.internal-c: Still creating... [10s elapsed]
yandex_vpc_subnet.internal-c: Creation complete after 10s [id=b0c2qhsj2vranoc9vhcq]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.E kholo! Re thehile marang-rang a rona mme joale re itokiselitse ho theha lits'ebeletso tsa rona tsa kahare.
Ho theha mechine ea sebele
Ho hlahloba kopo, re tla hloka feela ho etsa mechine e 'meli ea sebele - re tla hloka ea pele ho haha le ho tsamaisa kopo, ea bobeli ho tsamaisa kafka, eo re tla e sebelisa ho boloka melaetsa e kenang. Mme re tla theha mochini o mong moo re tla hlophisa prometheus ho beha leihlo ts'ebeliso.
Mechini ea sebele e tla hlophisoa ka mokhoa o utloahalang, kahoo pele u qala terraform, etsa bonnete ba hore u na le e 'ngoe ea liphetolelo tsa morao-rao tsa ansible. 'Me u kenye likarolo tse hlokahalang ka sehlopha sa linaleli se utloahalang:
vozerov@mba:~/events/terraform (master) $ cd ../ansible/
vozerov@mba:~/events/ansible (master) $ ansible-galaxy install -r requirements.yml
- cloudalchemy-prometheus (master) is already installed, skipping.
- cloudalchemy-grafana (master) is already installed, skipping.
- sansible.kafka (master) is already installed, skipping.
- sansible.zookeeper (master) is already installed, skipping.
- geerlingguy.docker (master) is already installed, skipping.
vozerov@mba:~/events/ansible (master) $Ka hare ho foldara e utloahalang ho na le mohlala oa .ansible.cfg configuration file eo ke e sebelisang. E ka ba molemo.
Pele o theha mechini e sebetsang, etsa bonnete ba hore o na le ssh-agent e sebetsang le senotlolo sa ssh se kentsoeng, ho seng joalo terraform e ke ke ea khona ho hokela mecheng e entsoeng. Ehlile, ke ile ka kopana le bothata ho os x: . Ho thibela sena hore se se ke sa etsahala hape, eketsa phetoho e nyane ho env pele o qala Terraform:
vozerov@mba:~/events/terraform (master) $ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YESKa foldareng e nang le terraform re theha lisebelisoa tse hlokahalang:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_compute_instance.build -target yandex_compute_instance.monitoring -target yandex_compute_instance.kafka
yandex_vpc_network.internal: Refreshing state... [id=enp2g2rhile7gbqlbrkr]
data.yandex_compute_image.ubuntu_image: Refreshing state...
yandex_vpc_subnet.internal-a: Refreshing state... [id=e9b1dad6mgoj2v4funog]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
... skipped ...
Plan: 3 to add, 0 to change, 0 to destroy.
... skipped ...Haeba ntho e 'ngoe le e' ngoe e felile ka katleho ('me e lokela ho ba joalo), joale re tla ba le mechine e meraro ea sebele:
- haha - mochini bakeng sa teko le ho aha kopo. Docker e kentsoe ka bo eona ke Ansible.
- tlhokomelo - mochini oa ho beha leihlo - prometheus & grafana e kentsoeng ho eona. Tekanyetso ea ho kena / password: admin / admin
- kafka ke mochini o monyane o kentsoeng kafka, o fumaneha boema-kepeng ba 9092.
Ha re etse bonnete ba hore kaofela li sebakeng:
vozerov@mba:~/events (master) $ yc compute instance list
+----------------------+------------+---------------+---------+---------------+-------------+
| ID | NAME | ZONE ID | STATUS | EXTERNAL IP | INTERNAL IP |
+----------------------+------------+---------------+---------+---------------+-------------+
| fhm081u8bkbqf1pa5kgj | monitoring | ru-central1-a | RUNNING | 84.201.159.71 | 172.16.1.35 |
| fhmf37k03oobgu9jmd7p | kafka | ru-central1-a | RUNNING | 84.201.173.41 | 172.16.1.31 |
| fhmt9pl1i8sf7ga6flgp | build | ru-central1-a | RUNNING | 84.201.132.3 | 172.16.1.26 |
+----------------------+------------+---------------+---------+---------------+-------------+
Lisebelisoa li teng, 'me ho tloha mona re ka fumana liaterese tsa bona tsa IP. Ho latela se latelang ke tla sebelisa liaterese tsa IP ho hokela ka ssh le ho leka ts'ebeliso. Haeba u na le ak'haonte ea cloudflare e amanang le terraform, ikutloe u lokolohile ho sebelisa mabitso a DNS a sa tsoa etsoa.
Ka tsela, ha u theha mochini o sebetsang, IP e ka hare le lebitso la ka hare la DNS li fanoa, kahoo o ka fihlella li-server ka har'a marang-rang ka mabitso:
ubuntu@build:~$ ping kafka.ru-central1.internal
PING kafka.ru-central1.internal (172.16.1.31) 56(84) bytes of data.
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=1 ttl=63 time=1.23 ms
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=2 ttl=63 time=0.625 ms
^C
--- kafka.ru-central1.internal ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.625/0.931/1.238/0.308 msSena se tla ba molemo ho rona ho bonts'a ts'ebeliso ea pheletso ka kafk.
Ho kopanya kopo
E kholo, ho na le li-server, ho na le kopo - se setseng ke ho e kopanya le ho e phatlalatsa. Bakeng sa mohaho re tla sebelisa mohaho o tloaelehileng oa li-docker, empa e le polokelo ea litšoantšo re tla sebelisa tšebeletso e tsoang ho Yandex - ngoliso ea lijana. Empa lintho tsa pele pele.
Re kopitsa kopo mochining oa kaho, kena ka ssh ebe re bokella setšoantšo:
vozerov@mba:~/events/terraform (master) $ cd ..
vozerov@mba:~/events (master) $ rsync -av app/ [email protected]:app/
... skipped ...
sent 3849 bytes received 70 bytes 7838.00 bytes/sec
total size is 3644 speedup is 0.93
vozerov@mba:~/events (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cd app
ubuntu@build:~/app$ sudo docker build -t app .
Sending build context to Docker daemon 6.144kB
Step 1/9 : FROM golang:latest AS build
... skipped ...
Successfully built 9760afd8ef65
Successfully tagged app:latestHalofo ea ntoa e felile - joale re ka sheba ts'ebetso ea ts'ebeliso ea rona ka ho e qala le ho e romella kafka:
ubuntu@build:~/app$ sudo docker run --name app -d -p 8080:8080 app /app/app -kafka=kafka.ru-central1.internal:9092</code>
С локальной машинки можно отправить тестовый event и посмотреть на ответ:
<code>vozerov@mba:~/events (master) $ curl -D - -s -X POST -d '{"key1":"data1"}' http://84.201.132.3:8080/post
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 13 Apr 2020 13:53:54 GMT
Content-Length: 41
{"status":"ok","partition":0,"Offset":0}
vozerov@mba:~/events (master) $Kopo e arabile ka katleho ea ho rekota le ho bontša id ea karohano le offset eo molaetsa o kentsoeng ho eona. Sohle se setseng ho etsa ke ho theha ngoliso ho Yandex.Cloud le ho kenya setšoantšo sa rona moo (mokhoa oa ho etsa sena u sebelisa mela e meraro e hlalositsoe faeleng ea registry.tf). Etsa polokelo:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_container_registry.events
... skipped ...
Plan: 1 to add, 0 to change, 0 to destroy.
... skipped ...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.Ho na le mekhoa e mengata ea ho netefatsa ho ngoliso ea sets'oants'o - ho sebelisa lets'oao la kano, lets'oao la iam, kapa senotlolo sa ak'haonte ea litšebeletso. Lintlha tse ling mabapi le mekhoa ena li ka fumanoa litokomaneng. . Re tla sebelisa senotlolo sa ak'haonte ea litšebeletso, kahoo re theha ak'haonte:
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_iam_service_account.docker -target yandex_resourcemanager_folder_iam_binding.puller -target yandex_resourcemanager_folder_iam_binding.pusher
... skipped ...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.Hona joale se setseng ke ho etsa senotlolo bakeng sa eona:
vozerov@mba:~/events/terraform (master) $ yc iam key create --service-account-name docker -o key.json
id: ajej8a06kdfbehbrh91p
service_account_id: ajep6d38k895srp9osij
created_at: "2020-04-13T14:00:30Z"
key_algorithm: RSA_2048Re fumana leseli mabapi le id ea polokelo ea rona, fetisetsa senotlolo ebe re kena:
vozerov@mba:~/events/terraform (master) $ scp key.json [email protected]:
key.json 100% 2392 215.1KB/s 00:00
vozerov@mba:~/events/terraform (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cat key.json | sudo docker login --username json_key --password-stdin cr.yandex
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
ubuntu@build:~$Ho kenya setšoantšo ho ngolisoeng, re hloka ID ea ngoliso ea setshelo, re e nka ho yc utility:
vozerov@mba:~ $ yc container registry get events
id: crpdgj6c9umdhgaqjfmm
folder_id:
name: events
status: ACTIVE
created_at: "2020-04-13T13:56:41.914Z"Kamora moo, re tšoaea setšoantšo sa rona ka lebitso le lecha ebe re se kenya:
ubuntu@build:~$ sudo docker tag app cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
ubuntu@build:~$ sudo docker push cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
The push refers to repository [cr.yandex/crpdgj6c9umdhgaqjfmm/events]
8c286e154c6e: Pushed
477c318b05cb: Pushed
beee9f30bc1f: Pushed
v1: digest: sha256:1dd5aaa9dbdde2f60d833be0bed1c352724be3ea3158bcac3cdee41d47c5e380 size: 946Re ka netefatsa hore setšoantšo se kentsoe ka katleho:
vozerov@mba:~/events/terraform (master) $ yc container repository list
+----------------------+-----------------------------+
| ID | NAME |
+----------------------+-----------------------------+
| crpe8mqtrgmuq07accvn | crpdgj6c9umdhgaqjfmm/events |
+----------------------+-----------------------------+Ka tsela, haeba u kenya ts'ebeliso ea yc mochining oa Linux, u ka sebelisa taelo
yc container registry configure-dockerho lokisa docker.
fihlela qeto e
Re entse mosebetsi o boima haholo mme ka lebaka leo:
- Re ile ra tla le moralo oa ts'ebeletso ea rona ea kamoso.
- Re ngotse kopo ka golang e sebelisang mohopolo oa rona oa khoebo.
- Re ile ra e bokella 'me ra e tšela ka har'a registry ea poraefete.
Karolong e latelang, re tla fetela linthong tse khahlisang - re tla lokolla kopo ea rona tlhahisong ebe qetellong re qala mojaro ho eona. Se ke oa chencha!
Boitsebiso bona bo rekotiloeng ka video ea workshop e bulehileng REBRAIN & Yandex.Cloud: Re amohela likopo tse 10 ka motsotsoana ho Yandex Cloud -
Haeba u thahasella ho ea liketsahalong tse joalo inthaneteng le ho botsa lipotso ka nako ea nnete, hokela ho.
Re rata ho leboha haholo Yandex.Cloud bakeng sa monyetla oa ho tšoara ketsahalo e joalo. Kopana le bona -
Haeba u hloka ho fallela lerung kapa u na le lipotso mabapi le lisebelisoa tsa hau, .
PS Re na le litlhahlobo tse 2 tsa mahala ka khoeli, mohlomong projeke ea hau e tla ba e 'ngoe ea tsona.
Source: www.habr.com