αα½ααααΈαα·αααααΆααα’ααααααΆ!
* α’ααααααααααΊααα’ααααΎαα·ααααΆααΆααΆααΎαα
αα α REBRAIN & Yandex.Cloud ααααα·αααΎα’αααα
αΌαα
α·αααααΎαααΈααα’αΌ α’αααα’αΆα
αααααααααΆαα
αααααα -
ααααΈααααααΎαααΆαα±ααΆαααΆαααααααααΆαα Yandex.Cloud α αααααΆαααΎαα
αααααΎαα’αααααααΌα αα·ααα·ααΆα ααΎαααΆαααααααα
ααααααΆααααΌααααα·αααααΆαα
αΆααααααΎαααααα Wordpress ααΆααααααΆαα½αααΉαααΌαααααΆαααα - ααΆαα½αα±αααα»ααααα αααααΆααααΈααΆααα·ααα½αα
ααα½α ααΎαααΆααααααα
α
α·αααααΆαααααααΆαα’αααΈαα½αααααααααααα
ααΉαααααΆααααααααααααΆααααααα·ααααααααααΆααααΆαααα½α αα·ααα·ααΆαααααΉαααα·ααΆααααα
αααα»ααααααααααααΆααΆαααααααα
αααα»αααααΆααααΆααααΆααΆαα
αααΎαααα’αΆααΈαααααα’αα‘αΆα (αα·αααααΉαααααα»ααααα) αααααααααΌαααααααΆαα’αααΈα’αααααααΎααααΆαα αα·ααααααααΆααααααα½αααα αααΆαα αα
ααΆαα ααΆα
αΆαααΆα
ααααααΆααααΆααααααα
α
α·αααααΆααααΆαα - α§ααΆα ααα ααααα·αααΎα’ααααααααααααα ααααα’αα‘αΆα α’αααα’αΆα
ααΎααααα·αα·αα
ααααα·αααΆαααα’αααααααΎααααΆααααΆαα
αααΎαααΆααααΆαα αα·ααα»ααααααΆααααααααααααααα’αααα α¬α ααα»α’αααΈααΆαααΆα’αααααααΎααααΆααα
αΆαα
ααααΈααα αααααααααα’αααααααα·ααα·αα’αααΈααΆααα’αα (αα½ααααΈ Yandex.Metrica)α
ααΌα αααα ααΏαααααααΎαα αααααααααΎαααααααααααα·ααΈαα αααα»αα αααααΆαα ααΆααααα kafka vs rabbitmq vs yqs ααΆααααααααΆαααααΆααα·αααααααα αααα»αα ααααα Clickhouse αα·αααΎαααΎααα·αααααααααααααΎ yandex datalens α ααΆαααααααΆαα· α’αααΈαααΆααα’αααααααααΌαααΆααααα αα‘αΎααααααΆαααΈαααΆαααα αααααΆαα ααΆαααααααααααα»αααααααααΆ docker, terraform, gitlab ci αα·αααΆααΆααα·αααΆαα prometheus α αααβαα !
αααα»αα ααααααΎααΆααααααααΆαααααααΎαααΉααα·αα’αΆα ααααααα ααΆααααααααα’αααΈαααΆααα’αααααα»ααα½αα’αααα»αααΆααα - αααααΆααααΏααααααΎαααΉαααααΌαααΆαα’αααααααΆα αααΎααα αααα»ααααααΈα ααααα·α α’αααΈαα ααΆααααααααα
αααααααΈ 1 (α’ααααααα»αα’αΆαααΆ) α ααΎαααΉααααααα
α
α·αααααΎααααααααΆααααΆαα αα·αααααΆαααααααααααααααααααΆα α αΎαααααααααααααα·ααΈααΆααΆααΆα ααααα‘αΆααααααα
αααααααΈ 2 α ααΎααααα
αααααααα·ααΈααααααΎααα
αααα»αααα·ααααα ααααΎα±ααααΆα’αΆα
ααααΎααΆαααααααΆαααΆα αα·αααΆαααααααααα»αα
αααααααΈ 3 α α
αΌαααΎαααααΆααΆαααααααααααΆα ααα»α’αααΈααΆαααΆααΎαααααΌαααΆααααααΆαα»αααΆααα
αααα»αααα·ααααααα’αΆαααα α αΎααα·ααα
αααα»αα―αααΆα α αΎαααααααααααααααΆαααα kafka, rabbitmq αα·α yandex αααααα
αααααααΈ 4 ααΎαααΉαααΆαααααααΆα Clickhouse cluster αααααααααΆααααααααααΈαααΎααααΈααααααα·ααααααααΈααα·ααααααα’αΆαααααα
ααΈααα α αΎααααα
αααΆαααΎαααΎααα
αααα»α datalensα
αααααααΈ 5 α
αΌαααΆαα αααααΆαα
ααΆααααααααααΆααααΌααα
ααΆααααααααααΉαααααΌα - αααα
α ci/cd αααααααΎ gitlab ci ααααΆααααΆααααα½ααα·αα·ααα αα·αααΆαααααΎαααααΆαααααααααααΎ prometheus αα·ααα»αααα»αα
Π’Π
ααΆααααΌα α αΌαααΎααααααΎαααααααααααααα ααααΈααα - α’αααΈαααααΎαα ααααα½αααΆαααΆααααααα
- ααΎαα ααααΆαα ααα»α αααα ααααΌα ααΆ events.kis.im (kis.im ααΊααΆαααααΆααααααααααΎαααΉαααααΎαααα»αα’αααααααΆααα’αα) ααααα½αααααα½αααΆαααααΉαααα·ααΆααααααααααΎ HTTPS α
- ααααΉαααα·ααΆαααααΊααΆ json ααΆααααααΌα ααΆα {βeventβ: βviewβ, βosβ: βlinuxβ, βbrowserβ: βchromeβ}α αα ααααΆααααΆαα α»αααααα ααΎαααΉαααααααααΆααα½αααααααααα ααα»ααααααΆααΉααα·αααΎααα½ααΆααΈαααααααα ααααα·αααΎα’αααα ααααΆα α’αααα’αΆα ααααΌααα protobuf α
- ααααΆααααααααΌαααα’αΆα ααααΎαααΆαααααΉαααα·ααΆααα 10 αααα»ααα½ααα·ααΆααΈα
- ααΆαα½αααα’αΆα ααααΎααΆαααααααΆαααααααααααααΆααααααααααααααΈααααΈαα αααα»ααααααααααΆαααααααΎαα α αΎαααΆααΉαααα’ααααα·αααΎααΎαα’αΆα ααααΆααααΈαααααααΆααα»ααα ααΈααΆααααΌαα·ααΆααααααααααα ααΎααααΈααΆαααααααααΆαααΊααααΆααααααΆααααααΎααααα’αα·αα·ααα
- ααΆαα’ααα±αα ααααααα α»αα αααααααααΆαααααΌαααααΆααααααααΆααααααααααΆαα αα·αα’αΆα αααααΆαααΆαααΈαα·αααΈααΆααα½αααααααααααααΆαα½α (ααΆααΆααα·αααΆαααα αΌααααα ααα½αααΆααααΆαα)α
ααααΆαααααααα
ααΆααΌαα αααααΆαααα·α αα ααΆαααααααααα ααααΆαααααααααα»ααΆαααααΌαααΆααααααΎαα‘αΎαααΆααΌαααα αΎα αααα’αα»ααααΆαα±ααααααΎααΆαααααααΆαααααααααααααα·αααααΆαα αα½ααααααα αΆαααΈα§ααΆα ααααααααααααααΆαααααααΎαα
ααΌα
ααααα’αααΈαααααΎαααΆαα
1. αα ααΆααααααααΊααΆα§αααααααααααΎαααααααααΎαααααΉαααα·ααΆααααααααα αα·αααΆααΆα’ααααααααααααααααααα·ααα αααα»ααααααΆαααααααααααα ααΎααααΆαα αααΌα α¬αααααΎαααΆααααααΆαα·ααα αααα»αα αΆαα’αα‘αΆαααΆαααααααααα·ααΈαα»αααααααααΆα ααααΉαααα·ααΆααααα½α ααΌα αααααΆααααααΆαααα αααα»αααΆααααααΆαα ααΊααΆ json ααΆαααααααααααΌαααΆαααααΎαα ααΆααα ααα»α αααα ααααααααΎα - events.kis.im α
2. αααΆαααΈαααααΈαααααΌαααΊαααα»αααααΆαααα ααΆααα·α αα α αααααααααα½αααααΊα
- α’αΆα ααααΆαααΆαα·α αα α ααΎααααΈααααΎααΌα αααα’αααα’αΆα ααααΎα§ααΆα ααα Keepalived αααααΉαααααΌα IP αα·αααα·ααααΆαααααΆαααααα»αααααΈααΆααααα αΆα
- αααα αα TLS α ααΆα/α αΆα ααΎαααΉααααα αα TLS ααΎαα½αααα ααΈαα½α ααΎααααΈα±αααααααααααΆαααααααΎαα’αα»αααααΆααααααααα αα ααααα αα·αααΈααΈα ααΎααααΈαααααΆαααααα»αααααΆααααααΎαααΆααααααΆαααααααΆαα’αα·αααααΈαααΈαααΆαααΈαααααΆααααααααααααΎαα
- ααααΎαααα»αααααΆαααααΎαα»αα αΌααα ααΆαααααΆαααΈαααααΆαααααααααααΆαα ααΆααααααααΉααα ααΈαααααΊα’αΆα α αΌαααααΎαααΆαααΆαα αααααα’ααααΎα ααα»α ααα ααΎαααααααααααααΆ load balancers ααααΌαααα’αΆα αααα½ααα·αα·ααααααΆαααΈαααααααααΎαααΆαα½αααΉααααααα·ααΈ αα·αααααααααΆαααααΎαα»αααααΆαα ααΆα ααααα ααΆααααααΆααααααααΆαααα
3. αααααΆααααΈ Balancers ααΎαααΆααααααα·ααΈ Servers ααααΎαααΆααααααα·ααΈααΆααααααααααα ααΆαα½αααα’αΆα ααα½αααααααΎα αΌαααΆαααα HTTP ααααΎα±ααααΆααα»ααααΆα json αααααΆαααααΎ αα·αααΆαααα·αααααααα αααα»αααα·ααααααα’αΆααααα
4. ααααΆααααΆααααα αΆα kafka ααΆααα·ααααααα’αΆαααα ααΎαααααΆααΆααΆααα·α ααααΆααααααααααΆααααααααα’αΆα ααααΌαααΆαααααΎαα ααααα·αααααααααα ααΎαααΉααααααααα Kafka, rabbitmq αα·α yqs αα αααα»αα’αααααααΈααΈα
5. α ααα»α α α»ααααααααααααΆααααααααααααααΎαααΊ Clickhouse - ααΌαααααΆααα·αααααα columnar αααα’αα»ααααΆαα±ααα’ααααααααΆαα»α αα·αααααΎαααΆααα·αααααααααΆαα αααΎαα αα ααααα·αααα ααΎαααααΌαααααααα·ααααααααΈααα·ααααααα’αΆαααααα αααααααααααα»αααααααα½αααΆ (ααααααααΎααααααα»αααΆααααΆ 4) α
ααΆααα ααΆαααα’αα»ααααΆαα±ααααΎαααααΎααΆαααααααΆααααααΆααααΈαα½αααααα―αααΆαααααααααααα αααΆαααΈααα backend αα·αα’αΆα αααααααΆαααΆααα - ααΌαααααααααΏααα½αααααααααα - αααααΆααααΈααΆααα’αα αα½αααΆααΆαααααα·ααΈααααΆααααα ααΌα ααααα αΎα ααΆα’αΆα ααααΌαααΆαααααΎαααααααααααααααα·α ααα·ααααααα’αΆαααααα ααΆααααα Kafka αα·αααααΎαααΆαααβααΌααααααααααΆαααΈαααααααααααα αα·ααααααααααααααααααααααΆαααααααααΎααα αα½αααα Clickhouse αα·αα’αΆα αααααααΆαααΆααΆααα - ααΆαα·αα’αΆα αα αα½α αα :) ααΆααα·αααΎαααΉαααααΆαααααΆαααΈααα αα·αααααααα·αααααααααααα
ααααα·ααΈααα ααααα·αααΎα’αααα ααα’αα»αααααααααααααα α α·ααααααααααααα αα ααααα αα·αααΆαααααααΆαααααααΎααα αααα»αααΈααΆααααΌαα·ααΆαααααααααααααααΆααα αα·αααΆαα’αααΈααΆααααααΆααααααα
αα
αααα»αααΈααΆααααΌαα·ααΆαααααααΈαα½αα ααΎαααΆαααααααΆααα»αααααΆαααααα»αααΆαα½αααΉααααααα·ααΈ αα·α kafka α ααΆααΌαα
αααΆαααΈααααααααα·ααΈ 2 ααααΆαα kafka 3 αα·αα§ααααααα»αααααΆαααα α§ααΆα ααα cloudflare ααΊαααααααααΆααα αΎα αααααΉααα·αα·αααααΎαααΆαα’αΆα
ααααΆαααααααΆαααααααα·ααΈ αα·αααααΎαααα»αααααΆαααΈααΆααααΌαα·ααΆααααααααααα’ααααΎα’αΆααααααΆα IP αααααααααα’αα·αα·ααα ααΌα
αααα αα·αααααααααααααΎαααα’αα·αα·ααα’αΆαααα·αααΉαα
α»αα
ααα
ααΎαααΆαααΈαααααααα’αΆαααα·αα α αΎααα·ααααααααΈααααΈαα’αΆα ααααα·αααΊαα
α’αΆα ααααα·αα
αααααΆααααα’αααΈααααααααΆαααΊααΆααααααΆαα - ααΎαααααΎα§ααααααααα ααααΈαααα»α Kafka α αΎαα αααααα·ααααααααΆααα’ααααΈαααααααΈααΆαααα ααααααααααα·αααααααααααΆαααααααΎααααααΆαααΈααΆαααα αααααααα»αααααΈα αα ααΆααααα»α ααΎαααααα·αααααα αα·ααααααααΆααΆαα αααα»α Clickhouse αααααΆααααΆαααΎαααΎαααΆαααααααααΆααα
ααΌα αααα ααΎαααΆαααααααα ααααΌαααααΆαααααααα - αααα αΆααααααΎαα’αααα½α Yandex.Cloud!
ααΆαααααααααααα·ααΈ
αα»αααα Cloud α’ααααα ααααααΌαα’αααααααααααα·α α αΎααααααααααΆααααααααΆαααααα½αααΎααααΈααααΎαααΆαααααΉαααα·ααΆααααααα αΌαααα ααΎαααΉαααααΎα αααααΆαα αααααααΆααΆααααα αΆαα±ααααΎααααΆαα αααΆααααΆααΆααΆααΆαααααΆααααΆαααααααααααα·ααΈαααααΆαα
αααααΆααααΈα
αααΆαααααα½ααααα (αααα ααααΆααΈαααΈαααα) ααΎαααα½αααΆαα’αααΈαα½αααΌα
αααα
ααΎα ααα»α ααααΆααα’αααΈααααααααααα»αα ααααααααααΆαααα ααΈαααα
1. αα αααα αΆααααααΎααααααα·ααΈ α’αααα’αΆα αααααΆαααααααΈαα αα½αααΊααα½ααα»αααααΌαα ααααα ααααααααΎαααΉαααααΆααααααΎ http α αΌα (-addr) α ααΈααΈαααΊαααααΆααα’αΆααααααΆααααΆαααΈααα kafka αααααΎαααΉααααααααΆααααΉαααα·ααΆαααααααααΎα (-kafka):
addr = flag.String("addr", ":8080", "TCP address to listen to")
kafka = flag.String("kafka", "127.0.0.1:9092", "Kafka endpointsβ)
2. αααααα·ααΈααααΎαααααΆααα sarama (
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForLocal
config.Producer.Compression = sarama.CompressionSnappy
config.Producer.Return.Successes = true
3. αααααα·ααΈααααααΎαααααΆααααΆαααΈαααααα prometheus ααααΆααααααΆαα½αααααα ααααααααΌααααααΆααααααααααΌα ααΆα
- α ααα½αααααΎαααααΆαααααααα·ααΈααααααΎα;
- α ααα½αααααα α»ααα αααααααα·ααααα·ααααΎ (αα·αα’αΆα α’αΆαααααΎαααααΆα, json ααΌα , αα·αα’αΆα ααααααα Kafka);
- αααααααΆααααΎαααΆααααααΆααααααΎαα½αααΈα’αα·αα·αα αα½αααΆαααααααααΆαααααΆααααΆααααααααΆααα Kafka α
4. α ααα»α αααα ααα ααα½αααΈαααααααΎαααΆααααααα·ααΈααααααΎαα
- / ααααΆαααΆα - ααααΆαααααααα‘αα ok ααΎααααΈαααα αΆαααΆααΎααα αααα αααααΈααΆα’αααα’αΆα ααααααααΆααααα½ααα·αα·ααααα½αα ααα½αααΌα ααΆααΆαα’αΆα ααααΆαααα ααααα Kafka αααααα
- /metrics - αααααΆα ββurl ααα α’αα·αα·αα prometheus ααΉααααα‘αααααααααααααΆααΆααααααΌαα
- /post ααΊααΆα ααα»α αααα ααααααΆααααα POST ααααΎαα»αααΆαα½α json ααΆααααα»αααΉαααααΌαααΆαααααΎα αααααα·ααΈααααααΎααα·αα·αααααΎα json αααααΆαααα»ααααΆα α αΎαααααα·αααΎα’αααΈαααΆααα’αααα·αα’αΈαα ααΆααααααα·αααααααα αααα»α Kafka α
αααα»αβααΉαβααααΎααΆαβαααβαα»αβαααβααΌαβαα·αβααα’βα₯αβααα αα - ααΆβα’αΆα (α αΎαβαα½αβ!) ααααΌαβααΆαβαααα ααα α§ααΆα ααα α’αααα’αΆα αααααααΎ net/http αααααααΆααααααΆαα½α α αΎαααααΌααα fasthttp ααΏαααΆααα»αα α¬α’αααα’αΆα ααα½αααΆααααααααΆααααΎαααΆα αα·αααααΆα cpu αααααααΆααααΈααΆααααα½ααα·αα·ααααα»ααααΆα json αα ααααΆααααΆαααααα - αα αααααααα·ααααααααααΌαααΆααααααααΈααααα»ααα clickhouse α
ααααααααΈααΎαααααα’αα·αααααααααααα αΆααα ααΎαααΆααα·αααααΆααα’αααΈα αααααΆαα
ααΆααααααααααΆαααα’ααΆααααααααΎα α αΎαααΆααααααα
α
α·αααααΆααα±ααααααΎααααΆαααααααα·ααΈααααααΎαααΆαααα docker α Dockerfile α
α»αααααααααααΆαααααααΎααααααα·ααΈααΊ
ααα αΆαααααΌααα αααα»αααα
ααΆααααΌαααααΆαααΆααα’ααα
α»αααααααα
ααΎ
αααααΆααααΈααΆαα α»αααααα αααααΆα ααααα‘αα αα·αααααααΆαααΎαααΉαααααΌαααΆααααααΎααααααΆααα’ααα αααα’αααα’αΆα α αΆααααααΎααααααΎαααααΆααααα ααΆααΌαα αα αααα»α Yandex.Cloud ααααΆααααααααααααΆαααΎααα ααΌα αααα
α’αααα’αΆα
αααααΎααααααΆα
αααΎααααααΆααααααΈαα½αα α αΎααα
ααΆααααα»αααα αααααΎαααααααααααααΆαααααΆαααααααααααα»αα αα»αααααααααααΆα α’αααα’αΆα
α’αΆαααααααα’αααΈααΏαααααα
αααα»αα―αααΆα -
ααΎααααΈαααααααααααα α’αααα’αΆα ααααΎααΆααα ααα»α αααααΆαααααααΆα αα·αα§αααααααααΎααααΆαααα»αααΌα - yc α ααΆαααα‘αΎαααααΌαααΆαα’αα»αααααααααααΎααΆααααααααΆαα½α (αααααΆαα Linux αα·α Mac Os)α
curl https://storage.yandexcloud.net/yandexcloud-yc/install.sh | bash
ααααα·αααΎα’αααα―ααααααααα·αα»ααααααααα»αααααα’ααααααα»αααΉααααααΆαα ααααααΆαααααΎαααΆαααααααΈαααΈα’ααΈαααΊαα·α αααααααΌα α’αααα’αΆα ααΎαααααααΈα α αΎαα’αΆαααΆ α αΎαααΈααΈα ααΎαααααΎαααΆαααΆαα αααααα’αααααααΎααααΆααααααααΎα αααααααΆααα·αααα·ααΆ root α
ααααα·αααΎα’αααα
ααααα‘αΎααααΆαααΈαααααααααααΆαα Windows α’αααα’αΆα
ααααΎααΆαααααΆα yc init
ααΎααααΈααααΌαααΆαααααααΆααΆαααααα»αα
vozerov@mba:~ $ yc init
Welcome! This command will take you through the configuration process.
Please go to https://oauth.yandex.ru/authorize?response_type=token&client_id= in order to obtain OAuth token.
Please enter OAuth token:
Please select cloud to use:
[1] cloud-b1gv67ihgfu3bp (id = b1gv67ihgfu3bpt24o0q)
[2] fevlake-cloud (id = b1g6bvup3toribomnh30)
Please enter your numeric choice: 2
Your current cloud has been set to 'fevlake-cloud' (id = b1g6bvup3toribomnh30).
Please choose folder to use:
[1] default (id = b1g5r6h11knotfr8vjp7)
[2] Create a new folder
Please enter your numeric choice: 1
Your current folder has been set to 'default' (id = b1g5r6h11knotfr8vjp7).
Do you want to configure a default Compute zone? [Y/n]
Which zone do you want to use as a profile default?
[1] ru-central1-a
[2] ru-central1-b
[3] ru-central1-c
[4] Don't set default zone
Please enter your numeric choice: 1
Your profile default Compute zone has been set to 'ru-central1-a'.
vozerov@mba:~ $
ααΆαααααΆααα ααααΎαααΆαααΊααΆαααα - ααααΌαα’αααααααΌαααα½αααΆααααααΆαααααΆαα oauth ααΎααααΈαααααααααααα ααααΎαααΎαααα αα·ααααααα’αααααΉαααααΎα
ααααα·αααΎα’αααααΆαααααΈ α¬ααα―αααΆαααΆα αααΎααα αααα»αααααααα½α α’αααα’αΆα αααααΎαααααααααααααααΆαα½αααΉαααΆααααααααΆα ααααα‘ααααΆαααα yc config profile αααααΎα αα·αααααΌααααΆααα½αααΆα
ααααααααΈααΎαα·ααΈααΆαααααααΆαααΎαααα»αααΆαααΆα Yandex.Cloud ααΆααααααααα’ααΆαα
vozerov@mba:~ $ git clone https://github.com/rebrainme/yandex-cloud-events/ events
Cloning into 'events'...
remote: Enumerating objects: 100, done.
remote: Counting objects: 100% (100/100), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 100 (delta 37), reused 89 (delta 26), pack-reused 0
Receiving objects: 100% (100/100), 25.65 KiB | 168.00 KiB/s, done.
Resolving deltas: 100% (37/37), done.
vozerov@mba:~ $ cd events/terraform/
α’αααααααΆαααααΆααα’αααααααααΎαααα»α terraform ααααΌαααΆαααααααα αααα»αα―αααΆα main.tf α ααΎααααΈα αΆααααααΎα ααΌααααααΎαα―αααΆα private.auto.tfvars αα αααα»ααα terraform αααααΆαααααΉαααΆαααΌα ααΆααααααα
# Yandex Cloud Oauth token
yc_token = ""
# Yandex Cloud ID
yc_cloud_id = ""
# Yandex Cloud folder ID
yc_folder_id = ""
# Default Yandex Cloud Region
yc_region = "ru-central1-a"
# Cloudflare email
cf_email = ""
# Cloudflare token
cf_token = ""
# Cloudflare zone id
cf_zone_id = ""
α’αααααΆααα’ααα’αΆα ααααΌαααΆαααα ααααΈαααααΈααααααα ααΆαααααααα yc α αΆααααΆααααΈααΎαααΆαααααααα ααΆααααααααα§αααααααααΎααααΆαααα»αααΌααα½α α αΎαα αααα»αααααΆαα’αααα±αααααααα private.auto.tfvars αα .gitignore ααΆαααααΆαα ααΎααααΈαα»αα±ααααααααααααΆααα·ααααααα―ααααααα αααααα
αα αααα»α private.auto.tfvars ααΎαααααΆααααααΆαααα·ααααααααΈ Cloudflare - ααΎααααΈαααααΎααααααααααΆ DNS αα·αααααΌααααΈαααααααΆαα events.kis.im αα ααΆαααααΆαααΈαααααααααΎαα ααααα·αααΎα’ααααα·αα ααααααΎ cloudflare ααααα ααΌααα»αααΆαα αΆααααααΎαααααα’ααααααααααααΆ cloudflare αα αααα»α main.tf αα·αα―αααΆα dns.tf αααααα½ααα»αααααΌααααα»αααΆααααααΎααααααααααΆ dns α αΆαααΆα αα
αα αααα»αααΆαααΆαααααααΎα ααΎαααΉααα½ααααα αΌαααααΆααΌααα·ααΈααΆαααααααΆααααΈ - α ααα»α αααααΆααααα ααααα α§αααααααααΎααααΆαααα»αααΌα αα·α terraform α
αααααΆααα·αααα·α
αα·ααΆαα±ααααααααα
α’αααα’αΆα
ααααααα αΆαααα ααααααα
αααα’ααααααααΎααααααααΈ α’αααααΉαααΆααααααΆαααΆα
ααααα‘αααα½ααααααααααααααααα· α αΎααααααΆαααα
ααα½α 3 ααααΌαααΆααααααΎαα‘αΎα - αα½ααααααΆααααααααααα’αΆα
ααααΆαααΈαα½ααα ααα»ααααααΎααα
ααα
αααααααΎααααααΆαααΆα
ααααα‘αααααααΆααααααααααααααΎαααΆαα½αααΉαα’αΆααααααΆαααααααΆααααΆααα ααααΆααααΆαααΌαα
ααααααα·ααααα·ααΆααααααΆααα
αααα»α Yandex.Cloud ααααΌαααΆααααα αΆααα
αααα»αααΌαααΆαααΆαααααα (αααααααααααααααααΈ
ααΌα
αααα α’ααααααααΎααααααΆααα½ααα½α αααααααΆαα’αΆα
ααααΆααααααααααΆαα
αα·ααα
ααα αααααΆααααααααααα’αΆα
ααααΆαααΈαα½αα αααααΆαααααααΌαααΆααααααΎααααααΆαα’αΆααααααΆαααααΆαααααα½αααααααΆ αα·αααααΆαααα
αααααΆαααΌαα
α ααΆαααααα ααααΆααααααΆααα’αααα
αααα»αααΆα’αΆα
ααααΆααααααααΆα αααααΈααΆααΆαααα·ααα
αααα»αααααααααα’αΆα
ααααΆααα»ααααααΆαααααα ααααΆααααααααΆαααα
αααααΆαααααααααααααΆα’αΆα
ααΎαααΎαααααΆαα
αα·ααα
ααααΆααααα’αΆααααααΆαααΆααααα
ααα»αααααα αα·ααΆαα’αΈααα
αΉα ααΎααααααααααααααΎαααΆααααΆαααΌα
ααααα
?
ααΆααααααΎααααααΆαααααΌαααΆααα·αααααΆαα αααα»αα―αααΆα network.tf ααΈααααΆααα αα ααΈαααααΎααααααΎααααααΆαα―αααααΌαα αα½ααα ααΆααααα»α α αΎαααααΆαααααααΆαααα ααα½αααΈαα ααΆαα αααα»αααααααααα’αΆα ααααΆαααααααααααΆ - internal-a (172.16.1.0/24), internal-b (172.16.2.0/24), internal-c (172.16.3.0/24) )
α αΆααααααΎα terraform αα·ααααααΎααααααΆαα
vozerov@mba:~/events/terraform (master) $ terraform init
... skipped ..
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_vpc_subnet.internal-a -target yandex_vpc_subnet.internal-b -target yandex_vpc_subnet.internal-c
... skipped ...
Plan: 4 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
yandex_vpc_network.internal: Creating...
yandex_vpc_network.internal: Creation complete after 3s [id=enp2g2rhile7gbqlbrkr]
yandex_vpc_subnet.internal-a: Creating...
yandex_vpc_subnet.internal-b: Creating...
yandex_vpc_subnet.internal-c: Creating...
yandex_vpc_subnet.internal-a: Creation complete after 6s [id=e9b1dad6mgoj2v4funog]
yandex_vpc_subnet.internal-b: Creation complete after 7s [id=e2liv5i4amu52p64ac9p]
yandex_vpc_subnet.internal-c: Still creating... [10s elapsed]
yandex_vpc_subnet.internal-c: Creation complete after 10s [id=b0c2qhsj2vranoc9vhcq]
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
α’ααα αΆααα! ααΎαααΆααααααΎααααααΆαααααααΎα α αΎαα₯α‘αΌααααααααααααα½αααΆααααα ααΎααααΈαααααΎαααααΆααααααΆααααα»αααααααΎαα
ααΆααααααΎααααΆαααΈααα·αααα·α
ααΎααααΈααΆααααααααααα·ααΈ ααΎαααΉαααααΌαααΆααααααΎααααΆαααΈααα·αααα·αααΈαααα»ααααα - ααΎαααΉαααααΌαααΆαααΈαα½αααΎααααΈαααααΎα αα·αααααΎαααΆααααααα·ααΈ ααΈααΈαααΎααααΈααααΎαααΆα kafka αααααΎαααΉαααααΎααΎααααΈαααααΆαα»αααΆαα αΌαα α αΎαααΎαααΉααααααΎααααΆαααΈααα½αααααααααΎαααΉαααααααα ααΆαααααααα prometheus ααΎααααΈαααα½ααα·αα·ααααααααα·ααΈα
αααΆαααΈααα·αααα·αααΉαααααΌαααΆαααααααα ααΆαααααααααααααααΎ ansible ααΌα αααααα»ααααα αΆααααααΎα terraform ααΌαααααΆααααΆα’αααααΆαααααα α»αααααααααα»ααα½ααα ansible α α αΎαααα‘αΎααα½ααΆααΈα αΆαααΆα αααΆαα½αααΆα‘αΆαααααΈ ansibleα
vozerov@mba:~/events/terraform (master) $ cd ../ansible/
vozerov@mba:~/events/ansible (master) $ ansible-galaxy install -r requirements.yml
- cloudalchemy-prometheus (master) is already installed, skipping.
- cloudalchemy-grafana (master) is already installed, skipping.
- sansible.kafka (master) is already installed, skipping.
- sansible.zookeeper (master) is already installed, skipping.
- geerlingguy.docker (master) is already installed, skipping.
vozerov@mba:~/events/ansible (master) $
αα ααΆααααα»α ansible folder ααΆαα§ααΆα ααα .ansible.cfg configuration file ααααααα»αααααΎα ααΆα’αΆα ααΆαααααααααα
αα»αααΉααααααΎααααΆαααΈααα·αααα·α ααααΌαααααΆααααΆα’αααααΆα ssh-agent αααα»αααααΎαααΆα α αΎαααΆααααααααα ssh ααΎαα·αααΌα
αααααα terraform ααΉααα·αα’αΆα
ααααΆαααα
αααΆαααΈααααααΆααααααΎαααΆαααα ααΆααΆααα·α αααα»αααΆααα½ααααα αΆαα
αααα»α os xα
vozerov@mba:~/events/terraform (master) $ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
αα αααα»αααααΆαα½α terraform ααΎααααααΎαααααΆαα αΆαααΆα αα
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_compute_instance.build -target yandex_compute_instance.monitoring -target yandex_compute_instance.kafka
yandex_vpc_network.internal: Refreshing state... [id=enp2g2rhile7gbqlbrkr]
data.yandex_compute_image.ubuntu_image: Refreshing state...
yandex_vpc_subnet.internal-a: Refreshing state... [id=e9b1dad6mgoj2v4funog]
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
... skipped ...
Plan: 3 to add, 0 to change, 0 to destroy.
... skipped ...
ααααα·αααΎα’αααΈαααΆααααα ααααααααααα (α αΎαααΆαα½αααααΆα) αααααΎαααΉαααΆααααΆαααΈααα·αααα·αα ααα½αααΈα
- build - αααΆαααΈααααααΆααααΆααααα αα·ααααααΎααααααα·ααΈα Docker ααααΌαααΆαααα‘αΎααααααααααααααααα·ααα Ansible α
- ααΆααααα½ααα·αα·ααα - αααΆαααΈααααα½ααα·αα·ααα - prometheus & grafana ααΆαααα‘αΎααα ααΎααΆα αααααααΆαα αΌα / ααΆααααααααΆααα ααααααααα / α’αααααααααααα
- kafka ααΊααΆαααΆαααΈαααΌα αα½ααααααΆα kafka αααααΆαααα‘αΎα α’αΆα α αΌαααΆααα ααΎα ααα 9092α
α αΌαααααΆααααΆαα½αααΆαα ααΉαααααααααΆααα’ααα
vozerov@mba:~/events (master) $ yc compute instance list
+----------------------+------------+---------------+---------+---------------+-------------+
| ID | NAME | ZONE ID | STATUS | EXTERNAL IP | INTERNAL IP |
+----------------------+------------+---------------+---------+---------------+-------------+
| fhm081u8bkbqf1pa5kgj | monitoring | ru-central1-a | RUNNING | 84.201.159.71 | 172.16.1.35 |
| fhmf37k03oobgu9jmd7p | kafka | ru-central1-a | RUNNING | 84.201.173.41 | 172.16.1.31 |
| fhmt9pl1i8sf7ga6flgp | build | ru-central1-a | RUNNING | 84.201.132.3 | 172.16.1.26 |
+----------------------+------------+---------------+---------+---------------+-------------+
ααααΆαααΊαα
ααΉααααααα α αΎαααΈααΈαααααΎαα’αΆα
ααα½αααΆαα’αΆααααααΆα IP αααααα½αααα ααΆααααα’αααΈααααααααΆαααααα»αααΉαααααΎα’αΆααααααΆα IP ααΎααααΈααααΆααααΆαααα ssh αα·αααΆααααααααααα·ααΈα ααααα·αααΎα’αααααΆαααααΈ cloudflare ααααΆαααα
terraform ααΌαααααΎααααα DNS αααααΎααααααΎαααααΈαα
ααααα·ααΈααα αα
ααααααααΎααααΆαααΈααα·αααα·α IP ααΆααααα»α αα·αααααα DNS ααΆααααα»αααααΌαααΆααααααα±αα ααΌα
ααααα’αααα’αΆα
α
αΌαααααΎαααΆαααΈααααα
αααα»ααααααΆαααΆααααααα
ubuntu@build:~$ ping kafka.ru-central1.internal
PING kafka.ru-central1.internal (172.16.1.31) 56(84) bytes of data.
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=1 ttl=63 time=1.23 ms
64 bytes from kafka.ru-central1.internal (172.16.1.31): icmp_seq=2 ttl=63 time=0.625 ms
^C
--- kafka.ru-central1.internal ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.625/0.931/1.238/0.308 ms
ααΆααΉαααΆααααααααααααααΆααααΎααααα»αααΆαα ααα’α»ααααα αΆααα αααααα·ααΈαααα ααααα kafk α
ααΆααααααΌααααα»ααααααα·ααΈ
α’ααα αΆαααααΆαα ααΆααααΆαααΈααα ααΆααααααα·ααΈαα½α - α’αααΈααααα ααααααααΊααααΌααααααΌααααα»αααΆ α αΎααααααααΆαααΆα αααααΆααααΆααααααΎα ααΎαααΉαααααΎ docker build ααααααΆ ααα»ααααααΆαααααααααα»αααΌαααΆα ααΎαααΉαααααΎααααΆααααααΈ Yandex - container registry α ααα»ααααααΏαααααΌαα
ααΎαα αααααααααα·ααΈαα αααΆαααΈααααααΎα α αΌαααΆαααα ssh α αΎααααααΌααααα»αααΌαααΆαα
vozerov@mba:~/events/terraform (master) $ cd ..
vozerov@mba:~/events (master) $ rsync -av app/ [email protected]:app/
... skipped ...
sent 3849 bytes received 70 bytes 7838.00 bytes/sec
total size is 3644 speedup is 0.93
vozerov@mba:~/events (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cd app
ubuntu@build:~/app$ sudo docker build -t app .
Sending build context to Docker daemon 6.144kB
Step 1/9 : FROM golang:latest AS build
... skipped ...
Successfully built 9760afd8ef65
Successfully tagged app:latest
ααΆαααααα»αααααΆαααααααΆαααααΌαααΆαααααΎαα½α - α₯α‘αΌααααααΎαα’αΆα αα·αα·αααααΎααα»αααΆααααααααα·ααΈααααααΎααααααΎαααααΎαααΆαααΆα αΎαααααΎααΆαα kafka:
ubuntu@build:~/app$ sudo docker run --name app -d -p 8080:8080 app /app/app -kafka=kafka.ru-central1.internal:9092</code>
Π‘ Π»ΠΎΠΊΠ°Π»ΡΠ½ΠΎΠΉ ΠΌΠ°ΡΠΈΠ½ΠΊΠΈ ΠΌΠΎΠΆΠ½ΠΎ ΠΎΡΠΏΡΠ°Π²ΠΈΡΡ ΡΠ΅ΡΡΠΎΠ²ΡΠΉ event ΠΈ ΠΏΠΎΡΠΌΠΎΡΡΠ΅ΡΡ Π½Π° ΠΎΡΠ²Π΅Ρ:
<code>vozerov@mba:~/events (master) $ curl -D - -s -X POST -d '{"key1":"data1"}' http://84.201.132.3:8080/post
HTTP/1.1 200 OK
Content-Type: application/json
Date: Mon, 13 Apr 2020 13:53:54 GMT
Content-Length: 41
{"status":"ok","partition":0,"Offset":0}
vozerov@mba:~/events (master) $
αααααα·ααΈααΆαααααΎααααααααααααααααΆααα αα·ααααα αΆαααααααααΆααααααΆαααΆα αα·αα’α»α αααα·ααααααΆαααααΌαααΆααα½ααααα αΌαα α’αααΈααααα αααααΎααααΈααααΎααΊαααααΎααααααΈααααααα αααα»α Yandex.Cloud α αΎααααα ααααΌαααΆαααααααΎααα ααΈααα (ααααααααΎααΆαααααααΎαααααΆααααΈααααΌαααΆααα·αααααΆαα αααα»αα―αααΆα registry.tf) α αααααΎααααααααααα»αα
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_container_registry.events
... skipped ...
Plan: 1 to add, 0 to change, 0 to destroy.
... skipped ...
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
ααΆααα·ααΈααΆα
αααΎααααα»αααΆααααααααααΆαααα
αααα»ααααααΈααααααα»αααΊααα - αααααααΎαααααΆαααααΆαα oauth αα·αα·ααααααααΆ iam α¬ααααααΈααααΆααααα ααααααΆααααα’α·αααααααα’αααΈαα·ααΈααΆαααααααΆαααααα’αΆα
ααααΆααα
αααα»αα―αααΆαα
vozerov@mba:~/events/terraform (master) $ terraform apply -target yandex_iam_service_account.docker -target yandex_resourcemanager_folder_iam_binding.puller -target yandex_resourcemanager_folder_iam_binding.pusher
... skipped ...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
α₯α‘αΌααααα’αααΈαααααα αααααΊααααΌααααααΎαααΌααααααααΆααααΆα
vozerov@mba:~/events/terraform (master) $ yc iam key create --service-account-name docker -o key.json
id: ajej8a06kdfbehbrh91p
service_account_id: ajep6d38k895srp9osij
created_at: "2020-04-13T14:00:30Z"
key_algorithm: RSA_2048
ααΎαααα½αααΆαααααααΆαα’αααΈααααααααΆααααααΆααααα»αααααααΎα ααααααα αα·αα αΌαα
vozerov@mba:~/events/terraform (master) $ scp key.json [email protected]:
key.json 100% 2392 215.1KB/s 00:00
vozerov@mba:~/events/terraform (master) $ ssh 84.201.132.3 -l ubuntu
ubuntu@build:~$ cat key.json | sudo docker login --username json_key --password-stdin cr.yandex
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
ubuntu@build:~$
ααΎααααΈαααα ααααΌαααΆααα αααα»ααααααΈααααα ααΎαααααΌαααΆαααααααααΆαααα»αααΊαααα α»ααααααΈ ααΎαααααΆα ααααΈα§αααααααααΎααααΆαα ycα
vozerov@mba:~ $ yc container registry get events
id: crpdgj6c9umdhgaqjfmm
folder_id:
name: events
status: ACTIVE
created_at: "2020-04-13T13:56:41.914Z"
αααααΆααααΈααα ααΎα tag ααΌαααΆαααααααΎαααΆαα½αααΉααααααααααΈ α αΎααααα ααα
ubuntu@build:~$ sudo docker tag app cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
ubuntu@build:~$ sudo docker push cr.yandex/crpdgj6c9umdhgaqjfmm/events:v1
The push refers to repository [cr.yandex/crpdgj6c9umdhgaqjfmm/events]
8c286e154c6e: Pushed
477c318b05cb: Pushed
beee9f30bc1f: Pushed
v1: digest: sha256:1dd5aaa9dbdde2f60d833be0bed1c352724be3ea3158bcac3cdee41d47c5e380 size: 946
ααΎαα’αΆα αααααααααΆααααΆααΌαααΆααααααΆααααα»ααααααααααα
vozerov@mba:~/events/terraform (master) $ yc container repository list
+----------------------+-----------------------------+
| ID | NAME |
+----------------------+-----------------------------+
| crpe8mqtrgmuq07accvn | crpdgj6c9umdhgaqjfmm/events |
+----------------------+-----------------------------+
ααααα·ααΈααα ααααα·αααΎα’αααααα‘αΎαα§αααααααααΎααααΆαα yc αα ααΎαααΆαααΈαααΈαα»α α’αααα’αΆα ααααΎααΆααααααααΆααΆαα
yc container registry configure-docker
ααΎααααΈααααααα ααΆαααααααα docker α
ααα ααααΈααααα·ααααΆα
ααΎαβααΆαβααααΎβααΆαβααααΆαβααΆβα αααΎα α αΎαβααΆβααααααα
- ααΎαααΆααααααΎαααααΆααααααααααααααΆααααααΆαααα’ααΆααααααααΎαα
- ααΎαααΆαααααααααααα·ααΈαα½ααα αααα»α golang αααα’αα»αααααααααα·ααααΆα’αΆααΈαααααααααααΎαα
- ααΎαααΆααααααΌαααΆα αΎαα αΆααααΆαα αααα»ααααααΈααααααα»αα―αααα
αα ααααααααααΆαα ααΎαααΉααααααα ααααα»ααααα½αα±ααα αΆααα’αΆαααααα - ααΎαααΉααααα αααααααα·ααΈααααααΎααα αααα»αααα·ααααα α αΎαααΈαααα»αα αΆααααααΎαααααΎαααΆαααΎααΆα αα»αααααΌα!
αααααΆαααααααΊαα
αααα»αααΆαααααΈααα’αΌαααα·ααααΆααΆααΆααΎαα
αα REBRAIN & Yandex.Cloud: ααΎαααα½αααααααΎ 10 αααα»ααα½ααα·ααΆααΈαα
ααΎ Yandex Cloud -
ααααα·αααΎα’αααα
αΆααα’αΆαααααααααα»αααΆαα
αΌααα½ααααα»αααααΉαααα·ααΆαααααααααααΆαα’ααΈαααΊαα·α αα·ααα½ααααα½ααααα»ααααααΆααααααα ααΌαααααΆαααα
ααΎαααΌααααααα’ααααα»ααααΆαααααΆααααα
α
αααα Yandex.Cloud αααααΆααα±ααΆααααα»αααΆαααααΎααΆααα
αΆααααααααααΉαααα·ααΆαααααααααα ααααΆαααα
αα½ααα -
ααααα·αααΎα’αααααααΌαααΆαααααΆααααΈαα
ααα α¬ααΆααααα½αα’αααΈα αααααΆαα
ααΆααααααααααααα’ααα
PS ααΎαααΆααααααααα₯ααα·αααααα
ααα½α 2 αααα»ααα½ααα αααα ααααΆααααααααααα’αααααΉαααααΆαααΆαα½ααααα»αα
αααααα½αααα
ααααα: www.habr.com