Gbigbe awọn iṣẹ pinpin ni Yandex.Cloud ni lilo Grafana gẹgẹbi apẹẹrẹ

Bawo ni gbogbo eniyan! Gẹgẹbi apakan ti iṣẹ iṣẹ mi, Mo ṣe iwadii awọn agbara ti iru iru ẹrọ awọsanma inu ile bii Yandex.Cloud. Syeed nfunni ni awọn iṣẹ lọpọlọpọ lati yanju awọn iṣoro to wulo. Sibẹsibẹ, nigbakan o nilo lati ṣeto ohun elo awọsanma tirẹ pẹlu awọn amayederun lọpọlọpọ ti o da lori awọn iṣẹ wọnyi. Ninu nkan yii Mo fẹ pin iriri mi ni gbigbe iru ohun elo bẹẹ.

Kini o fẹ lati gba?

Grafana - ohun elo ti o lagbara lati yanju awọn iṣoro itupalẹ tabi awọn iṣoro ibojuwo ti awọn eto eyikeyi. Ninu iṣeto ipilẹ rẹ, eyi jẹ ẹrọ foju kan pẹlu olupin wẹẹbu Grafana, bakanna bi data data (ClickHouse, InfluxDB, ati bẹbẹ lọ) pẹlu dataset kan eyiti awọn atupale yoo da lori.

Lẹhin ifilọlẹ ẹrọ foju kan pẹlu olupin wẹẹbu, o le lọ si agbalejo rẹ ki o gba UI ẹlẹwa kan, pato awọn apoti isura infomesonu bi awọn orisun fun iṣẹ siwaju, ṣẹda awọn dashboards ati awọn aworan.

Awọn ipilẹ ti ikede ni o ni ọkan significant drawback - o jẹ ko ẹbi-ọlọdun ni gbogbo. Iyẹn ni, gbogbo iṣẹ ṣiṣe ti ohun elo da lori ṣiṣeeṣe ti ẹrọ foju kan. Ti o ba kọ tabi awọn eniyan 10 ṣii UI ni akoko kanna, lẹhinna awọn iṣoro yoo dide.

Wọn le yanju ni irọrun: o kan nilo lati… ran ọpọlọpọ awọn ẹrọ foju kanna ṣiṣẹ pẹlu olupin wẹẹbu kan ki o gbe wọn labẹ iwọntunwọnsi L3 kan. Ṣugbọn kii ṣe ohun gbogbo jẹ kedere nibi. Grafana tọju awọn eto olumulo (awọn ọna si awọn apoti isura data, dashboards, awọn aworan, ati bẹbẹ lọ) taara lori disiki ti ẹrọ foju rẹ. Nitorinaa, ti a ba yipada diẹ ninu awọn eto ni UI, awọn ayipada wọnyi yoo han nikan lori ẹrọ foju nibiti iwọntunwọnsi ti firanṣẹ wa. Eyi yoo ja si awọn eto aisedede fun ohun elo wa, nfa awọn iṣoro pẹlu ifilọlẹ ati lilo.

Nibi data data miiran yoo wa si igbala, fun apẹẹrẹ, MySQL tabi deede rẹ. A sọ fun Grafana pe o yẹ ki o tọju awọn eto olumulo sinu ibi ipamọ data “apaja” yii. Lẹhinna, yoo to lati pato ọna si aaye data yii lẹẹkan lori ẹrọ kọọkan, ati ṣatunkọ gbogbo awọn eto olumulo miiran lori eyikeyi awọn ẹrọ foju; wọn yoo fa si awọn miiran.

Eyi ni aworan atọka ti awọn amayederun ohun elo ikẹhin:

Jẹ ki a kọ ẹkọ lati gbe pẹlu ọwọ wa

MySQL ati ClickHouse

Ṣaaju ki o to gbe iru ohun elo bẹ pẹlu titẹ bọtini kan, o jẹ dandan lati kọ ẹkọ bi o ṣe le mu ọkọọkan awọn paati rẹ ati ṣepọ wọn pẹlu ara wọn.

Nibi Yandex.Cloud yoo ṣe iranlọwọ fun wa, eyiti o pese awọn iwọntunwọnsi L3, ClickHouse ati MySQL bi awọn iṣẹ iṣakoso. Olumulo nikan nilo lati pato awọn paramita ati duro titi ti pẹpẹ yoo mu ohun gbogbo wa sinu aṣẹ iṣẹ.

Mo forukọsilẹ, ṣẹda awọsanma ati akọọlẹ isanwo kan. Lẹhin iyẹn, Mo lọ si awọsanma ati ṣeto MySQL ati awọn iṣupọ ClickHouse pẹlu awọn eto to kere julọ. Mo duro titi wọn fi di alakitiyan.

O tun nilo lati ranti lati ṣẹda data data ninu iṣupọ kọọkan ati tunto iraye si rẹ nipa lilo iwọle ati ọrọ igbaniwọle kan. Emi kii yoo lọ sinu awọn alaye nibi - ohun gbogbo jẹ kedere ni wiwo.

Awọn alaye ti kii ṣe kedere ni pe awọn apoti isura infomesonu ni ọpọlọpọ awọn ọmọ-ogun, eyiti o ṣe idaniloju ifarada aṣiṣe wọn. Sibẹsibẹ, Grafana nilo ogun kan pato fun ibi ipamọ data kọọkan ti o ṣiṣẹ pẹlu. Iwe kika gigun iwe Awọn awọsanma mu mi wá si ipinnu. O wa ni jade wipe ogun ti awọn eya c-<cluster_id>.rw.mdb.yandexcloud.net ti ya aworan si agbalejo oluwa ti nṣiṣe lọwọ lọwọlọwọ ti iṣupọ pẹlu ID ti o baamu. Eyi ni ohun ti a yoo fun Grafana.

olupin ayelujara

Bayi o to olupin ayelujara. Jẹ ki a gbe ẹrọ foju deede kan pẹlu Linux ati tunto Grafana pẹlu ọwọ lori rẹ.

Jẹ ki a sopọ nipasẹ ssh ki o fi awọn idii pataki sii.

sudo apt-get install -y apt-transport-https software-properties-common wget
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo add-apt-repository "deb https://packages.grafana.com/enterprise/deb stable main"
sudo apt-get update
sudo apt-get install -y grafana-enterprise 

Lẹhin iyẹn, jẹ ki a ṣiṣẹ Grafana labẹ systemctl ki o fi ohun itanna sori ẹrọ fun ṣiṣẹ pẹlu ClickHouse (bẹẹni, ko pese ni package ipilẹ).

sudo systemctl start grafana-server
sudo systemctl enable grafana-server
sudo grafana-cli plugins install vertamedia-clickhouse-datasource

Iyẹn ni, lẹhin iyẹn pẹlu aṣẹ ti o rọrun

sudo service grafana-server start

a yoo bẹrẹ olupin wẹẹbu naa. Bayi o le tẹ awọn ita IP adirẹsi ti awọn foju ẹrọ ni awọn kiri ayelujara, pato ibudo 3000 ati ki o wo awọn lẹwa Grafana UI.

Ṣugbọn maṣe yara, ṣaaju ki o to ṣeto Grafana, o gbọdọ ranti lati sọ fun ọna si MySQL lati le fi awọn eto pamọ sibẹ.

Gbogbo iṣeto ti olupin wẹẹbu Grafana wa ninu faili naa /etc/grafana/grafana.ini. Laini ti a beere dabi eyi:

;url =

A ṣeto ogun naa si iṣupọ MySQL. Faili kanna ni iwọle ati ọrọ igbaniwọle fun iraye si Grafana ninu aworan loke, eyiti nipasẹ aiyipada jẹ mejeeji dọgba admin.

O le lo awọn aṣẹ sed:

sudo sed -i "s#.*;url =.*#url = mysql://${MYSQL_USERNAME}:${MYSQL_PASSWORD}@${MYSQL_CLUSTER_URI}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_user =.*#admin_user = ${GRAFANA_USERNAME}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_password =.*#admin_password = ${GRAFANA_PASSWORD}#" /etc/grafana/grafana.ini

O to akoko lati tun olupin wẹẹbu bẹrẹ!

sudo service grafana-server restart

Bayi ni Grafana UI a yoo pato ClickHouse bi DataSource kan.

Mo ni anfani lati ṣaṣeyọri iṣeto iṣẹ pẹlu awọn eto atẹle:

Mo pato bi URL https://c-<cluster_id>.rw.mdb.yandexcloud.net:8443

Gbogbo! A ni ẹrọ foju kan ti n ṣiṣẹ pẹlu olupin wẹẹbu ti o sopọ si CH ati MySQL. O le ti gbejade dataset tẹlẹ si ClickHouse ati kọ awọn dasibodu. Sibẹsibẹ, a ko tii ṣaṣeyọri ibi-afẹde wa ati pe a ko ti gbe awọn amayederun ti o ni kikun ranṣẹ.

Apoti

Yandex.Cloud gba ọ laaye lati ṣẹda aworan disiki ti ẹrọ foju ti o wa tẹlẹ, ati lori ipilẹ rẹ - nọmba eyikeyi ti awọn ẹrọ ti o jọra si ara wọn. Eleyi jẹ gangan ohun ti a yoo lo. Lati ṣe apejọ aworan ni irọrun, mu ohun elo naa Apoti lati HashiCorp. Yoo gba bi titẹ faili json kan pẹlu awọn ilana fun iṣakojọpọ aworan naa.

Faili json wa yoo ni awọn bulọọki meji: awọn akọle ati awọn olupese. Àkọsílẹ akọkọ ṣe apejuwe awọn aye ti aworan funrararẹ bi nkan kan, ati bulọọki keji ṣe apejuwe awọn ilana fun kikun rẹ pẹlu akoonu pataki.

Awọn akọle

{
"builders": [
    {
      "type": "yandex",
      "endpoint": "{{user `endpoint`}}",
      "folder_id": "<folder_id>",
      "subnet_id": "{{user `subnet_id`}}",
      "zone": "{{user `zone`}}",
      "labels": {},
      "use_ipv4_nat": true,
      "use_internal_ip": false,
      "service_account_key_file": "<service_account_key_file>",
      "image_name": "grafana-{{timestamp}}",
      "image_family": "grafana",
      "image_labels": {},
      "image_description": "GRAFANA",
      "source_image_family": "ubuntu-1804-lts",
      "disk_size_gb": 3,
      "disk_type": "network-hdd",
      "ssh_username": "ubuntu"
    }
  ],
...
}

Ninu awoṣe yii, o nilo lati ṣeto idanimọ ti apakan ninu awọsanma nibiti o fẹ ṣẹda aworan naa, bakanna bi ọna si faili pẹlu awọn bọtini lati akọọlẹ iṣẹ ti a ṣẹda tẹlẹ ni apakan yii. O le ka diẹ sii nipa ṣiṣẹda awọn akọọlẹ iṣẹ ati awọn bọtini ni irisi faili ni apakan ti o baamu iwe.

Yi iṣeto ni wi pe awọn disk image yoo wa ni itumọ ti da lori awọn Syeed ubuntu-1804-lts, ti a gbe sinu apakan olumulo ti o yẹ ni idile aworan GRAFANA labẹ orukọ grafana-{{timestamp}}.

Olupese

Bayi ba wa ni diẹ awon apa ti awọn iṣeto ni. Yoo ṣe apejuwe lẹsẹsẹ awọn iṣe ti yoo nilo lati ṣe lori ẹrọ foju ṣaaju didi ipo rẹ sinu aworan disiki kan.

{
...,
"provisioners": [
    {
            "type": "shell",
            "pause_before": "5s",
            "scripts": [
                "prepare-ctg.sh"
            ]
        },
    {
            "type": "file",
            "source": "setup.sh",
            "destination": "/opt/grafana/setup.sh"
        },
        {
            "type": "shell",
        "execute_command": "sudo {{ .Vars }} bash '{{ .Path }}'",
            "pause_before": "5s",
            "scripts": [
                "install-packages.sh",
                "grafana-setup.sh",
                "run-setup-at-reboot.sh"
        ]
        }
  ]
}

Nibi gbogbo awọn iṣe ti pin si awọn ipele mẹta. Ni ipele akọkọ, iwe afọwọkọ ti o rọrun ti wa ni ṣiṣe ti o ṣẹda itọsọna oluranlọwọ.

mura-ctg.sh:

#!/bin/bash
sudo mkdir -p /opt/grafana
sudo chown -R ubuntu:ubuntu /opt/grafana

Ni ipele ti o tẹle, a gbe iwe afọwọkọ kan sinu itọsọna yii, eyiti yoo nilo lati ṣe ifilọlẹ lẹsẹkẹsẹ lẹhin ti o bẹrẹ ẹrọ foju. Iwe afọwọkọ yii yoo fi awọn oniyipada olumulo ti o nilo lati forukọsilẹ ni atunto Grafana ki o tun olupin wẹẹbu bẹrẹ.

setup.sh:

#!/bin/bash
CLUSTER_ID="<cluster_id>"
USERNAME="<username>"
PASSWORD="<password>"
sudo sed -i "s#.*;url =.*#url = mysql://${USERNAME}:${PASSWORD}@c-${CLUSTER_ID}.rw.mdb.yandexcloud.net#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_user =.*#admin_user = ${USERNAME}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_password =.*#admin_password = ${PASSWORD}#" /etc/grafana/grafana.ini
sudo service grafana-server restart

Lẹhin eyi awọn nkan mẹta lo ku lati ṣe:
1) fi sori ẹrọ awọn idii
2) ṣiṣe Grafana labẹ systemctl ki o fi ohun itanna ClickHouse sori ẹrọ
3) fi iwe afọwọkọ setup.sh sinu isinyi ifilọlẹ lẹsẹkẹsẹ lẹhin titan ẹrọ foju.

install-packages.sh:

#!/bin/bash
sudo systemd-run --property='After=apt-daily.service apt-daily-upgrade.service' --wait /bin/true
sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo add-apt-repository "deb https://packages.grafana.com/enterprise/deb stable main"
sudo apt-get update
sudo apt-get install -y grafana-enterprise 

grafana-setup.sh:

#!/bin/bash
sudo systemctl start grafana-server
sudo systemctl enable grafana-server
sudo grafana-cli plugins install vertamedia-clickhouse-datasource

run-setup-at-reboot.sh:

#!/bin/bash
chmod +x /opt/grafana/setup.sh
cat > /etc/cron.d/first-boot <<EOF
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
@reboot root /bin/bash /opt/grafana/setup.sh > /var/log/yc-setup.log 2>&1
EOF
chmod +x /etc/cron.d/first-boot;

Bayi gbogbo ohun ti o ku ni lati ṣiṣẹ Packer ati gba aworan ti o wujade ti a gbe sinu apakan ti a sọ. Nigbati o ba ṣẹda ẹrọ foju kan, o le yan bi disiki bata ati lẹhin ifilọlẹ iwọ yoo gba olupin wẹẹbu Grafana ti o ti ṣetan.

Ẹgbẹ apẹẹrẹ ati iwontunwonsi

Ni kete ti a ba ni aworan disiki ti o fun wa laaye lati ṣẹda ọpọlọpọ awọn olupin wẹẹbu Grafana kanna, a le ṣẹda ẹgbẹ apẹẹrẹ kan. Lori Syeed Yandex.Cloud, ọrọ yii n tọka si iṣọkan ti awọn ẹrọ foju ti o ni awọn abuda kanna. Nigbati o ba ṣẹda ẹgbẹ apẹẹrẹ, a tunto apẹẹrẹ ti gbogbo awọn ẹrọ inu ẹgbẹ yii, lẹhinna awọn abuda ti ẹgbẹ funrararẹ (fun apẹẹrẹ, o kere julọ ati nọmba ti o pọju ti awọn ẹrọ ti nṣiṣe lọwọ). Ti nọmba lọwọlọwọ ko ba pade awọn ibeere wọnyi, lẹhinna ẹgbẹ apẹẹrẹ funrararẹ yoo yọ awọn ẹrọ ti ko wulo tabi ṣẹda awọn tuntun ni aworan tirẹ.

Gẹgẹbi apakan ti iṣẹ-ṣiṣe wa, a yoo ṣẹda ẹgbẹ apẹẹrẹ ti awọn olupin wẹẹbu ti yoo ṣe ipilẹṣẹ lati aworan disk ti a ṣẹda tẹlẹ.

Ohun ti o jẹ iyalẹnu gaan ni iṣeto ẹgbẹ apẹẹrẹ ti o kẹhin. Ẹgbẹ ibi-afẹde ni iṣọpọ pẹlu Load Balancer yoo ṣe iranlọwọ fun ọ lati tunto iwọntunwọnsi L3 kan lori awọn ẹrọ foju ti ẹgbẹ yii nipa titẹ awọn bọtini meji kan.

Nigbati o ba ṣeto iwọntunwọnsi, Mo ṣe awọn aaye pataki meji:

1. Mo ti ṣe awọn iwontunwonsi gba olumulo ijabọ lori ibudo 80 ati ki o àtúnjúwe o si ibudo 3000 ti awọn foju ero, pato ibi ti Grafana ngbe.
2. Mo ṣeto iṣayẹwo ṣiṣeeṣe ti awọn ẹrọ nipa pingi wọn si ibudo 3000.

Mini Lakotan

Nikẹhin, a ni anfani lati ran awọn amayederun ohun elo ti o fẹ lọ pẹlu ọwọ, ati ni bayi a ni iṣẹ Grafana resilient giga. Iwọ nikan nilo lati mọ adiresi IP ti iwọntunwọnsi bi aaye iwọle si ohun elo ati agbalejo ti iṣupọ ClickHouse lati le ṣajọpọ dataset sinu rẹ.

Yoo dabi ẹnipe iṣẹgun? Beeni isegun. Sugbon ohun kan tun da mi loju. Gbogbo ilana ti o wa loke nilo ọpọlọpọ awọn igbesẹ afọwọṣe ati pe ko ṣe iwọn rara; Emi yoo fẹ lati ṣe adaṣe adaṣe ti o ba ṣeeṣe. Eleyi jẹ ohun ti nigbamii ti apakan yoo wa ni ti yasọtọ si.

Terraform Integration

A yoo tun lo ọpa kan lati HashiCorp ti a pe Ilana ipilẹ. Yoo ṣe iranlọwọ fun ọ lati mu gbogbo awọn amayederun ohun elo ṣiṣẹ pẹlu titẹ bọtini kan, da lori ọpọlọpọ awọn oniyipada ti olumulo kọja. Jẹ ki a kọ ohunelo kan ti o le ṣiṣẹ ni awọn akoko pupọ ni awọn apakan oriṣiriṣi ti awọn olumulo oriṣiriṣi.

Gbogbo iṣẹ pẹlu Terraform wa si kikọ faili iṣeto ni (*.tf) ati ṣiṣẹda awọn amayederun ti o da lori rẹ.

Awọn oniyipada

Ni ibẹrẹ akọkọ ti faili, a yoo pẹlu awọn oniyipada ti o pinnu ibiti ati bii awọn amayederun iwaju yoo ṣe ran lọ.

variable "oauth_token" {
    type = string
    default = "<oauth-token>"
}
variable "cloud_id" {
    type = string
    default = "<cloud-id>"
}
variable "folder_id" {
    type = string
    default = "<folder_id>"
}
variable "service_account_id" {
    type = string
    default = "<service_account_id>"
}
variable "image_id" {
    type = string
    default = "<image_id>"
}
variable "username" {
    type = string
    default = "<username>"
}
variable "password" {
    type = string
    default = "<password>"
}
variable "dbname" {
    type = string
    default = "<dbname>"
}
variable "public_key_path" {
    type = string
    default = "<path to ssh public key>"
}

Gbogbo ilana imuṣiṣẹ ohun elo yoo wa silẹ lati kọ aworan disk kan ati ṣeto awọn oniyipada wọnyi. Jẹ ki n ṣe alaye ohun ti wọn jẹ lodidi fun:

oauth_tokini - ami kan fun iwọle si awọsanma. Le ti wa ni gba nipa ọna asopọ.
awọsanma_id - idanimọ awọsanma nibiti a yoo fi ohun elo naa ranṣẹ
folda_id - idamo apakan ibi ti a yoo ran awọn ohun elo
service_account_id - idamo iroyin iṣẹ ni apakan ti o baamu ti awọsanma.
aworan_id - idamo ti disk image gba lilo Packer
olumulo и ọrọigbaniwọle - orukọ olumulo ati ọrọ igbaniwọle lati wọle si awọn apoti isura data mejeeji ati olupin wẹẹbu Grafana
dbname - orukọ data inu CH ati awọn iṣupọ MySQL
ọna_bọtini_gbangba - ọna si faili pẹlu bọtini ssh ti gbogbo eniyan, eyiti o le lo lati sopọ labẹ orukọ ubuntu si awọn ẹrọ foju pẹlu awọn olupin wẹẹbu

Eto olupese

Bayi o nilo lati tunto olupese Terraform - ninu ọran wa, Yandex:

provider "yandex" {
  token     = var.oauth_token
  cloud_id  = var.cloud_id
  folder_id = var.folder_id
  zone      = "ru-central1-a"
}

Iwọ yoo ṣe akiyesi pe nibi a nlo awọn oniyipada ti asọye loke.

Nẹtiwọọki ati awọn iṣupọ

Bayi a yoo ṣẹda nẹtiwọọki kan ninu eyiti awọn eroja ti awọn amayederun wa yoo ṣe ibaraẹnisọrọ, awọn subnets mẹta (ọkan ni agbegbe kọọkan) ati gbe awọn iṣupọ CH ati MySQL dide.

resource "yandex_vpc_network" "grafana_network" {}

resource "yandex_vpc_subnet" "subnet_a" {
  zone           = "ru-central1-a"
  network_id     = yandex_vpc_network.grafana_network.id
  v4_cidr_blocks = ["10.1.0.0/24"]
}

resource "yandex_vpc_subnet" "subnet_b" {
  zone           = "ru-central1-b"
  network_id     = yandex_vpc_network.grafana_network.id
  v4_cidr_blocks = ["10.2.0.0/24"]
}

resource "yandex_vpc_subnet" "subnet_c" {
  zone           = "ru-central1-c"
  network_id     = yandex_vpc_network.grafana_network.id
  v4_cidr_blocks = ["10.3.0.0/24"]
}

resource "yandex_mdb_clickhouse_cluster" "ch_cluster" {
  name        = "grafana-clickhouse"
  environment = "PRODUCTION"
  network_id  = yandex_vpc_network.grafana_network.id

  clickhouse {
    resources {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 16
    }
  }

  zookeeper {
    resources {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 10
    }
  }

  database {
    name = var.dbname
  }

  user {
    name     = var.username
    password = var.password
    permission {
      database_name = var.dbname
    }
  }

  host {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-a"
    subnet_id = yandex_vpc_subnet.subnet_a.id
  }

  host {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-b"
    subnet_id = yandex_vpc_subnet.subnet_b.id
  }

  host {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-c"
    subnet_id = yandex_vpc_subnet.subnet_c.id
  }

  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-a"
    subnet_id = yandex_vpc_subnet.subnet_a.id
  }

  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-b"
    subnet_id = yandex_vpc_subnet.subnet_b.id
  }

  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-c"
    subnet_id = yandex_vpc_subnet.subnet_c.id
  }
}

resource "yandex_mdb_mysql_cluster" "mysql_cluster" {
  name        = "grafana_mysql"
  environment = "PRODUCTION"
  network_id  = yandex_vpc_network.grafana_network.id
  version     = "8.0"

  resources {
    resource_preset_id = "s2.micro"
    disk_type_id       = "network-ssd"
    disk_size          = 16
  }

  database {
    name = var.dbname
  }

  user {
    name     = var.username
    password = var.password
    permission {
      database_name = var.dbname
      roles         = ["ALL"]
    }
  }

  host {
    zone      = "ru-central1-a"
    subnet_id = yandex_vpc_subnet.subnet_a.id
  }
  host {
    zone      = "ru-central1-b"
    subnet_id = yandex_vpc_subnet.subnet_b.id
  }
  host {
    zone      = "ru-central1-c"
    subnet_id = yandex_vpc_subnet.subnet_c.id
  }
}

Gẹgẹbi o ti le rii, ọkọọkan awọn iṣupọ meji naa ni a ṣẹda ni ifarada-ẹbi pupọ nipa wiwa ni awọn agbegbe wiwa mẹta.

Awọn olupin ayelujara

Ó dà bíi pé a lè máa bá a lọ nínú ẹ̀mí kan náà, ṣùgbọ́n mo sá lọ sínú ìṣòro. Ṣaaju eyi, Mo kọkọ gbe iṣupọ MySQL kan ati pe lẹhin iyẹn nikan, ni mimọ ID rẹ, Mo gba aworan disk kan pẹlu iṣeto ti o nilo, nibiti Mo ti ṣalaye agbalejo si iṣupọ naa. Ṣugbọn ni bayi a ko mọ ID iṣupọ ṣaaju ifilọlẹ Terraform, pẹlu ni akoko kikọ aworan naa. Nitorinaa, Mo ni lati lo si atẹle naa omoluabi.

Lilo iṣẹ metadata Amazon, a yoo kọja diẹ ninu awọn paramita si ẹrọ foju, eyiti yoo gba ati ṣe ilana. A nilo ẹrọ naa lati lọ si metadata lẹhin ogun iṣupọ MySQL ati ọrọ igbaniwọle orukọ olumulo, eyiti olumulo pato ninu faili Terraform, lẹhin ti o bẹrẹ. Jẹ ki a yi awọn akoonu inu faili pada diẹ diẹ setup.sh, eyi ti nṣiṣẹ nigbati awọn foju ẹrọ ti wa ni titan.

setup.sh:

#!/bin/bash
CLUSTER_URI="$(curl -H 'Metadata-Flavor:Google' http://169.254.169.254/computeMetadata/v1/instance/attributes/mysql_cluster_uri)"
USERNAME="$(curl -H 'Metadata-Flavor:Google' http://169.254.169.254/computeMetadata/v1/instance/attributes/username)"
PASSWORD="$(curl -H 'Metadata-Flavor:Google' http://169.254.169.254/computeMetadata/v1/instance/attributes/password)"
sudo sed -i "s#.*;url =.*#url = mysql://${USERNAME}:${PASSWORD}@${CLUSTER_URI}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_user =.*#admin_user = ${USERNAME}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_password =.*#admin_password = ${PASSWORD}#" /etc/grafana/grafana.ini
sudo service grafana-server restart

Intance Ẹgbẹ ati iwontunwonsi

Lẹhin ti tun aworan disiki titun ṣe, a le nipari ṣafikun faili wa fun Terraform.

Jẹ ki a tọka pe a fẹ lo aworan disk ti o wa tẹlẹ:

data "yandex_compute_image" "grafana_image" {
  image_id = var.image_id
}

Bayi jẹ ki a ṣẹda ẹgbẹ apẹẹrẹ:

resource "yandex_compute_instance_group" "grafana_group" {
  name               = "grafana-group"
  folder_id          = var.folder_id
  service_account_id = var.service_account_id
  instance_template {
    platform_id = "standard-v1"
    resources {
      memory = 1
      cores  = 1
    }
    boot_disk {
      mode = "READ_WRITE"
      initialize_params {
        image_id = data.yandex_compute_image.grafana_image.id
        size     = 4
      }
    }
    network_interface {
      network_id = yandex_vpc_network.grafana_network.id
      subnet_ids = [yandex_vpc_subnet.subnet_a.id, yandex_vpc_subnet.subnet_b.id, yandex_vpc_subnet.subnet_c.id]
      nat = "true"
    }
    metadata = {
      mysql_cluster_uri = "c-${yandex_mdb_mysql_cluster.mysql_cluster.id}.rw.mdb.yandexcloud.net:3306/${var.dbname}"
      username = var.username
      password = var.password
      ssh-keys = "ubuntu:${file("${var.public_key_path}")}"
    }
    network_settings {
      type = "STANDARD"
    }
  }

  scale_policy {
    fixed_scale {
      size = 6
    }
  }

  allocation_policy {
    zones = ["ru-central1-a", "ru-central1-b", "ru-central1-c"]
  }

  deploy_policy {
    max_unavailable = 2
    max_creating    = 2
    max_expansion   = 2
    max_deleting    = 2
  }

  load_balancer {
    target_group_name = "grafana-target-group"
  }
}

O tọ lati san ifojusi si bi a ti kọja sinu metadata cluster_uri, username и password. O jẹ iwọnyi pe ẹrọ foju yoo mu jade ni ibẹrẹ ati fi sinu atunto Grafana.

O to iwọntunwọnsi.

resource "yandex_lb_network_load_balancer" "grafana_balancer" {
  name = "grafana-balancer"

  listener {
    name = "grafana-listener"
    port = 80
    target_port = 3000
    external_address_spec {
      ip_version = "ipv4"
    }
  }

  attached_target_group {
    target_group_id = yandex_compute_instance_group.grafana_group.load_balancer.0.target_group_id

    healthcheck {
      name = "healthcheck"
      tcp_options {
        port = 3000
      }
    }
  }
}

Suga kekere kan

O ku die. Lẹhin ti a ti gbe awọn amayederun lọ, iwọ yoo ni lati lọ si Grafana UI ki o ṣafikun iṣupọ CH pẹlu ọwọ (ID eyiti o tun nilo lati gba) bi Orisun Data. Ṣugbọn Terraform mọ ID iṣupọ. Ẹ jẹ́ ká gbé ọ̀rọ̀ náà lé e lọ́wọ́.

Jẹ ki a ṣafikun olupese tuntun kan - Grafana, ki o fun ni adiresi IP iwọntunwọnsi bi agbalejo. Gbogbo awọn iyipada ti Terraform ṣe lori ẹrọ nibiti awọn ipinnu iwọntunwọnsi rẹ yoo dagba ni MySQL, ati nitorinaa lori gbogbo awọn ẹrọ miiran.

provider "grafana" {
  url  = "http://${[for s in yandex_lb_network_load_balancer.grafana_balancer.listener: s.external_address_spec.0.address].0}"
  auth = "${var.username}:${var.password}"
}

resource "grafana_data_source" "ch_data_source" {
  type          = "vertamedia-clickhouse-datasource"
  name          = "grafana"
  url           = "https://c-${yandex_mdb_clickhouse_cluster.ch_cluster.id}.rw.mdb.yandexcloud.net:8443"
  basic_auth_enabled = "true"
  basic_auth_username = var.username
  basic_auth_password = var.password
  is_default = "true"
  access_mode = "proxy"
}

Jẹ ki a fọ ​​irun wa

Jẹ ki a ṣe afihan adiresi IP iwọntunwọnsi ati ogun ti akopọ ClickHouse

output "grafana_balancer_ip_address" {
  value = [for s in yandex_lb_network_load_balancer.grafana_balancer.listener: s.external_address_spec.0.address].0
}

output "clickhouse_cluster_host" {
  value = "https://c-${yandex_mdb_clickhouse_cluster.ch_cluster.id}.rw.mdb.yandexcloud.net:8443"
}

O le ṣiṣe

Gbogbo! Faili iṣeto wa ti ṣetan ati pe a le, nipa ṣeto awọn oniyipada, sọ fun Terraform lati gbe ohun gbogbo ti a ṣalaye loke. Gbogbo ilana gba mi nipa iṣẹju 15.
Ni ipari o le rii ifiranṣẹ lẹwa kan:

Apply complete! Resources: 9 added, 0 changed, 0 destroyed.
Outputs:

clickhouse_cluster_host = https://c-c9q14ipa2ngadqsbp2iq.rw.mdb.yandexcloud.net:8443
grafana_balancer_ip_address = 130.193.50.25

Ati ninu awọsanma, awọn eroja ti awọn amayederun ti o dide yoo han:

Jẹ ki a ṣe idajọ awọn esi

Bayi, ni lilo Grafana gẹgẹbi apẹẹrẹ, ọkọọkan rẹ le ran awọn ohun elo ṣiṣẹ pẹlu faaji awọsanma ti o tan kaakiri lori pẹpẹ Yandex.Cloud. Awọn irinṣẹ iranlọwọ lati HashiCorp bi Packer ati Terraform le ṣe iranlọwọ fun ọ pẹlu eyi. Mo nireti pe ẹnikan rii pe nkan yii wulo :)

PS Ni isalẹ Emi yoo so ọna asopọ kan si ibi ipamọ nibiti o ti le rii awọn ilana ti a ti ṣetan fun Packer ati Terraform, awọn ajẹkù eyiti Mo pese ninu nkan yii.

ibi ipamọ

orisun: www.habr.com