Tuuina atu o auaunaga tufatufaina i Yandex.Cloud faʻaaoga Grafana e fai ma faʻataʻitaʻiga

Talofa uma! I le avea ai ma se vaega o laʻu galuega, sa ou suʻesuʻeina le gafatia o sea faʻavae ao faʻalotoifale e pei o Yandex.Cloud. O lo'o tu'uina atu e le fa'avae auaunaga eseese mo le fo'ia o fa'afitauli fa'atino. Ae ui i lea, o nisi taimi e te manaʻomia le faʻatulagaina o lau lava faʻaoga ao faʻatasi ai ma se faʻalauteleina o atinaʻe e faʻavae i luga o nei auaunaga. I lenei tusiga ou te fia faʻasoa atu loʻu poto masani i le faʻaogaina o sea talosaga.

O le a le mea e te mana'o e maua?

tusifana - o se meafaigaluega mamana mo le foia o faʻafitauli auʻiliʻili poʻo le mataʻituina o faʻafitauli o soʻo se faiga. I lona faʻatulagaga faʻavae, o se masini faʻapipiʻi ma se 'upega tafaʻilagi a Grafana, faʻapea foʻi ma se faʻamaumauga (ClickHouse, InfluxDB, ma isi) faʻatasi ai ma se faʻamaumauga e faʻavae ai auʻiliʻiliga.

A maeʻa ona faʻalauiloa se masini komepiuta ma se 'upega tafaʻilagi, e mafai ona e alu i lona talimalo ma maua se UI matagofie, faʻamaonia faʻamaumauga e fai ma faʻapogai mo nisi galuega, fatuina dashboards ma kalafi.

O le fa'ata'ita'iga fa'avae e tasi le fa'aletonu taua - e le fa'apalepale i se mea sese. O lona uiga, o le faʻatinoga atoa o le talosaga e faʻalagolago i le faʻaogaina o le tasi masini masini. Afai e musu pe 10 tagata e tatalaina le UI i le taimi e tasi, ona tulaʻi mai lea o faʻafitauli.

E mafai ona foia faigofie: e te manaʻomia ... faʻapipiʻi le tele o masini komepiuta tutusa ma se 'upega tafaʻilagi ma tuʻu i lalo ole paleni L3. Ae le o mea uma e matua manino iinei. O lo'o teuina e Grafana le fa'aogaina o tagata fa'aoga (ala i fa'amaumauga, dashboards, kalafi, ma isi) sa'o i luga o le tisiki o lana masini masini. O lea la, afai tatou te suia nisi tulaga i le UI, o nei suiga o le a atagia na o le masini komepiuta lea na auina mai ai i matou e le paleni. O lenei mea o le a taʻitaʻia ai tulaga le fetaui mo la matou talosaga, mafua ai faʻafitauli i le faʻalauiloaina ma le faʻaaogaina.

O iinei o le a sau ai se isi faʻamaumauga e laveai, mo se faʻataʻitaʻiga, MySQL poʻo lona tutusa. Matou te taʻu atu ia Grafana e tatau ona ia teuina tulaga faʻaoga i totonu o lenei "faʻasao" faʻamaumauga. A maeʻa, o le a lava le faʻamaonia o le ala i lenei faʻamaumauga i luga o masini taʻitasi, ma faʻasaʻo uma isi faʻaoga faʻaoga i luga o soʻo se masini masini; latou te faʻalautele atu i isi.

Ole fa'ata'ita'iga lenei o le fa'aogaina mulimuli o atina'e:

Tatou aoao e sii i luga i o tatou lima

MySQL ma ClickHouse

Aʻo leʻi faʻapipiʻiina sea talosaga ma le kiliki o se ki, sa tatau ona aʻoaʻo pe faʻafefea ona taulimaina ona vaega taʻitasi ma tuʻufaʻatasia i latou.

O iinei o le a fesoasoani ai Yandex.Cloud ia i matou, lea e tuʻuina atu ai L3 paleni, ClickHouse ma MySQL e pei o auaunaga faʻatautaia. E na'o le tagata fa'aoga e mana'omia le fa'amaonia o tapula'a ma fa'atali se'ia o'o ina fa'aulu e le fa'avae mea uma i le fa'atonuga.

Na ou lesitala, faia se ao ma se teugatupe totogi. Ina ua uma lena, sa ou alu i le ao ma seti MySQL ma ClickHouse fuifui ma ni tulaga laiti. Sa ou faatali seia oo ina latou toaaga.

E tatau foi ona e manatua e fai se database i fuifui taitasi ma fetuutuunai avanoa i ai e faaaoga ai se login ma upu faataga. O le a ou le alu i auiliiliga iinei - o mea uma e matua manino lava i le atinaʻe.

O faʻamatalaga e le o manino o nei faʻamaumauga o loʻo i ai le tele o 'au, e faʻamautinoa ai lo latou faʻapalepale sese. Ae ui i lea, e manaʻomia e Grafana le tasi talimalo mo faʻamaumauga taʻitasi e galue ai. Faitauga umi fa'amaumauga Na aumaia e ao ia te au se faaiuga. E foliga mai o le talimalo o le ituaiga c-<cluster_id>.rw.mdb.yandexcloud.net fa'afanua i le matai matai o lo'o galue nei o le fuifui ma le ID talafeagai. O le mea lea o le a matou avatu ia Grafana.

Web server

O lea la ua o'o i le 'upega tafa'ilagi. Se'i o tatou si'i a'e se masini komepiuta masani ma Linux ma fa'atulaga ma le lima le Grafana i luga.

Tatou fa'afeso'ota'i e ala i le ssh ma fa'apipi'i afifi mana'omia.

sudo apt-get install -y apt-transport-https software-properties-common wget
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo add-apt-repository "deb https://packages.grafana.com/enterprise/deb stable main"
sudo apt-get update
sudo apt-get install -y grafana-enterprise 

A maeʻa lena, seʻi o tatou taʻavale Grafana i lalo o le systemctl ma faʻapipiʻi le plugin mo le galulue faatasi ma ClickHouse (ioe, e le o tuʻuina atu i le pusa autu).

sudo systemctl start grafana-server
sudo systemctl enable grafana-server
sudo grafana-cli plugins install vertamedia-clickhouse-datasource

O le mea lena, a uma lena ma se poloaiga faigofie

sudo service grafana-server start

o le a matou amataina le upega tafaʻilagi. O lea e mafai ona e ulufale i le tuatusi IP fafo o le masini komepiuta i totonu o le masini, faʻamaonia le taulaga 3000 ma vaʻai i le matagofie o le Grafana UI.

Ae aua le faanatinati, aʻo leʻi faʻatulagaina le Grafana, e tatau ona e manatua e taʻu atu i ai le ala i MySQL ina ia teu ai faʻatulagaga iina.

O le faʻatulagaga atoa o le upega tafaʻilagi a Grafana o loʻo i totonu o le faila /etc/grafana/grafana.ini. Ole laina mana'omia e pei o lenei:

;url =

Matou te setiina le talimalo i le MySQL cluster. O le faila lava lea e tasi o loʻo i ai le saini ma le upega tafaʻilagi mo le mauaina o Grafana i le ata o loʻo i luga, lea e le mafai ona tutusa uma admin.

E mafai ona e faʻaogaina sed commands:

sudo sed -i "s#.*;url =.*#url = mysql://${MYSQL_USERNAME}:${MYSQL_PASSWORD}@${MYSQL_CLUSTER_URI}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_user =.*#admin_user = ${GRAFANA_USERNAME}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_password =.*#admin_password = ${GRAFANA_PASSWORD}#" /etc/grafana/grafana.ini

Ua oo i le taimi e toe amata ai le upega tafailagi!

sudo service grafana-server restart

I le taimi nei i le Grafana UI o le a matou faʻamaonia ClickHouse o se DataSource.

Na mafai ona ou ausia se faʻatulagaga galue ma tulaga nei:

Na ou faʻamaonia o le URL https://c-<cluster_id>.rw.mdb.yandexcloud.net:8443

Uma! E tasi le matou masini komepiuta galue ma se upega tafaʻilagi e fesoʻotaʻi ma CH ma MySQL. E mafai ona e tuʻuina atu le faʻamaumauga i le ClickHouse ma fausia faʻailoga. Ae ui i lea, matou te leʻi ausia la matou sini ma e leʻi faʻapipiʻiina se atinaʻe atoatoa.

Afifiina

Yandex.Cloud e mafai ai ona e fatuina se ata tisiki o se masini komepiuta o loʻo i ai, ma i luga o lona faʻavae - soʻo se numera o masini e tutusa le tasi ma le isi. O le mea tonu lea o le a matou faʻaaogaina. Ina ia faigofie ona faʻapipiʻi le ata, ave le meafaigaluega Afifiina mai HashiCorp. E manaʻomia e tuʻuina atu se faila json ma faʻatonuga mo le faʻapipiʻiina o le ata.

O la matou faila json o le a aofia ai ni poloka se lua: faufale ma tuʻuina atu. O le poloka muamua o loʻo faʻamatalaina faʻamaufaʻailoga o le ata lava ia o se faʻalapotopotoga, ma o le poloka lona lua o loʻo faʻamatalaina faatonuga mo le faʻatumuina o mea e manaʻomia.

tufuga

{
"builders": [
    {
      "type": "yandex",
      "endpoint": "{{user `endpoint`}}",
      "folder_id": "<folder_id>",
      "subnet_id": "{{user `subnet_id`}}",
      "zone": "{{user `zone`}}",
      "labels": {},
      "use_ipv4_nat": true,
      "use_internal_ip": false,
      "service_account_key_file": "<service_account_key_file>",
      "image_name": "grafana-{{timestamp}}",
      "image_family": "grafana",
      "image_labels": {},
      "image_description": "GRAFANA",
      "source_image_family": "ubuntu-1804-lts",
      "disk_size_gb": 3,
      "disk_type": "network-hdd",
      "ssh_username": "ubuntu"
    }
  ],
...
}

I totonu o lenei faʻataʻitaʻiga, e tatau ona e setiina le faʻamatalaga o le vaega i le ao e te manaʻo e fai ai le ata, faʻapea foʻi ma le ala i le faila ma ki mai le tala o le tautua na faia muamua i lenei vaega. E mafai ona e faitau atili e uiga i le fatuina o tala o auaunaga ma ki i foliga o se faila i le vaega tutusa fa'amaumauga.

O lenei faʻatulagaga e fai mai o le ata o le disk o le a fausia e faʻavae i luga o le faʻavae ubuntu-1804-lts, tu'u i le vaega fa'aoga talafeagai i le aiga ata GRAFANA lalo ole igoa grafana-{{timestamp}}.

Tagata saunia

Ua o'o mai nei le vaega sili ona manaia o le fa'atulagaina. O le a faʻamatalaina le faʻasologa o gaioiga e manaʻomia ona faia i luga o le masini masini aʻo leʻi faʻamaisa lona tulaga i se ata tisiki.

{
...,
"provisioners": [
    {
            "type": "shell",
            "pause_before": "5s",
            "scripts": [
                "prepare-ctg.sh"
            ]
        },
    {
            "type": "file",
            "source": "setup.sh",
            "destination": "/opt/grafana/setup.sh"
        },
        {
            "type": "shell",
        "execute_command": "sudo {{ .Vars }} bash '{{ .Path }}'",
            "pause_before": "5s",
            "scripts": [
                "install-packages.sh",
                "grafana-setup.sh",
                "run-setup-at-reboot.sh"
        ]
        }
  ]
}

O fa'agaioiga uma e vaevaeina i ni vaega se 3. I le laasaga muamua, o loʻo faʻatinoina se faʻamatalaga faigofie e fatuina ai se lisi fesoasoani.

saunia-ctg.sh:

#!/bin/bash
sudo mkdir -p /opt/grafana
sudo chown -R ubuntu:ubuntu /opt/grafana

I le isi laasaga, matou te tuʻuina se tusitusiga i totonu o lenei lisi, lea e manaʻomia ona faʻalauiloa vave pe a uma ona amata le masini masini. O lenei tusitusiga o le a tuʻuina ai le tagata faʻaoga fesuiaiga e manaʻomia ona resitalaina i le Grafana config ma toe amata le upega tafaʻilagi.

setup.sh:

#!/bin/bash
CLUSTER_ID="<cluster_id>"
USERNAME="<username>"
PASSWORD="<password>"
sudo sed -i "s#.*;url =.*#url = mysql://${USERNAME}:${PASSWORD}@c-${CLUSTER_ID}.rw.mdb.yandexcloud.net#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_user =.*#admin_user = ${USERNAME}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_password =.*#admin_password = ${PASSWORD}#" /etc/grafana/grafana.ini
sudo service grafana-server restart

A maeʻa lenei mea e 3 mea e totoe e fai:
1) faʻapipiʻi afifi
2) taʻavale Grafana i lalo o le systemctl ma faʻapipiʻi le ClickHouse plugin
3) tuʻu le setup.sh script i le faʻailoga faʻalauiloa i le taimi lava e uma ai ona ki luga le masini masini.

install-packages.sh:

#!/bin/bash
sudo systemd-run --property='After=apt-daily.service apt-daily-upgrade.service' --wait /bin/true
sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo add-apt-repository "deb https://packages.grafana.com/enterprise/deb stable main"
sudo apt-get update
sudo apt-get install -y grafana-enterprise 

grafana-setup.sh:

#!/bin/bash
sudo systemctl start grafana-server
sudo systemctl enable grafana-server
sudo grafana-cli plugins install vertamedia-clickhouse-datasource

run-setup-at-reboot.sh:

#!/bin/bash
chmod +x /opt/grafana/setup.sh
cat > /etc/cron.d/first-boot <<EOF
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
@reboot root /bin/bash /opt/grafana/setup.sh > /var/log/yc-setup.log 2>&1
EOF
chmod +x /etc/cron.d/first-boot;

O mea uma e totoe o le taʻavale Packer ma faʻapipiʻi le ata o loʻo tuʻuina i le vaega faʻamaonia. Pe a fatuina se masini komepiuta, e mafai ona e filifilia e avea o se boot disk ma a maeʻa le faʻalauiloaina o le ae mauaina se faʻapipiʻi faʻapipiʻi Grafana web server.

Fa'ata'ita'iga vaega ma paleni

O le taimi lava e maua ai se ata tisiki e mafai ai ona tatou fatuina le tele o sapalai uepesite Grafana tutusa, e mafai ona tatou fatuina se faʻataʻitaʻiga vaega. I luga o le Yandex.Cloud platform, o lenei faaupuga e faʻatatau i le tuʻufaʻatasia o masini masini e tutusa uiga. Pe a fatuina se faʻataʻitaʻiga faʻataʻitaʻiga, o le faʻataʻitaʻiga o masini uma i totonu o lenei vaega e faʻapipiʻiina, ona faʻataʻitaʻiina lea o uiga o le vaega lava ia (mo se faʻataʻitaʻiga, le numera aupito maualalo ma le maualuga o masini galue). Afai e le ausia e le numera o loʻo iai nei taʻiala, o le vaega faʻataʻitaʻi lava ia o le a aveesea masini e le manaʻomia pe fatuina ni mea fou i lona lava ata.

I le avea ai o se vaega o la matou galuega, o le a matou fatuina se faʻataʻitaʻiga vaega o 'upega tafaʻilagi o le a gaosia mai le ata na faia muamua.

O le mea e matua'i mata'ina o le fa'ata'ita'iga mulimuli fa'atulagaina vaega. Ole vaega fa'atatau ile tu'ufa'atasia ma le Load Balancer ole a fesoasoani ia te oe e fa'atulaga se L3 paleni i luga o masini masini o lenei vaega e ala i le kilikiina o ni fa'amau.

I le setiina o le paleni, sa ou faʻatinoina ni mea taua se lua:

1. Na ou faia le paleni e talia le felauaiga o tagata i luga o le taulaga 80 ma toe faʻafeiloaʻi i le taulaga 3000 o masini masini, i le mea tonu e nofo ai Grafana.
2. Na ou setiina le siakiina o le gafatia o masini e ala i le pinging i latou i le port 3000.

Aotelega laiti

Mulimuli ane, na mafai ona matou faʻaogaina ma le lima le faʻaogaina o atinaʻe o loʻo manaʻomia, ma o lea ua matou maua se auaunaga sili ona malosi Grafana. E manaʻomia ona e iloa le tuatusi IP o le paleni e avea ma mea e ulufale ai i le talosaga ma le 'au o le ClickHouse cluster ina ia mafai ai ona uta le faʻamaumauga i totonu.

E foliga mai o se manumalo? Ioe, manumalo. Ae o loo iai lava se mea o loo fenumiai ai lava au. O le faagasologa atoa o loʻo i luga e manaʻomia ai le tele o laasaga tusi lesona ma e le mafai ona faʻaogaina; Ou te manaʻo e faʻautometi pe a mafai. O le mea lea o le a tuuto atu i le isi vaega.

Terraform tu'ufa'atasi

Matou te toe faʻaaogaina se meafaigaluega mai le HashiCorp ua taʻua Terraform. O le a fesoasoani ia te oe e faʻapipiʻi uma le faʻaogaina o mea faʻaoga i le kiliki o se ki, faʻavae i luga o le tele o fesuiaiga na pasia e le tagata faʻaoga. Se'i o tatou tusia se fua e mafai ona fa'atino fa'atele i vaega eseese o tagata fa'aoga eseese.

O galuega uma ma Terraform e sau i lalo i le tusiaina o se faila faila (*.tf) ma le fausiaina o atinaʻe e faʻavae i luga.

Fesuiaʻiga

I le amataga lava o le faila, o le a matou aofia ai fesuiaiga e fuafua ai poʻo fea ma pe faʻafefea ona faʻapipiʻiina atinaʻe i le lumanaʻi.

variable "oauth_token" {
    type = string
    default = "<oauth-token>"
}
variable "cloud_id" {
    type = string
    default = "<cloud-id>"
}
variable "folder_id" {
    type = string
    default = "<folder_id>"
}
variable "service_account_id" {
    type = string
    default = "<service_account_id>"
}
variable "image_id" {
    type = string
    default = "<image_id>"
}
variable "username" {
    type = string
    default = "<username>"
}
variable "password" {
    type = string
    default = "<password>"
}
variable "dbname" {
    type = string
    default = "<dbname>"
}
variable "public_key_path" {
    type = string
    default = "<path to ssh public key>"
}

O le faagasologa atoa o le faʻaogaina o talosaga o le a oʻo mai i le fausiaina o se ata tisiki ma faʻatulagaina nei fesuiaiga. Sei ou faamatala atu le mea latou te nafa ma:

oauth_token — o se faailoga mo le mauaina o le ao. E mafai ona maua e fesoʻotaʻiga.
cloud_id - faʻailoga ao o le a matou faʻaogaina ai le talosaga
folder_id - fa'asinomaga vaega o le a matou fa'aogaina ai le talosaga
service_account_id - faʻamatalaga faʻamatalaga auaunaga i le vaega tutusa o le ao.
image_id - fa'ailoaina o le ata tisiki na maua e fa'aaoga ai le Packer
username и numera e le iloa e sesi - igoa ole igoa ma upu fa'aulu e maua uma ai fa'amaumauga ma le upegatafa'ilagi a le Grafana
dbname - igoa faʻamaumauga i totonu ole CH ma MySQL fuifui
public_key_path - ala i le faila ma lau ssh key lautele, lea e mafai ona e faʻaogaina e faʻafesoʻotaʻi ai i lalo o le igoa ubuntu i masini mata'utia ma 'upega tafa'ilagi

Seti a le kamupani

Ole taimi nei e te manaʻomia le faʻatulagaina o le kamupani Terraform - i la matou tulaga, Yandex:

provider "yandex" {
  token     = var.oauth_token
  cloud_id  = var.cloud_id
  folder_id = var.folder_id
  zone      = "ru-central1-a"
}

O le ae matauina o iinei o loʻo matou faʻaogaina ai fesuiaiga o loʻo faʻamatalaina i luga.

Fesootaiga ma fuifui

O le taimi nei o le a matou fatuina se fesoʻotaʻiga e fesoʻotaʻi ai elemene oa tatou atinaʻe, tolu subnets (tasi i itulagi taʻitasi) ma faʻatupuina CH ma MySQL fuifui.

resource "yandex_vpc_network" "grafana_network" {}

resource "yandex_vpc_subnet" "subnet_a" {
  zone           = "ru-central1-a"
  network_id     = yandex_vpc_network.grafana_network.id
  v4_cidr_blocks = ["10.1.0.0/24"]
}

resource "yandex_vpc_subnet" "subnet_b" {
  zone           = "ru-central1-b"
  network_id     = yandex_vpc_network.grafana_network.id
  v4_cidr_blocks = ["10.2.0.0/24"]
}

resource "yandex_vpc_subnet" "subnet_c" {
  zone           = "ru-central1-c"
  network_id     = yandex_vpc_network.grafana_network.id
  v4_cidr_blocks = ["10.3.0.0/24"]
}

resource "yandex_mdb_clickhouse_cluster" "ch_cluster" {
  name        = "grafana-clickhouse"
  environment = "PRODUCTION"
  network_id  = yandex_vpc_network.grafana_network.id

  clickhouse {
    resources {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 16
    }
  }

  zookeeper {
    resources {
      resource_preset_id = "s2.micro"
      disk_type_id       = "network-ssd"
      disk_size          = 10
    }
  }

  database {
    name = var.dbname
  }

  user {
    name     = var.username
    password = var.password
    permission {
      database_name = var.dbname
    }
  }

  host {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-a"
    subnet_id = yandex_vpc_subnet.subnet_a.id
  }

  host {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-b"
    subnet_id = yandex_vpc_subnet.subnet_b.id
  }

  host {
    type      = "CLICKHOUSE"
    zone      = "ru-central1-c"
    subnet_id = yandex_vpc_subnet.subnet_c.id
  }

  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-a"
    subnet_id = yandex_vpc_subnet.subnet_a.id
  }

  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-b"
    subnet_id = yandex_vpc_subnet.subnet_b.id
  }

  host {
    type      = "ZOOKEEPER"
    zone      = "ru-central1-c"
    subnet_id = yandex_vpc_subnet.subnet_c.id
  }
}

resource "yandex_mdb_mysql_cluster" "mysql_cluster" {
  name        = "grafana_mysql"
  environment = "PRODUCTION"
  network_id  = yandex_vpc_network.grafana_network.id
  version     = "8.0"

  resources {
    resource_preset_id = "s2.micro"
    disk_type_id       = "network-ssd"
    disk_size          = 16
  }

  database {
    name = var.dbname
  }

  user {
    name     = var.username
    password = var.password
    permission {
      database_name = var.dbname
      roles         = ["ALL"]
    }
  }

  host {
    zone      = "ru-central1-a"
    subnet_id = yandex_vpc_subnet.subnet_a.id
  }
  host {
    zone      = "ru-central1-b"
    subnet_id = yandex_vpc_subnet.subnet_b.id
  }
  host {
    zone      = "ru-central1-c"
    subnet_id = yandex_vpc_subnet.subnet_c.id
  }
}

E pei ona e va'aia, o fuifui ta'itasi e lua ua faia e fai si fa'apalepale sese e ala i le tu'u i totonu o sone avanoa e tolu.

Web servers

E foliga mai e mafai ona ma faaauau pea i le agaga lava e tasi, ae na ou feagai ma faigata. Aʻo leʻi oʻo i lenei mea, na ou faʻatūina muamua se MySQL fuifui ma naʻo le maeʻa ai, i le iloaina o lona ID, na ou aoina ai se ata tisiketi ma le faʻatulagaina manaʻomia, lea na ou faʻamaonia ai le talimalo i le fuifui. Ae o lenei matou te le iloa le ID faʻapipiʻi aʻo leʻi faʻalauiloaina Terraform, e aofia ai ma le taimi o le fausiaina o le ata. O le mea lea, na tatau ai ona ou faʻaogaina mea nei togafiti.

I le faʻaaogaina o le auaunaga metadata a Amazon, o le a matou pasi atu nisi o tapulaʻa i le masini masini, lea o le a talia ma faʻagasolo. Matou te manaʻomia le masini e alu i metadata i tua o le MySQL cluster host ma le username-password, lea na faʻamaonia e le tagata faʻaoga i le faila Terraform, pe a uma ona amata. Se'i o tatou sui teisi mea o lo'o i totonu o le faila setup.sh, lea e tamo'e pe a fa'aola le masini masini.

setup.sh:

#!/bin/bash
CLUSTER_URI="$(curl -H 'Metadata-Flavor:Google' http://169.254.169.254/computeMetadata/v1/instance/attributes/mysql_cluster_uri)"
USERNAME="$(curl -H 'Metadata-Flavor:Google' http://169.254.169.254/computeMetadata/v1/instance/attributes/username)"
PASSWORD="$(curl -H 'Metadata-Flavor:Google' http://169.254.169.254/computeMetadata/v1/instance/attributes/password)"
sudo sed -i "s#.*;url =.*#url = mysql://${USERNAME}:${PASSWORD}@${CLUSTER_URI}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_user =.*#admin_user = ${USERNAME}#" /etc/grafana/grafana.ini
sudo sed -i "s#.*;admin_password =.*#admin_password = ${PASSWORD}#" /etc/grafana/grafana.ini
sudo service grafana-server restart

Vaega fa'amasani ma paleni

I le toe fausia o se ata fou tisiki, e mafai ona faʻaopoopoina le matou faila mo Terraform.

Sei o tatou faailoa atu tatou te mananao e faaaoga se ata tisiki oi ai nei:

data "yandex_compute_image" "grafana_image" {
  image_id = var.image_id
}

Se'i o tatou faia se vaega fa'ata'ita'i:

resource "yandex_compute_instance_group" "grafana_group" {
  name               = "grafana-group"
  folder_id          = var.folder_id
  service_account_id = var.service_account_id
  instance_template {
    platform_id = "standard-v1"
    resources {
      memory = 1
      cores  = 1
    }
    boot_disk {
      mode = "READ_WRITE"
      initialize_params {
        image_id = data.yandex_compute_image.grafana_image.id
        size     = 4
      }
    }
    network_interface {
      network_id = yandex_vpc_network.grafana_network.id
      subnet_ids = [yandex_vpc_subnet.subnet_a.id, yandex_vpc_subnet.subnet_b.id, yandex_vpc_subnet.subnet_c.id]
      nat = "true"
    }
    metadata = {
      mysql_cluster_uri = "c-${yandex_mdb_mysql_cluster.mysql_cluster.id}.rw.mdb.yandexcloud.net:3306/${var.dbname}"
      username = var.username
      password = var.password
      ssh-keys = "ubuntu:${file("${var.public_key_path}")}"
    }
    network_settings {
      type = "STANDARD"
    }
  }

  scale_policy {
    fixed_scale {
      size = 6
    }
  }

  allocation_policy {
    zones = ["ru-central1-a", "ru-central1-b", "ru-central1-c"]
  }

  deploy_policy {
    max_unavailable = 2
    max_creating    = 2
    max_expansion   = 2
    max_deleting    = 2
  }

  load_balancer {
    target_group_name = "grafana-target-group"
  }
}

E taua le faʻalogo i le auala na matou pasi atu ai i metadata cluster_uri, username и password. O mea ia o le a ave e le masini komepiuta i le amataga ma tuʻu i totonu o le Grafana config.

E pule lava le tagata paleni.

resource "yandex_lb_network_load_balancer" "grafana_balancer" {
  name = "grafana-balancer"

  listener {
    name = "grafana-listener"
    port = 80
    target_port = 3000
    external_address_spec {
      ip_version = "ipv4"
    }
  }

  attached_target_group {
    target_group_id = yandex_compute_instance_group.grafana_group.load_balancer.0.target_group_id

    healthcheck {
      name = "healthcheck"
      tcp_options {
        port = 3000
      }
    }
  }
}

O sina suka

E na o sina mea itiiti o totoe. A maeʻa ona faʻapipiʻiina le atinaʻe, e tatau ona e alu i le Grafana UI ma faʻaopoopo ma le lima le CH cluster (le ID e manaʻomia pea ona maua) o se Faʻamatalaga Faʻamatalaga. Ae e iloa e Terraform le ID kulupu. Tatou tuu atu ia te ia le faataunuuina o le mataupu.

Sei o tatou faʻaopoopoina se kamupani fou - Grafana, ma tuʻuina atu ia te ia le tuatusi IP o le paleni o se talimalo. O suiga uma e faia e Terraform i luga o le masini lea e fuafua ai lona paleni o le a tupu i MySQL, ma o lea i luga o isi masini uma.

provider "grafana" {
  url  = "http://${[for s in yandex_lb_network_load_balancer.grafana_balancer.listener: s.external_address_spec.0.address].0}"
  auth = "${var.username}:${var.password}"
}

resource "grafana_data_source" "ch_data_source" {
  type          = "vertamedia-clickhouse-datasource"
  name          = "grafana"
  url           = "https://c-${yandex_mdb_clickhouse_cluster.ch_cluster.id}.rw.mdb.yandexcloud.net:8443"
  basic_auth_enabled = "true"
  basic_auth_username = var.username
  basic_auth_password = var.password
  is_default = "true"
  access_mode = "proxy"
}

Tatou selu o tatou lauulu

Sei o tatou faʻaalia le tuatusi IP paleni ma le 'au o le ClickHouse cluster

output "grafana_balancer_ip_address" {
  value = [for s in yandex_lb_network_load_balancer.grafana_balancer.listener: s.external_address_spec.0.address].0
}

output "clickhouse_cluster_host" {
  value = "https://c-${yandex_mdb_clickhouse_cluster.ch_cluster.id}.rw.mdb.yandexcloud.net:8443"
}

E mafai ona tamoe

Uma! Ua saunia la matou faila faila ma e mafai, e ala i le setiina o fesuiaiga, taʻu atu ia Terraform e sii i luga mea uma na matou faamatalaina i luga. O le faagasologa atoa na ou alu ai pe a ma le 15 minute.
I le faaiuga e mafai ona e vaʻai i se savali matagofie:

Apply complete! Resources: 9 added, 0 changed, 0 destroyed.
Outputs:

clickhouse_cluster_host = https://c-c9q14ipa2ngadqsbp2iq.rw.mdb.yandexcloud.net:8443
grafana_balancer_ip_address = 130.193.50.25

Ma i totonu o le ao, o elemene o atinaʻe faʻaleleia o le a faʻaalia:

Sei o tatou faʻasalaga iuga

I le taimi nei, faʻaaoga Grafana e fai ma faʻataʻitaʻiga, e mafai e oe taʻitoʻatasi ona faʻapipiʻi tusi talosaga ma se fausaga faʻapipiʻi ao i luga o le Yandex.Cloud platform. Meafaigaluega fesoasoani mai HashiCorp pei o Packer ma Terraform e mafai ona fesoasoani ia te oe i lenei mea. Ou te faʻamoemoe e maua e se tasi lenei tusiga aoga :)

PS O loʻo i lalo o le a ou faʻapipiʻiina se fesoʻotaʻiga i le fale teu oloa lea e mafai ai ona e mauaina fua saunia mo Packer ma Terraform, vaega na ou saunia i lenei tusiga.

faleteuoloa

puna: www.habr.com