ProHoster > Pūnaewele > Nā Administration > Ke kūkulu nei i kahi pūʻulu PostgreSQL i loaʻa loa me ka hoʻohana ʻana iā Patroni, etcd, HAProxy

Ke kūkulu nei i kahi pūʻulu PostgreSQL i loaʻa loa me ka hoʻohana ʻana iā Patroni, etcd, HAProxy

ʻO ia wale nō i ka manawa i hoʻopuka ʻia ai ka pilikia, ʻaʻole lawa kaʻu ʻike e hoʻomohala a hoʻomaka i kēia hoʻonā wale nō. A laila hoʻomaka wau iā Google.

ʻAʻole wau i ʻike i ke ʻano o ka hopu ʻana, akā no ka manawa ʻumi ʻumi ke kū nei au i ka ʻoiaʻiʻo inā hana wau i nā mea āpau i kēlā me kēia pae e like me ke aʻo ʻana, e hoʻomākaukau i kahi ʻano like me ka mea kākau, a laila ʻaʻohe mea e hana. ʻAʻohe oʻu manaʻo he aha ia mea, akā i koʻu hālāwai ʻana me kēia, ua hoʻoholo wau e kākau wau i kaʻu kumu aʻo ponoʻī ke holo nā mea āpau. ʻO kekahi e hana maoli.

Nā alakaʻi ma ka Pūnaewele

ʻAʻole pilikia ka Pūnaewele i ka nele o nā alakaʻi like ʻole, nā aʻoaʻo, nā ʻanuʻu a me nā mea like. ʻO ia wale nō ka mea i hana ʻia iaʻu e hoʻomohala i kahi hopena no ka hoʻonohonoho pono ʻana a me ke kūkulu ʻana i kahi puʻupuʻu failover PostgreSQL, nā koi nui no ka hoʻoheheʻe ʻana mai ka server Master i nā replicas a me ka hoʻolako ʻana i kahi mālama i ka wā o kahi kikowaena Master. hāʻule.

I kēia pae, ua hoʻoholo ʻia ka pūʻulu o nā ʻenehana i hoʻohana ʻia:

  • ʻO PostgreSQL ma ke ʻano he DBMS
  • Patroni ma ke ʻano he hāʻina huikau
  • etcd ma ke ʻano he waihona puʻupuʻu no Patroni
  • HAproxy no ka hoʻonohonoho ʻana i kahi helu komo hoʻokahi no nā noi e hoʻohana ana i ka waihona

Kāu Mau Koho Paʻamau

No kou nānā - kūkulu ʻana i kahi puʻupuʻu PostgreSQL i loaʻa me ka hoʻohana ʻana iā Patroni, etcd, HAProxy.

Ua hana ʻia nā hana āpau ma nā mīkini virtual me Debian 10 OS i kau ʻia.

a pēlā aku

ʻAʻole wau e manaʻo e hoʻokomo i ka etcd ma nā mīkini like kahi e loaʻa ai ka patroni a me ka postgresql, no ka mea he mea nui ka hoʻouka disk no etcd. Akā no nā kumu hoʻonaʻauao, e hana mākou i kēlā.
E hoʻouka etcd.

#!/bin/bash
apt-get update
apt-get install etcd

E hoʻohui i ka ʻike i /etc/default/etcd file

[lālā]

ETCD_NAME=datanode1 # inoa inoa o kāu mīkini
ETCD_DATA_DIR = "/var/lib/etcd/default.etcd"

PONO KA PALAPALA IP A PAU. LISTER PEER, CLIENT etc Pono e hoʻonoho i IP ADDRESS O HOST

ETCD_LISTEN_PEER_URLS="http://192.168.0.143:2380» # helu wahi o kāu kaʻa
ETCD_LISTEN_CLIENT_URLS="http://192.168.0.143:2379,http://127.0.0.1:2379» # helu wahi o kāu kaʻa

[hui]

ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.0.143:2380» # helu wahi o kāu kaʻa
ETCD_INITIAL_CLUSTER=»datanode1=http://192.168.0.143:2380,datanode2=http://192.168.0.144:2380,datanode3=http://192.168.0.145:2380» # helu wahi o nā mīkini a pau i ka hui etcd
ETCD_INITIAL_CLUSTER_STATE="hou"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-1″
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.0.143:2379» # helu wahi o kāu kaʻa

Holo i ke kauoha

systemctl restart etcd

ʻO PostgreSQL 9.6 + patroni

ʻO ka mea mua āu e hana ai, ʻo ia ka hoʻonohonoho ʻana i ʻekolu mīkini virtual e hoʻokomo i ka polokalamu pono ma luna o lākou. Ma hope o ka hoʻokomo ʻana i nā mīkini, inā ʻoe e hahai i kaʻu kumu aʻo, hiki iā ʻoe ke holo i kēia palapala maʻalahi e (kokoke) e hana i nā mea āpau no ʻoe. Holo e like me ke kumu.

E ʻoluʻolu e hoʻohana ka palapala i ka PostgreSQL version 9.6, ma muli o nā koi kūloko o kā mākou hui. ʻAʻole i hoʻāʻo ʻia ka hopena ma nā mana ʻē aʻe o PostgreSQL.

#!/bin/bash
apt-get install gnupg -y
echo "deb http://apt.postgresql.org/pub/repos/apt/ buster-pgdg main" >> /etc/apt/sources.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update
apt-get install postgresql-9.6 python3-pip python3-dev libpq-dev -y
systemctl stop postgresql
pip3 install --upgrade pip
pip install psycopg2
pip install patroni[etcd]
echo "
[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL
After=syslog.target network.target

[Service]
Type=simple

User=postgres
Group=postgres

ExecStart=/usr/local/bin/patroni /etc/patroni.yml

KillMode=process

TimeoutSec=30

Restart=no

[Install]
WantedBy=multi-user.targ
" > /etc/systemd/system/patroni.service
mkdir -p /data/patroni
chown postgres:postgres /data/patroni
chmod 700 /data/patroniпо
touch /etc/patroni.yml

A laila, i loko o ka faila /etc/patroni.yml āu i hana ai, pono ʻoe e kau i kēia mau ʻike, ʻoiaʻiʻo e hoʻololi i nā IP address ma nā wahi āpau i nā helu āu e hoʻohana ai.
E nānā i nā manaʻo ma kēia yaml. E hoʻololi i nā helu wahi i kāu iho ma kēlā me kēia mīkini i ka pūʻulu.

/etc/patroni.yml

scope: pgsql # должно быть одинаковым на всех нодах
namespace: /cluster/ # должно быть одинаковым на всех нодах
name: postgres1 # должно быть разным на всех нодах

restapi:
    listen: 192.168.0.143:8008 # адрес той ноды, в которой находится этот файл
    connect_address: 192.168.0.143:8008 # адрес той ноды, в которой находится этот файл

etcd:
    hosts: 192.168.0.143:2379,192.168.0.144:2379,192.168.0.145:2379 # перечислите здесь все ваши ноды, в случае если вы устанавливаете etcd на них же

# this section (bootstrap) will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
# and all other cluster members will use it as a `global configuration`
bootstrap:
    dcs:
        ttl: 100
        loop_wait: 10
        retry_timeout: 10
        maximum_lag_on_failover: 1048576
        postgresql:
            use_pg_rewind: true
            use_slots: true
            parameters:
                    wal_level: replica
                    hot_standby: "on"
                    wal_keep_segments: 5120
                    max_wal_senders: 5
                    max_replication_slots: 5
                    checkpoint_timeout: 30

    initdb:
    - encoding: UTF8
    - data-checksums
    - locale: en_US.UTF8
    # init pg_hba.conf должен содержать адреса ВСЕХ машин, используемых в кластере
    pg_hba:
    - host replication postgres ::1/128 md5
    - host replication postgres 127.0.0.1/8 md5
    - host replication postgres 192.168.0.143/24 md5
    - host replication postgres 192.168.0.144/24 md5
    - host replication postgres 192.168.0.145/24 md5
    - host all all 0.0.0.0/0 md5

    users:
        admin:
            password: admin
            options:
                - createrole
                - createdb

postgresql:
    listen: 192.168.0.143:5432 # адрес той ноды, в которой находится этот файл
    connect_address: 192.168.0.143:5432 # адрес той ноды, в которой находится этот файл
    data_dir: /data/patroni # эту директорию создаст скрипт, описанный выше и установит нужные права
    bin_dir:  /usr/lib/postgresql/9.6/bin # укажите путь до вашей директории с postgresql
    pgpass: /tmp/pgpass
    authentication:
        replication:
            username: postgres
            password: postgres
        superuser:
            username: postgres
            password: postgres
    create_replica_methods:
        basebackup:
            checkpoint: 'fast'
    parameters:
        unix_socket_directories: '.'

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false

Pono e holo ka palapala ma nā mīkini ʻekolu o ka pūʻulu, a pono e hoʻokomo ʻia ka hoʻonohonoho i luna ma ka faila /etc/patroni.yml ma nā mīkini āpau.

Ke hoʻopau ʻoe i kēia mau hana ma nā mīkini āpau i ka hui, e holo i kēia kauoha ma luna o kekahi o lākou

systemctl start patroni
systemctl start postgresql

E kali ma kahi o 30 kekona, a laila e holo i kēia kauoha ma nā mīkini i koe o ka hui.

HAproxy

Hoʻohana mākou i ka HAproxy maikaʻi e hāʻawi i kahi helu komo. E loaʻa mau ka haku kikowaena ma ka helu wahi o ka mīkini kahi i kau ʻia ai ʻo HAproxy.

I ʻole e hoʻolilo i ka mīkini me HAproxy i hoʻokahi wahi o ka hāʻule ʻole, e hoʻokuʻu mākou iā ia i loko o kahi pahu Docker; i ka wā e hiki mai ana hiki ke hoʻokuʻu ʻia i loko o ka pūʻulu K8 a hoʻolilo i kā mākou failover cluster i ʻoi aku ka hilinaʻi.

E hana i papa kuhikuhi kahi e mālama ai ʻoe i ʻelua faila - Dockerfile a me haproxy.cfg. E hele i laila.

dockerfile

FROM ubuntu:latest

RUN apt-get update 
    && apt-get install -y haproxy rsyslog 
    && rm -rf /var/lib/apt/lists/*

RUN mkdir /run/haproxy

COPY haproxy.cfg /etc/haproxy/haproxy.cfg

CMD haproxy -f /etc/haproxy/haproxy.cfg && tail -F /var/log/haproxy.log

E akahele, ʻo nā laina hope ʻekolu o ka faila haproxy.cfg e helu i nā helu wahi o kāu mau mīkini. E hoʻokaʻaʻike ʻo HAproxy iā Pattroni, ma nā poʻomanaʻo HTTP e hoʻihoʻi mau ka server master i 200, a e hoʻihoʻi mau ka replica i 503.

haproxy.cfg

global
    maxconn 100

defaults
    log global
    mode tcp
    retries 2
    timeout client 30m
    timeout connect 4s
    timeout server 30m
    timeout check 5s

listen stats
    mode http
    bind *:7000
    stats enable
    stats uri /

listen postgres
    bind *:5000
    option httpchk
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server postgresql1 192.168.0.143:5432 maxconn 100 check port 8008
    server postgresql2 192.168.0.144:5432 maxconn 100 check port 8008
    server postgresql3 192.168.0.145:5432 maxconn 100 check port 8008

Aia i loko o ka papa kuhikuhi kahi o kā mākou mau faila ʻelua e "wahaheʻe," e hoʻokō pono i nā kauoha no ka hoʻopili ʻana i ka ipu, a me ka hoʻokuʻu ʻana me ka hoʻouna ʻana i nā awa kūpono:

docker build -t my-haproxy .
docker run -d -p5000:5000 -p7000:7000 my-haproxy

I kēia manawa, ma ka wehe ʻana i ka helu wahi o kāu mīkini me HAproxy i ka polokalamu kele pūnaewele a me ka wehewehe ʻana i ke awa 7000, e ʻike ʻoe i nā helu ma kāu hui.

Aia ke kikowaena ʻo ia ka haku ma ka mokuʻāina ʻo UP, a aia nā kope i ka mokuʻāina DOWN. He mea maʻamau kēia, ʻoiaʻiʻo, hana lākou, akā ʻike ʻia lākou ma kēia ʻano no ka mea hoʻihoʻi lākou i 503 no nā noi mai HAproxy. ʻAe kēia iā mākou e ʻike mau i ka mea o nā kikowaena ʻekolu ka haku o kēia manawa.

hopena

Nani ʻoe! I loko o nā minuke 30 wale nō, ua hoʻonoho ʻoe i kahi pūʻulu waihona waihona hoʻomanawanui maikaʻi loa me ka hoʻopiʻi kahe ʻana a me ka hāʻule ʻakomi. Inā hoʻolālā ʻoe e hoʻohana i kēia hoʻonā, e nānā me nā palapala patroni kūhelu, a me kāna ʻāpana e pili ana i ka pono patronictl, e hāʻawi ana i ke ala kūpono i ka hoʻokele ʻana i kāu hui.

Hoʻomaikaʻi!

Source: www.habr.com

Pākuʻi i ka manaʻo hoʻopuka