ProHoster > Блог > Kev tswj hwm > Tsim kom muaj PostgreSQL pawg siv Patroni, thiab lwm yam, HAProxy

Tsim kom muaj PostgreSQL pawg siv Patroni, thiab lwm yam, HAProxy

Nws tsuas yog tshwm sim thaum lub sijhawm teeb meem tshwm sim, kuv tsis muaj kev paub txaus los tsim thiab tso tawm qhov kev daws teeb meem no ib leeg. Thiab ces kuv pib Googling.

Kuv tsis paub tias qhov ntes yog dab tsi, tab sis rau lub sijhawm umpteenth kuv tau ntsib nrog qhov tseeb tias txawm tias kuv ua txhua kauj ruam los ntawm kauj ruam raws li hauv kev qhia, npaj ib puag ncig zoo li tus sau, ces tsis muaj dab tsi ua haujlwm. Kuv tsis paub dab tsi yog qhov teeb meem, tab sis thaum kuv ntsib qhov no dua, kuv txiav txim siab tias kuv yuav sau kuv tus kheej cov lus qhia thaum txhua yam ua haujlwm. Ib qho uas yuav twv yuav raug hu ua haujlwm.

Cov lus qhia hauv Internet

Nws tsuas yog tshwm sim tias Internet tsis raug kev txom nyem los ntawm qhov tsis muaj ntau yam kev qhia, kev qhia, ib kauj ruam dhau los thiab lwm yam. Nws tsuas yog tshwm sim uas kuv tau ua haujlwm los tsim cov kev daws teeb meem rau kev teeb tsa yooj yim thiab txhim kho qhov ua tsis tiav PostgreSQL pawg, cov kev cai tseem ceeb uas tau tshaj tawm cov ntawv luam tawm los ntawm Master server rau txhua qhov kev hloov pauv thiab tsis siv neeg muab kev khaws cia hauv qhov xwm txheej ntawm Master server. ua tsis tiav.

Nyob rau theem no, pawg ntawm technologies siv tau txiav txim siab:

  • PostgreSQL ua DBMS
  • Patroni raws li kev daws teeb meem
  • etcd raws li faib cia rau Patroni
  • HAproxy rau kev teeb tsa ib qho chaw nkag rau cov ntawv thov siv cov ntaub ntawv

chaw

Rau koj mloog - tsim kom muaj PostgreSQL pawg uas siv Patroni, thiab lwm yam, HAProxy.

Txhua qhov haujlwm tau ua tiav ntawm cov tshuab virtual nrog Debian 10 OS ntsia.

lwm yam

Kuv tsis pom zoo kom txhim kho etcd ntawm tib lub tshuab uas patroni thiab postgresql yuav nyob, vim tias disk load tseem ceeb heev rau lwm yam. Tab sis rau lub hom phiaj kev kawm, peb yuav ua li ntawd xwb.
Cia peb nruab thiab lwm yam.

#!/bin/bash
apt-get update
apt-get install etcd

Ntxiv cov ntsiab lus rau /etc/default/etcd file

[tus tswv cuab]

ETCD_NAME=datanode1 # hostname ntawm koj lub tshuab
ETCD_DATA_DIR = "/var/lib/etcd/default.etcd"

Txhua tus IP chaw nyob yuav tsum muaj cai. LISTER PEER, Client thiab lwm yam yuav tsum tau teeb tsa rau IP chaw nyob ntawm tus tswv tsev

ETCD_LISTEN_PEER_URLS="http://192.168.0.143:2380» # chaw nyob ntawm koj lub tsheb
ETCD_LISTEN_CLIENT_URLS="http://192.168.0.143:2379,http://127.0.0.1:2379» # chaw nyob ntawm koj lub tsheb

[cluster]

ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.0.143:2380» # chaw nyob ntawm koj lub tsheb
ETCD_INITIAL_CLUSTER=»datanode1=http://192.168.0.143:2380,datanode2=http://192.168.0.144:2380,datanode3=http://192.168.0.145:2380» # chaw nyob ntawm txhua lub tshuab hauv pawg thiab lwm yam
ETCD_INITIAL_CLUSTER_STATE = "tshiab"
ETCD_INITIAL_CLUSTER_TOKEN = "etcd-cluster-1"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.0.143:2379» # chaw nyob ntawm koj lub tsheb

Khiav cov lus txib

systemctl restart etcd

PostgreSQL 9.6 + txhawb nqa

Thawj qhov koj yuav tsum tau ua yog teeb tsa peb lub tshuab virtual los nruab cov software tsim nyog rau lawv. Tom qab txhim kho cov tshuab, yog tias koj ua raws li kuv cov lus qhia, koj tuaj yeem khiav cov ntawv yooj yim no uas yuav (yuav luag) ua txhua yam rau koj. Khiav li hauv paus.

Thov nco ntsoov tias tsab ntawv siv PostgreSQL version 9.6, qhov no yog vim qhov xav tau sab hauv ntawm peb lub tuam txhab. Cov kev daws teeb meem tsis tau raug sim rau lwm cov versions ntawm PostgreSQL.

#!/bin/bash
apt-get install gnupg -y
echo "deb http://apt.postgresql.org/pub/repos/apt/ buster-pgdg main" >> /etc/apt/sources.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update
apt-get install postgresql-9.6 python3-pip python3-dev libpq-dev -y
systemctl stop postgresql
pip3 install --upgrade pip
pip install psycopg2
pip install patroni[etcd]
echo "
[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL
After=syslog.target network.target

[Service]
Type=simple

User=postgres
Group=postgres

ExecStart=/usr/local/bin/patroni /etc/patroni.yml

KillMode=process

TimeoutSec=30

Restart=no

[Install]
WantedBy=multi-user.targ
" > /etc/systemd/system/patroni.service
mkdir -p /data/patroni
chown postgres:postgres /data/patroni
chmod 700 /data/patroniпо
touch /etc/patroni.yml

Tom ntej no, hauv /etc/patroni.yml cov ntaub ntawv koj nyuam qhuav tsim, koj yuav tsum tso cov ntsiab lus hauv qab no, tau kawg hloov chaw nyob IP hauv txhua qhov chaw mus rau qhov chaw nyob uas koj siv.
Ua tib zoo mloog cov lus hauv no yaml. Hloov chaw nyob rau koj tus kheej ntawm txhua lub tshuab hauv pawg.

/etc/patroni.yml

scope: pgsql # должно быть одинаковым на всех нодах
namespace: /cluster/ # должно быть одинаковым на всех нодах
name: postgres1 # должно быть разным на всех нодах

restapi:
    listen: 192.168.0.143:8008 # адрес той ноды, в которой находится этот файл
    connect_address: 192.168.0.143:8008 # адрес той ноды, в которой находится этот файл

etcd:
    hosts: 192.168.0.143:2379,192.168.0.144:2379,192.168.0.145:2379 # перечислите здесь все ваши ноды, в случае если вы устанавливаете etcd на них же

# this section (bootstrap) will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
# and all other cluster members will use it as a `global configuration`
bootstrap:
    dcs:
        ttl: 100
        loop_wait: 10
        retry_timeout: 10
        maximum_lag_on_failover: 1048576
        postgresql:
            use_pg_rewind: true
            use_slots: true
            parameters:
                    wal_level: replica
                    hot_standby: "on"
                    wal_keep_segments: 5120
                    max_wal_senders: 5
                    max_replication_slots: 5
                    checkpoint_timeout: 30

    initdb:
    - encoding: UTF8
    - data-checksums
    - locale: en_US.UTF8
    # init pg_hba.conf должен содержать адреса ВСЕХ машин, используемых в кластере
    pg_hba:
    - host replication postgres ::1/128 md5
    - host replication postgres 127.0.0.1/8 md5
    - host replication postgres 192.168.0.143/24 md5
    - host replication postgres 192.168.0.144/24 md5
    - host replication postgres 192.168.0.145/24 md5
    - host all all 0.0.0.0/0 md5

    users:
        admin:
            password: admin
            options:
                - createrole
                - createdb

postgresql:
    listen: 192.168.0.143:5432 # адрес той ноды, в которой находится этот файл
    connect_address: 192.168.0.143:5432 # адрес той ноды, в которой находится этот файл
    data_dir: /data/patroni # эту директорию создаст скрипт, описанный выше и установит нужные права
    bin_dir:  /usr/lib/postgresql/9.6/bin # укажите путь до вашей директории с postgresql
    pgpass: /tmp/pgpass
    authentication:
        replication:
            username: postgres
            password: postgres
        superuser:
            username: postgres
            password: postgres
    create_replica_methods:
        basebackup:
            checkpoint: 'fast'
    parameters:
        unix_socket_directories: '.'

tags:
    nofailover: false
    noloadbalance: false
    clonefrom: false
    nosync: false

Tsab ntawv yuav tsum tau khiav ntawm tag nrho peb lub tshuab ntawm pawg, thiab cov kev teeb tsa saum toj no yuav tsum tau muab tso rau hauv cov ntaub ntawv /etc/patroni.yml ntawm txhua lub tshuab.

Thaum koj ua tiav cov haujlwm no ntawm txhua lub tshuab hauv pawg, khiav cov lus txib hauv qab no rau ib qho ntawm lawv

systemctl start patroni
systemctl start postgresql

Tos li 30 vib nas this, ces khiav cov lus txib no ntawm cov tshuab seem hauv pawg.

HAproxy

Peb siv HAproxy zoo heev los muab ib qho chaw nkag. Tus tswv server yuav ib txwm muaj nyob ntawm qhov chaw nyob ntawm lub tshuab uas HAproxy raug xa mus.

Txhawm rau kom tsis txhob ua kom lub tshuab nrog HAproxy ib qho kev ua tsis tiav, peb yuav tso nws rau hauv lub thawv Docker; yav tom ntej nws tuaj yeem tso rau hauv K8 pawg thiab ua rau peb pawg tsis ua haujlwm ntau dua.

Tsim ib daim ntawv teev npe uas koj tuaj yeem khaws ob cov ntaub ntawv - Dockerfile thiab haproxy.cfg. Mus rau nws.

Dockerfile

FROM ubuntu:latest

RUN apt-get update 
    && apt-get install -y haproxy rsyslog 
    && rm -rf /var/lib/apt/lists/*

RUN mkdir /run/haproxy

COPY haproxy.cfg /etc/haproxy/haproxy.cfg

CMD haproxy -f /etc/haproxy/haproxy.cfg && tail -F /var/log/haproxy.log

Ceev faj, peb kab kawg ntawm cov ntaub ntawv haproxy.cfg yuav tsum sau cov chaw nyob ntawm koj lub tshuab. HAproxy yuav hu rau Patroni, nyob rau hauv HTTP headers tus tswv server yuav ib txwm rov qab 200, thiab cov replica yeej ib txwm rov 503.

haproxy.cfg ua

global
    maxconn 100

defaults
    log global
    mode tcp
    retries 2
    timeout client 30m
    timeout connect 4s
    timeout server 30m
    timeout check 5s

listen stats
    mode http
    bind *:7000
    stats enable
    stats uri /

listen postgres
    bind *:5000
    option httpchk
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
    server postgresql1 192.168.0.143:5432 maxconn 100 check port 8008
    server postgresql2 192.168.0.144:5432 maxconn 100 check port 8008
    server postgresql3 192.168.0.145:5432 maxconn 100 check port 8008

Nyob rau hauv phau ntawv teev npe nyob rau hauv uas ob peb cov ntaub ntawv "dej," cia peb ua raws li cov lus txib rau ntim lub thawv, nrog rau kev xa tawm nrog xa mus rau cov chaw nres nkoj tsim nyog:

docker build -t my-haproxy .
docker run -d -p5000:5000 -p7000:7000 my-haproxy

Tam sim no, los ntawm kev qhib qhov chaw nyob ntawm koj lub tshuab nrog HAproxy hauv qhov browser thiab qhia qhov chaw nres nkoj 7000, koj yuav pom cov txheeb cais ntawm koj pawg.

Tus neeg rau zaub mov uas yog tus tswv yuav nyob rau hauv lub xeev UP, thiab cov replicas yuav nyob rau hauv lub xeev DOWN. Qhov no yog qhov qub, qhov tseeb lawv ua haujlwm, tab sis lawv tshwm sim li no vim tias lawv rov qab 503 rau kev thov los ntawm HAproxy. Qhov no tso cai rau peb ib txwm paub qhov tseeb ntawm peb lub servers yog tus tswv tam sim no.

xaus

Koj zoo nkauj! Hauv 30 feeb xwb koj tau siv qhov kev ua txhaum zoo-hloov siab thiab ua haujlwm siab ntawm pawg database nrog streaming replication thiab tsis siv neeg poob. Yog tias koj tab tom npaj siv qhov kev daws teeb meem no, kos tawm nrog cov ntaub ntawv pov thawj Patroni, thiab tshwj xeeb tshaj yog nrog nws feem txog kev siv hluav taws xob patronictl, uas muab kev yooj yim rau kev tswj hwm koj pawg.

Nrog koj zoo siab!

Tau qhov twg los: www.hab.com

Ntxiv ib saib