Kedaden nalika masalah kasebut, aku ora duwe pengalaman sing cukup kanggo ngembangake lan ngluncurake solusi iki. Banjur aku miwiti Googling.
Aku ora ngerti apa nyekel, nanging kanggo umpteenth wektu aku ngadhepi karo kasunyatan sing malah yen aku kabeh langkah dening langkah ing tutorial, nyiapake lingkungan padha penulis, banjur ora tau bisa. Aku ora ngerti apa masalah, nanging nalika aku ketemu iki maneh, Aku mutusaké sing aku bakal nulis tutorial dhewe nalika kabeh bisa metu. Siji sing mesthi bakal bisa.
Panuntun ing Internet
Iku mung mengkono sing Internet ora nandhang sangsara marga saka lack saka macem-macem Panuntun, tutorial, step-by-step lan liya-liyane. Kedaden aku ditugasi ngembangake solusi kanggo ngatur lan mbangun kluster PostgreSQL failover, syarat utama yaiku streaming replikasi saka server Master menyang kabeh replika lan nyedhiyakake cadangan kanthi otomatis yen ana server Master. gagal.
Ing tahap iki, tumpukan teknologi sing digunakake ditemtokake:
- PostgreSQL minangka DBMS
- minangka solusi clustering
- etcd minangka panyimpenan mbagekke kanggo Patroni
- HAproxy kanggo ngatur titik entri siji kanggo aplikasi nggunakake database
Instalasi
Kanggo perhatian sampeyan - mbangun kluster PostgreSQL sing kasedhiya kanthi nggunakake Patroni, etcd, HAProxy.
Kabeh operasi ditindakake ing mesin virtual kanthi Debian 10 OS diinstal.
lsp
Aku ora nyaranake nginstal etcd ing mesin padha ngendi patroni lan postgresql bakal dumunung, wiwit mbukak disk penting banget kanggo etcd. Nanging kanggo tujuan pendhidhikan, kita bakal nindakake iki.
Ayo nginstal etcd.
#!/bin/bash
apt-get update
apt-get install etcdTambah konten menyang file /etc/default/etcd
[anggota]
ETCD_NAME=datanode1 # jeneng host mesin sampeyan
ETCD_DATA_DIR=”/var/lib/etcd/default.etcd”
KABEH ALAMAT IP KUDU VALID. LISTER PEER, KLIEN, lsp kudu disetel menyang ALAMAT IP Host
ETCD_LISTEN_PEER_URLS="» # alamat mobil sampeyan
ETCD_LISTEN_CLIENT_URLS="» # alamat mobil sampeyan
[kluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="» # alamat mobil sampeyan
ETCD_INITIAL_CLUSTER=»datanode1=» # alamat kabeh mesin ing kluster etcd
ETCD_INITIAL_CLUSTER_STATE="anyar"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-1″
ETCD_ADVERTISE_CLIENT_URLS="» # alamat mobil sampeyan
Mbukak printah
systemctl restart etcdPostgreSQL 9.6 + patroni
Wangsulan: Bab ingkang sapisanan kudu dilakoni yaiku nyiyapake telung mesin virtual kanggo nginstal piranti lunak sing dibutuhake. Sawise nginstal mesin, yen sampeyan tindakake tutorial sandi, sampeyan bisa mbukak script prasaja iki sing bakal (meh) nindakake kabeh kanggo sampeyan. Mlaku minangka root.
Wigati dimangerteni manawa skrip nggunakake PostgreSQL versi 9.6, iki amarga syarat internal perusahaan kita. Solusi kasebut durung diuji ing versi PostgreSQL liyane.
#!/bin/bash
apt-get install gnupg -y
echo "deb http://apt.postgresql.org/pub/repos/apt/ buster-pgdg main" >> /etc/apt/sources.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update
apt-get install postgresql-9.6 python3-pip python3-dev libpq-dev -y
systemctl stop postgresql
pip3 install --upgrade pip
pip install psycopg2
pip install patroni[etcd]
echo "
[Unit]
Description=Runners to orchestrate a high-availability PostgreSQL
After=syslog.target network.target
[Service]
Type=simple
User=postgres
Group=postgres
ExecStart=/usr/local/bin/patroni /etc/patroni.yml
KillMode=process
TimeoutSec=30
Restart=no
[Install]
WantedBy=multi-user.targ
" > /etc/systemd/system/patroni.service
mkdir -p /data/patroni
chown postgres:postgres /data/patroni
chmod 700 /data/patroniпо
touch /etc/patroni.ymlSabanjure, ing file /etc/patroni.yml sing lagi wae digawe, sampeyan kudu nyelehake isi ing ngisor iki, mesthi ngganti alamat IP ing kabeh panggonan menyang alamat sing sampeyan gunakake.
Perhatikan komentar ing yaml iki. Ngganti alamat menyang dhewe ing saben mesin ing kluster.
/etc/patroni.yml
scope: pgsql # должно быть одинаковым на всех нодах
namespace: /cluster/ # должно быть одинаковым на всех нодах
name: postgres1 # должно быть разным на всех нодах
restapi:
listen: 192.168.0.143:8008 # адрес той ноды, в которой находится этот файл
connect_address: 192.168.0.143:8008 # адрес той ноды, в которой находится этот файл
etcd:
hosts: 192.168.0.143:2379,192.168.0.144:2379,192.168.0.145:2379 # перечислите здесь все ваши ноды, в случае если вы устанавливаете etcd на них же
# this section (bootstrap) will be written into Etcd:/<namespace>/<scope>/config after initializing new cluster
# and all other cluster members will use it as a `global configuration`
bootstrap:
dcs:
ttl: 100
loop_wait: 10
retry_timeout: 10
maximum_lag_on_failover: 1048576
postgresql:
use_pg_rewind: true
use_slots: true
parameters:
wal_level: replica
hot_standby: "on"
wal_keep_segments: 5120
max_wal_senders: 5
max_replication_slots: 5
checkpoint_timeout: 30
initdb:
- encoding: UTF8
- data-checksums
- locale: en_US.UTF8
# init pg_hba.conf должен содержать адреса ВСЕХ машин, используемых в кластере
pg_hba:
- host replication postgres ::1/128 md5
- host replication postgres 127.0.0.1/8 md5
- host replication postgres 192.168.0.143/24 md5
- host replication postgres 192.168.0.144/24 md5
- host replication postgres 192.168.0.145/24 md5
- host all all 0.0.0.0/0 md5
users:
admin:
password: admin
options:
- createrole
- createdb
postgresql:
listen: 192.168.0.143:5432 # адрес той ноды, в которой находится этот файл
connect_address: 192.168.0.143:5432 # адрес той ноды, в которой находится этот файл
data_dir: /data/patroni # эту директорию создаст скрипт, описанный выше и установит нужные права
bin_dir: /usr/lib/postgresql/9.6/bin # укажите путь до вашей директории с postgresql
pgpass: /tmp/pgpass
authentication:
replication:
username: postgres
password: postgres
superuser:
username: postgres
password: postgres
create_replica_methods:
basebackup:
checkpoint: 'fast'
parameters:
unix_socket_directories: '.'
tags:
nofailover: false
noloadbalance: false
clonefrom: false
nosync: falseSkrip kudu mbukak ing kabeh telung mesin kluster, lan konfigurasi ndhuwur uga kudu diselehake ing file /etc/patroni.yml ing kabeh mesin.
Sawise sampeyan wis rampung operasi iki ing kabeh mesin ing kluster, mbukak printah ing ngisor iki ing samubarang mau
systemctl start patroni
systemctl start postgresqlEnteni kira-kira 30 detik, banjur jalanake printah iki ing mesin sing isih ana ing kluster.
HAproxy
Kita nggunakake HAproxy apik kanggo nyedhiyani siji titik entri. Server master bakal tansah kasedhiya ing alamat mesin sing HAproxy disebarake.
Supaya ora nggawe mesin karo HAproxy minangka titik kegagalan, kita bakal mbukak ing wadhah Docker; ing mangsa ngarep bisa diluncurake menyang kluster K8 lan nggawe kluster failover luwih dipercaya.
Nggawe direktori ing ngendi sampeyan bisa nyimpen rong file - Dockerfile lan haproxy.cfg. Pindhah menyang.
file docker
FROM ubuntu:latest
RUN apt-get update
&& apt-get install -y haproxy rsyslog
&& rm -rf /var/lib/apt/lists/*
RUN mkdir /run/haproxy
COPY haproxy.cfg /etc/haproxy/haproxy.cfg
CMD haproxy -f /etc/haproxy/haproxy.cfg && tail -F /var/log/haproxy.logAti-ati, telung baris pungkasan file haproxy.cfg kudu nyathet alamat mesin sampeyan. HAproxy bakal ngubungi Patroni, ing header HTTP server master mesthi bakal ngasilake 200, lan replika bakal bali 503.
haproxy.cfg
global
maxconn 100
defaults
log global
mode tcp
retries 2
timeout client 30m
timeout connect 4s
timeout server 30m
timeout check 5s
listen stats
mode http
bind *:7000
stats enable
stats uri /
listen postgres
bind *:5000
option httpchk
http-check expect status 200
default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions
server postgresql1 192.168.0.143:5432 maxconn 100 check port 8008
server postgresql2 192.168.0.144:5432 maxconn 100 check port 8008
server postgresql3 192.168.0.145:5432 maxconn 100 check port 8008
Dadi ing direktori ing ngendi loro file kita "ngapusi", ayo nglakokake perintah kanggo ngemas wadhah kasebut kanthi urutan, uga diluncurake kanthi nerusake port sing dibutuhake:
docker build -t my-haproxy .
docker run -d -p5000:5000 -p7000:7000 my-haproxy Saiki, kanthi mbukak alamat mesin sampeyan nganggo HAproxy ing browser lan nemtokake port 7000, sampeyan bakal weruh statistik ing kluster sampeyan.
Server sing dadi master bakal ana ing negara UP, lan replika bakal ana ing negara DOWN. Iki normal, nyatane kerjane, nanging katon kaya ngono amarga padha bali 503 kanggo panjaluk saka HAproxy. Iki ngidini kita tansah ngerti persis endi saka telung server sing dadi master saiki.
kesimpulan
Koe ayu! Mung 30 menit sampeyan wis masang kluster database sing tahan kesalahan lan kinerja dhuwur kanthi replikasi streaming lan mundur otomatis. Yen sampeyan arep nggunakake solusi iki, priksa , lan utamane babagan utilitas patronictl, sing nyedhiyakake akses sing trep kanggo ngatur kluster sampeyan.
Sugeng rawuh!
Source: www.habr.com