I kēia lā, koi ʻia ka loaʻa kiʻekiʻe o nā lawelawe i nā manawa a pau, ʻaʻole wale i nā papahana kumukūʻai nui. Loaʻa ʻole ʻia nā pūnaewele me ka memo "E kala mai, ke hoʻomau nei ka mālama ʻana" e ʻike mau ʻia ana, akā maʻa mau ka ʻakaʻaka hoʻohaʻahaʻa. E hoʻohui i ke ola i loko o nā ao i kēia, i ka wā hoʻokahi wale nō kelepona i ka API e pono ai e hoʻomaka i kahi kikowaena hou, a ʻaʻohe pono e noʻonoʻo e pili ana i ka hana "hao". A ʻaʻohe kumu kumu i hana ʻole ʻia ai kahi ʻōnaehana koʻikoʻi me ka hilinaʻi me ka hoʻohana ʻana i nā ʻenehana cluster a me ka redundancy.
E haʻi mākou iā ʻoe i nā hopena a mākou i noʻonoʻo ai e hōʻoia i ka hilinaʻi o nā ʻikepili i kā mākou lawelawe a me nā mea a mākou i hana ai. Hoʻohui i kahi demo me nā hopena lōʻihi.
Hoʻoilina ma ka Hoʻolālā Loaʻa Kiʻekiʻe
ʻOi aku ka maikaʻi o kēia ma ke ʻano o ka hoʻomohala ʻana i nā ʻōnaehana open source. Pono nā ʻenehana hoʻoilina e hoʻohui i nā ʻenehana loaʻa kiʻekiʻe e like me ka piʻi ʻana o ka noi. A ʻokoʻa ko lākou ʻano. Hāʻawi nā hāʻina o nā hanauna e hiki mai ana i ka loaʻa kiʻekiʻe ma ke kumu o kā lākou hoʻolālā. No ka laʻana, ʻo ka hui ʻana o nā kūlana MongoDB ma kāna hihia hoʻohana nui. Hoʻopaʻa ʻia ka pūʻulu ma ka ʻaoʻao, ʻo ia ka lanakila hoʻokūkū ikaika o kēia DBMS.
E hoʻi kāua i PostgreSQL. ʻO kēia kekahi o nā papahana opensource kaulana loa, ʻo ka hoʻokuʻu mua ʻana i hana ʻia i ka makahiki 95th o ke kenekulia i hala. No ka manawa lōʻihi, ʻaʻole i noʻonoʻo ka hui papahana i ka loaʻa kiʻekiʻe he pilikia e pono e hoʻoponopono ʻia e ka ʻōnaehana. No laila, ua hoʻokumu ʻia ka ʻenehana replication no ka hana ʻana i nā kope o ka ʻikepili i loko wale nō i ka mana 8.2 i ka makahiki 2006, akā ua hoʻokumu ʻia i ka faila (log shipping). I ka makahiki 2010, ua hoʻokomo ka mana 9.0 i ka hoʻopiʻi kahe ʻana, a ʻo ia ke kumu no ka hana ʻana i nā ʻano hui like ʻole. ʻO kēia, ʻoiaʻiʻo, he mea kāhāhā loa i ka poʻe e kamaʻāina me PostgreSQL ma hope o Enterprise SQL a i ʻole NoSQL hou - ʻo ka hoʻonā maʻamau mai ke kaiāulu ʻo ia wale nō ka master-replica pair me ka synchronous a asynchronous replication. I ka manawa like, ma ka waihona, hana lima ʻia ka hoʻololi ʻana o ka haku, a ua manaʻo ʻia hoʻi ka pilikia o ka hoʻololi ʻana i nā mea kūʻai aku e hoʻoholo kūʻokoʻa.
Pehea mākou i hoʻoholo ai e hana i ka PostgreSQL hilinaʻi a me ka mea a mākou i koho ai no kēia
Eia nō naʻe, ʻaʻole i kaulana loa ʻo PostgreSQL inā ʻaʻole no ka nui o nā papahana a me nā mea hana e kōkua ai i ke kūkulu ʻana i kahi hoʻonā hewa ʻole ʻaʻole pono ka nānā mau. I ke ao (MCS) mai ka hoʻomaka ʻana o DBaaS, ua loaʻa nā kikowaena PostgreSQL hoʻokahi a me nā hui master-replica me ka replication asynchronous.
Ma keʻano maʻamau, makemake mākou e hoʻomaʻamaʻa i ke ola no kēlā me kēia kanaka a hoʻolako i kahi hoʻonohonoho PostgreSQL i hiki ke lilo i kumu no nā lawelawe i loaʻa ʻole e nānā mau a ala i ka pō e hana i kahi hoʻololi. Ma kēia ʻāpana, aia nā hoʻonā kahiko i hōʻoia ʻia a me kahi hanauna o nā pono hana hou e hoʻohana ana i nā mea hou loa.
I kēia lā, ʻaʻole i kau ʻia ka pilikia o ka loaʻa kiʻekiʻe ma ka hoʻopaʻa ʻana (ʻo kēia me ka ʻole o ka ʻōlelo ʻana), akā ma ka ʻae ʻana - ka algorithm no ke koho ʻana i kahi alakaʻi (koho alakaʻi). ʻO ka pinepine, ʻaʻole i loaʻa nā ulia pōpilikia ma muli o ka nele o nā kikowaena, akā no nā pilikia me ka ʻae ʻana: ʻaʻole i puka kahi alakaʻi hou, ʻelua mau alakaʻi i ʻike ʻia i nā kikowaena data like ʻole, etc. ʻO kahi hiʻohiʻona kahi hāʻule ma kahi pūʻulu Github MySQL - ua kākau lākou .
He koʻikoʻi ke kumu makemakika ma kēia mea. Ma kekahi ʻaoʻao, aia , ka mea e kau ana i nā kaohi o ka manaʻo i ka hiki ke kūkulu ʻana i nā haʻina HA, ma ka ʻaoʻao ʻē aʻe, nā algorithms i hōʻoia ʻia ma ka makemakika no ka hoʻoholo ʻana i ka ʻaelike, e like me и . Ma kēia kumu, aia nā DCS kaulana loa (decentralized consensus system) - Zookeeper, etcd, Consul. No laila, inā hana ka ʻōnaehana hoʻoholo i kekahi o kāna mau algorithms, i kākau kūʻokoʻa, pono e makaʻala loa kekahi. Ma hope o ka nānā ʻana i ka nui o nā ʻōnaehana, ua noho mākou ma Patroni, kahi ʻōnaehana open source i hoʻomohala nui ʻia e Zalando.
Ma ke ʻano he digression lyrical, e ʻōlelo wau ua noʻonoʻo mākou i nā hoʻonā multi-master, ʻo ia hoʻi, nā puʻupuʻu i hiki ke hoʻonui ʻia no ka hoʻopaʻa ʻana. Eia naʻe, no nā kumu nui ʻelua, ua hoʻoholo mākou ʻaʻole e hana i kēlā ʻano hui. ʻO ka mea mua, ʻoi aku ka paʻakikī o ia mau hoʻonā a, no laila, ʻoi aku ka nāwaliwali. E paʻakikī ke hana i kahi hoʻonā paʻa no nā hihia āpau. ʻO ka lua, i kēia hihia, ua pau ka PostgreSQL i ka maʻemaʻe (maʻemaʻe), ʻaʻole loaʻa kekahi mau hana, ʻike paha kekahi mau noi i nā pōpoki huna i ka wā o ka hana.
Patroni
No laila pehea e hana ai ʻo Patroni? ʻAʻole i hana hou nā mea hoʻomohala i ka huila a manaʻo e hoʻohana i kekahi o nā hoʻonā DCS i hōʻoia ʻia i kumu. Ma ke aloha o ia e hāʻawi ʻia i nā pilikia āpau me ka synchronization o nā hoʻonohonoho, ke koho ʻana i kahi alakaʻi a me kahi kōrum. Ua koho mākou etcd no kēia.
A laila, pili ʻo Patroni i ka noi kūpono o nā hoʻonohonoho PostgreSQL āpau a me nā hoʻonohonoho replication, a me ka hoʻokō ʻana i nā kauoha no ka hoʻololi a me ka failover (ʻo ia hoʻi, ka hoʻololi maʻamau a me ka hoʻololi ʻole o ka haku). Ma ke ʻano kikoʻī, ma ke ao MCS, hiki iā ʻoe ke hana i kahi pūʻulu mai kahi haku, kahi hoʻopiʻi synchronous, a hoʻokahi a ʻoi aʻe paha nā kope asynchronous. ʻO ka loaʻa ʻana o kahi replica synchronous e hōʻoia i ka palekana o ka ʻikepili ma ka liʻiliʻi o 2 mau kikowaena, a ʻo kēia replica ka mea nui "moho haku".
No ka mea ua kau ʻia ʻo etcd ma nā kikowaena like, 3 a i ʻole 5 mau kikowaena i manaʻo ʻia no ka quorum maikaʻi loa. ʻO ia ʻano puʻupuʻu unahi ʻaʻe no ka heluhelu ʻana (kākau wau e pili ana i ka scaling no ke kākau ʻana ma luna). Eia nō naʻe, e makaʻala e lohi nā replicas asynchronous, ʻoi aku ma lalo o nā haʻahaʻa kaumaha.
Hoʻohana ʻia ka hoʻohana ʻana i ia mau kope no ka heluhelu ʻana (hot standby) no ka hōʻike ʻana a i ʻole nā hana loiloi a hoʻokuʻu i ke kikowaena master.
Inā makemake ʻoe e hana i kēlā ʻano hui iā ʻoe iho, pono ʻoe:
- e hoʻomākaukau i nā kikowaena 3 a i ʻole, e hoʻonohonoho i ka leka uila IP a me nā lula ahi ma waena o lākou;
- hoʻokomo i nā pūʻolo no nā lawelawe etcd, Patroni, PostgreSQL;
- hoʻonohonoho i kahi hui etcd;
- hoʻonohonoho i ka lawelawe patroni e hana pū me PostgreSQL.
ʻO ia hoʻi, i ka huina, pono ʻoe e kākau pololei i nā faila hoʻonohonoho he ʻumi a ʻaʻole e hana hewa i nā wahi āpau. No ka hana ʻana i kēia, pono ʻoe e hoʻohana i kahi mea hoʻokele hoʻonohonoho hoʻonohonoho, e like me Ansible, no ka laʻana. Eia naʻe, ʻaʻohe mea hoʻohālikelike TCP i loaʻa loa. He hana kaawale ka hana ana.
No ka poʻe e pono ana i kahi pūʻulu mākaukau, akā ʻaʻole makemake e ʻoki a puni i kēia mau mea a pau, ua hoʻāʻo mākou e hoʻomaʻamaʻa i ke ola a hana i kahi pūʻulu mākaukau i hana ʻia ma Patroni i kā mākou ao, hiki iā ʻoe ke hoʻāʻo iā ia no ka manuahi. Ma kahi o ka cluster pono'ī, ua hana mākou:
- TCP kaulike; ma nā awa like ʻole, kuhikuhi mau ʻo ia i ka haku o kēia manawa, synchronous a asynchronous replica, kēlā me kēia;
- API e hoʻololi i ka haku Patroni ikaika.
Hiki iā lākou ke hoʻokaʻawale ʻia ma o ka MCS cloud API a me ka console pūnaewele.
Hōʻike
No ka hoʻāʻo ʻana i ka hiki o kahi puʻupuʻu PostgreSQL i ke ao MCS, e ʻike kākou i ka hana ʻana o kahi noi ola inā pilikia me ka DBMS.
ʻO kēia ka palapala noi e hoʻopaʻa inoa i nā hanana artificial a hōʻike iā lākou i ka pale. Inā he hewa, e hōʻike ʻo ia i kēia a hoʻomau i kāna hana ma kahi loop a hiki i ka wā e hoʻōki ai mākou me ka hui pū Ctrl + C.
from __future__ import print_function
from datetime import datetime
from random import randint
from time import sleep
import psycopg2
def main():
try:
connection = psycopg2.connect(user = "admin",
password = "P@ssw0rd",
host = "89.208.87.38",
port = "5432",
database = "myproddb")
cursor = connection.cursor()
cursor.execute("SELECT version();")
record = cursor.fetchone()
print("Connection opened to", record[0])
cursor.execute(
"INSERT INTO log VALUES ({});".format(randint(1, 10000)))
connection.commit()
cursor.execute("SELECT COUNT(event_id) from log;")
record = cursor.fetchone()
print("Logged a value, overall count: {}".format(record[0]))
except Exception as error:
print ("Error while connecting to PostgreSQL", error)
finally:
if connection:
cursor.close()
connection.close()
print("Connection closed")
if __name__ == '__main__':
try:
while True:
try:
print(datetime.now())
main()
sleep(3)
except Exception as e:
print("Caught error:n", e)
sleep(1)
except KeyboardInterrupt:
print("exit")
Pono ka palapala noi i ka PostgreSQL e hana. E hana kākou i pūʻulu ma ke ao MCS me ka API. Ma kahi kikowaena maʻamau, kahi o ka OS_TOKEN variable i loaʻa kahi hōʻailona no ke komo ʻana i ka API (hiki ke loaʻa me ka openstack token issue command), e kākau mākou i nā kauoha:
E hana i kahi hui:
cat <<EОF > pgc10.json
{"cluster":{"name":"postgres10","allow_remote_access":true,"datastore":{"type":"postgresql","version":"10"},"databases":[{"name":"myproddb"}],"users":[{"databases":[{"name":"myproddb"}],"name":"admin","password":"P@ssw0rd"}],"instances":[{"key_name":"shared","availability_zone":"DP1","flavorRef":"d659fa16-c7fb-42cf-8a5e-9bcbe80a7538","nics":[{"net-id":"b91eafed-12b1-4a46-b000-3984c7e01599"}],"volume":{"size":50,"type":"DP1"}},{"key_name":"shared","availability_zone":"DP1","flavorRef":"d659fa16-c7fb-42cf-8a5e-9bcbe80a7538","nics":[{"net-id":"b91eafed-12b1-4a46-b000-3984c7e01599"}],"volume":{"size":50,"type":"DP1"}},{"key_name":"shared","availability_zone":"DP1","flavorRef":"d659fa16-c7fb-42cf-8a5e-9bcbe80a7538","nics":[{"net-id":"b91eafed-12b1-4a46-b000-3984c7e01599"}],"volume":{"size":50,"type":"DP1"}}]}}
EOF
curl -s -H "X-Auth-Token: $OS_TOKEN"
-H 'Accept: application/json'
-H 'Content-Type: application/json'
-d @pgc10.json https://infra.mail.ru:8779/v1.0/ce2a41bbd1434013b85bdf0ba07c770f/clusters
Ke hoʻololi ka puʻupuʻu i ke kūlana ACTIVE, e loaʻa i nā māla āpau nā waiwai o kēia manawa - ua mākaukau ka pūʻulu.
Ma GUI:
E ho'āʻo kākou e hoʻohui a hana i papaʻaina:
psql -h 89.208.87.38 -U admin -d myproddb
Password for user admin:
psql (11.1, server 10.7)
Type "help" for help.
myproddb=> CREATE TABLE log (event_id integer NOT NULL);
CREATE TABLE
myproddb=> INSERT INTO log VALUES (1),(2),(3);
INSERT 0 3
myproddb=> SELECT * FROM log;
event_id
----------
1
2
3
(3 rows)
myproddb=>
Ma ka noi, e kuhikuhi mākou i nā hoʻonohonoho o kēia manawa no ka hoʻopili ʻana iā PostgreSQL. E wehewehe mākou i ka helu o ka TCP balancer, a laila e hoʻopau i ka pono no ka hoʻololi lima i ka helu o ka haku. E hoʻomaka kākou. E like me kāu e ʻike ai, ua hoʻopaʻa ʻia nā hanana i ka waihona.
Hoʻololi Luna Hoʻonohonoho
I kēia manawa e hoʻāʻo mākou i ka hana o kā mākou noi i ka wā o ka hoʻololi ʻana o ka haku:
Ke nānā nei mākou i ka noi. ʻIke mākou ua hoʻopau maoli ʻia ka noi, akā he mau kekona wale nō, ma kēia hihia, ʻoi aku ka nui o 9.
ulia kaʻa
I kēia manawa e hoʻāʻo kākou e hoʻohālikelike i ka hāʻule ʻana o ka mīkini virtual, ka haku o kēia manawa. Hiki ke hoʻopau wale i ka mīkini virtual ma o ka interface Horizon, ʻo ia wale nō ka pani maʻamau. E hana ʻia kēlā ʻano hoʻololi e nā lawelawe āpau, me ka Patroni.
Pono mākou i kahi pani hiki ʻole. No laila, ua noi au i kā mākou mau luna hoʻokele no ka hoʻāʻo ʻana e pani i ka mīkini virtual - ka haku o kēia manawa - ma ke ʻano ʻokoʻa.
I ka manawa like, hoʻomau kā mākou noi i ka hana. ʻO ka mea maʻamau, ʻaʻole hiki ke ʻike ʻia kahi hoʻololi ulia pōpilikia o ka haku.
2019-03-29 10:45:56.071234
Connection opened to PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
Logged a value, overall count: 453
Connection closed
2019-03-29 10:45:59.205463
Connection opened to PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
Logged a value, overall count: 454
Connection closed
2019-03-29 10:46:02.661440
Error while connecting to PostgreSQL server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Caught error:
local variable 'connection' referenced before assignment
……………………………………………………….. - здесь какое-то количество ошибок
2019-03-29 10:46:30.930445
Error while connecting to PostgreSQL server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Caught error:
local variable 'connection' referenced before assignment
2019-03-29 10:46:31.954399
Connection opened to PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
Logged a value, overall count: 455
Connection closed
2019-03-29 10:46:35.409800
Connection opened to PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
Logged a value, overall count: 456
Connection closed
^Cexit
E like me kāu e ʻike ai, hiki i ka noi ke hoʻomau i kāna hana ma lalo o 30 kekona. ʻAe, e loaʻa i kahi helu o nā mea hoʻohana lawelawe ka manawa e ʻike ai i nā pilikia. Eia naʻe, he haʻihaʻi koʻikoʻi kēia o ke kikowaena, ʻaʻole hiki pinepine kēia. I ka manawa like, ʻaʻole i loaʻa i kahi kanaka (luna hoʻomalu) ka manawa e pane koke ai, ke ʻole ʻo ia e noho ana i loko o ka console me ka mākaukau me kahi palapala hoʻololi.
hopena
Me he mea lā iaʻu e hāʻawi ana kēlā ʻano hui i kahi pōmaikaʻi nui no nā luna. ʻO ka ʻoiaʻiʻo, ʻaʻole ʻike ʻia nā haʻihaʻi koʻikoʻi a me nā hemahema o nā kikowaena waihona i ka noi a, no laila, i ka mea hoʻohana. ʻAʻole pono ʻoe e hoʻoponopono i kahi mea me ka wikiwiki a hoʻololi i nā hoʻonohonoho manawaleʻa, server, etc. A inā hoʻohana ʻia kēlā ʻano hopena ma ke ʻano he lawelawe mākaukau i ke ao, a laila ʻaʻole pono e hoʻolilo i ka manawa e hoʻomākaukau ai. Hiki iā ʻoe ke hana i kahi mea hoihoi.
Source: www.habr.com
