A yau, ana buƙatar babban wadatar sabis koyaushe kuma a ko'ina, ba kawai a cikin manyan ayyuka masu tsada ba. Shafukan da ba su da wani ɗan lokaci tare da saƙon "Yi haƙuri, ana ci gaba da ci gaba" har yanzu ana ci karo da su, amma yawanci suna haifar da murmushi. Bari mu ƙara rayuwa a cikin gajimare zuwa wannan, lokacin da ake buƙatar kira ɗaya kawai zuwa API don ƙaddamar da ƙarin uwar garke, kuma babu buƙatar yin tunani game da aikin "ƙarfe". Kuma babu wani uzuri da ya sa ba a samar da wani muhimmin tsari ta hanyar amfani da fasahohin tari da sakewa ba.
Za mu gaya muku hanyoyin da muka yi la'akari don tabbatar da amincin bayanan bayanai a cikin ayyukanmu da abin da muka fito da su. Ƙara demo tare da sakamako mai nisa.
Legacy a cikin Babban Samuwar Gine-gine
An fi ganin wannan a cikin mahallin ci gaban tsarin tushen buɗe ido iri-iri. Maganganun gadon dole ne su ƙara yawan fasahar samarwa yayin da buƙata ta ƙaru. Kuma ingancinsu ya bambanta. Maganganun ƙarni na gaba sun sanya wadatuwa mai yawa a cikin tushen gine-ginen su. Misali, MongoDB matsayi gungu a matsayin babban yanayin amfaninsa. Tarin ma'auni a kwance, wanda shine babban fa'idar gasa na wannan DBMS.
Bari mu koma PostgreSQL. Wannan yana ɗaya daga cikin tsoffin mashahuran ayyukan buɗaɗɗen tushe, wanda farkon fitowar sa ya faru a cikin shekara ta 95 na ƙarni na ƙarshe. Ƙungiyar aikin na dogon lokaci ba su yi la'akari da samuwa mai yawa a matsayin aikin da za a warware ta hanyar tsarin ba. Don haka, fasahar kwafi don ƙirƙirar kwafin bayanai ta zama an gina ta a cikin sigar 8.2 a cikin 2006 kawai, amma fayil ce (shipping log). A cikin 2010, kwafin yawo ya bayyana a cikin sigar 9.0, kuma shine tushen ƙirƙirar gungu iri-iri. Wannan, a zahiri, abin mamaki ne sosai ga mutanen da suka saba da PostgreSQL bayan Enterprise SQL ko NoSQL na zamani - daidaitaccen bayani daga al'umma shine kawai babban nau'i-nau'i-nau'i tare da kwafi na aiki tare ko asynchronous. A lokaci guda kuma, a cikin magudanar ruwa, maigidan yana canzawa da hannu, kuma batun canza abokan ciniki kuma ana ba da shawarar a warware shi da kansa.
Yadda muka yanke shawarar yin abin dogaro PostgreSQL da abin da muka zaɓa don wannan
Koyaya, PostgreSQL ba zai zama sananne sosai ba idan ba don ɗimbin ayyuka da kayan aikin da ke taimakawa gina mafita mai jurewa da kuskure wanda baya buƙatar kulawa akai-akai. A cikin gajimare (MCS) tun lokacin ƙaddamar da DBaaS, ana samun sabar PostgreSQL guda ɗaya da nau'i-nau'i-nau'i-nau'i-nau'i-nau'i tare da kwafi asynchronous.
A zahiri, muna so mu sauƙaƙa rayuwa ga kowa da kowa kuma mu samar da shigarwar PostgreSQL wanda zai iya zama tushen tushen ayyuka masu yawa waɗanda ba za su ci gaba da saka idanu da tashi da dare don yin canji ba. A cikin wannan sashin, akwai duka tsoffin ingantattun hanyoyin magancewa da kuma ƙarni na sabbin abubuwan amfani waɗanda ke amfani da sabbin abubuwan ci gaba.
A yau, matsalar babban samuwa ba ta dogara ne akan ajiyar ba (wannan yana tafiya ba tare da faɗi ba), amma akan yarjejeniya - algorithm don zaɓar shugaba (zaben shugabanni). Mafi sau da yawa, manyan hatsarori suna faruwa ba saboda rashin sabobin ba, amma saboda matsaloli tare da yarjejeniya: sabon shugaban bai fita ba, shugabannin biyu sun bayyana a cibiyoyin bayanai daban-daban, da dai sauransu. Misali shine karo akan gungu na Github MySQL - sun rubuta .
Tushen lissafi a cikin wannan lamari yana da matukar muhimmanci. A gefe guda, akwai , wanda ke sanya ƙuntatawa na ka'idar akan yuwuwar gina hanyoyin HA, a gefe guda, algorithms da aka tabbatar ta hanyar lissafi don ƙayyade yarjejeniya, kamar su. и . A kan wannan tushen, akwai sanannen DCS (tsarin ra'ayi da aka raba) - Zookeeper, da sauransu, Consul. Saboda haka, idan tsarin yanke shawara yana aiki akan wasu algorithms na kansa, wanda aka rubuta da kansa, ya kamata mutum yayi taka tsantsan game da shi. Bayan nazarin ɗimbin tsarin, mun daidaita akan Patroni, tsarin tushen buɗe ido wanda Zalando ya haɓaka.
A matsayin digression lyrical, zan ce mun kuma yi la'akari da Multi-master mafita, wato, gungu da za a iya a kwance a sikelin don rikodi. Koyaya, saboda manyan dalilai guda biyu, mun yanke shawarar ba za mu yi irin wannan tari ba. Da fari dai, irin waɗannan mafita suna da babban rikitarwa kuma, daidai da haka, ƙarin rauni. Zai yi wuya a samar da tsayayyen bayani ga kowane yanayi. Abu na biyu, a cikin wannan yanayin, PostgreSQL ya daina kasancewa mai tsabta (an ƙasa), wasu ayyuka ba za su kasance ba, wasu aikace-aikacen na iya fuskantar ɓoyayyun kwari yayin aiki.
Majiɓinci
To yaya Patroni yake aiki? Masu haɓakawa ba su sake ƙirƙira dabaran ba kuma sun ba da shawarar yin amfani da ɗayan ingantattun hanyoyin magance DCS a matsayin tushe. A cikin jinƙansa, ana ba da duk batutuwa tare da daidaitawa na daidaitawa, zaɓin jagora da ƙungiyar ƙira. Mun zaɓi etcd don wannan.
Bayan haka, Patroni yana hulɗa da daidaitaccen aikace-aikacen duk saitunan PostgreSQL da saitunan maimaitawa, da kuma aiwatar da umarni don sauyawa da gazawar (wato, sauyawa na yau da kullun da na yau da kullun na maigidan). Musamman, a cikin gajimare na MCS, zaku iya ƙirƙirar tari daga maigidan, kwafi na aiki tare, da kwafi ɗaya ko fiye da asynchronous. Kasancewar kwafi na aiki tare yana tabbatar da amincin bayanan akan aƙalla sabobin 2, kuma wannan kwafin ne zai zama babban “manan takara”.
Tunda etcd aka tura akan sabar iri ɗaya, ana bada shawarar sabar 3 ko 5 don mafi kyawun ƙididdiga. Irin wannan gungu yana daidaita a kwance don karantawa (Na rubuta game da sikelin rubutu a sama). Koyaya, ku sani cewa kwafi na asynchronous yakan yi kasala, musamman a ƙarƙashin kaya masu nauyi.
Amfani da irin waɗannan kwafin don karantawa (jiran jiran aiki mai zafi) ya dace don bayar da rahoto ko ayyukan nazari da sauke babban sabar.
Idan kuna son yin irin wannan gungu da kanku, to kuna buƙatar:
- shirya 3 ko fiye sabobin, saita adireshin IP da ka'idodin tacewar zaɓi tsakanin su;
- shigar da fakiti don etcd, Patroni, sabis na PostgreSQL;
- kafa gungu na etcd;
- saita sabis na majiɓinci don aiki tare da PostgreSQL.
Wato, gabaɗaya, kuna buƙatar tsara fayilolin daidaitawa dozin daidai kuma kada ku yi kuskure a ko'ina. Don yin wannan, lallai ya kamata ku yi amfani da kayan aikin sarrafa sanyi, kamar Mai yiwuwa, misali. Koyaya, har yanzu babu wani ma'auni na TCP sosai. Yin shi aiki ne daban.
Ga waɗanda suke buƙatar gungu da aka shirya, amma ba sa son yin wasa a cikin duk wannan, mun yi ƙoƙarin sauƙaƙe rayuwa kuma mun yi gungu na shirye-shiryen akan Patroni a cikin girgijenmu, zaku iya gwada shi kyauta. Baya ga gungu kanta, mun yi:
- Ma'aunin TCP; a kan tashoshin jiragen ruwa daban-daban, koyaushe yana nuni ga maigidan na yanzu, kwafi na aiki tare ko asynchronous, bi da bi;
- API don canza mai sarrafa Patroni mai aiki.
Ana iya raba su ta hanyar MCS girgije API da na'urar wasan bidiyo na yanar gizo.
Demo
Don gwada iyawar gungu na PostgreSQL a cikin gajimare na MCS, bari mu ga yadda aikace-aikacen kai tsaye ke aiki idan akwai matsaloli tare da DBMS.
Mai zuwa shine lambar aikace-aikacen da za ta shiga abubuwan da suka faru na wucin gadi kuma su ba da rahoton su ga allo. Idan akwai kurakurai, zai ba da rahoton wannan kuma ya ci gaba da aikinsa a cikin madauki har sai mun dakatar da shi tare da haɗin Ctrl + C.
from __future__ import print_function
from datetime import datetime
from random import randint
from time import sleep
import psycopg2
def main():
try:
connection = psycopg2.connect(user = "admin",
password = "P@ssw0rd",
host = "89.208.87.38",
port = "5432",
database = "myproddb")
cursor = connection.cursor()
cursor.execute("SELECT version();")
record = cursor.fetchone()
print("Connection opened to", record[0])
cursor.execute(
"INSERT INTO log VALUES ({});".format(randint(1, 10000)))
connection.commit()
cursor.execute("SELECT COUNT(event_id) from log;")
record = cursor.fetchone()
print("Logged a value, overall count: {}".format(record[0]))
except Exception as error:
print ("Error while connecting to PostgreSQL", error)
finally:
if connection:
cursor.close()
connection.close()
print("Connection closed")
if __name__ == '__main__':
try:
while True:
try:
print(datetime.now())
main()
sleep(3)
except Exception as e:
print("Caught error:n", e)
sleep(1)
except KeyboardInterrupt:
print("exit")
Aikace-aikacen yana buƙatar PostgreSQL don aiki. Bari mu ƙirƙiri gungu a cikin gajimare na MCS ta amfani da API. A cikin tasha na yau da kullun, inda OS_TOKEN m ya ƙunshi alama don samun dama ga API (za'a iya samuwa tare da umarnin fitowar alamar buɗewa), za mu buga umarni:
Ƙirƙiri tari:
cat <<EОF > pgc10.json
{"cluster":{"name":"postgres10","allow_remote_access":true,"datastore":{"type":"postgresql","version":"10"},"databases":[{"name":"myproddb"}],"users":[{"databases":[{"name":"myproddb"}],"name":"admin","password":"P@ssw0rd"}],"instances":[{"key_name":"shared","availability_zone":"DP1","flavorRef":"d659fa16-c7fb-42cf-8a5e-9bcbe80a7538","nics":[{"net-id":"b91eafed-12b1-4a46-b000-3984c7e01599"}],"volume":{"size":50,"type":"DP1"}},{"key_name":"shared","availability_zone":"DP1","flavorRef":"d659fa16-c7fb-42cf-8a5e-9bcbe80a7538","nics":[{"net-id":"b91eafed-12b1-4a46-b000-3984c7e01599"}],"volume":{"size":50,"type":"DP1"}},{"key_name":"shared","availability_zone":"DP1","flavorRef":"d659fa16-c7fb-42cf-8a5e-9bcbe80a7538","nics":[{"net-id":"b91eafed-12b1-4a46-b000-3984c7e01599"}],"volume":{"size":50,"type":"DP1"}}]}}
EOF
curl -s -H "X-Auth-Token: $OS_TOKEN"
-H 'Accept: application/json'
-H 'Content-Type: application/json'
-d @pgc10.json https://infra.mail.ru:8779/v1.0/ce2a41bbd1434013b85bdf0ba07c770f/clusters
Lokacin da gungu ya canza zuwa matsayin ACTIVE, duk filayen za su sami ƙimar halin yanzu - tarin ya shirya.
A cikin GUI:
Bari mu yi ƙoƙarin haɗi da ƙirƙirar tebur:
psql -h 89.208.87.38 -U admin -d myproddb
Password for user admin:
psql (11.1, server 10.7)
Type "help" for help.
myproddb=> CREATE TABLE log (event_id integer NOT NULL);
CREATE TABLE
myproddb=> INSERT INTO log VALUES (1),(2),(3);
INSERT 0 3
myproddb=> SELECT * FROM log;
event_id
----------
1
2
3
(3 rows)
myproddb=>
A cikin aikace-aikacen, za mu ƙayyade saitunan yanzu don haɗawa zuwa PostgreSQL. Za mu ƙayyade adireshin ma'auni na TCP, ta haka ne za mu kawar da buƙatar sauyawa ta hannu zuwa adireshin maigidan. Mu kaddamar da shi. Kamar yadda kuke gani, an sami nasarar shiga abubuwan da suka faru a cikin rumbun adana bayanai.
Canjin Jagora Mai Tsara
Yanzu bari mu gwada aikin aikace-aikacen mu yayin shirin sauya maigidan:
Muna kallon aikace-aikacen. Mun ga cewa an katse aikace-aikacen da gaske, amma yana ɗaukar 'yan daƙiƙa kaɗan kawai, a cikin wannan yanayin, matsakaicin matsakaicin 9.
hadarin mota
Yanzu bari mu yi ƙoƙari mu kwaikwayi faɗuwar injin kama-da-wane, maigidan na yanzu. Zai yuwu a kashe na'urar kama-da-wane ta hanyar Horizon interface, kawai wannan zai zama rufewa na yau da kullun. Irin wannan canjin za a sarrafa ta duk sabis, gami da Patroni.
Muna buƙatar rufewar mara tabbas. Saboda haka, na tambayi masu gudanar da mu don dalilai na gwaji don rufe na'ura mai mahimmanci - maigidan na yanzu - ta hanyar da ba ta dace ba.
A lokaci guda, aikace-aikacenmu ya ci gaba da aiki. A dabi'a, irin wannan canjin gaggawa na maigidan ba zai iya wucewa ba a sani ba.
2019-03-29 10:45:56.071234
Connection opened to PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
Logged a value, overall count: 453
Connection closed
2019-03-29 10:45:59.205463
Connection opened to PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
Logged a value, overall count: 454
Connection closed
2019-03-29 10:46:02.661440
Error while connecting to PostgreSQL server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Caught error:
local variable 'connection' referenced before assignment
……………………………………………………….. - здесь какое-то количество ошибок
2019-03-29 10:46:30.930445
Error while connecting to PostgreSQL server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Caught error:
local variable 'connection' referenced before assignment
2019-03-29 10:46:31.954399
Connection opened to PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
Logged a value, overall count: 455
Connection closed
2019-03-29 10:46:35.409800
Connection opened to PostgreSQL 10.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit
Logged a value, overall count: 456
Connection closed
^Cexit
Kamar yadda kuke gani, aikace-aikacen ya sami damar ci gaba da aikinsa cikin ƙasa da daƙiƙa 30. Ee, takamaiman adadin masu amfani da sabis za su sami lokaci don lura da matsaloli. Koyaya, wannan babban rushewar uwar garken ne, wannan baya faruwa sau da yawa. A lokaci guda kuma, da wuya mutum (mai gudanarwa) ya sami lokacin da zai amsa da sauri, sai dai idan yana zaune a cikin na'ura mai kwakwalwa a shirye tare da rubutun canzawa.
ƙarshe
Da alama a gare ni cewa irin wannan tari yana ba da babbar fa'ida ga masu gudanarwa. A haƙiƙa, ɓarna mai tsanani da gazawar sabobin bayanai ba za su zama abin lura ga aikace-aikacen ba kuma, bisa ga haka, ga mai amfani. Ba dole ba ne ka gyara wani abu cikin gaggawa kuma ka canza zuwa saitunan wucin gadi, sabobin, da sauransu. Kuma idan an yi amfani da irin wannan bayani a matsayin sabis ɗin da aka shirya a cikin girgije, to, ba za a yi amfani da lokacin shirya shi ba. Kuna iya yin wani abu mafi ban sha'awa.
source: www.habr.com
