Ke hoʻohana nei iā Ceph ma ke ʻano he mālama pūnaewele i nā papahana me nā ukana ʻē aʻe, hiki iā mākou ke hālāwai me nā hana like ʻole i ka nānā mua ʻana ʻaʻole i maʻalahi a i ʻole mea ʻole. ʻo kahi laʻana:
- ka neʻe ʻana o ka ʻikepili mai Ceph kahiko i ka mea hou me ka hoʻohana hapa o nā kikowaena mua i ka hui hou;
- ka hoʻoponopono i ka pilikia o ka hoʻokaʻawale ʻana i ka disk space ma Ceph.
Ke hana nei me ia mau pilikia, ke alo nei mākou i ka pono e wehe pono i ka OSD me ka nalowale ʻole o ka ʻikepili, ʻo ia ka mea nui loa i ka wā e pili ana i ka nui o ka ʻikepili. E kūkākūkā ʻia kēia ma ka ʻatikala.
ʻO nā ʻano hana i hōʻike ʻia ma lalo nei e pili ana i kekahi ʻano o Ceph. Eia hou,ʻo kaʻoiaʻiʻo e hiki iā Ceph ke mālama i ka nui o nāʻikepili e noʻonoʻoʻia: no ka paleʻana i ka nalowale o kaʻikepili a me nā pilikia'ē aʻe, e "hoʻokaʻawaleʻia" kekahi mau hana i kekahi mau mea'ē aʻe.
Olelo mua e pili ana i ka OSD
No ka mea ua hoʻolaʻa ʻia ʻelua o nā ʻano ʻekolu i kūkākūkā ʻia no OSD (), ma mua o ka luʻu ʻana i ka ʻāpana hana - pōkole e pili ana i ke ʻano o Ceph a me ke kumu he mea nui ia.
ʻO ka mea mua, pono e ʻōlelo ʻia ʻo ka hui Ceph holoʻokoʻa he nui nā OSD. ʻOi aku ka nui, ʻoi aku ka nui o ka nui o ka ʻikepili manuahi ma Ceph. He maʻalahi ke hoʻomaopopo mai ʻaneʻi hana OSD nui: Mālama ʻo ia i ka ʻikepili mea Ceph ma nā ʻōnaehana faila o nā nodes cluster āpau a hāʻawi i ke komo pūnaewele iā ia (no ka heluhelu, kākau, a me nā noi ʻē aʻe).
Ma ka pae hoʻokahi, hoʻonohonoho ʻia nā ʻāpana replication ma ke kope ʻana i nā mea ma waena o nā OSD like ʻole. A maanei hiki iā ʻoe ke hālāwai me nā pilikia like ʻole, nā hoʻonā e kūkākūkā ʻia ma lalo nei.
Hihia helu 1. Wehe i ka OSD mai ka hui Ceph me ka nalowale ʻole o ka ʻikepili
ʻO ka pono e wehe i ka OSD ma muli o ka wehe ʻana i ke kikowaena mai ka pūʻulu - no ka laʻana, e hoʻololi iā ia me kahi kikowaena ʻē aʻe - ʻo ia ka mea i loaʻa iā mākou, e hāpai ana i ke kākau ʻana i kēia ʻatikala. No laila, ʻo ka pahuhopu hope loa o ka hoʻoponopono ʻana, ʻo ia ka unuhi ʻana i nā OSD a me nā mons a pau ma kahi kikowaena i hāʻawi ʻia i hiki ke pani ʻia.
No ka maʻalahi a me ka pale ʻana i kahi kūlana kahi e hana hewa ai mākou i ka wehewehe ʻana i ka OSD i koi ʻia i ka wā e hoʻokō ai i nā kauoha, e hoʻonohonoho mākou i kahi ʻokoʻa ʻokoʻa, ʻo ia ka helu o ka OSD e holoi ʻia. E kāhea kākou iā ia ${ID} - ma ʻaneʻi a ma lalo, hoʻololi kēlā ʻano hoʻololi i ka helu o ka OSD a mākou e hana nei.
E nānā kākou i ke kūlana ma mua o ka hoʻomaka ʻana i ka hana:
root@hv-1 ~ # ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 0.46857 root default
-3 0.15619 host hv-1
-5 0.15619 host hv-2
1 ssd 0.15619 osd.1 up 1.00000 1.00000
-7 0.15619 host hv-3
2 ssd 0.15619 osd.2 up 1.00000 1.00000
No ka hoʻomaka ʻana i ka wehe ʻana o OSD, pono ʻoe e hana maʻalahi reweight ma luna o laila i ka ʻole. Ma kēia ala mākou e hōʻemi ai i ka nui o ka ʻikepili i ka OSD ma ke kaupaona ʻana iā ia i nā OSD ʻē aʻe. No ka hana ʻana i kēia, e holo i kēia mau kauoha:
ceph osd reweight osd.${ID} 0.98
ceph osd reweight osd.${ID} 0.88
ceph osd reweight osd.${ID} 0.78... a pēlā aku a hiki i ka ʻole.
Pono ke kaulike maʻemaʻei ʻole e nalowale ka ʻikepili. He ʻoiaʻiʻo kēia inā loaʻa i ka OSD ka nui o ka ʻikepili. E hōʻoia ma hope o ka hoʻokō ʻana i nā kauoha reweight ua holo pono nā mea a pau, hiki iā ʻoe ke hoʻopau ceph -s a i ʻole ma kahi puka makani ʻokoʻa holo ceph -w i mea e nānā ai i nā loli i ka manawa maoli.
Ke "hoʻopau ʻia ka OSD", hiki iā ʻoe ke hoʻomau i ka hana maʻamau e wehe iā ia. No ka hana ʻana i kēia, e hoʻololi i ka OSD i makemake ʻia i ka mokuʻāina down:
ceph osd down osd.${ID}E "huki" i ka OSD mai loko mai o ka pūʻulu:
ceph osd out osd.${ID}E ho'ōki i ka lawelawe OSD a wehe i kona ʻāpana i ka FS:
systemctl stop ceph-osd@${ID}
umount /var/lib/ceph/osd/ceph-${ID}Wehe iā OSD mai :
ceph osd crush remove osd.${ID}E holoi mākou i ka mea hoʻohana OSD:
ceph auth del osd.${ID}A ʻo ka hope, e wehe kākou i ka OSD ponoʻī:
ceph osd rm osd.${ID}i hoʻopuka: Inā ʻoe e hoʻohana ana i ka mana Ceph Luminous a i ʻole ka mea kiʻekiʻe, a laila hiki ke hoʻemi ʻia nā ʻanuʻu hoʻoneʻe OSD ma luna i ʻelua kauoha:
ceph osd out osd.${ID}
ceph osd purge osd.${ID}
Inā, ma hope o ka hoʻopau ʻana i nā ʻanuʻu i hōʻike ʻia ma luna, e holo ʻoe i ke kauoha ceph osd tree, a laila e akaka ma ka kikowaena kahi i hana ʻia ai ka hana ʻaʻohe OSDs i hana ʻia ai nā hana i luna.
root@hv-1 ~ # ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 0.46857 root default
-3 0.15619 host hv-1
-5 0.15619 host hv-2
-7 0.15619 host hv-3
2 ssd 0.15619 osd.2 up 1.00000 1.00000
Ma ke ala, e hoʻomaopopo i ka mokuʻāina o ka hui Ceph e hele i HEALTH_WARN, a e ʻike pū mākou i ka emi ʻana o ka nui o nā OSD a me ka nui o nā wahi disk i loaʻa.
E wehewehe ka mea aʻe i nā ʻanuʻu e koi ʻia inā makemake ʻoe e hoʻōki loa i ka kikowaena a, no laila, e wehe iā ia mai Ceph. I kēia hihia, he mea nui e hoʻomanaʻo i kēlā Ma mua o ka pani ʻana i ke kikowaena, pono ʻoe e wehe i nā OSD a pau ma keia kikowaena.
Inā ʻaʻohe OSD i koe ma kēia kikowaena, a laila ma hope o ka wehe ʻana iā lākou pono ʻoe e haʻalele i ke kikowaena mai ka palapala OSD. hv-2ma ka holo ʻana i kēia kauoha:
ceph osd crush rm hv-2
Hoʻopau mon mai ke kikowaena hv-2ma ka holo ʻana i ke kauoha ma lalo ma kahi kikowaena ʻē aʻe (ʻo ia hoʻi ma kēia hihia, ma hv-1):
ceph-deploy mon destroy hv-2Ma hope o kēia, hiki iā ʻoe ke hoʻōki i ka kikowaena a hoʻomaka i nā hana ma hope (hoʻihoʻi hou ʻia, etc.).
Helu 2. Ka hoʻokaʻawale ʻana o kahi diski i kahi hui Ceph i hana mua ʻia
E hoʻomaka wau i ka moʻolelo ʻelua me kahi ʻōlelo mua e pili ana iā PG (). ʻO ka hana nui o PG ma Ceph ka mea nui e hōʻuluʻulu i nā mea Ceph a hoʻopili hou iā lākou ma OSD. ʻO ke kumu e hiki ai iā ʻoe ke helu i ka nui o ka PG i makemake ʻia Nā palapala Ceph. Kūkākūkā ʻia kēia pilikia ma laila me nā laʻana kikoʻī.
No laila: ʻo kekahi o nā pilikia maʻamau i ka hoʻohana ʻana iā Ceph ka helu kaulike ʻole o OSD a me PG ma waena o nā loko i Ceph.
ʻO ka mea mua, ma muli o kēia, hiki mai ke kūlana ke hōʻike ʻia ka nui o nā PG i loko o kahi wai liʻiliʻi, ʻo ia hoʻi ka hoʻohana pono ʻole o ka wahi disk i loko o ka pūpū. ʻO ka lua, ma ka hoʻomaʻamaʻa aia kahi pilikia koʻikoʻi: ka nui o ka ʻikepili i kekahi o nā OSD. Hoʻopili kēia i ka hoʻololi mua ʻana o ka hui i ka mokuʻāina HEALTH_WARN, a laila HEALTH_ERR. ʻO ke kumu o kēia ʻo Ceph, i ka helu ʻana i ka nui o ka ʻikepili i loaʻa (hiki iā ʻoe ke ʻike ma o MAX AVAIL ma ka puka kauoha ceph df no kēlā me kēia loko i hoʻokaʻawale ʻia) ma muli o ka nui o nā ʻikepili i loaʻa i ka OSD. Inā ʻaʻole lawa ka hakahaka ma ka liʻiliʻi o hoʻokahi OSD, ʻaʻole hiki ke kākau ʻia nā ʻikepili hou a hiki i ka puʻunaue pono ʻana o ka ʻikepili i waena o nā OSD a pau.
He mea pono e wehewehe i keia mau pilikia ua hoʻoholo nui ʻia ma ke kahua hoʻonohonoho Ceph cluster. ʻO kekahi o nā mea hana āu e hoʻohana ai . Me kāna kōkua, ʻike maopopo ʻia ka nui o ka PG i koi ʻia. Eia nō naʻe, hiki iā ʻoe ke hoʻohana iā ia i kahi kūlana kahi o ka Ceph cluster ua i hoʻonohonoho hewa ʻia. Pono e wehewehe ma aneʻi ma ke ʻano he ʻāpana o ka hana hoʻoponopono e pono paha ʻoe e hōʻemi i ka nui o nā PG, a ʻaʻole i loaʻa kēia hiʻohiʻona i nā mana kahiko o Ceph (ʻike wale ʻia ma ka mana. ).
No laila, e noʻonoʻo kākou i ke kiʻi aʻe: he kūlana ko ka pūpū HEALTH_WARN ma muli o kekahi o ka OSD e pau ana i waho. E hōʻike ʻia kēia e kahi hewa HEALTH_WARN: 1 near full osd. Aia ma lalo kahi algorithm no ka puka ʻana i waho o kēia kūlana.
ʻO ka mea mua, pono ʻoe e puʻunaue i ka ʻikepili i loaʻa ma waena o nā OSD i koe. Ua hana mua mākou i kahi hana like i ka hihia mua, i ka wā i "hoʻokahe" ai mākou i ka node - me ka ʻokoʻa wale nō i kēia manawa pono mākou e hōʻemi iki. reweight. No ka laʻana, a hiki i 0.95:
ceph osd reweight osd.${ID} 0.95Hoʻokuʻu kēia i kahi diski i ka OSD a hoʻoponopono i ka hewa i ke olakino ceph. Eia nō naʻe, e like me ka mea i haʻi mua ʻia, hiki mai kēia pilikia ma muli o ka hoʻonohonoho hewa ʻole ʻana o Ceph i nā pae mua: he mea koʻikoʻi ke hana hou ʻana i ʻole e ʻike ʻia i ka wā e hiki mai ana.
I kā mākou hihia kūikawā, ua hele mai nā mea a pau i:
- kiʻekiʻe loa ka waiwai
replication_counti loko o kekahi o nā loko, - ʻoi aku ka nui o ka PG i loko o ka wai hoʻokahi a ʻaʻole liʻiliʻi i kekahi.
E hoʻohana kākou i ka helu helu i ʻōlelo ʻia. Hōʻike maopopo ia i nā mea e pono ai ke komo a, ma ke kumu, ʻaʻohe mea paʻakikī. Ke hoʻonohonoho nei i nā ʻāpana kūpono, loaʻa iā mākou nā ʻōlelo aʻe:
i hoʻopuka: Inā ʻoe e hoʻonohonoho ana i kahi puʻupuʻu Ceph mai ka ʻohi ʻana, ʻo kahi hana pono ʻē aʻe o ka calculator ka hana o nā kauoha e hana ai i nā loko mai ka wā ʻōpala me nā ʻāpana i kuhikuhi ʻia ma ka papaʻaina.
Kōkua ke kolamu hope iā ʻoe e hoʻokele - Helu PG manaʻo ʻia. I kā mākou hihia, he mea pono hoʻi ka lua, kahi i hōʻike ʻia ai ke ʻano replication, ʻoiai ua hoʻoholo mākou e hoʻololi i ka mea hoʻonui replication.
No laila, pono ʻoe e hoʻololi i nā ʻāpana replication - pono e hana mua kēia, no ka mea ma ka hōʻemi ʻana i ka multiplier, e hoʻokuʻu mākou i kahi disk. Ke hoʻokō nei ke kauoha, e ʻike ʻoe e piʻi aʻe ka wahi disk i loaʻa:
ceph osd pool $pool_name set $replication_size
A ma hope o kona hoʻopau ʻana, hoʻololi mākou i nā koina parameter pg_num и pgp_num penei:
ceph osd pool set $pool_name pg_num $pg_number
ceph osd pool set $pool_name pgp_num $pg_numbernui: Pono mākou e hoʻololi i ka helu o nā PG i kēlā me kēia pūnāwai a ʻaʻole e hoʻololi i nā waiwai i loko o nā loko ʻē aʻe a pau nā ʻōlelo luhi. "Hoʻohaʻahaʻa ʻia ka ʻikepili hōʻemi" и "N-helu o nā pgs i hoʻohaʻahaʻa ʻia".
Hiki iā ʻoe ke nānā ua holo maikaʻi nā mea a pau me ka hoʻohana ʻana i nā kauoha ceph health detail и ceph -s.
Helu 3. Ke neʻe nei i kahi mīkini virtual mai LVM i Ceph RBD
Ma ke kūlana kahi e hoʻohana ai kahi papahana i nā mīkini virtual i hoʻokomo ʻia ma nā kikowaena metala hoʻolimalima ʻia, kū pinepine ka pilikia o ka mālama ʻana i ka hewa. He mea makemake loa ia i ka nui o ka wahi i loko o kēia waihona ... ʻO kekahi kūlana maʻamau: aia kahi mīkini virtual me kahi waihona kūloko ma ka kikowaena a pono ʻoe e hoʻonui i ka disk, akā ʻaʻohe wahi e hele ai, no ka mea ʻaʻohe waiho ʻia ka hakahaka diski ma ke kikowaena.
Hiki ke hoʻoholo i ka pilikia ma nā ʻano like ʻole - no ka laʻana, ma ka neʻe ʻana i kahi kikowaena ʻē aʻe (inā loaʻa kekahi) a i ʻole ka hoʻohui ʻana i nā disks hou i ke kikowaena. Akā ʻaʻole hiki ke hana i kēia, no laila ʻo ka neʻe ʻana mai LVM a Ceph hiki ke lilo i mea hoʻonā maikaʻi loa i kēia pilikia. Ma ke koho ʻana i kēia koho, hoʻomaʻamaʻa mākou i ke kaʻina hana hou o ka neʻe ʻana ma waena o nā kikowaena, no ka mea ʻaʻole pono e neʻe i kahi waihona kūloko mai kahi hypervisor i kekahi. ʻO ka hopu wale nō, pono ʻoe e hoʻōki i ka VM i ka wā e hana ʻia ana ka hana.
Lawe ʻia kēia ʻano meaʻai mai , ua ho'āʻoʻia nā kuhikuhi i ka hana. No laila, Ua wehewehe ʻia ke ʻano o ka neʻe ʻana me ka pilikia ʻole, akā naʻe, i kā mākou hihia ʻaʻole pono ia, no laila ʻaʻole mākou i nānā. Inā he mea koʻikoʻi kēia no kāu papahana, e hauʻoli mākou e lohe i nā hopena ma nā ʻōlelo.
E neʻe kākou i ka ʻāpana hana. I ka laʻana, hoʻohana mākou i ka virsh a, no laila, libvirt. ʻO ka mea mua, e hōʻoia i ka pili ʻana o ka wai Ceph kahi e neʻe ai ka ʻikepili i libvirt:
virsh pool-dumpxml $ceph_poolPono e loaʻa i ka wehewehe wai ka ʻikepili pili iā Ceph me ka ʻikepili ʻae.
ʻO ka hana aʻe e hoʻohuli ʻia ke kiʻi LVM i Ceph RBD. ʻO ka manawa hoʻokō e pili ana i ka nui o ke kiʻi:
qemu-img convert -p -O rbd /dev/main/$vm_image_name rbd:$ceph_pool/$vm_image_nameMa hope o ka hoʻololi ʻana, e hoʻomau ʻia kahi kiʻi LVM, kahi mea e pono ai ke neʻe i ka VM i RBD a pono ʻoe e hoʻihoʻi i nā loli. Eia kekahi, i hiki iā ʻoe ke hoʻololi koke i nā loli, e hana mākou i kahi waihona o ka faila hoʻonohonoho mīkini virtual:
virsh dumpxml $vm_name > $vm_name.xml
cp $vm_name.xml $vm_name_backup.xml
... a hoʻoponopono i ke kumu (vm_name.xml). E ʻimi kākou i kahi poloka me ka wehewehe ʻana o ka disk (hoʻomaka me ka laina <disk type='file' device='disk'> a pau me </disk>) a e hoemi i ke ano penei:
<disk type='network' device='disk'>
<driver name='qemu'/>
<auth username='libvirt'>
<secret type='ceph' uuid='sec-ret-uu-id'/>
</auth>
<source protocol='rbd' name='$ceph_pool/$vm_image_name>
<host name='10.0.0.1' port='6789'/>
<host name='10.0.0.2' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>E nānā kākou i kekahi mau kikoʻī:
- I ka protocol
sourcehōʻike ʻia ka helu wahi i ka mālama ʻana ma Ceph RBD (ʻo ia ka helu e hōʻike ana i ka inoa o ka kolamu Ceph a me ke kiʻi RBD, i hoʻoholo ʻia ma ka pae mua). - I ka palaka
secrethōʻike ʻia ke ʻanoceph, a me ka UUID o ka mea huna e hoʻohui iā ia. Hiki ke loaʻa kona uuid me ke kauohavirsh secret-list. - I ka palaka
hosthōʻike ʻia nā ʻōlelo i nā māka Ceph.
Ma hope o ka hoʻoponopono ʻana i ka faila hoʻonohonoho a hoʻopau i ka hoʻololi ʻana o LVM i RBD, hiki iā ʻoe ke hoʻohana i ka faila hoʻonohonoho hoʻololi ʻia a hoʻomaka i ka mīkini virtual:
virsh define $vm_name.xml
virsh start $vm_name
ʻO ka manawa kēia e nānā ai ua hoʻomaka pololei ka mīkini virtual: hiki iā ʻoe ke ʻike, no ka laʻana, ma ka hoʻopili ʻana iā ia ma o SSH a i ʻole virsh.
Inā hana pololei ka mīkini virtual a ʻaʻole ʻoe i ʻike i nā pilikia ʻē aʻe, a laila hiki iā ʻoe ke holoi i ke kiʻi LVM i hoʻohana ʻole ʻia:
lvremove main/$vm_image_namehopena
Ua hālāwai mākou i nā hihia i wehewehe ʻia ma ka hoʻomaʻamaʻa - manaʻolana mākou e kōkua nā ʻōlelo aʻoaʻo i nā luna ʻē aʻe e hoʻoponopono i nā pilikia like. Inā loaʻa iā ʻoe nā manaʻo a i ʻole nā moʻolelo like ʻole mai kāu ʻike e hoʻohana ana iā Ceph, e hauʻoli mākou e ʻike iā lākou ma nā ʻōlelo!
PS
E heluhelu pū ma kā mākou blog:
- «";
- «";
- «";
- «".
Source: www.habr.com