ื ืืฆื Ceph ืืื ื ืขืฅ ืกืืึธืจืืืืฉ ืืื ืคึผืจืึทืืืฉืขืงืก ืืื ืคืึทืจืฉืืืขื ืข ืืึธืืื, ืืืจ ืงืขื ืืจืขืคื ืคืึทืจืฉืืื ืืึทืกืงืก ืืืึธืก ืืืึท ืขืจืฉืืขืจ ืืืืง ืืึธื ื ืื ืืืกืงืืืขื ืคึผืฉืื ืึธืืขืจ ื ืืฉืืืง. ืืืฉื:
- ืืืืจืึทืืืึธื ืคืื ืืึทืื ืคืื ืึทืื ืกืขืฃ ืฆื ื ืืึท ืืื ืคึผืึทืจืืืืืฉ ื ืืฆื ืคืื ืคืจืืึทืขืจืืืง ืกืขืจืืืขืจืก ืืื ืื ื ืืึท ืงื ืืื;
- ืืืืืื ื ืฆื ืื ืคึผืจืึธืืืขื ืคืื ืืืกืง ืคึผืืึทืฅ ืึทืืึทืงืืืฉืึทื ืืื Ceph.
ืืืืื ื ืืื ืึทืืึท ืคึผืจืึธืืืขืืก, ืืืจ ืืขื ืขื ืคืืืกื ืืื ืื ื ืืื ืฆื ืจืืืืืง ืืึทืืืึทืืืงื ืื OSD ืึธื ืืืืื ื ืืึทืื, ืืืึธืก ืืื ืกืคึผืขืฆืืขื ืืืืืืืง ืืืขื ืืืืื ื ืืื ืืจืืืก ืึทืืึทืื ืฅ ืคืื ืืึทืื. ืืึธืก ืืืขื ืืืื ืืืกืงืึทืกื ืืื ืืขื ืึทืจืืืงื.
ืื ืืขืืืึธืืก ืืืกืงืจืืืื ืืื ืื ืืขื ืขื ืืึทืืืึทืืืง ืคึฟืึทืจ ืงืืื ืืืขืจืกืืข ืคืื โโCeph. ืืื ืืขืจืฆื, ืื ืคืึทืงื ืึทื Ceph ืงืขื ืขื ืงืจืึธื ืึท ืืจืืืก ืกืืืข ืคืื โโืืึทืื ืืืขื ืืืื ืืขื ืืืขื ืืื ืืฉืืื: ืฆื ืคืึทืจืืืึทืื ืืึทืื ืึธื ืืืขืจ ืืื ืื ืืขืจืข ืคึผืจืึธืืืขืืก, ืขืืืขืืข ืึทืงืฉืึทื ื ืืืขื ืืืื "ืฉืคึผืึทืืื" ืืื ืขืืืขืืข ืื ืืขืจืข.
ืืงืืื ืืืขืื OSD
ืืื ื ืฆืืืื ืคืื ืื ืืจืื ืืืกืงืึทืกื ืจืขืกืึทืคึผืื ืืขื ืขื ืืขืืึทืงืืืืึทื ืฆื OSD (), ืืืืืขืจ ืืืืืืื ื ืืื ืื ืคึผืจืึทืงืืืฉ ืืืื - ืืขืงืืฆืขืจ ืืืขืื ืืืึธืก ืขืก ืืื ืืื Ceph ืืื ืืืึธืก ืขืก ืืื ืึทืืื ืืืืืืืง.
ืขืจืฉืืขืจ ืคืื ืึทืืข, ืขืก ืืึธื ืืืื ืืขืืึธืื ืึทื ืื ืืื ืฆืข Ceph ืงื ืืื ืืืฉืืืื ืคืื ืคืืืข OSDs. ืื ืืขืจ ืขืก ืืขื ืขื, ืื ืืจืขืกืขืจ ืื ืคืจืื ืืึทืื ืืึทื ื ืืื Ceph. ืขืก ืืื ืืจืื ื ืฆื ืคึฟืึทืจืฉืืืื ืคืื ืืึธ ืืืืคึผื ืึธืกื ืคึฟืื ืงืฆืืข: ืขืก ืกืืึธืจื Ceph ืึทืืืืฉืขืงืฅ ืืึทืื ืืืืฃ ืื ืืขืงืข ืกืืกืืขืืขื ืคืื ืึทืืข ืงื ืืื ื ืึธืืื ืืื ืืื ื ืขืฅ ืึทืงืกืขืก ืฆื ืขืก (ืคึฟืึทืจ ืืืืขื ืขื, ืฉืจืืืื ืืื ืื ืืขืจืข ืจืืงืืืขืก).
ืืื ืืขืจ ืืขืืืืงืขืจ ืืืจืื, ืจืขืคึผืืึทืงืืืฉืึทื ืคึผืึทืจืึทืืขืืขืจืก ืืขื ืขื ืืึทืฉืืืื ืืืจื ืงืึทืคึผืืื ื ืึทืืืืฉืขืงืฅ ืฆืืืืฉื ืคืึทืจืฉืืืขื ืข OSDs. ืืื ืืึธ ืืืจ ืงืขื ืขื ืืจืขืคื ืคืึทืจืฉืืื ืคึผืจืึธืืืขืืก, ืื ืกืึทืืืฉืึทื ื ืฆื ืืืึธืก ืืืขื ืืืื ืืืกืงืึทืกื ืืื ืื.
ืคืึทื ื ืืืขืจ 1. ืืขืฉืึธืืขื ืืึทืืืึทืืืงื OSD ืคืื Ceph ืงื ืืื ืึธื ืืืืื ื ืืึทืื
ืื ื ืืื ืฆื ืืึทืืืึทืืืงื ืื OSD ืงืขื ืืืื ืืขืคึฟืืจื ืืืจื ืจืืืืืืื ื ืื ืกืขืจืืืขืจ ืคืื ืืขื ืงื ืืื - ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืฆื ืคืึทืจืืืึทืื ืขืก ืืื ืื ืื ืืขืจ ืกืขืจืืืขืจ - ืืืึธืก ืืื ืืขืฉืขื ืืื ืืื ืื, ืืืึธืก ืืื ืืขืคึฟืืจื ืฆื ืื ืฉืจืืืื ืคืื ืืขื ืึทืจืืืงื. ืืืื, ืื ืืขืฆื ืฆืื ืคืื ืืึทื ืืคึผืืึทืืืืฉืึทื ืืื ืฆื ืขืงืกืืจืึทืงื ืึทืืข ืึธืกืืก ืืื ืืึธื ืก ืืืืฃ ืึท ืืขืืขืื ืกืขืจืืืขืจ ืึทืืื ืึทื ืขืก ืงืขื ืขื ืืืื ืกืืึทืคึผื.
ืคึฟืึทืจ ืงืึทื ืืืื ืืึทื ืก ืืื ืฆื ืืืกืืืืื ืึท ืกืืืืึทืฆืืข ืืืขื ืืืจ ืืึทืื ืึท ืืจืืึทื ืืืขื ืืืจ ืืืจืืคืืจื ืงืึทืืึทื ืื ืืื ืึธื ืืืืึทืื ืื ืคืืจืืื ืื OSD, ืืืจ ืฉืืขืื ืึท ืืึทืืื ืืขืจ ืืืึทืืขืืืืืง, ืื ืืืขืจื ืคืื ืืืึธืก ืืืขื ืืืื ืื ื ืืืขืจ ืคืื ืื OSD ืฆื ืืืกืืขืงื. ืืืืืจ ืื ืจืืคื ${ID} - ืืึธ ืืื ืืื ืื, ืึทืืึท ืึท ืืืึทืืขืืืืืง ืจืืคึผืืืืกืื ืื ื ืืืขืจ ืคืื ืื OSD ืืื ืืืึธืก ืืืจ ืึทืจืืขืื.
ืืื ืก ืงืืง ืืื ืื ืฆืืฉืืึทื ื ืืืืืขืจ ืืืจ ืึธื ืืืืื ืึทืจืืขื:
root@hv-1 ~ # ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 0.46857 root default
-3 0.15619 host hv-1
-5 0.15619 host hv-2
1 ssd 0.15619 osd.1 up 1.00000 1.00000
-7 0.15619 host hv-3
2 ssd 0.15619 osd.2 up 1.00000 1.00000
ืฆื ืึธื ืืืืื ืื ืืึทืืืึทืืืงืื ื ืคืื OSD, ืืืจ ืืืขื ืืึทืจืคึฟื ืฆื ืืืจืืคืืจื ืกืืืืื reweight ืืืืฃ ืขืก ืฆื ื ืื. ืืขื ืืืขื ืืืจ ืจืขืืืฆืืจื ืื ืกืืืข ืคืื โโืืึทืื ืืื ืื OSD ืืืจื ืืึทืืึทื ืกืื ื ืขืก ืฆื ืื ืืขืจืข OSDs. ืฆื ืืึธื ืืึธืก, ืืืืคื ืื ืคืืืืขื ืืข ืงืึทืืึทื ืื:
ceph osd reweight osd.${ID} 0.98
ceph osd reweight osd.${ID} 0.88
ceph osd reweight osd.${ID} 0.78... ืืื ืึทืืื ืืืืฃ ืืื ื ืื.
ืืืึทื ืืึทืืึทื ืกืื ื ืคืืจืืื ืืืึทืืื ืืื ื ืืฉื ืฆื ืคืึทืจืืืจื ืืึทืื. ืืึธืก ืืื ืกืคึผืขืฆืืขื ืืืช ืืืื ืื OSD ืึผืืื ืึท ืืจืืืก ืกืืืข ืคืื โโืืึทืื. ืฆื ืืึทืื ืืืืขืจ ืึทื ื ืึธื ืขืงืกืึทืงืืืืื ื ืื ืงืึทืืึทื ืื reweight ืึทืืฅ ืืื ืืื, ืืืจ ืงืขื ืขื ืคืึทืจืขื ืืืงื ืขืก ceph -s ืึธืืขืจ ืืืืคื ืืื ืึท ืืึทืืื ืืขืจ ืืืึธืงืืึทื ืคึฟืขื ืฆืืขืจ ceph -w ืืื ืกืืจ ืฆื ืึธืืกืขืจืืืืจื ืขื ืืขืจืื ืืขื ืืื ืคืึทืงืืืฉ ืฆืืื.
ืืืขื ืื OSD ืืื "ืขืืคึผืืื", ืืืจ ืงืขื ืขื ืืืื ืืืืึทืืขืจ ืืื ืื ื ืึธืจืืึทื ืึธืคึผืขืจืึทืฆืืข ืฆื ืืึทืืืึทืืืงื ืขืก. ืฆื ืืึธื ืืึธืก, ืึทืจืืืขืจืคืืจื ืื ืืขืืขืื OSD ืฆื ืื ืฉืืึทื down:
ceph osd down osd.${ID}ืืึธืืืจ "ืฆืืขื" ืื OSD ืคึฟืื ืืขื ืงื ืืื:
ceph osd out osd.${ID}ืืื ืก ืืึทืืื ืื OSD ืืื ืกื ืืื ืึทื ืืึธืื ื ืืืึทื ืฆืขืืืืืื ื ืืื ืื FS:
systemctl stop ceph-osd@${ID}
umount /var/lib/ceph/osd/ceph-${ID}ืึทืจืึธืคึผื ืขืืขื OSD ืคืื :
ceph osd crush remove osd.${ID}ืืื ืก ืืืกืืขืงื ืื OSD ืืึทื ืืฆืขืจ:
ceph auth del osd.${ID}ืืื ืืขืกืึธืฃ, ืืึธืื ืืื ืื ืืึทืืืึทืืืงื ืื OSD ืืื:
ceph osd rm osd.${ID}ืืึธื: ืืืื ืืืจ ื ืืฆื Ceph Luminous ืืืขืจืกืืข ืึธืืขืจ ืืขืืขืจ, ืื ืืืืื OSD ืืึทืืืึทืืืงืื ื ืกืืขืคึผืก ืงืขื ืขื ืืืื ืจืืืืกื ืฆื ืฆืืืื ืงืึทืืึทื ืื:
ceph osd out osd.${ID}
ceph osd purge osd.${ID}
ืืืื, ื ืึธื ืงืึทืืคึผืืืืื ื ืื ืกืืขืคึผืก ืืืกืงืจืืืื ืืืืื, ืืืจ ืืืืคื ืื ืืึทืคึฟืขื ceph osd tree, ืขืก ืืึธื ืืืื ืงืืึธืจ ืึทื ืืืืฃ ืื ืกืขืจืืืขืจ ืืื ืื ืึทืจืืขื ืืื ืืืจืืืขืงืึธืื ืขืก ืืขื ืขื ื ืื ืืขืจ OSDs ืคึฟืึทืจ ืืืึธืก ืื ืืืืื ืึทืคึผืขืจืืืฉืึทื ื ืืขื ืขื ืืืจืืืขืงืึธืื:
root@hv-1 ~ # ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 0.46857 root default
-3 0.15619 host hv-1
-5 0.15619 host hv-2
-7 0.15619 host hv-3
2 ssd 0.15619 osd.2 up 1.00000 1.00000
ืืืืฃ ืืขื ืืืขื, ืืึธื ืึทื ืื ืฉืืึทื ืคืื ืื Ceph ืงื ืืื ืืืขื ืืืื ืฆื HEALTH_WARN, ืืื ืืืจ ืืืขืื ืืืื ืืขื ืึท ืคืึทืจืงืืขื ืขืจื ืืื ืื ื ืืืขืจ ืคืื OSDs ืืื ืื ืกืืืข ืคืื โโโโืื ืื ืืืฆื ืืืกืง ืคึผืืึทืฅ.
ืื ืคืืืืขื ืืข ืืืขื ืืึทืฉืจืืึทืื ืื ืกืืขืคึผืก ืืืึธืก ืืืขื ืืืื ืคืืจืืื ืื ืืืื ืืืจ ืืืืื ืฆื ืืึธืจ ืืึทืืื ืื ืกืขืจืืืขืจ ืืื, ืึทืงืึธืจืืื ืืื, ืืึทืืืึทืืืงื ืขืก ืคืื Ceph. ืืื ืืขื ืคืึทื, ืขืก ืืื ืืืืืืืง ืฆื ืืขืืขื ืงืขื ืึทื ืืืืืขืจ ืืืจ ืคืึทืจืืึทืื ืื ืกืขืจืืืขืจ, ืืืจ ืืืื ืืึทืืืึทืืืงื ืึทืืข OSDs ืืืืฃ ืืขื ืกืขืจืืืขืจ.
ืืืื ืขืก ืืขื ืขื ื ืื ืืขืจ OSDs ืืื ืงืก ืืืืฃ ืืขื ืกืขืจืืืขืจ, ื ืึธื ืจืืืืืืื ื ืืื, ืืืจ ืืึทืจืคึฟื ืฆื ืืืกืฉืืืกื ืื ืกืขืจืืืขืจ ืคืื ืื OSD ืืึทืคึผืข hv-2ืืืจื ืืืืคื ืื ืคืืืืขื ืืข ืืึทืคึฟืขื:
ceph osd crush rm hv-2
ืืืกืืขืงื mon ืคืื ืื ืกืขืจืืืขืจ hv-2ืืืจื ืืืืคื ืื ืืึทืคึฟืขื ืืื ืื ืืืืฃ ืื ืื ืืขืจ ืกืขืจืืืขืจ (ื"ื ืืื ืืขื ืคืึทื, ืืืืฃ hv-1):
ceph-deploy mon destroy hv-2ื ืึธื ืืขื, ืืืจ ืงืขื ืขื ืืึทืืื ืื ืกืขืจืืืขืจ ืืื ืึธื ืืืืื ืกืึทืืกืึทืงืืืึทื ื ืึทืงืฉืึทื ื (ืฉืืึทืขื-ืืืคึผืืืืื ื ืขืก, ืืื"ื ื).
ืคืึทื ื ืืืขืจ 2. ืคืึทืจืฉืคึผืจืืืืื ื ืคืื ืืืกืง ืคึผืืึทืฅ ืืื ืึท ืฉืืื ืืืฉืืคื Ceph ืงื ืืื
ืืื ืืืขื ืึธื ืืืืื ืื ืจืืข ืืขืฉืืืืข ืืื ืึท ืืงืืื ืืืขืื ืคึผื (). ืื ืืืืคึผื ืจืึธืืข ืคืื โโPG ืืื Ceph ืืื ืืคึฟืจื ืฆื ืฆืื ืืืคืืืกื Ceph ืึทืืืืฉืขืงืฅ ืืื ืืืืึทืืขืจ ืจืขืคึผืืึทืงืืื ืืื ืืื OSD. ืื ืคืึธืจืืืืข ืืื ืืืึธืก ืืืจ ืงืขื ืขื ืจืขืืขื ืขื ืื ืคืืจืืื ืื ืกืืืข ืคืื โโPG ืืื ืืื Ceph ืืึทืงืืืืขื ืืืืฉืึทื. ื ื ืืืืืง ืข ืคืจืื ืข ืฐืขืจ ื ืืื ื ืืืจ ื ืืืกืงืืืืจื , ืื ื ืกืคืขืฆืืคืืฉ ืข ืืืืฉืคืืื .
ืึทืืื: ืืืื ืขืจ ืคืื ืื ืคึผืจืึธืกื ืคึผืจืึธืืืขืืก ืืืขื ืืืจ ื ืืฆื Ceph ืืื ืื ืึทื ืืึทืืึทื ืกื ื ืืืขืจ ืคืื OSD ืืื PG ืฆืืืืฉื ืคึผืึธืึธืืก ืืื Ceph.
ืคืืจืกืืื, ืืืืึทื ืคืื ืืขื, ืึท ืกืืืืึทืฆืืข ืงืขื ืืืืคืฉืืืื ืืืขื ืฆื ืคืืืข ืคึผืืก ืืขื ืขื ืกืคึผืขืกืืคืืขื ืืื ืึท ืงืืืื ืืขืงื, ืืืึธืก ืืื ืืกืขื ืฉืึทืื ืึท ืืจืึทืฉืึทื ืึทื ื ืืฆื ืคืื ืืืกืง ืคึผืืึทืฅ ืืื ืืขื ืงื ืืื. ืฆืืืืืื ืก, ืืื ืคืืจ ืขืก ืืื ืึท ืืขืจ ืขืจื ืกื ืคึผืจืึธืืืขื: ืืึทืื ืืืืคื ืืื ืืืื ืขืจ ืคืื ืื OSDs. ืืึธืก ืื ืืืืื ืื ืงื ืืื ืขืจืฉืืขืจ ืืืขืจืืึทื ื ืฆื ืื ืฉืืึทื HEALTH_WARN, ืืื ืืื HEALTH_ERR. ืื ืกืืื ืคึฟืึทืจ ืืขื ืืื ืึทื Ceph, ืืืขื ืงืึทืืงืืึทืืืืืื ื ืื ืื ืืืฆื ืกืืืข ืคืื โโืืึทืื (ืืืจ ืงืขื ืขื ืืขืคึฟืื ืขื ืขืก ืืืจื MAX AVAIL ืืื ืื ืืึทืคึฟืขื ืจืขืืืืืึทื ceph df ืคึฟืึทืจ ืืขืืขืจ ืืขืงื ืกืขืคึผืขืจืึทืืื) ืืื ืืืืืจื ืืืืฃ ืื ืกืืืข ืคืื โโืื ืืืฆื ืืึทืื ืืื ืื OSD. ืืืื ืขืก ืืื ื ืืฉื ืืขื ืื ืคึผืืึทืฅ ืืื ืืคึผืืืช ืืืื OSD, ื ืื ืืขืจ ืืึทืื ืงืขื ืขื ืืืื ืืขืฉืจืืื ืืื ืื ืืึทืื ืืขื ืขื ืจืขืื ืฆืขืฉืืงื ืฆืืืืฉื ืึทืืข OSDs.
ืขืก ืืื ืืืขืจื ืงืืขืจืึทืคืืืื ื ืึทื ืื ืคึผืจืึธืืืขืืก ืืขื ืขื ืืึทืจืืขืื ืืึทืฉืืึธืกื ืืื ืื Ceph ืงื ืืื ืงืึทื ืคืืืืขืจืืืฉืึทื ืืื ืข. ืืืื ืขืจ ืคืื ืื ืืืฉืืจืื ืืืจ ืงืขื ืขื ื ืืฆื ืืื . ืืื ืืืึทื ืืืืฃ, ืื ืคืืจืืื ืื ืกืืืข ืคืื โโืคึผื ืืื ืงืืืจ ืงืึทืืงืืึทืืืืืื. ืึธืืขืจ, ืืืจ ืงืขื ืขื ืืืื ืจืืืึธืจื ืฆื ืขืก ืืื ืึท ืกืืืืึทืฆืืข ืืื ืื Ceph ืงื ืืื ืฉืืื ืงืึทื ืคืืืืขืจื ืคืึทืืฉ. ืขืก ืืื ืืืขืจื ืฆื ืืขืจืงืืขืจื ืืึธ ืึทื ืืื ืึท ืืืื ืคืื ืื ืคืึทืจืจืืืื ืึทืจืืขื ืืืจ ืืืขื ืืืกืืึธืืข ืืึทืจืคึฟื ืฆื ืจืขืืืฆืืจื ืื ื ืืืขืจ ืคืื ืคึผืืก, ืืื ืื ืฉืืจืื ืืื ื ืืฉื ืื ืืืฆื ืืื ืขืืืขืจืข ืืืขืจืกืืขืก ืคืื Ceph (ืขืก ืืื ืืืืื ืืจืืืก ืืื ืืืขืจืกืืข ).
ืึทืืื, ืืึธืื ืืื ืื ืืืึทืืืฉืึทื ืื ืคืืืืขื ืืข ืืืื: ืืขืจ ืงื ืืื ืืื ืึท ืกืืึทืืืก HEALTH_WARN ืจืขืื ืฆื ืืืื ืขืจ ืคืื ืื OSD ืคืืืกื ืืืง ืืืืก ืคืื ืคึผืืึทืฅ. ืืึธืก ืืืขื ืืืื ืื ืืขืืืืื ืืืจื ืึท ืืขืืช HEALTH_WARN: 1 near full osd. ืื ืืขืจ ืืื ืึท ืึทืืืขืจืืืึทื ืคึฟืึทืจ ืืึทืงืืืขื ืืืืก ืคืื ืืขื ืกืืืืึทืฆืืข.
ืขืจืฉืืขืจ ืคืื ืึทืืข, ืืืจ ืืึทืจืคึฟื ืฆื ืคืึทืจืฉืคึผืจืืืื ืื ืื ืืืฆื ืืึทืื ืฆืืืืฉื ืื ืจืืขื OSDs. ืืืจ ืืึธืื ืฉืืื ืืืจืืืขืงืึธืื ืึท ืขื ืืขื ืึธืคึผืขืจืึทืฆืืข ืืื ืืขืจ ืขืจืฉืืขืจ ืคืึทื, ืืืขื ืืืจ "ืืืกืืขืฉืขืคึผื" ืื ื ืึธืืข - ืืื ืืขืจ ืืืืื ืืืืืง ืึทื ืืืฆื ืืืจ ืืึทืจืคึฟื ืฆื ืึท ืืืกื ืจืขืืืฆืืจื reweight. ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืึทืจืืืฃ ืฆื 0.95:
ceph osd reweight osd.${ID} 0.95ืืึธืก ืคืจืื ืืืกืง ืคึผืืึทืฅ ืืื ืื OSD ืืื ืคืืงืกืื ืื ืืขืืช ืืื ืกืขืฃ ืืขืืื ื. ืึธืืขืจ, ืืื ืฉืืื ืืขืจืืื ื, ืืขื ืคึผืจืึธืืืขื ืืขืจ ืืืืคึผื ืึทืงืขืจื ืจืขืื ืฆื ืคืึทืืฉ ืงืึทื ืคืืืืขืจืืืฉืึทื ืคืื Ceph ืืื ืื ืขืจืฉื ืกืืึทืืขืก: ืขืก ืืื ืืืืขืจ ืืืืืืืง ืฆื ืืึทืื ืึท ืจืืงืึทื ืคืืืืขืจืืืฉืึทื ืึทืืื ืึทื ืขืก ืืืขื ื ืืฉื ืืขืจืฉืืึทื ืขื ืืื ืืขืจ ืฆืืงืื ืคึฟื.
ืืื ืืื ืืืขืจ ืืึทืืื ืืขืจ ืคืึทื, ืขืก ืึทืืข ืืขืงืืืขื ืึทืจืึธืคึผ ืฆื:
- ืืืขืจื ืฆื ืืืื
replication_countืืื ืืืื ืขืจ ืคืื ืื ืืขืงื, - ืฆื ืคืื ืคึผื ืืื ืืืื ืืขืงื ืืื ืฆื ืงืืืื ืืื ืื ืืขืจื.
ืืื ืก ื ืืฆื ืื ืฉืืื ืืขืจืืื ื ืงืึทืืงืืืึทืืึธืจ. ืขืก ืืืืืื ืงืืืจ ืืืึธืก ืืึทืจืฃ ืืืื ืืจืืื ืืื ืืื ืคึผืจืื ืฆืืคึผ, ืขืก ืืื ืืึธืจื ืืฉื ืงืึธืืคึผืืืฆืืจื. ื ืึธื ืฉืืขืื ืื ื ืืืืืง ืคึผืึทืจืึทืืขืืขืจืก, ืืืจ ืืึทืงืืืขื ืื ืคืืืืขื ืืข ืจืขืงืึทืืึทื ืืืืฉืึทื ื:
ืืึธื: ืืืื ืืืจ ืฉืืขืื ืึทืจืืืฃ ืึท Ceph ืงื ืืื ืคึฟืื ืงืจืึทืฆื, ืื ืื ืืขืจ ื ืืฆืืง ืคึฟืื ืงืฆืืข ืคืื โโืื ืงืึทืืงืืืึทืืึธืจ ืืื ืื ืืืจ ืคืื ืงืึทืืึทื ืื ืืืึธืก ืืึทืื ืคึผืึธืึธืืก ืคึฟืื ืงืจืึทืฆื ืืื ืื ืคึผืึทืจืึทืืขืืขืจืก ืกืคึผืขืกืืคืืขื ืืื ืื ืืืฉ.
ืื ืืขืฆืืข ืืืึทื ืืขืืคึผืก ืืืจ ื ืึทืืืืืืจื - ืกืึทืืืืฉืขืกืืื ืคึผื ืืจืืฃ. ืืื ืืื ืืืขืจ ืคืึทื, ืื ืจืืข ืืืื ืขืจ ืืื ืืืื ื ืืฆืืง, ืืื ืื ืจืขืคึผืืึทืงืืืฉืึทื ืคึผืึทืจืึทืืขืืขืจ ืืื ืื ืืขืืืืื, ืืื ื ืืืจ ืืึทืฉืืึธืกื ืฆื ืืืืฉื ืื ืจืขืคึผืืึทืงืืืฉืึทื ืืึทืืืึทืคึผืืืืขืจ.
ืึทืืื, ืขืจืฉืืขืจ ืืืจ ืืึทืจืคึฟื ืฆื ืืืืฉื ืื ืจืขืคึผืืึทืงืืืฉืึทื ืคึผืึทืจืึทืืขืืขืจืก - ืืึธืก ืืื ืืืขืจื ืืื ืขืจืฉืืขืจ, ืืืืึทื ืืืจื ืจืืืืกืื ื ืื ืืึทืืืึทืคึผืืืืขืจ, ืืืจ ืคืจืื ืืืกืง ืคึผืืึทืฅ. ืืืขื ืื ืืึทืคึฟืขื ืขืงืกืึทืงืืืืึทื, ืืืจ ืืืขื ืืึทืืขืจืงื ืึทื ืื ืคืึทืจืึทื ืขื ืืืกืง ืคึผืืึทืฅ ืืืขื ืคืึทืจืืจืขืกืขืจื:
ceph osd pool $pool_name set $replication_size
ืืื ื ืึธื ืืืึทื ืงืึทืืคึผืืืฉืึทื, ืืืจ ืืืืฉื ืื ืคึผืึทืจืึทืืขืืขืจ ืืืึทืืืขืก pg_num ะธ pgp_num ืืื ืืืื:
ceph osd pool set $pool_name pg_num $pg_number
ceph osd pool set $pool_name pgp_num $pg_numberืขืก ืก ืืืืืืืง: ืืืจ ืืืื ืืืืฉื ืื ื ืืืขืจ ืคืื ืคึผืืก ืกืึทืงืืืขื ืืฉืึทืื ืืื ืืขืืขืจ ืืขืงื ืืื ื ืืฉื ืืืืฉื ืื ืืืึทืืืขืก ืืื ืื ืืขืจืข ืคึผืึธืึธืืก ืืื ืื ืืืึธืจื ืื ืื ืคืึทืจืฉืืืื ืื "ืืขืืจืึทืืขื ืืึทืื ืืืขืจืืงืืึทื" ะธ "n-ื ืืืขืจ ืคืื ืคึผืืก ืืืืจืืืืื".
ืืืจ ืงืขื ื ืืืื ืงืึธื ืืจืึธืืืจื ืึทื ืึทืืฅ ืืื ืืื ืืื ืื ืืึทืคึฟืขื ืึทืืืคึผืืฅ ceph health detail ะธ ceph -s.
ืคืื ื ืื 3 . ืืืืจืืืืื ื ืึท ืืืืจืืืึทื ืืึทืฉืื ืคืื LVM ืฆื Ceph RBD
ืืื ืึท ืกืืืืึทืฆืืข ืืืขื ืึท ืคึผืจืืืขืงื ื ืืฆื ืืืืจืืืึทื ืืืฉืื ืขื ืืื ืกืืึทืืืจื ืืืืฃ ืจืขื ืืึทื ื ืึทืงืขื-ืืขืืึทื ืกืขืจืืืขืจืก, ืื ืึทืจืืืกืืขืื ืคืื ืฉืืื-ืืึธืืขืจืึทื ื ืกืืึธืจืืืืฉ ืึธืคื ืขืจืืืืึทื. ืขืก ืืื ืืืื ืืืืขืจ ืืืืืืขืจืึทืืึทื ืึทื ืขืก ืืื ืืขื ืื ืคึผืืึทืฅ ืืื ืืขื ืกืืึธืจืืืืฉ ... ืื ืื ืืขืจ ืคึผืจืึธืกื ืกืืืืึทืฆืืข: ืขืก ืืื ืึท ืืืืจืืืึทื ืืึทืฉืื ืืื ืืืืข ืกืืึธืจืืืืฉ ืืืืฃ ืื ืกืขืจืืืขืจ ืืื ืืืจ ืืึทืจืคึฟื ืฆื ืืงืกืคึผืึทื ื ืื ืืืกืง, ืึธืืขืจ ืขืก ืืื ืื ืขืจืืขืฆ ื ืื ืฆื ืืืื, ืืืืึทื ืขืก ืืื ืงืืื ืคืจืื ืืืกืง ืคึผืืึทืฅ ืืื ืงืก ืืืืฃ ืื ืกืขืจืืืขืจ.
ืื ืคึผืจืึธืืืขื ืงืขื ืขื ืืืื ืกืึทืืืื ืืื ืคืึทืจืฉืืืขื ืข ืืืขืื - ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืืจื ืืืืืจืืืืื ื ืฆื ืื ืื ืืขืจ ืกืขืจืืืขืจ (ืืืื ืขืก ืืื ืืืื ืขืจ) ืึธืืขืจ ืึทืืื ื ื ืืึท ืืืกืงืก ืฆื ืื ืกืขืจืืืขืจ. ืึธืืขืจ ืขืก ืืื ื ืื ืฉืืขื ืืืง ืืขืืืขื ืฆื ืืึธื ืืึธืก, ืึทืืื ืืืืจืืืืื ื ืคืื LVM ืฆื Ceph ืงืขื ืขื ืืืื ืึท ืืืกืืขืฆืืืื ื ืืืืืื ื ืฆื ืืขื ืคึผืจืึธืืืขื. ืืืจื ืืฉืืืื ื ืืขื ืึธืคึผืฆืืข, ืืืจ ืืืื ืคืึทืจืคึผืึธืฉืขืืขืจื ืื ืืืืึทืืขืจ ืคึผืจืึธืฆืขืก ืคืื ืืืืืจืืืฉืึทื ืฆืืืืฉื ืกืขืจืืืขืจืก, ืืืืึทื ืขืก ืืืขื ืืืื ื ืื ืืึทืจืคึฟื ืฆื ืึทืจืืืขืจืคืืจื ืืืืข ืกืืึธืจืืืืฉ ืคืื ืืืื ืืืืคึผืขืจืืืืืืขืจ ืฆื ืื ืืขืจื. ืืขืจ ืืืืื ืืึทืคึผื ืืื ืึทื ืืืจ ืืืขื ืืึธืื ืฆื ืืึทืืื ืื VM ืืฉืขืช ืื ืึทืจืืขื ืืื ืืืจืืืขืงืึธืื.
ืื ืคืืืืขื ืืข ืจืขืฆืขืคึผื ืืื ืืขื ืืืขื ืคืื , ืื ืื ืกืืจืึทืงืฉืึทื ื ืคืื ืืืึธืก ืืขื ืขื ืืขืกืืขื ืืื ืงืึทืืฃ. ืืื ืืืื ืืืขืืก, ืืขืจ ืืืคึฟื ืคืื ืืึทืกืึทื-ืคืจืื ืืืืจืึทืืืึธื ืืื ืืืื ืืืกืงืจืืืื ืืึธืจืืืืขืจ ืืื ืืื ืืืขืจ ืคืึทื ืขืก ืืื ืคืฉืื ื ืืฉื ืืืจืฃ, ืึทืืื ืืืจ ืืึธืื ื ืืฉื ืงืึธื ืืจืึธืืืจื ืขืก. ืืืื ืืึธืก ืืื ืงืจืืืืฉ ืคึฟืึทืจ ืืืื ืคึผืจืืืขืงื, ืืืจ ืืืขืื ืืืื ืฆืืคืจืืื ืฆื ืืขืจื ืืืขืื ืื ืจืขืืืืืึทืื ืืื ืื ืืึทืืขืจืงืื ืืขื.
ืืึธืืืจ ืืืื ืฆื ืื ืคึผืจืึทืงืืืฉ ืืืื. ืืื ืืขื ืืืึทืฉืคึผืื ืืืจ ื ืืฆื virsh ืืื, ืึทืงืึธืจืืื ืืื, libvirt. ืขืจืฉืืขืจ, ืืึทืื ืืืืขืจ ืึทื ืื Ceph ืืขืงื ืฆื ืืืึธืก ืื ืืึทืื ืืืขื ืืืื ืืืืืจืืืืื ืืื ืงืึธื ื ืขืงืืขื ืฆื libvirt:
virsh pool-dumpxml $ceph_poolืื ืืขืงื ืืึทืฉืจืืึทืืื ื ืืืื ืึทื ืืืึทืืื ืงืฉืจ ืืึทืื ืฆื Ceph ืืื ืืขืจืืืืืขื ืืฉ ืืึทืื.
ืืขืจ ืืืืึทืืขืจ ืฉืจืื ืืื ืึทื ืื LVM ืืืื ืืื ืงืึธื ืืืขืจืืขื ืฆื Ceph RBD. ืืืจืืคืืจืื ื ืฆืืื ืืขืคึผืขื ืืก ืืคึฟืจื ืืืืฃ ืื ืืจืืืก ืคืื ืืขื ืืืื:
qemu-img convert -p -O rbd /dev/main/$vm_image_name rbd:$ceph_pool/$vm_image_nameื ืึธื ืื ืงืึทื ืืืขืจืืฉืึทื, ืึท LVM ืืืื ืืืขื ืืืืึทืื, ืืืึธืก ืืืขื ืืืื ื ืืฆืืง ืืืื ืืืืจืืืืื ื ืื VM ืฆื RBD ืคืืืื ืืื ืืืจ ืืึธืื ืฆื ืฆืืจืืงืงืจืืื ืื ืขื ืืขืจืื ืืขื. ืืืื, ืฆื ืงืขื ืขื ืืขืฉืืืื ื ืฆืืจืืงืงืจืืื ืขื ืืขืจืื ืืขื, ืืึธืื ืืื ืื ืืึทืื ืึท ืืึทืงืึทืคึผ ืคืื ืื ืืืืจืืืึทื ืืึทืฉืื ืงืึทื ืคืืืืขืจืืืฉืึทื ืืขืงืข:
virsh dumpxml $vm_name > $vm_name.xml
cp $vm_name.xml $vm_name_backup.xml
... ืืื ืจืขืืึทืืืจื ืื ืึธืจืืืื ืขื (vm_name.xml). ืืึธืืืจ ืืขืคึฟืื ืขื ืึท ืืืึธืง ืืื ืึท ืืึทืฉืจืืึทืืื ื ืคืื ืื ืืืกืง (ืกืืึทืจืฅ ืืื ืื ืฉืืจื <disk type='file' device='disk'> ืืื ืขื ืืก ืืื </disk>) ืืื ืจืขืืืฆืืจื ืขืก ืฆื ืื ืคืืืืขื ืืข ืคืึธืจืขื:
<disk type='network' device='disk'>
<driver name='qemu'/>
<auth username='libvirt'>
<secret type='ceph' uuid='sec-ret-uu-id'/>
</auth>
<source protocol='rbd' name='$ceph_pool/$vm_image_name>
<host name='10.0.0.1' port='6789'/>
<host name='10.0.0.2' port='6789'/>
</source>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</disk>ืืื ืก ืงืืง ืืืึท ืขืืืขืืข ืคืจืืื:
- ืฆื ืืขื ืคึผืจืึธืืึธืงืึธื
sourceืื ืึทืืจืขืก ืฆื ืื ืกืืึธืจืืืืฉ ืืื Ceph RBD ืืื ืื ืืขืืืืื (ืืึธืก ืืื ืื ืึทืืจืขืก ืืืึธืก ืื ืืืงืืืฅ ืื ื ืึธืืขื ืคืื ืื Ceph ืืขืงื ืืื ืื RBD ืืืื, ืืืึธืก ืืื ืืขืืืขื ืืืฉืืืกื ืืื ืืขืจ ืขืจืฉืืขืจ ืืื ืข). - ืืื ืื ืืืึธืง
secretืืืคึผ ืืื ืื ืืขืืืืืceph, ืืื ืืขืืื ื ืืื ืื UUID ืคืื ืื ืกืื ืฆื ืคืึทืจืืื ืื ืฆื ืขืก. ืืืื ืืืื ืงืขื ืขื ืืืื ืืขืคึฟืื ืขื ืืื ืืขื ืืึทืคึฟืขืvirsh secret-list. - ืืื ืื ืืืึธืง
hostืื ืืืขื ืื ืฆื ืื Ceph ืืึธื ืืืึธืจืก ืืขื ืขื ืื ืืขืืืืื.
ื ืึธื ืขืืืืื ื ืื ืงืึทื ืคืืืืขืจืืืฉืึทื ืืขืงืข ืืื ืงืึทืืคึผืืืืื ื ืื LVM ืฆื RBD ืงืึทื ืืืขืจืืฉืึทื, ืืืจ ืงืขื ืขื ืฆืืืืืื ืื ืืึทืืึทืคืืื ืงืึทื ืคืืืืขืจืืืฉืึทื ืืขืงืข ืืื ืึธื ืืืืื ืื ืืืืจืืืึทื ืืึทืฉืื:
virsh define $vm_name.xml
virsh start $vm_name
ืขืก ืืื ืฆืืื ืฆื ืงืึธื ืืจืึธืืืจื ืึทื ืื ืืืืจืืืึทื ืืึทืฉืื ืกืืึทืจืืขื ืจืืืืืง: ืืืจ ืงืขื ืขื ืืขืคึฟืื ืขื ืขืก, ืืืฉื, ืืืจื ืงืึทื ืขืงืืื ื ืฆื ืขืก ืืืจื SSH ืึธืืขืจ ืืืจื virsh.
ืืืื ืื ืืืืจืืืึทื ืืึทืฉืื ืึทืจืืขื ืจืืืืืง ืืื ืืืจ ืืึธื ื ืืฉื ืืขืคึฟืื ืขื ืงืืื ืื ืืขืจืข ืคึผืจืึธืืืขืืก, ืืืจ ืงืขื ืขื ืืืกืืขืงื ืื LVM ืืืื ืืืึธืก ืืื ื ืื ืืขืจ ืืขืืืืื ื:
lvremove main/$vm_image_nameืกืึธืฃ
ืืืจ ืืึธืื ืืขืคึผืืึธื ืืขืจื ืึทืืข ืื ืืืกืงืจืืืื ืงืึทืกืขืก ืืื ืคืืจ - ืืืจ ืืึธืคึฟื ืึทื ืื ืื ืกืืจืึทืงืฉืึทื ื ืืืขื ืืขืืคื ืื ืืขืจืข ืึทืืืื ืืกืืจืึทืืึธืจืก ืกืึธืืืืข ืขื ืืขื ืคึผืจืึธืืืขืืก. ืืืื ืืืจ ืืึธื ืืึทืืขืจืงืื ืืขื ืึธืืขืจ ืื ืืขืจืข ืขื ืืขื ืืขืฉืืืช ืคืื ืืืื ืืขืจืคืึทืจืื ื ืืื Ceph, ืืืจ ืืืขืื ืืืื ืฆืืคืจืืื ืฆื ืืขื ืืื ืืื ืื ืืึทืืขืจืงืื ืืขื!
ืคึผืก
ืืืืขื ืขื ืืืื ืืืืฃ ืืื ืืืขืจ ืืืึธื:
- ยซ";
- ยซ";
- ยซ";
- ยซ'.
ืืงืืจ: www.habr.com