ืืขื ืขื ืขืก ืฆืืืืฉื ืืื ืื (ืฆืขืคืึธืืืึธืืึธืื) ืืืึธืก ืืึธื ื ืื ืืื "ืคืึทืืืึทื ืขืงืกืืจืขื"?
ืขืก ืืื ืึทื ืืืืงืื - ืึทื ืืขืจืฉ ืืืจ ืืืึธืื ื ืืฉื ืืืื ืืึทืืืึทืืื ื ืืื ืืขื ืืึธืจ ืืฉืืงืึทืืืข ืืื ืืึธืื ืข ืคึผืจืึธืืืงื.
ืคืืืข ืคืื โโโโืื ืืืึธืก ืืขื ืขื ืื ืืืึทืืืื ืืื ืื ืึธืคึผืขืจืึทืฆืืข ืคืื โโCeph ืืึธืื ืืขืคึฟืื ืขื ืึท ื ืื ืืืืขืจ ืึธืคื (ืึธืืขืจ ืืื ืืคืืื ืืืืขืจ ืืขืืื) ืึธืืขืจ ืืื ืืื ืืึธื ืขื ืคืึทื - ืงืึทื ืขืงืืื ื Ceph ืืืจื iSCSI ืึธืืขืจ FC. ืฆื ืืืืก? ื ื, ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืคืึธืจืืืืื ืึท ืืืื ืคึฟืื Ceph ืฆื ืึท Windows ืึธืืขืจ ืกืึธืืึทืจืืก ืกืขืจืืืขืจ ืืืึธืก ืืื ื ืึธื ื ืืฉื ืืืืจืืืึทืืืืขื ืคึฟืึทืจ ืขืืืขืืข ืกืืื. ืึธืืขืจ ืึท ืืืืจืืืึทืืืืขื, ืึธืืขืจ ื ืืฆื ืึท ืืืืคึผืขืจืืืืืืขืจ ืืืึธืก ืงืขื ื ืืฉื ืืึธื Ceph - ืืื, ืืื ืืืจ ืืืืกื, ืขืก ืืขื ืขื ืคืืืข ืคืื โโโโืืื. ืืืฉื? ื ื, ืคึฟืึทืจ ืืืึทืฉืคึผืื, HyperV ืึธืืขืจ ESXi, ืืืึธืก ืืขื ืขื ืึทืงืืืืืื ืืขื ืืฆื. ืืื ืืืื ืื ืึทืจืืขื ืขืจืืืืึทื ืคืื ืืื ืขื ืึท ืืืื ืคืื ืกืขืฃ ืฆื ืึท ืืึทืกื ืืึทืฉืื, ืืึธืก ืืืจื ืก ืืื ืึท ืืืืขืจ ืืงืกืืืืื ื ืึทืจืืขื.
ืึทืืื, ืืขืืขืื:
- ืึท ืฉืืื ืคืืืกื ืืืง ืกืขืฃ ืงื ืืื
- ืึท ืฉืืื ืืืืืกืืื ื ืืืื ืืืึธืก ืืืื ืืืื ืืขืืื ื ืืืจื iSCSI
- ืืขืงื ื ืึธืืขื mypool, ืืืื ื ืึธืืขื ืืืื ืืืื
ืึธื ืืืืื?
ืขืจืฉืืขืจ ืคืื ืึทืืข, ืืืขื ืืืจ ืจืขืื ืืืขืื FC ืึธืืขืจ iSCSI, ืืืจ ืืึธืื ืึทืืึท ืขื ืืืืื ืืื ืื ืืฉืืืืืขืจ ืืื ืฆืื. ืฆืื ืืื ืคืืงืืืฉ ืึท ืกืขืจืืืขืจ, ืื ืืฉืืืืืขืจ ืืื ืึท ืงืืืขื ื. ืืื ืืืขืจ ืึทืจืืขื ืืื ืฆื ืคืึธืจืืืืื ืื Ceph ืืืื ืฆื ืื ืื ืืฉืืืืืขืจ ืืื ืืื ืืืึทื ืื. ืืขื ืืืื ืืืจ ืืืื ืืงืกืคึผืึทื ื ืฆืื. ืึธืืขืจ ืืื, ืืืืฃ ืืืึธืก ืงืึธืืคึผืืืืขืจ?
ืฆืื ืืืืง, ืืื ืึท Ceph ืงื ืืื ืืืจ ืืึธืื ืืืึท ืืื ืืกืืขืจ ืืืื ืงืึธืืคึผืึธื ืขื ื ืืืขืืขื ืก IP ืึทืืจืขืก ืืื ืคืึทืจืคืขืกืืืงื ืืื ืืืืฃ ืืืึธืก ืืืื ืขืจ ืคืื ืื ืืขืจืกื ืืืืืืืง ืงืึทืืคึผืึธืื ืึทื ืฅ ืคืื Ceph ืืื ืงืึทื ืคืืืืขืจื, ืืื ืืขืจ ืงืึธืืคึผืึธื ืขื ื ืืื ืื ืืึธื ืืืึธืจ. ืึทืงืงืึธืจืืื ืืื, ืืืจ ืื ืกืืึทืืืจื ืึท iSCSI ืฆืื ืืืืฃ ืื ืืึธื ืืืึธืจ (ืืื ืึทื ืื ืืฉืืืืืขืจ ืืื ืืขืจ ืืขืืืืงืขืจ ืฆืืื, ืืคึผืืืช ืคึฟืึทืจ ืืขืกืฅ). ืืื ืืึธื ืืึธืก ืืขืืึธื ืืืืฃ CentOS, ืึธืืขืจ ืื ืืืืืื ื ืืื ืืืื ืคึผืึทืกืืง ืคึฟืึทืจ ืงืืื ืื ืืขืจืข ืคืึทืจืฉืคึผืจืืืืื ื - ืืืจ ื ืึธืจ ืืึทืจืคึฟื ืฆื ืื ืกืืึทืืืจื ืื ืคึผืึทืงืึทืืืฉืึทื ืืื ืคึผืึทืกืืง ืืื ืืืื ืคืึทืจืฉืคึผืจืืืืื ื.
# yum -y install iscsi-initiator-utils targetcli
ืืืึธืก ืืื ืืขืจ ืฆืื ืคืื ืื ืืื ืกืืึทืืืจื ืคึผืึทืงืึทืืืฉืึทื?
- targetcli - ืึท ื ืืฆื ืคึฟืึทืจ ืึธื ืคืืจืื ื ืื SCSI ืฆืื ืืขืืืื ืืื ืื ืืื ืืงืก ืงืขืจื
- iscsi-initiator-utils - ืึท ืคึผืขืงื ืืื ืืืืืืึทืืื ืืขื ืืฆื ืฆื ืคืืจื ืื iSCSI ืื ืืฉืืืืืขืจ ืืขืืืื ืืื ืื ืืื ืืงืก ืงืขืจื
ืืื ืกืืจ ืฆื ืคืึธืจืืืืื ืึท ืืืื ืืืจื iSCSI ืฆื ืื ืื ืืฉืืืืืขืจ, ืขืก ืืขื ืขื ืฆืืืื ืึธืคึผืฆืืขืก ืคึฟืึทืจ ืึทื ืืืืืงืืื ื ืคืื ืืขืฉืขืขื ืืฉื - ื ืืฆื ืื ืืึทื ืืฆืขืจ-ืกืคึผืึทืกืข ืืึทืงืขื ื ืคืื ืื ืฆืื ืึธืืขืจ ืคืึทืจืืื ืื ืื ืืืื ืืื ืึท ืืืึธืง ืืืื ืงืขื ืืืง ืฆื ืื ืึธืคึผืขืจืืืืื ื ืกืืกืืขื ืืื ืึทืจืืืกืคืืจื ืขืก ืืืจื iSCSI. ืืืจ ืืืขืื ืืืื ืื ืจืืข ืืืขื - ืื ืืึทื ืืฆืขืจ-ืกืคึผืึทืกืข ืืึทืงืขื ื ืืื ื ืึธื ืืื ืึท "ืขืงืกืคึผืขืจืืืขื ืืึทื" ืฉืืึทื ืืื ืืื ืึท ืืืกื ื ืืฉื ืืจืืื ืคึฟืึทืจ ืคึผืจืึธืืืงืืืื ื ืืฆื. ืืื ืืขืจืฆื, ืขืก ืืขื ืขื ืคึผืืืคืึธืื ืืื ืืื, ืืืขืื ืืืึธืก ืืืจ ืงืขื ืขื ืจืขืื ืึท ืคึผืืึทืฅ ืืื (ืึธื ืืจืืื!) ืืืึทื ืขื.
ืืืื ืืืจ ื ืืฆื ืืคืืื ืึท ืืืกื ืกืืึทืืื ืคืึทืจืฉืคึผืจืืืืื ื ืืื ืึท ืืึทื ื ืฉืืืฆื ืฆืืงื, ืืขืจ ืงืขืจื ืืืึธืก ืืืจ ืืึธืื ืืื ืึทื ืืืืข, ืืืืข ืืืขืจืกืืข. ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืื CentOS7 ืขืก ืืื 3.10.*, ืืื CentOS8 ืขืก ืืื 4.19. ืืื ืืืจ ืืขื ืขื ืืื ืืขืจืขืกืืจื ืืื ืึท ืงืขืจื ืคืื ืืืึท ืืื ืืกืืขืจ 5.3 (ืึธืืขืจ ืืึทื ืฅ 5.4) ืืื ื ืืึทืขืจ. ืคืืจืืืืก? ืืืืึทื ืืืจื ืคืขืืืงืืึทื Ceph ืืืืืขืจ ืืึธืื ืึท ืืึทื ื ืคืื ืึธืคึผืฆืืขืก ืืืึธืก ืืขื ืขื ื ืืฉื ืงืึทืืคึผืึทืืึทืืึทื ืืื ืขืืืขืจืข ืงืขืจื ืึทืื. ืืขื ืืืื ืึทื ืืืจ ืคืึทืจืืื ืื ืึท ืจืืคึผืึทืืึทืืึธืจื ืืื ืึท ื ืืึทืข ืงืขืจื ืคึฟืึทืจ ืืื ืืืขืจ ืคืึทืจืฉืคึผืจืืืืื ื (ืืืฉื ืคึฟืึทืจ CentOS ืืึธืก ืืื ืขืืจืขืคึผืึธ), ืื ืกืืึทืืืจื ืื ื ืืึทืข ืงืขืจื ืืื ืจืขืืึธืึธื ืื ืกืืกืืขื ืฆื ืึทืจืืขืื ืืื ืื ื ืืึทืข ืงืขืจื:
- ืคืึทืจืืื ืื ืฆื ืื ืืึธื ืืืึธืจ ืืืืกืืขืงืืืื ืคึฟืึทืจ ืืขืจ ืขืงืกืคึผืขืจืืืขื ื
- ืืืจ ืคืึทืจืืื ืื ืขืืจืขืคึผืึธ ืจืืคึผืึทืืึทืืึธืจืื ืืืื ืื ืื ืกืืจืึทืงืฉืึทื ื -
- ืื ืกืืึทืืืจื ืื ืงืขืจื: yum -y โenablerepo=elrepo-kernel install kernel-ml
- ืจืขืืึธืึธื ืื ืกืขืจืืืขืจ ืืื ืื ืืึธื ืืืึธืจ (ืืืจ ืืึธืื ืืจืื ืืึธื ืืืึธืจืก, ืจืขืื?)
ืงืึทื ืขืงืืื ื ืื ืืืื ืืื ืึท ืืืึธืง ืืืื
# rbd map mypool/myimage
/dev/rbd0
ืึทืืข ืืืึธืก ืืืืืื ืืื ืฆื ืงืึทื ืคืืืืขืจ ืื ืฆืื. ืืื ืืขื ืืืึทืฉืคึผืื, ืืื ืืืขื ืงืึทื ืคืืืืขืจ ืื ืฆืื ืืื ืื ืึทืืื ืืขืจืืคืขื ืข. ืืขืืึธ ืืึธืืข - ืึธื ืึธืืขื ืืึทืงืืืฉืึทื, ืงืขื ืืืง ืืื ืฆืืืจืืืืขื ืคึฟืึทืจ ืึทืืขืืขื. ืืื ืึท ืคึผืจืึธืืืงืฆืืข ืกืืืืืืข, ืืืจ ืืืขื ืืกืชึผืื ืืืืื ืฆื ืงืึทื ืคืืืืขืจ ืึธืืขื ืืึทืงืืืฉืึทื - ืึธืืขืจ ืืึธืก ืืื ืึท ืืืกื ื ืื-ืคืื-ืคืึทืจื ืขื ืคึฟืึทืจ ืืืึทื ื ืก ื ืึธืจ-ืคึฟืึทืจ-ืฉืคึผืึทืก ืืขื ืืืื ื.
ืฉืึทืคึฟื ืึท ืืึทืงืขื ื ืืืื ื ืึธืืขื ืืืกืง1 ืคึฟืึทืจืืื ืื ืืื ืืขืจ ืืขืงืข /dev/rbd/mypool/myimage. ืื ืกืคึผืขืกืืคืืขื ืืขืงืข ืืื ืึท ืกืืืืึธืืืฉ ืืื ืง ืืืืืึธืืึทืืืฉ ืืืฉืืคื ืืืจื ืื udev ืืขืืึธื ืฆื /dev/rbd0. ืืืจ ื ืืฆื ืึท ืกืืืืึธืืืฉ ืืื ืง ืืืืึทื ืื ื ืึธืืขื ืคืื ืื ืจืื ืืืื ืงืขื ืขื ืืืืฉื ืืืคึผืขื ืืื ื ืืืืฃ ืื ืกืืจ ืืื ืืืึธืก ืื Ceph ืืืืืขืจ ืืขื ืขื ืงืึธื ื ืขืงืืขื ืฆื ืื ืืึทืืขืืึธืก.
ืฉืึทืคึฟื ืึท ืืึทืงืขื ื:
# targetcli /backstores/block create disk1 /dev/rbd/mypool/myimage
ืฉืึทืคึฟื ืึท iSCSI ืฆืื:
# targetcli /iscsi create iqn.2020-01.demo.ceph:mypool
ืืืจ ืคืึทืจืืื ืื ืื ืืึทืงืขื ื ืืื ืึท LUN ืฆื ืื ืฆืื:
# targetcli /iscsi/iqn.2020-01.demo.ceph:mypool/tpg1/luns create /backstores/block/disk1
ืืื ืก ืงืึทื ืคืืืืขืจ ืื ืฆืื ืคึฟืึทืจ ืืขืืึธ ืืึธืืข:
# targetcli /iscsi/iqn.2020-01.demo.ceph:mypool/tpg1/ set
> attribute demo_mode_write_protect=0
# targetcli /iscsi/iqn.2020-01.demo.ceph:mypool/tpg1/ set
> attribute generate_node_acls=1
# targetcli /iscsi/iqn.2020-01.demo.ceph:mypool/tpg1/ set
> attribute cache_dynamic_acls=1
ืืื ืื ืงืึทื ืคืืืืขืจืืืฉืึทื:
# targetcli saveconfig
ืงืึธื ืืจืึธืืืจืื ื ืื ืืืึทืืืึทื ืคืื ืื ืฆืื:
# iscsiadm -m discovery -t st -p 127.0.0.1:3260
127.0.0.1:3260,1 iqn.2020-01.demo.ceph:mypool
ืืืจ ืคืึทืจืืื ืื ืื ืฆืื:
# iscsiadm -m node --login
Logging in to [iface: default, target: iqn.2020-01.demo.ceph:mypool, portal: 127.0.0.1,3260] (multiple)
Login to [iface: default, target: iqn.2020-01.demo.ceph:mypool, portal: 127.0.0.1,3260] successful.
ืืืื ืืืจ ืืึธื ืึทืืฅ ืจืืืืืง, ืึท ื ืืึท ืืืกืง ืืืขื ืืขืจืฉืืึทื ืขื ืืืืฃ ืื ืกืขืจืืืขืจ, ืืืึธืก ืงืืงื ืืื ืึท SCSI ืืืื, ืึธืืขืจ ืืื ืคืืงืืืฉ ืึท ืืืื ืคืื Ceph, ืึทืงืกืขืกื ืืืจื ืึท iSCSI ืฆืื. ืฆื ืืืกืืืืื ืฉืืืืื ืคึผืจืึธืืืขืืก, ืขืก ืืื ืืขืกืขืจ ืฆื ืืึทืืืึทืืืงื ืื ืงืึธื ื ืขืงืืขื ืืืกืง ืืื ืื ืืืืขืงืืึทื ืฆืื ืคืื ืื ืืืืข ืื ืืฉืืืืืขืจ:
# iscsiadm -m node --logout
# iscsiadm -m discoverydb -o delete -t st -p 127.0.0.1:3260
ืึทืืข ืืืึธืก ืืืืืื ืืื ืฆื ืึธื ืืึทืืื ืื ืงืึทื ืคืืืืขืจืืืฉืึทื ืึทืืื ืึทื ืื ืืืื ืืื ืงืึธื ื ืขืงืืขื ืืืืืึธืืึทืืืฉ ืืื, ื ืึธื ืงืฉืจ, ืื ืฆืื ืืื ืกืืจืึทืืึทืคืืื. ืืึธื ืืฉืื ื ืึท ืฆืื ืืืฉืืืื ืคืื ืฆืืืื ืกืืขืคึผืก - ืงืึทื ืขืงืืื ื ืื ืจืื ืืื ืคืืงืืืฉ ืืึธื ืืฉืื ื ืื ืฆืื.
ืขืจืฉืืขืจ, ืืึธืื ืืื ืื ืงืึทื ืคืืืืขืจ ืื ืึธืืึทืืึทืืืง ืงืฉืจ ืคืื RBD ืืืืืขืจ ืฆื ืืขืจ ืืึทืืขืืึธืก. ืืึธืก ืืื ืืขืืื ืืืจื ืึทืืื ื ืื ืคืืืืขื ืืข ืฉืืจืืช ืฆื ืื /etc/ceph/rbdmap ืืขืงืข:
# cat /etc/ceph/rbdmap
# RbdDevice Parameters
mypool/myimage id=admin
# systemctl enable rbdmap
ืจืืกืืึธืจืื ื ืื ืฆืื ืงืึทื ืคืืืืขืจืืืฉืึทื ืืื ืึท ืืืกื ืืขืจ ืงืึธืืคึผืืืฆืืจื - ืืืจ ืืึทืจืคึฟื ืฆื ืฉืจืืึทืื ืึท ืึทืคึผืึทืจืึทื ืคึฟืึทืจ ืกืืกืืขื ืืืึธืก ืืืขื ืืืงืขืจื ืื ืงืึทื ืคืืืืขืจืืืฉืึทื:
# cat /usr/lib/systemd/system/scsi-target.service
[Unit]
Description=Start iSCSI target
After=network-online.target rbdmap.service
Before=remote-fs-pre.target
Wants=network-online.target remote-fs-pre.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/targetcli restoreconfig
[Install]
WantedBy=multi-user.target
# systemctl daemon-reload
# systemctl enable scsi-target
ืื ืืขืฆื ืคึผืจืึธืืข ืืื ืฆื ืจืขืืึธืึธื ืืื ืืืขืจ ืืึธื ืืืึธืจ ืืืืืขืจ (ืขืก ืืื ืืืฆื ืึทื iSCSI ืฆืื). ืขืก ืืึธื ืืืื ืื ืืขืืืืื ืึทื ืืืื ืืืจ ืืึธืื ื ืืฉื ืงืืืจื ืื ืืึทืืึทืืืืก ืคืื ืื ืื ืืฉืืืืืขืจ ืืื ืื ืืึทืคึฟืขื iscsiadm -n discoverydb -o ืืืกืืขืงื ... ืืืจ ืงืขื ืกืืฃ ืึทืจืืืฃ ืืื ืึท ืกืขืจืืืขืจ ืืืึธืก ืืื ื ืืฉื ืืึธืื ืึธืืขืจ ืขืก ื ืขืื ืึท ืืึทื ื ืฆืืึทื ืฆื ืืึทืกืข.
ืืืืก ืืื ืืขืืืืื?
ืงืึทื ืคืืืืขืจ ืื ืื ืืฉืืืืืขืจ ืืืืฃ ืื ืกืขืจืืืขืจ ืืื ืืืจ ืืืืื ืฆื ืฉืืงื ืืขื ืฆืื.
ืืื ืฆื ืขื ืฉืืจ ืฉืืื ืืึธืืขืจืึทื ืฅ ืคืื ืืื ืืืขืจ ืฆืื?
ืืืจ ืงืขื ืขื ืกืืืืืึทืจืื ืงืึทื ืคืืืืขืจ ืืึทืจืืึทืฅ ืืืืฃ ืื ืืขืจืข ืืึธื ืืืึธืจืก ืืื ืฉืืขืื ืืืืืืคึผืึทืื (vmware ืืืขื ืคึฟืึทืจืฉืืืื ืืขื ืืื ืืคืืื ืึทืจืืขื, Hyper-V ืืืขื ื ืืฉื ืคึฟืึทืจืฉืืืื - ืขืก ืจืืงืืืืืขืจื SCSI ืืึทืงืก). ืืื ื ืื Ceph ืงืืืขื ื ืคืื ืื ืงืขืจื ืืื ื ืืฉื ื ืืฆื ืงืึทืืฉืื ื, ืืึธืก ืืื ืืึทื ืฅ ืืืขืจืงืึทืืึทื. ืึธืืขืจ ืื ืื ืืขืจ ืึธืคึผืฆืืข ืืื ืฆื ืฉืึทืคึฟื ืึท ืงื ืืื ืืืื ืคืื ืืจืื ืงืึทืืคึผืึธืื ืึทื ืฅ - ืึท ืืขืืึทืงืืืืึทื ืฆืื ืืคึผ ืึทืืจืขืก ืืื rbdmap ืืื scsi-target ืืึทืืื ืื ืืก, ืืื ืคืืจื ืืขื ืืืื ืืืจื ืงืืึทืกืืขืจืื ื ืืืฉืืจืื (ืืืขืจ ืืื ืืขืืืื ืคึผืืืกืืืืงืขืจ?)
ืึทื ืฉืืึธื ืคืื ืึท ืขืคึผืืืึธื
ืืื ืงืืึธืจ, ืืขืจ ืึทืจืืืงื ืืื ืึท ืืืกื ืคืื ืึท ืืืืฅ - ืึธืืขืจ ืืื ืขืก ืืื ืืขืคืจืืืื ืฆื "ืืื ืืื ืืื ืืืืฉืคืืื" ืืึทืืจืึทืืื ืขืืืขืืข ืคืขืจืื ืคืึธืืงืก ืืขืืขืก ืืื ืืขืจ ืืขืืืืงืขืจ ืฆืืื - iSCSI ืฆืื, ืืืึธืก ืงืขื ื ืืฉื ืืึทืืืงืข ืึทืจืืืกืคืืจื Ceph ืืืืืขืจ - ืึธืืขืจ ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืึทืจืืืกืคืืจื LVM ืืืึทืืืืื, ืื ืืึทืกืืงืก ืคืื ืืจืืขืื ืืื ืึทื iSCSI ืื ืืฉืืืืืขืจ (ืืื ืฆื ืืืขืจืงืืงื ืึท ืฆืื, ืืื ืฆื ืคืึทืจืืื ืื ืฆื ืึท ืฆืื, ืืืกืงืึทื ืขืงื, ืืืกืืขืงื ืึท ืฆืื ืคึผืึธืืืฆืืข ืคืื โโืื ืืึทืืึทืืืืก), ืฉืจืืืื ืืืื ืืืืืขื ืข ืึทืคึผืึทืจืึทื ืคึฟืึทืจ ืกืืกืืขื ืืื ืขืืืขืืข ืื ืืขืจืข
ืืื ืืึธืคึฟื ืึทื ืืคืืื ืืืื ืืืจ ืืึธื ื ืื ืืืืขืจืืืจื ืืขื ืืึทื ืฆื ืขืงืกืคึผืขืจืืืขื ื ืืื ืคืื, ืืคึผืืืช ืขืคึผืขืก ืคืื ืืขื ืึทืจืืืงื ืืืขื ืืืื ื ืืฆืืง ืคึฟืึทืจ ืืืจ.
ืืงืืจ: www.habr.com