Sumala sa : --apod-apod sa sayop-tolerant butang storage , compatible sa mga kliyente nga naggamit sa Amazon S3 API ug REST-API, ug nagsuporta usab sa NFS server mode. Adunay mga pag-optimize alang sa pagtipig sa gagmay ug dagko kaayo nga mga butang, adunay built-in nga mekanismo sa caching, ug posible ang pagkopya sa pagtipig tali sa mga sentro sa datos. Ang mga tumong sa proyekto naglakip sa pagkab-ot sa 99.9999999% nga kasaligan pinaagi sa sobra nga pagkopya sa mga duplicate ug pagwagtang sa usa ka punto sa kapakyasan. Ang code sa proyekto gisulat sa Erlang.
Ang LeoFS naglangkob sa tulo ka mga sangkap:
- β nag-alagad sa mga operasyon sa pagdugang, pagbawi ug pagtangtang sa mga butang ug metadata, responsable sa paghimo sa pagkopya, pagbawi ug pagpila sa mga hangyo sa kliyente.
- β nagsilbi sa mga hangyo sa HTTP ug nag-redirect sa mga tubag sa mga kliyente gamit ang REST-API o S3-API, nagsiguro sa pag-cache sa pinakapopular nga datos sa memorya ug sa disk.
- β nagmonitor sa operasyon sa LeoFS Gateway ug LeoFS Storage nodes, nagmonitor sa kahimtang sa mga node ug nagsusi sa mga checksum. Gigarantiya ang integridad sa datos ug taas nga pagkaanaa sa pagtipig.
Niini nga post atong i-install ang Leofs gamit ang ansible-playbook ug pagsulay sa S3, NFS.
Kung sulayan nimo nga i-install ang LeoFS gamit ang opisyal nga mga playbook, makasugat ka og lainlaing mga sayup: ,. Niini nga post akong isulat kung unsa ang kinahanglan buhaton aron malikayan kini nga mga sayup.
Kung asa ka magpadagan sa ansible-playbook, kinahanglan nimo nga i-install ang netcat.
Pananglitan nga imbentaryo
Pananglitan nga imbentaryo (sa hosts.sample repository):
# Please check roles/common/vars/leofs_releases for available versions
[all:vars]
leofs_version=1.4.3
build_temp_path="/tmp/leofs_builder"
build_install_path="/tmp/"
build_branch="master"
source="package"
#[builder]
#172.26.9.177
# nodename of leo_manager_0 and leo_manager_1 are set at group_vars/all
[leo_manager_0]
172.26.9.176
# nodename of leo_manager_0 and leo_manager_1 are set at group_vars/all
[leo_manager_1]
172.26.9.178
[leo_storage]
172.26.9.179 [email protected]
172.26.9.181 [email protected]
172.26.9.182 [email protected]
172.26.9.183 [email protected]
[leo_gateway]
172.26.9.180 [email protected]
172.26.9.184 [email protected]
[leofs_nodes:children]
leo_manager_0
leo_manager_1
leo_gateway
leo_storagePag-andam sa server
Pag-disable sa Selinux. Nanghinaut ko nga ang komunidad maghimo sa mga palisiya sa Selinux para sa LeoFS.
- name: Install libselinux as prerequisite for SELinux Ansible module
yum:
name: "{{item}}"
state: latest
with_items:
- libselinux-python
- libsemanage-python
- name: Disable SELinux at next reboot
selinux:
state: disabled
- name: Set SELinux in permissive mode until the machine is rebooted
command: setenforce 0
ignore_errors: true
changed_when: false
Pag-instalar netcat ΠΈ redhat-lsb-core. netcat gikinahanglan alang sa leofs-adm, redhat-lsb-core gikinahanglan aron mahibal-an ang bersyon sa OS .
- name: Install Packages
yum: name={{ item }} state=present
with_items:
- nmap-ncat
- redhat-lsb-corePaghimo og user leof ug idugang kini sa wheel group
- name: Create user leofs
group:
name: leofs
state: present
- name: Allow 'wheel' group to have passwordless sudo
lineinfile:
dest: /etc/sudoers
state: present
regexp: '^%wheel'
line: '%wheel ALL=(ALL) NOPASSWD: ALL'
validate: 'visudo -cf %s'
- name: Add the user 'leofs' to group 'wheel'
user:
name: leofs
groups: wheel
append: yesPag-instalar sa Erlang
- name: Remote erlang-20.3.8.23-1.el7.x86_64.rpm install with yum
yum: name=https://github.com/rabbitmq/erlang-rpm/releases/download/v20.3.8.23/erlang-20.3.8.23-1.el7.x86_64.rpm
Ang tibuok nga bersyon sa gitul-id nga Ansible playbook makita dinhi:
Pag-instalar, pag-configure, paglansad
Sunod among gihimo ingon nga nahisulat sa walay build_leofs.yml
## Install LeoFS
$ ansible-playbook -i hosts install_leofs.yml
## Config LeoFS
$ ansible-playbook -i hosts config_leofs.yml
## Start LeoFS
$ ansible-playbook -i hosts start_leofs.ymlPagsusi sa kahimtang sa cluster sa Primary LeoManager
leofs-adm statusAng Primary ug Secondary makita sa ansible-playbook logs
Ang output mahimong sama niini
[System Confiuration]
-----------------------------------+----------
Item | Value
-----------------------------------+----------
Basic/Consistency level
-----------------------------------+----------
system version | 1.4.3
cluster Id | leofs_1
DC Id | dc_1
Total replicas | 2
number of successes of R | 1
number of successes of W | 1
number of successes of D | 1
number of rack-awareness replicas | 0
ring size | 2^128
-----------------------------------+----------
Multi DC replication settings
-----------------------------------+----------
[mdcr] max number of joinable DCs | 2
[mdcr] total replicas per a DC | 1
[mdcr] number of successes of R | 1
[mdcr] number of successes of W | 1
[mdcr] number of successes of D | 1
-----------------------------------+----------
Manager RING hash
-----------------------------------+----------
current ring-hash | a0314afb
previous ring-hash | a0314afb
-----------------------------------+----------
[State of Node(s)]
-------+----------------------+--------------+---------+----------------+----------------+----------------------------
type | node | state | rack id | current ring | prev ring | updated at
-------+----------------------+--------------+---------+----------------+----------------+----------------------------
S | [email protected] | running | | a0314afb | a0314afb | 2019-12-05 10:33:47 +0000
S | [email protected] | running | | a0314afb | a0314afb | 2019-12-05 10:33:47 +0000
S | [email protected] | running | | a0314afb | a0314afb | 2019-12-05 10:33:47 +0000
S | [email protected] | attached | | | | 2019-12-05 10:33:58 +0000
G | [email protected] | running | | a0314afb | a0314afb | 2019-12-05 10:33:49 +0000
G | [email protected] | running | | a0314afb | a0314afb | 2019-12-05 10:33:49 +0000
-------+----------------------+--------------+---------+----------------+----------------+----------------------------Paghimo og user
Paghimo og leofs user:
leofs-adm create-user leofs leofs
access-key-id: 9c2615f32e81e6a1caf5
secret-access-key: 8aaaa35c1ad78a2cbfa1a6cd49ba8aaeb3ba39ebListahan sa mga tiggamit:
leofs-adm get-users
user_id | role_id | access_key_id | created_at
------------+---------+------------------------+---------------------------
_test_leofs | 9 | 05236 | 2019-12-02 06:56:49 +0000
leofs | 1 | 9c2615f32e81e6a1caf5 | 2019-12-02 10:43:29 +0000Paghimo ug Balde
Naghimo ug balde
leofs-adm add-bucket leofs 9c2615f32e81e6a1caf5
OKListahan sa balde:
leofs-adm get-buckets
cluster id | bucket | owner | permissions | created at
-------------+----------+--------+------------------+---------------------------
leofs_1 | leofs | leofs | Me(full_control) | 2019-12-02 10:44:02 +0000Pag-configure sa s3cmd
Sa sa kapatagan HTTP Proxy server name ipiho ang Gateway server IP
s3cmd --configure
Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key [9c2615f32e81e6a1caf5]:
Secret Key [8aaaa35c1ad78a2cbfa1a6cd49ba8aaeb3ba39eb]:
Default Region [US]:
Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint [s3.amazonaws.com]:
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: leofs
Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program [/usr/bin/gpg]:
When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [No]:
On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name [172.26.9.180]:
HTTP Proxy server port [8080]:
New settings:
Access Key: 9c2615f32e81e6a1caf5
Secret Key: 8aaaa35c1ad78a2cbfa1a6cd49ba8aaeb3ba39eb
Default Region: US
S3 Endpoint: s3.amazonaws.com
DNS-style bucket+hostname:port template for accessing a bucket: leofs
Encryption password:
Path to GPG program: /usr/bin/gpg
Use HTTPS protocol: False
HTTP Proxy server name: 172.26.9.180
HTTP Proxy server port: 8080
Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)
Now verifying that encryption works...
Not configured. Never mind.
Save settings? [y/N] y
Configuration saved to '/home/user/.s3cfg'Kung makuha nimo ang sayup ERROR: S3 error: 403 (AccessDenied): Access Denied:
s3cmd put test.py s3://leofs/
upload: 'test.py' -> 's3://leofs/test.py' [1 of 1]
382 of 382 100% in 0s 3.40 kB/s done
ERROR: S3 error: 403 (AccessDenied): Access DeniedUnya kinahanglan nimo nga usbon ang signature_v3 sa Tinuod sa s2cmd config. Mga detalye niini .
Kung ang signature_v2 False, nan adunay usa ka sayup nga sama niini:
WARNING: Retrying failed request: /?delimiter=%2F (getaddrinfo() argument 2 must be integer or string)
WARNING: Waiting 3 sec...
WARNING: Retrying failed request: /?delimiter=%2F (getaddrinfo() argument 2 must be integer or string)
WARNING: Waiting 6 sec...
ERROR: Test failed: Request failed for: /?delimiter=%2FPagsulay sa load
Paghimo og 1GB nga file
fallocate -l 1GB 1gbI-upload kini sa Leofs
time s3cmd put 1gb s3://leofs/
real 0m19.099s
user 0m7.855s
sys 0m1.620sStatistics
leofs-adm du alang sa 1 node:
leofs-adm du [email protected]
active number of objects: 156
total number of objects: 156
active size of objects: 602954495
total size of objects: 602954495
ratio of active size: 100.0%
last compaction start: ____-__-__ __:__:__
last compaction end: ____-__-__ __:__:__Atong nakita nga ang konklusyon dili kaayo informative.
Atong tan-awon kung asa nahimutang kini nga file.
leofs-adm diin ang leofs/1gb
leofs-adm whereis leofs/1gb
-------+----------------------+--------------------------------------+------------+--------------+----------------+----------------+----------------+----------------------------
del? | node | ring address | size | checksum | has children | total chunks | clock | when
-------+----------------------+--------------------------------------+------------+--------------+----------------+----------------+----------------+----------------------------
| [email protected] | 657a9f3a3db822a7f1f5050925b26270 | 976563K | a4634eea55 | true | 64 | 598f2aa976a4f | 2019-12-05 10:48:15 +0000
| [email protected] | 657a9f3a3db822a7f1f5050925b26270 | 976563K | a4634eea55 | true | 64 | 598f2aa976a4f | 2019-12-05 10:48:15 +0000I-aktibo ang NFS
Among gi-activate ang NFS sa Leo Gateway 172.26.9.184 server.
I-install ang nfs-utils sa server ug kliyente
sudo yum install nfs-utilsSumala sa mga instruksyon, atong tul-iron ang configuration file /usr/local/leofs/current/leo_gateway/etc/leo_gateway.conf
protocol = nfsSa server 172.26.9.184 run rpcbind ug leofs-gateway
sudo service rpcbind start
sudo service leofs-gateway restartSa server diin nagdagan ang leo_manager, paghimo usa ka balde alang sa NFS ug paghimo usa ka yawe alang sa pagkonekta sa NFS
leofs-adm add-bucket test 05236
leofs-adm gen-nfs-mnt-key test 05236 ip-Π°Π΄ΡΠ΅Ρ-nfs-ΠΊΠ»ΠΈΠ΅Π½ΡΠ°Pagkonektar sa NFS
sudo mkdir /mnt/leofs
## for Linux - "sudo mount -t nfs -o nolock <host>:/<bucket>/<token> <dir>"
sudo mount -t nfs -o nolock ip-Π°Π΄ΡΠ΅Ρ-nfs-ΡΠ΅ΡΠ²Π΅ΡΠ°-ΡΠ°ΠΌ-Π³Π΄Π΅-Ρ-Π²Π°Ρ-ΡΡΡΠ°Π½ΠΎΠ²Π»Π΅Π½-gateway:/bucket/access_key_id/ΠΊΠ»ΡΡ-ΠΏΠΎΠ»ΡΡΠ΅Π½Π½ΡΠΉ-ΠΎΡ-gen-nfs-mnt-key /mnt/leofs
sudo mount -t nfs -o nolock 172.26.9.184:/test/05236/bb5034f0c740148a346ed663ca0cf5157efb439f /mnt/leofsTan-awa ang wanang sa disk pinaagi sa kliyente sa NFS
Disk space, nga gikonsiderar nga ang matag storage node adunay 40GB nga disk (3 running node, 1 attached node):
df -hP
Filesystem Size Used Avail Use% Mounted on
172.26.9.184:/test/05236/e7298032e78749149dd83a1e366afb328811c95b 60G 3.6G 57G 6% /mnt/leofsPag-instalar sa LeoFS nga adunay 6 nga mga node sa pagtipig.
Imbentaryo (walay magtutukod):
# Please check roles/common/vars/leofs_releases for available versions
[all:vars]
leofs_version=1.4.3
build_temp_path="/tmp/leofs_builder"
build_install_path="/tmp/"
build_branch="master"
source="package"
# nodename of leo_manager_0 and leo_manager_1 are set at group_vars/all
[leo_manager_0]
172.26.9.177
# nodename of leo_manager_0 and leo_manager_1 are set at group_vars/all
[leo_manager_1]
172.26.9.176
[leo_storage]
172.26.9.178 [email protected]
172.26.9.179 [email protected]
172.26.9.181 [email protected]
172.26.9.182 [email protected]
172.26.9.183 [email protected]
172.26.9.185 [email protected]
[leo_gateway]
172.26.9.180 [email protected]
172.26.9.184 [email protected]
[leofs_nodes:children]
leo_manager_0
leo_manager_1
leo_gateway
leo_storageOutput leofs-adm status
Output leofs-adm status
[System Confiuration]
-----------------------------------+----------
Item | Value
-----------------------------------+----------
Basic/Consistency level
-----------------------------------+----------
system version | 1.4.3
cluster Id | leofs_1
DC Id | dc_1
Total replicas | 2
number of successes of R | 1
number of successes of W | 1
number of successes of D | 1
number of rack-awareness replicas | 0
ring size | 2^128
-----------------------------------+----------
Multi DC replication settings
-----------------------------------+----------
[mdcr] max number of joinable DCs | 2
[mdcr] total replicas per a DC | 1
[mdcr] number of successes of R | 1
[mdcr] number of successes of W | 1
[mdcr] number of successes of D | 1
-----------------------------------+----------
Manager RING hash
-----------------------------------+----------
current ring-hash | d8ff465e
previous ring-hash | d8ff465e
-----------------------------------+----------
[State of Node(s)]
-------+----------------------+--------------+---------+----------------+----------------+----------------------------
type | node | state | rack id | current ring | prev ring | updated at
-------+----------------------+--------------+---------+----------------+----------------+----------------------------
S | [email protected] | running | | d8ff465e | d8ff465e | 2019-12-06 05:18:29 +0000
S | [email protected] | running | | d8ff465e | d8ff465e | 2019-12-06 05:18:29 +0000
S | [email protected] | running | | d8ff465e | d8ff465e | 2019-12-06 05:18:30 +0000
S | [email protected] | running | | d8ff465e | d8ff465e | 2019-12-06 05:18:29 +0000
S | [email protected] | running | | d8ff465e | d8ff465e | 2019-12-06 05:18:29 +0000
S | [email protected] | running | | d8ff465e | d8ff465e | 2019-12-06 05:18:29 +0000
G | [email protected] | running | | d8ff465e | d8ff465e | 2019-12-06 05:18:31 +0000
G | [email protected] | running | | d8ff465e | d8ff465e | 2019-12-06 05:18:31 +0000
-------+----------------------+--------------+---------+----------------+----------------+----------------------------Disk space, nga gikonsiderar nga ang matag storage node adunay 40GB disk (6 running node):
df -hP
Filesystem Size Used Avail Use% Mounted on
172.26.9.184:/test/05236/e7298032e78749149dd83a1e366afb328811c95b 120G 3.6G 117G 3% /mnt/leofsKung gigamit ang 5 nga mga node sa pagtipig
[leo_storage]
172.26.9.178 [email protected]
172.26.9.179 [email protected]
172.26.9.181 [email protected]
172.26.9.182 [email protected]
172.26.9.183 [email protected]df -hP
172.26.9.184:/test/05236/e7298032e78749149dd83a1e366afb328811c95b 100G 3.0G 97G 3% /mnt/leofsMga troso
Ang mga log nahimutang sa mga direktoryo /usr/local/leofs/current/*/log
Telegram channel:
Source: www.habr.com