ื™ื ืงืจืขืึทืกื™ื ื’ ืงืึทื ื˜ื™ื™ื ืขืจ ื’ืขื“ื™ื›ื˜ืงื™ื™ึทื˜ ืื•ื™ืฃ ืึท ื ืึธื“ืข ื ื™ืฆืŸ PFCACHE ื˜ืขื›ื ืึธืœืึธื’ื™ืข

ื™ื ืงืจืขืึทืกื™ื ื’ ืงืึทื ื˜ื™ื™ื ืขืจ ื’ืขื“ื™ื›ื˜ืงื™ื™ึทื˜ ืื•ื™ืฃ ืึท ื ืึธื“ืข ื ื™ืฆืŸ PFCACHE ื˜ืขื›ื ืึธืœืึธื’ื™ืข

ืื™ื™ื ืขืจ ืคื•ืŸ ื“ื™ ื’ืึธื•ืœื– ืคื•ืŸ ื“ื™ ื”ืึธืกื˜ื™ื ื’ ืฉืคึผื™ื™ึทื–ืขืจ ืื™ื– ืฆื• ืžืึทืงืกืึทืžื™ื™ื– ื“ื™ ื™ื•ื˜ืึทืœืึทื–ื™ื™ืฉืึทืŸ ืคื•ืŸ ื™ื’ื–ื™ืกื˜ื™ื ื’ ื•ื™ืกืจื™ื›ื˜ ืื™ืŸ ืกื“ืจ ืฆื• ืฆื•ืฉื˜ืขืœืŸ ื”ื•ื™ืš-ืงื•ื•ืึทืœื™ื˜ืขื˜ ื“ื™ื ืกื˜ ืฆื• ืกื•ืฃ ื ื™ืฆืขืจืก. ื“ื™ ืจืขืกื•ืจืกืŸ ืคื•ืŸ ืกื•ืฃ ืกืขืจื•ื•ืขืจืก ื–ืขื ืขืŸ ืฉื˜ืขื ื“ื™ืง ืœื™ืžื™ื˜ืขื“, ืึธื‘ืขืจ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ื›ืึธื•ืกื˜ื™ื“ ืงืœื™ืขื ื˜ ื‘ืึทื“ื™ื ื•ื ื’ืก, ืื•ืŸ ืื™ืŸ ืื•ื ื“ื–ืขืจ ืคืึทืœ ืžื™ืจ ืจืขื“ืŸ ื•ื•ืขื’ืŸ ื•ื•ืคึผืก, ืงืขื ืขืŸ ื–ื™ื™ืŸ ื‘ืื˜ื™ื™ื˜ื™ืง ืึทื ื“ืขืจืฉ. ืœื™ื™ืขื ืขืŸ ื•ื•ืขื’ืŸ ื•ื•ื™ ืฆื• ืงืจื™ื›ืŸ ื“ืขื ื‘ื•ื™ื ืื•ืŸ ืขืกืŸ ืึท ื‘ืขืจื’ืขืจ ืื•ื ื˜ืขืจ ื“ื™ ืฉื ื™ื™ึทื“ืŸ.

ืงืึธืžืคึผืึทืงื˜ื™ื ื’ VPS ืื•ื™ืฃ ืึท ื ืึธื“ืข ืื™ืŸ ืึทื–ืึท ืึท ื•ื•ืขื’ ืึทื– ืงืœื™ื™ืึทื ืฅ ื˜ืึธืŸ ื ื™ื˜ ืคื™ืœืŸ ืขืก ื‘ื™ื™ึท ืึทืœืข ื–ื™ื™ืขืจ ื”ืขืœืคึผืก ืฆื• ืคืึทืจื’ืจืขืกืขืจืŸ ื“ื™ ืขืงืึธื ืึธืžื™ืฉ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ืงื™ื™ืŸ ื”ืึธืกื˜ื™ื ื’ ืฉืคึผื™ื™ึทื–ืขืจ. ืคื•ืŸ ืงื•ืจืก, ืึท ื ืึธื“ืข ื–ืึธืœ ื ื™ืฉื˜ ืคึผืœืึทืฆืŸ ืื™ืŸ ื“ื™ ืกื™ืžื– ืื•ื™ื‘ ืขืก ืื™ื– ืงืจืึทืžื“ ืคื•ืœ ืคื•ืŸ ืงืึทื ื˜ื™ื™ื ืขืจื–, ืื•ืŸ ืงื™ื™ืŸ ืกืขืจื“ื–ืฉ ืื™ืŸ ืžืึทืกืข ืื™ื– ืžื™ื“ ืคึผืขืœืฅ ื“ื•ืจืš ืึทืœืข ืงืœื™ื™ืึทื ืฅ.

ื•ื•ื™ ืคื™ืœืข ื•ื•ืคึผืก ืงืขื ืขืŸ ื–ื™ื™ืŸ ื›ืึธื•ืกื˜ื™ื“ ืื•ื™ืฃ ืื™ื™ืŸ ื ืึธื“ืข ื“ืขืคึผืขื ื“ืก ืื•ื™ืฃ ืคื™ืœืข ืกื™ื‘ื•ืช, ืึทื–ืึท ืงืœืึธืจ ื•ื•ื™ ื“ืขืจ ื˜ืึธื’ ืึธื ืขืก ื•ื•ื™:

1. ื˜ืฉืึทืจืึทืงื˜ืขืจื™ืกื˜ื™ืงืก ืคื•ืŸ ื“ื™ ื™ื™ึทื–ื ื•ื•ืึทืจื’ ืคื•ืŸ ื“ื™ ื ืึธื“ืข ื–ื™ืš
2. ื•ื•ืคึผืก ื’ืจื™ื™ืก
3. ื ืึทื˜ื•ืจ ืคื•ืŸ ื“ื™ ืžืึทืกืข ืื•ื™ืฃ ื“ื™ ื•ื•ืคึผืก
4. ื•ื•ื™ื™ื›ื•ื•ืืจื’ ื˜ืขืงื ืึทืœืึทื“ื–ืฉื™ื– ืึทื– ื”ืขืœืคืŸ ืึทืคึผื˜ืึทืžื™ื™ื– ื’ืขื“ื™ื›ื˜ืงื™ื™ึทื˜

ืื™ืŸ ื“ืขื ืคืึทืœ, ืžื™ืจ ื•ื•ืขืœืŸ ื˜ื™ื™ืœืŸ ืื•ื ื“ื–ืขืจ ื“ืขืจืคืึทืจื•ื ื’ ืคื•ืŸ ื ื™ืฆืŸ Pfcache ื˜ืขื›ื ืึธืœืึธื’ื™ืข ืคึฟืึทืจ Virtuozzo.
ืžื™ืจ ื ื•ืฆืŸ ื“ื™ 6 ืฆื•ื•ื™ื™ึทื’, ืึธื‘ืขืจ ืึทืœืฅ ื’ืขื–ืึธื’ื˜ ืื™ื– ืื•ื™ืš ืืžืช ืคึฟืึทืจ ื“ื™ 7.

Pfcache - ืึท ื•ื•ื™ืจื˜ื•ืึธื–ื–ืึธ ืžืขืงืึทื ื™ื–ืึทื ื•ื•ืึธืก ื”ืขืœืคึผืก ื“ื™ื“ื•ืคึผืœื™ืงืึทื˜ IOPS ืื•ืŸ ื‘ืึทืจืึทืŸ ืื™ืŸ ืงืึทื ื˜ื™ื™ื ืขืจื–, ืึทืœืึทืงื™ื™ื˜ื™ื ื’ ื™ื™ื“ืขื ื™ืงืึทืœ ื˜ืขืงืขืก ืื™ืŸ ืงืึทื ื˜ื™ื™ื ืขืจื– ืื™ืŸ ืึท ื‘ืึทื–ื•ื ื“ืขืจ ืคึผืจืึธืกื˜ ื’ืขื’ื ื˜.

ืื™ืŸ ืคืึทืงื˜ ืขืก ื‘ืืฉื˜ื™ื™ื˜ ืคื•ืŸ:
1. ืงืขืจื ืขืœ ืงืึธื“
2. ื‘ืึทื ื™ืฆืขืจ-ืึธืจื˜ ืฉืขื“
3. ื‘ืึทื ื™ืฆืขืจ-ืคึผืœืึทืฅ ื™ื•ื˜ื™ืœืึทื˜ื™ื–

ืื•ื™ืฃ ื“ื™ ื ืึธื“ืข ื–ื™ื™ึทื˜, ืžื™ืจ ืึทืœืึทืงื™ื™ื˜ ืึท ื’ืึทื ืฅ ืึธืคึผื˜ื™ื™ืœื•ื ื’ ืื™ืŸ ื•ื•ืึธืก ื˜ืขืงืขืก ื•ื•ืขื˜ ื–ื™ื™ืŸ ื‘ืืฉืืคืŸ ื•ื•ืึธืก ื•ื•ืขื˜ ื–ื™ื™ืŸ ื’ืœื™ื™ืš ื’ืขื ื™ืฆื˜ ื“ื•ืจืš ืึทืœืข VPS ืื•ื™ืฃ ื“ื™ ื ืึธื“ืข. ื ื‘ืœืึธืง ืคึผืœืึธืึธืคึผ ืžื™ื˜ืœ ืื™ื– ืžืึธื•ื ื˜ืขื“ ืื™ืŸ ื“ืขื ืึธืคึผื˜ื™ื™ืœื•ื ื’. ื“ืขืจื ืึธืš, ื•ื•ืขืŸ ื“ืขืจ ืงืึทื ื˜ื™ื™ื ืขืจ ืกื˜ืึทืจืฅ, ืขืก ื ืขืžื˜ ืึท ืจืขืคึฟืขืจืขื ืฅ ืคึฟืึทืจ ื“ืขื ืึธืคึผื˜ื™ื™ืœื•ื ื’:

[root@pcs13 ~]# cat /proc/mounts
...
/dev/ploop62124p1 /vz/pfcache ext4 rw,relatime,barrier=1,data=ordered,balloon_ino=12 0 0
...
/dev/ploop22927p1 /vz/root/418 ext4 rw,relatime,barrier=1,data=ordered,balloon_ino=12,pfcache_csum,pfcache=/vz/pfcache 0 0
/dev/ploop29642p1 /vz/root/264 ext4 rw,relatime,barrier=1,data=ordered,balloon_ino=12,pfcache_csum,pfcache=/vz/pfcache 0 0
...

ื“ืึธ ื–ืขื ืขืŸ ื“ืขืจื ืขื ื˜ืขืจื  ืกื˜ืึทื˜ื™ืกื˜ื™ืง ืื•ื™ืฃ ื“ื™ ื ื•ืžืขืจ ืคื•ืŸ ื˜ืขืงืขืก ืื•ื™ืฃ ืื™ื™ื ืขืจ ืคื•ืŸ ืื•ื ื“ื–ืขืจ ื ืึธื•ื“ื–:

[root@pcs13 ~]# find /vz/pfcache -type f | wc -l
45851
[root@pcs13 ~]# du -sck -h /vz/pfcache
2.4G    /vz/pfcache
2.4G    total

ื“ืขืจ ืคึผืจื™ื ืฆื™ืคึผ ืคื•ืŸ pfcache ืื™ื– ื•ื•ื™ ื’ื™ื™ื˜:
โ€ข ื“ืขืจ ื‘ืึทื ื™ืฆืขืจ-ืคึผืœืึทืฅ ื“ื™ื™ืžืึทืŸ Pfcached ืฉืจื™ื™ื‘ื˜ ื“ื™ ืฉืึท-1 ื”ืึทืฉ ืคื•ืŸ ื“ืขืจ ื˜ืขืงืข ืฆื• ื“ื™ xattr ืึทื˜ืจื™ื‘ื™ื•ื˜ ืคื•ืŸ ื“ืขื ื˜ืขืงืข. ื ื™ื˜ ืึทืœืข ื˜ืขืงืขืก ื–ืขื ืขืŸ ืคึผืจืึทืกืขืกื˜, ืึธื‘ืขืจ ื‘ืœื•ื™ื– ืื™ืŸ ื“ื™ /usr, /bin, /usr/sbin, /sbin, /lib, /lib64 ื“ื™ืจืขืงื˜ืขืจื™ื–

โ€ข ืขืก ืื™ื– ืจื•ื‘ึฟ ืžืกืชึผืžื ืึทื– ื“ื™ ื˜ืขืงืขืก ืื™ืŸ ื“ื™ ื“ื™ืจืขืงื˜ืขืจื™ื– ื•ื•ืขื˜ ื–ื™ื™ืŸ "ืฉืขืจื“" ืื•ืŸ ื•ื•ืขื˜ ื–ื™ื™ืŸ ื’ืขื•ื•ื™ื™ื ื˜ ื“ื•ืจืš ืขื˜ืœืขื›ืข ืงืึทื ื˜ื™ื™ื ืขืจื–;

โ€ข Pfcached ืงืึทืœืขืงืฅ ืคึผื™ืจื™ืึทื“ื™ืงืœื™ ืกื˜ืึทื˜ื™ืกื˜ื™ืง ืื•ื™ืฃ ืœื™ื™ืขื ืขืŸ ื˜ืขืงืขืก ืคื•ืŸ ื“ื™ ืงืขืจืŸ, ืึทื ืึทืœื™ื–ืขืก ืขืก, ืื•ืŸ ืžื•ืกื™ืฃ ื˜ืขืงืขืก ืฆื• ื“ื™ ืงืึทืฉ ืื•ื™ื‘ ื–ื™ื™ ื–ืขื ืขืŸ ืึธืคื˜ ื’ืขื ื™ืฆื˜;

โ€ข ื“ื™ ื“ื™ืจืขืงื˜ืขืจื™ื– ืงืขืŸ ื–ื™ื™ืŸ ืึทื ื“ืขืจืฉ ืื•ืŸ ื–ืขื ืขืŸ ืงืึทื ืคื™ื’ื™ืขืจื“ ืื™ืŸ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ื˜ืขืงืขืก.

โ€ข ื•ื•ืขืŸ ืœื™ื™ืขื ืขืŸ ืึท ื˜ืขืงืข, ืขืก ืื™ื– ืึธืคึผื’ืขืฉื˜ืขืœื˜ ืฆื™ ืขืก ื›ึผื•ืœืœ ื“ื™ ืกืคึผืขืกื™ืคื™ืขื“ ื”ืึทืฉ ืื™ืŸ ื“ื™ xattr ืขืงืกื˜ืขื ื“ืขื“ ืึทื˜ืจื™ื‘ื™ื•ืฅ. ืื•ื™ื‘ ืขืก ื›ึผื•ืœืœ, ื“ื™ "ืึทืœื’ืขืžื™ื™ื ืข" ื˜ืขืงืข ืื™ื– ื’ืขืขืคื ื˜ ืึทื ืฉื˜ืึธื˜ ืคื•ืŸ ื“ื™ ืงืึทื ื˜ื™ื™ื ืขืจ ื˜ืขืงืข. ื“ืขื ืกืึทื‘ืกื˜ื™ื˜ื•ืฉืึทืŸ ืึทืงืขืจื– ืึทื ื ืึธื•ื˜ื™ืกื˜ ื“ื•ืจืš ื“ื™ ืงืึทื ื˜ื™ื™ื ืขืจ ืงืึธื“ ืื•ืŸ ืื™ื– ืคืึทืจื‘ืึธืจื’ืŸ ืื™ืŸ ื“ื™ ืงืขืจืŸ;

โ€ข ื•ื•ืขืŸ ืฉืจื™ื™ื‘ืŸ ืฆื• ืึท ื˜ืขืงืข, ื“ื™ ื”ืึทืฉ ืื™ื– ื™ื ื•ื•ืึทืœืึทื“ื™ื™ื˜ืึทื“. ืื–ื•ื™, ื“ืขืจ ื•ื•ื™ื™ึทื˜ืขืจ ืžืึธืœ ืื™ืจ ืขืคึฟืขื ืขืŸ ืขืก, ื“ื™ ืงืึทื ื˜ื™ื™ื ืขืจ ื˜ืขืงืข ื–ื™ืš ื•ื•ืขื˜ ื–ื™ื™ืŸ ื’ืขืขืคื ื˜, ืื•ืŸ ื ื™ืฉื˜ ื–ื™ื™ืŸ ืงืึทืฉ.

ื“ื•ืจืš ื”ืืœื˜ืŸ ืฉืขืจื“ ื˜ืขืงืขืก ืคื•ืŸ /vz/pfcache ืื™ืŸ ื“ื™ ื‘ืœืึทื˜ ืงืึทืฉ, ืžื™ืจ ื“ืขืจื’ืจื™ื™ื›ืŸ ืกื™ื™ื•ื•ื™ื ื’ื– ืื™ืŸ ื“ืขื ืงืึทืฉ ื–ื™ืš, ื•ื•ื™ ื’ืขื–ื•ื ื˜ ื•ื•ื™ ืกื™ื™ื•ื•ื™ื ื’ื– ืื™ืŸ IOPS. ืึทื ืฉื˜ืึธื˜ ืคื•ืŸ ืœื™ื™ืขื ืขืŸ ืฆืขืŸ ื˜ืขืงืขืก ืคื•ืŸ ื“ื™ืกืง, ืžื™ืจ ืœื™ื™ืขื ืขืŸ ืื™ื™ื ืขืจ, ื•ื•ืึธืก ื’ืœื™ื™ืš ื’ื™ื™ื˜ ืฆื• ื“ื™ ื‘ืœืึทื˜ ืงืึทืฉ.

struct inode {
...
 struct file             *i_peer_file;
...
};
struct address_space {
...
 struct list_head        i_peer_list;
...
}

ื“ื™ VMA ืจืฉื™ืžื” ืคึฟืึทืจ ื“ืขืจ ื˜ืขืงืข ื‘ืœื™ื™ื‘ื˜ ืื™ื™ืŸ (ืžื™ืจ ื“ืขื“ื•ืคึผืœื™ืงืึทื˜ ื–ื›ึผืจื•ืŸ) ืื•ืŸ ืื™ื– ื•ื•ื™ื™ื ื™ืงืขืจ ืึธืคื˜ ืœื™ื™ืขื ืขืŸ ืคึฟื•ืŸ ื“ื™ืกืง (ืฉืคึผืึธืจืŸ ื™ืึธืคึผืก). ืื•ื ื“ื–ืขืจ ืคึผืจืึธืกื˜ ืคืึธื ื“ ืื™ื– ืœื™ื’ืŸ ืื•ื™ืฃ ืึท ืกืกื“ - ืึทืŸ ื ืึธืš ื’ืขื•ื•ื™ื ืขืŸ ืื™ืŸ ื’ื™ื›ืงื™ื™ึทื˜.

ื‘ื™ื™ึทืฉืคึผื™ืœ ืคึฟืึทืจ ืงืึทื˜ืฉื™ื ื’ ื“ื™ /bin/bash ื˜ืขืงืข:

[root@pcs13 ~]# ls -li /vz/root/2388/bin/bash
524650 -rwxr-xr-x 1 root root 1021112 Oct  7  2018 /vz/root/2388/bin/bash
[root@pcs13 ~]# pfcache dump /vz/root/2388 | grep 524650
8e3aa19fdc42e87659746f6dc8ea3af74ab30362 i:524650      g:1357611108  f:CP
[root@pcs13 ~]# sha1sum /vz/root/2388/bin/bash
8e3aa19fdc42e87659746f6dc8ea3af74ab30362  /vz/root/2388/bin/bash
[root@pcs13 /]# getfattr -ntrusted.pfcache /vz/root/2388/bin/bash
# file: vz/root/2388/bin/bash
trusted.pfcache="8e3aa19fdc42e87659746f6dc8ea3af74ab30362"
[root@pcs13 ~]# sha1sum /vz/pfcache/8e/3aa19fdc42e87659746f6dc8ea3af74ab30362
8e3aa19fdc42e87659746f6dc8ea3af74ab30362  /vz/pfcache/8e/3aa19fdc42e87659746f6dc8ea3af74ab30362

ืžื™ืจ ืจืขื›ืขื ืขืŸ ื“ื™ ืขืคืขืงื˜ื™ื•ื•ืงื™ื™ึทื˜ ืคื•ืŸ ื ื•ืฆืŸ ืคืึทืจื˜ื™ืง ืฉืจื™ืคื˜.

ื“ืขืจ ืฉืจื™ืคื˜ ื’ื™ื™ื˜ ื“ื•ืจืš ืึทืœืข ื“ื™ ืงืึทื ื˜ื™ื™ื ืขืจื– ืื•ื™ืฃ ื“ื™ ื ืึธื“ืข, ืงืึทืœืงื™ืึทืœื™ื™ื˜ื™ื ื’ ื“ื™ ืงืึทืฉื˜ ื˜ืขืงืขืก ืคื•ืŸ ื™ืขื“ืขืจ ืงืึทื ื˜ื™ื™ื ืขืจ.

[root@pcs16 ~]# /pcs/distr/pfcache-examine.pl
...
Pfcache cache uses 831 MB of memory
Total use of pfcached files in containers is 39837 MB of memory
Pfcache effectiveness: 39006 MB

ืื–ื•ื™, ืคึฟื•ืŸ ื–ื›ึผืจื•ืŸ ืžื™ืจ ืฉืคึผืึธืจืŸ ื•ื•ืขื’ืŸ 40 ื’ื™ื’ืื‘ื™ื™ื˜ ืคื•ืŸ ื˜ืขืงืขืก ืื™ืŸ ืงืึทื ื˜ื™ื™ื ืขืจื–; ื–ื™ื™ ื•ื•ืขืœืŸ ื–ื™ื™ืŸ ืœืึธื•ื“ื™ื“ ืคึฟื•ืŸ ื“ื™ ืงืึทืฉ.

ืคึฟืึทืจ ื“ืขื ืžืขืงืึทื ื™ื–ืึทื ืฆื• ืึทืจื‘ืขื˜ืŸ ืืคื™ืœื• ื‘ืขืกืขืจ, ืขืก ืื™ื– ื ื™ื™ื˜ื™ืง ืฆื• ืฉื˜ืขืœืŸ ื“ื™ ืžืขืจืกื˜ "ื™ื™ื“ืขื ื˜ื™ืงืึทืœ" ื•ื•ืคึผืก ืื•ื™ืฃ ื“ื™ ื ืึธื“ืข. ืคึฟืึทืจ ื‘ื™ื™ึทืฉืคึผื™ืœ, ื“ื™ ื•ื•ืึธืก ื“ืขืจ ื‘ืึทื ื™ืฆืขืจ ื˜ื•ื˜ ื ื™ืฉื˜ ื”ืึธื‘ืŸ ื•ื•ืึธืจืฆืœ ืึทืงืกืขืก ืื•ืŸ ืื•ื™ืฃ ื•ื•ืึธืก ื“ื™ ืกื•ื•ื™ื•ื•ืข ืคึฟื•ืŸ ื“ื™ ื“ื™ืคึผืœื•ื™ื“ ื‘ื™ืœื“ ืื™ื– ืงืึทื ืคื™ื’ื™ืขืจื“.

ืื™ืจ ืงืขื ืขืŸ ืกื˜ืจื•ื™ืขืจืŸ pfcache ื“ื•ืจืš ื“ื™ ืงืึธื ืคื™ื’ ื˜ืขืงืข
/etc/vz/pfcache.conf

MINSIZE, MAXSIZE - ืžื™ื ื™ืžื•ื / ืžืึทืงืกื™ืžื•ื ื˜ืขืงืข ื’ืจื™ื™ืก ืคึฟืึทืจ ืงืึทื˜ืฉื™ื ื’
TIMEOUT - ื˜ื™ื™ืžืึทื•ื˜ ืฆื•ื•ื™ืฉืŸ ืงืึทื˜ืฉื™ื ื’ ืคืจื•ื•ื•ืŸ

ืื™ืจ ืงืขื ืขืŸ ื–ืขืŸ ื“ื™ ืคื•ืœ ืจืฉื™ืžื” ืคื•ืŸ ืคึผืึทืจืึทืžืขื˜ืขืจืก ะฟะพ ััั‹ะปะบะต.

ืžืงื•ืจ: www.habr.com

ืœื™ื™ื’ืŸ ืึท ื‘ืึทืžืขืจืงื•ื ื’