{"id":77266,"date":"2020-04-09T13:42:08","date_gmt":"2020-04-09T11:42:08","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/tips-tricks-v-rabote-s-ceph-v-nagruzhennyh-proektah"},"modified":"2020-04-09T13:42:08","modified_gmt":"2020-04-09T11:42:08","slug":"tips-tricks-v-rabote-s-ceph-v-nagruzhennyh-proektah","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/tips-tricks-v-rabote-s-ceph-v-nagruzhennyh-proektah","title":{"rendered":"Tips &amp; tricks for working with Ceph in high-load projects","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Tips &amp; tricks for working with Ceph in demanding projects\" src=\"\/wp-content\/uploads\/2020\/04\/1d60f91e55b20a5dc4346bd8087bc11e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nUsing Ceph as a network storage solution in projects with varying loads, we may encounter various challenges that at first glance do not seem simple or trivial. For example:<\/p>\n<ul>\n<li> data migration from an old Ceph to a new one, partially utilizing previous servers in the new cluster;<\/li>\n<li> addressing the issue of disk space distribution in Ceph.<\/li>\n<\/ul>\n<p>\nWhen dealing with such tasks, we face the need to correctly extract OSD without data loss, which is particularly relevant when handling large volumes of data. This is what this article will discuss.<\/p>\n<p>The methods described below are applicable to any version of Ceph. Additionally, it will be taken into account that Ceph may store a large amount of data: to prevent data loss and other issues, some actions will be 'broken down' into several others.<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h3>Preface about OSD<\/h3>\n<p>\nSince two out of the three recipes under consideration are dedicated to OSD (<noindex><a rel=\"nofollow\" href=\"https:\/\/docs.ceph.com\/docs\/master\/man\/8\/ceph-osd\/\">Object Storage Daemon<\/a><\/noindex>), before diving into the practical part\u2014let's briefly discuss what it actually is in Ceph and why it is so important.<\/p>\n<p>First and foremost, it should be noted that the entire Ceph cluster consists of multiple OSDs. The more there are, the greater the free data capacity in Ceph. From this, it is easy to understand <b>the main function of OSD<\/b>: it stores Ceph object data on the file systems of all nodes in the cluster and provides network access to them (for reading, writing, and other requests).<\/p>\n<p>At this level, replication parameters are set by copying objects between different OSDs. Here, various problems may arise, the solutions to which will be discussed next.<\/p>\n<h2>Case #1. Safe extraction of OSD from Ceph cluster without data loss<\/h2>\n<p>\nThe need to extract OSD may arise from removing the server from the cluster\u2014 for example, to replace it with another server\u2014which is what happened in our case and served as the impetus for writing this article. Thus, the ultimate goal of the manipulations is to extract all OSDs and monitors on this server so that it can be shut down.<\/p>\n<p>For convenience and to avoid a situation where we mistakenly point to the wrong OSD during command execution, we will set a separate variable, the value of which will be the number of the OSD being removed. Let's name it <code>${ID}<\/code> \u2014 here and below such a variable replaces the number of the OSD we are working with.<\/p>\n<p>Let's look at the state before starting the work:<\/p>\n<pre><code class=\"bash\">root@hv-1 ~ # ceph osd tree\nID CLASS WEIGHT  TYPE NAME      STATUS REWEIGHT PRI-AFF\n-1       0.46857 root default\n-3       0.15619      host hv-1\n-5       0.15619      host hv-2\n 1   ssd 0.15619      osd.1     up     1.00000  1.00000\n-7       0.15619      host hv-3\n 2   ssd 0.15619      osd.2     up     1.00000  1.00000<\/code><\/pre>\n<p>\nTo initiate the removal of an OSD, it is necessary to smoothly execute <code>reweight<\/code> it down to zero. This reduces the amount of data in the OSD by redistributing it to other OSDs. The following commands are executed for this purpose:<\/p>\n<pre><code class=\"bash\">ceph osd reweight osd.${ID} 0.98\nceph osd reweight osd.${ID} 0.88\nceph osd reweight osd.${ID} 0.78<\/code><\/pre>\n<p>\n\u2026 and so on down to zero.<\/p>\n<p><b>Smooth rebalancing is necessary<\/b>, to avoid data loss. This is especially relevant if there is a large volume of data in the OSD. To ensure that the commands <code>reweight<\/code> have executed successfully, you can run <code>ceph -s<\/code> or alternatively launch in a separate terminal window <code>ceph -w<\/code> to monitor changes in real-time.<\/p>\n<p>When the OSD is 'emptied', you can proceed with the standard operation to remove it. For this, we will set the desired OSD to a state of <code>down<\/code>:<\/p>\n<pre><code class=\"bash\">ceph osd down osd.${ID}<\/code><\/pre>\n<p>\nWe will 'take out' the OSD from the cluster:<\/p>\n<pre><code class=\"bash\">ceph osd out osd.${ID}<\/code><\/pre>\n<p>\nWe will stop the OSD service and unmount its partition in the FS:<\/p>\n<pre><code class=\"bash\">systemctl stop ceph-osd@${ID}\numount \/var\/lib\/ceph\/osd\/ceph-${ID}<\/code><\/pre>\n<p>\nWe will remove the OSD from <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.ceph.com\/docs\/nautilus\/rados\/operations\/crush-map\/\">CRUSH map<\/a><\/noindex>:<\/p>\n<pre><code class=\"bash\">ceph osd crush remove osd.${ID}<\/code><\/pre>\n<p>\nWe will delete the OSD user:<\/p>\n<pre><code class=\"bash\">ceph auth del osd.${ID}<\/code><\/pre>\n<p>\nAnd finally, we will delete the OSD itself:<\/p>\n<pre><code class=\"bash\">ceph osd rm osd.${ID}<\/code><\/pre>\n<p>\n<b>Note<\/b>: if you are using Ceph Luminous version or higher, then the above actions for removing the OSD can be reduced to two commands:<\/p>\n<pre><code class=\"bash\">ceph osd out osd.${ID}\nceph osd purge osd.${ID}\n<\/code><\/pre>\n<p>\nIf after executing the above commands you run the command <code>ceph osd tree<\/code>, it should be visible that there are no longer any OSDs on the server where the operations were performed:<\/p>\n<pre><code class=\"bash\">root@hv-1 ~ # ceph osd tree\nID CLASS WEIGHT  TYPE NAME     STATUS REWEIGHT PRI-AFF\n-1       0.46857      root default\n-3       0.15619      host hv-1\n-5       0.15619      host hv-2\n-7       0.15619      host hv-3\n 2   ssd 0.15619      osd.2    up     1.00000  1.00000<\/code><\/pre>\n<p>\nMeanwhile, we will note that the state of the Ceph cluster will transition to <code>HEALTH_WARN<\/code>, and we will also see a decrease in the number of OSDs and available disk space. <\/p>\n<p>Next, actions will be described that will be required if you want to completely stop the server and, accordingly, remove it from Ceph. In this case, it is important to remember that <b>before disconnecting the server, you must remove all OSDs<\/b> on this server.<\/p>\n<p>If there are no remaining OSDs on this server, then after their removal, you need to exclude the OSD server from the map <code>hv-2<\/code>, by executing the following command:<\/p>\n<pre><code class=\"bash\">ceph osd crush rm hv-2<\/code><\/pre>\n<p>\nRemove <code>mon<\/code> from the server <code>hv-2<\/code>, running the command below on another server (i.e., in this case \u2014 on <code>hv-1<\/code>):<\/p>\n<pre><code class=\"bash\">ceph-deploy mon destroy hv-2<\/code><\/pre>\n<p>\nAfter this, you can stop the server and proceed to the next steps (its redeployment, etc.).<\/p>\n<h2>Case #2. Distributing disk space in an already created Ceph cluster<\/h2>\n<p>\nI'll start the second story with a preface about PG (<noindex><a rel=\"nofollow\" href=\"https:\/\/docs.ceph.com\/docs\/mimic\/rados\/operations\/placement-groups\/\">Placement Groups<\/a><\/noindex>). The primary role of PG in Ceph is primarily to aggregate Ceph objects and further replicate them in OSD. The formula to calculate the necessary number of PG can be found in the <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.ceph.com\/docs\/mimic\/rados\/operations\/placement-groups\/#choosing-the-number-of-placement-groups\">relevant section<\/a><\/noindex> of the Ceph documentation. This issue is also discussed there with concrete examples.<\/p>\n<p>So, one of the common problems during the operation of Ceph is an unbalanced number of OSD and PG among pools in Ceph.<\/p>\n<p>Firstly, this may lead to a situation where too many PGs are specified in a small-sized pool, which is essentially an irrational use of disk space in the cluster. Secondly, in practice, a more serious problem arises: data overflow in one of the OSDs. This leads to the cluster first entering a state <code>HEALTH_WARN<\/code>, and then <code>HEALTH_ERR<\/code>. The cause is that Ceph, when calculating the available data volume (you can find out by using <code>MAX AVAIL<\/code> in the output of the command <code>ceph df<\/code> for each pool individually) relies on the volume of available data in OSD. If there is insufficient space in even one OSD, data cannot be written until it is properly distributed among all OSDs.<\/p>\n<p>It should be clarified that these problems <b>are mostly solved during the configuration stage of the Ceph cluster.<\/b>One of the tools you can use is <noindex><a rel=\"nofollow\" href=\"http:\/\/blog.easter-eggs.org\/index.php\/post\/2013\/09\/27\/%5BLibvirt%5D-Migrating-from-on-disk-raw-images-to-RBD-storage\">Ceph PGCalc<\/a><\/noindex>. It visually calculates the necessary number of PG. However, it can also be used in situations where the Ceph cluster <i>already<\/i> is misconfigured. Here it should be noted that while fixing issues, you will likely need to reduce the number of PG, which is not possible in older versions of Ceph (this feature only appeared starting with version <noindex><a rel=\"nofollow\" href=\"https:\/\/ceph.com\/rados\/new-in-nautilus-pg-merging-and-autotuning\/\">Nautilus<\/a><\/noindex>).<\/p>\n<p>. So, imagine the following picture: the cluster has a status <code>HEALTH_WARN<\/code> because one of the OSDs is running out of space. This will be indicated by the error <code>HEALTH_WARN: 1 near full osd<\/code>. Below is the algorithm for resolving such a situation.<\/p>\n<p>First, it is necessary to redistribute the existing data among the other OSDs. We have already performed such an operation in the first case when we 'drained' the node \u2014 the only difference being that now we will need to slightly reduce the weight. <code>reweight<\/code>For example, to 0.95:<\/p>\n<pre><code class=\"bash\">ceph osd reweight osd.${ID} 0.95<\/code><\/pre>\n<p>\nThis frees up disk space in the OSD and fixes the error in ceph health. However, as mentioned earlier, this problem mainly arises due to incorrect Ceph configuration in the initial stages: it is very important to reconfigure it to prevent this from occurring in the future.<\/p>\n<p>In our specific case, everything boiled down to:<\/p>\n<ul>\n<li> too high a value <code>replication_count<\/code> in one of the pools,<\/li>\n<li> too high a number of PGs in one pool and too low in another. <\/li>\n<\/ul>\n<p>\nWe will use the previously mentioned calculator. It clearly shows what needs to be entered, and overall, it's not complicated. By setting the necessary parameters, we obtain the following recommendations:<\/p>\n<p><i><b>Note<\/b>: if you are setting up a Ceph cluster from scratch, another useful feature of the calculator will be the generation of commands that will create pools from scratch with the parameters specified in the table.<\/i><\/p>\n<p>The last column helps orient: <i>Suggested PG Count<\/i>. In our case, the second one is also useful, which indicates the replication parameter, since we decided to change the replication factor.<\/p>\n<p>So, first, we need to change the replication parameters \u2014 this should be done first, as reducing the factor will free up disk space. During the execution of the command, you may notice that the available disk space increases:<\/p>\n<pre><code class=\"bash\">ceph osd pool $pool_name set $replication_size<\/code><\/pre>\n<p>\nAnd after its completion \u2014 we change the values of the parameters <code>pg_num<\/code> and <code>pgp_num<\/code> as follows:<\/p>\n<pre><code class=\"bash\">ceph osd pool set $pool_name pg_num $pg_number\nceph osd pool set $pool_name pgp_num $pg_number<\/code><\/pre>\n<p>\n<b>Important<\/b>: we must sequentially change the number of PGs in each pool and not change the values in other pools until the warnings disappear <i>\u2018Degraded data redundancy\u2019<\/i> and <i>\u2018n-number of PGs degraded\u2019<\/i>.<\/p>\n<p>You can also verify that everything went successfully by examining the outputs of the commands <code>ceph health detail<\/code> and <code>ceph -s<\/code>.<\/p>\n<h2>Case #3. Migration of a virtual machine from LVM to Ceph RBD<\/h2>\n<p>\nIn situations where virtual machines are used on rented bare-metal servers, questions about fault-tolerant storage often arise. Additionally, it is highly desirable to have enough space in this storage... Another common scenario is having a virtual machine with local storage on the server and needing to expand the disk, but there is nowhere to do so because there\u2019s no free disk space left on the server.<\/p>\n<p>This problem can be solved in various ways\u2014such as migrating to another server (if one is available) or adding new disks to the server. However, this is not always feasible, so migration from LVM to Ceph can be an excellent solution to this issue. Choosing this option also simplifies the further migration process between servers since there will be no need to move local storage from one hypervisor to another. The only catch is that the VM will have to be stopped during the work.<\/p>\n<p>The following recipe is taken from <noindex><a rel=\"nofollow\" href=\"http:\/\/blog.easter-eggs.org\/index.php\/post\/2013\/09\/27\/%5BLibvirt%5D-Migrating-from-on-disk-raw-images-to-RBD-storage\">an article from this blog<\/a><\/noindex>, whose instructions have been tested in practice. By the way, <b>it also describes a method for simple migration<\/b>, but in our case, it wasn't needed, so we didn't validate it. However, if this is critical for your project, we would be happy to hear the results in the comments.<\/p>\n<p>Let's get to the practical part. In this example, we will use virsh and, accordingly, libvirt. First, make sure that the Ceph pool, into which the data will be migrated, is connected to libvirt:<\/p>\n<pre><code class=\"bash\">virsh pool-dumpxml $ceph_pool<\/code><\/pre>\n<p>\nThe pool description should contain connection details to Ceph with authorization data. <\/p>\n<p>The next step involves converting the LVM image to Ceph RBD. The execution time primarily depends on the size of the image:<\/p>\n<pre><code class=\"bash\">qemu-img convert -p -O rbd \/dev\/main\/$vm_image_name rbd:$ceph_pool\/$vm_image_name<\/code><\/pre>\n<p>\nAfter the conversion, the LVM image will remain, which will be useful if migrating the VM to RBD fails and we need to roll back the changes. Also, to allow for a quick rollback, we will create a backup of the virtual machine's configuration file:<\/p>\n<pre><code class=\"bash\">virsh dumpxml $vm_name &gt; $vm_name.xml\ncp $vm_name.xml $vm_name_backup.xml<\/code><\/pre>\n<p>\n\u2026 and we will edit the original (<code>vm_name.xml<\/code>). We will find the block describing the disk (starting from the line <code>&lt;disk type='file' device='disk'&gt;<\/code> and ending with <code>&lt;\/disk&gt;<\/code>) and we will bring it to the following form:<\/p>\n<pre><code class=\"xml\">\\n\n  \\n \n\n  \\n  \\n\n \n\\n<address \/>\\n<\/code><\/pre>\n<p>\nLet's break down some details:<\/p>\n<ol>\n<li> In the protocol <code>source<\/code> the address to the storage in Ceph RBD is specified (this is the address indicating the name of the Ceph pool and the RBD image defined in the first step).<\/li>\n<li> In the section <code>secret<\/code> the type is specified as <code>ceph<\/code>, as well as the UUID of the secret for connecting to it. Its uuid can be obtained using the command <code>virsh secret-list<\/code>.<\/li>\n<li> In the section <code>host<\/code> the addresses of the Ceph monitors are specified.<\/li>\n<\/ol>\n<p>\nAfter editing the configuration file and completing the conversion of LVM to RBD, you can apply the modified configuration file and start the virtual machine:<\/p>\n<pre><code class=\"bash\">virsh define $vm_name.xml\\nvirsh start $vm_name<\/code><\/pre>\n<p>\nNow it's time to check that the virtual machine started correctly: you can find this out, for example, by connecting to it via SSH or through <code>virsh<\/code>. <\/p>\n<p>If the virtual machine is running correctly and you have not found any other issues, you can remove the LVM image that is no longer in use:<\/p>\n<pre><code class=\"bash\">lvremove main\/$vm_image_name<\/code><\/pre>\n<p><\/p>\n<h2>Conclusion<\/h2>\n<p>\nWe have encountered all the described cases in practice \u2014 we hope that the instructions will help other administrators solve similar problems. If you have comments or other similar stories from your experience with Ceph \u2014 we would be delighted to see them in the comments!<\/p>\n<h2>P.S.<\/h2>\n<p>\nAlso read in our blog:<\/p>\n<ul>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/477680\/\">Our hands are not for idleness: restoring the Rook cluster in K8s<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/451818\/\">To Rook or not to Rook \u2014 that is the question<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/348044\/\">Rook is a \u2018self-service\u2019 storage solution for Kubernetes<\/a><\/noindex>\u00bb;<\/li>\n<li> \u00ab<noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/329666\/\">Creating persistent storage with provisioning in Kubernetes based on Ceph<\/a><\/noindex>\u00bb.<\/li>\n<\/ul>\n<p>Source: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/flant\/blog\/495870\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044f Ceph \u043a\u0430\u043a \u0441\u0435\u0442\u0435\u0432\u043e\u0435 \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435 \u0432 \u0440\u0430\u0437\u043d\u044b\u0445 \u043f\u043e \u043d\u0430\u0433\u0440\u0443\u0436\u0435\u043d\u043d\u043e\u0441\u0442\u0438 \u043f\u0440\u043e\u0435\u043a\u0442\u0430\u0445, \u043c\u044b \u043c\u043e\u0436\u0435\u043c \u0441\u0442\u043e\u043b\u043a\u043d\u0443\u0442\u044c\u0441\u044f \u0441 \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u043c\u0438 \u0437\u0430\u0434\u0430\u0447\u0430\u043c\u0438, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0441 \u043f\u0435\u0440\u0432\u043e\u0433\u043e \u0432\u0437\u0433\u043b\u044f\u0434\u0430 \u043d\u0435 \u043a\u0430\u0436\u0443\u0442\u0441\u044f \u043f\u0440\u043e\u0441\u0442\u044b\u043c\u0438 \u0438\u043b\u0438 \u0442\u0440\u0438\u0432\u0438\u0430\u043b\u044c\u043d\u044b\u043c\u0438. \u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440: \u043c\u0438\u0433\u0440\u0430\u0446\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445 \u0438\u0437 \u0441\u0442\u0430\u0440\u043e\u0433\u043e Ceph \u0432 \u043d\u043e\u0432\u044b\u0439 \u0441 \u0447\u0430\u0441\u0442\u0438\u0447\u043d\u044b\u043c \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c \u043f\u0440\u0435\u0434\u044b\u0434\u0443\u0449\u0438\u0445 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0432 \u043d\u043e\u0432\u043e\u043c \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0435; \u0440\u0435\u0448\u0435\u043d\u0438\u0435 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u044b \u0440\u0430\u0441\u043f\u0440\u0435\u0434\u0435\u043b\u0435\u043d\u0438\u044f \u0434\u0438\u0441\u043a\u043e\u0432\u043e\u0433\u043e \u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0441\u0442\u0432\u0430 \u0432 Ceph. \u0420\u0430\u0437\u0431\u0438\u0440\u0430\u044f\u0441\u044c \u0441 \u0442\u0430\u043a\u0438\u043c\u0438 \u0437\u0430\u0434\u0430\u0447\u0430\u043c\u0438, \u043c\u044b \u0441\u0442\u0430\u043b\u043a\u0438\u0432\u0430\u0435\u043c\u0441\u044f \u0441 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":77267,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-77266","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044f Ceph \u043a\u0430\u043a \u0441\u0435\u0442\u0435\u0432\u043e\u0435 \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435 \u0432.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/tips-tricks-v-rabote-s-ceph-v-nagruzhennyh-proektah\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47Tips &amp; tricks \u0432 \u0440\u0430\u0431\u043e\u0442\u0435 \u0441 Ceph \u0432 \u043d\u0430\u0433\u0440\u0443\u0436\u0435\u043d\u043d\u044b\u0445 \u043f\u0440\u043e\u0435\u043a\u0442\u0430\u0445 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044f Ceph \u043a\u0430\u043a \u0441\u0435\u0442\u0435\u0432\u043e\u0435 \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435 \u0432.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/tips-tricks-v-rabote-s-ceph-v-nagruzhennyh-proektah\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-04-09T11:42:08+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-04-09T11:42:08+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Tips &amp; tricks for working with Ceph in demanding projects | ProHoster","description":"Using Ceph as network storage in.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/tips-tricks-v-rabote-s-ceph-v-nagruzhennyh-proektah","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47Tips &amp; tricks \u0432 \u0440\u0430\u0431\u043e\u0442\u0435 \u0441 Ceph \u0432 \u043d\u0430\u0433\u0440\u0443\u0436\u0435\u043d\u043d\u044b\u0445 \u043f\u0440\u043e\u0435\u043a\u0442\u0430\u0445 | ProHoster","og:description":"\u0418\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044f Ceph \u043a\u0430\u043a \u0441\u0435\u0442\u0435\u0432\u043e\u0435 \u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435 \u0432.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/tips-tricks-v-rabote-s-ceph-v-nagruzhennyh-proektah","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-04-09T11:42:08+00:00","article:modified_time":"2020-04-09T11:42:08+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"77266","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 17:20:51","updated":"2022-10-02 10:23:09","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/77266","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=77266"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/77266\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/77267"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=77266"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=77266"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=77266"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}