{"id":35800,"date":"2019-10-31T22:06:24","date_gmt":"2019-10-31T19:06:24","guid":{"rendered":"https:\/\/prohoster.info\/blog\/dizajn-virtualizovannogo-tsod\/"},"modified":"2019-10-31T22:06:24","modified_gmt":"2019-10-31T19:06:24","slug":"dizajn-virtualizovannogo-tsod","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/dizajn-virtualizovannogo-tsod","title":{"rendered":"Design of the Virtualized Data Center","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Design
\n
\nIntroduction<\/b><\/p>\n

An information system, from the user's perspective, is well-defined in GOST R V 51987 as an \"automated system whose functioning results in the presentation of output information for subsequent use.\" If we consider the internal structure, any IS essentially constitutes a system of interconnected algorithms implemented in code. In a broader understanding, Turing's-Church thesis suggests that an algorithm (and consequently the IS) performs a transformation of a multitude of input data into a multitude of output data.
\nOne could even say that the transformation of input data is the very purpose of the existence of an information system. Accordingly, the value of an IS and the entire complex of ISs is determined by the value of the input and output data.
\nBased on this, design should begin by focusing on the data, tailoring the architecture and methods to the structure and significance of the data.<\/p>\n

Stored Data<\/b>
\nA key stage in the preparation for design is obtaining the characteristics of all data sets planned for processing and storage. These characteristics include:
\n \u2014 Volume of data;
\n \u2014 Information about the data lifecycle (increase of new data, lifespan, handling of outdated data);
\n \u2014 Classification of data from the perspective of its impact on the company's core business (the triad of confidentiality, integrity, availability) along with financial metrics (e.g., the cost of data loss over the last hour);
\n \u2014 Geography of data processing (physical location of processing systems);
\n \u2014 Regulatory requirements for each class of data (e.g., Federal Law 152, PCI DSS).<\/p>\n

<\/a><\/noindex><\/p>\n

Information Systems<\/b><\/p>\n

Data is not only stored but also processed (transformed) by information systems. The next step after obtaining data characteristics is to conduct a comprehensive inventory of information systems, their architectural features, interdependencies, and infrastructure requirements in conditional units for four types of resources:
\n \u2014 Processor computing power;
\n \u2014 Volume of RAM;
\n \u2014 Requirements for the volume and performance of data storage systems;
\n \u2014 Requirements for data transmission networks (external channels, channels between IS components).
\nRequirements must be defined for each service\/microservice within the information system.
\nIt is essential to note the need for data on the impact of the information system on the company's core business in terms of the cost of downtime (in currency per hour).<\/p>\n

Threat Model<\/b><\/p>\n

A formal threat model must be available, detailing the threats against which data\/services will be protected. This model includes not only confidentiality aspects but also integrity and availability. For example:
\n \u2014 Failure of the physical server;
\n \u2014 Failure of the top-of-the-rack switch;
\n \u2014 Disruption of the optical communication channel between data centers;
\n \u2014 Complete failure of the operational storage area network.
\nIn some cases, threat models are developed not only for infrastructure components but also for specific information systems or their components, such as the failure of a database management system with logical data structure corruption.
\nAll decisions made within the project to protect against unspecified threats are excessive.<\/p>\n

Regulatory requirements<\/b><\/p>\n

If the data being processed falls under special rules established by regulators, it is mandatory to have information about the data sets and processing\/storage rules.<\/p>\n

Target Indicators RPO \/ RTO<\/b><\/p>\n

Designing any type of protection requires the establishment of target data loss indicators and target service recovery time for each described threat.
\nIdeally, the RPO and RTO should have associated costs for data loss and downtime per unit of time.<\/p>\n

\"Design
\n
\nSegregation into Resource Pools<\/b><\/p>\n

After gathering all the preliminary input information, the first step is to group data sets and information systems into pools based on threat models and regulatory requirements. The type of segregation for different pools is determined \u2013 either programmatic at the system software level or physical.
\nExamples:
\n \u2014 The contour processing personal data is completely physically separated from other systems;
\n \u2014 Backup copies are stored on a separate storage area network.<\/p>\n

At the same time, pools can have partial independence; for example, two computing resource pools (processing power + RAM) can use a single storage data pool and a single data transmission resource pool.<\/p>\n

Processing Power<\/b><\/p>\n

\"Design
\n
\nThe abstract needs for processor power in a virtualized data center are measured by the number of virtual CPUs (vCPU) and their consolidation ratio on physical processors (pCPU). In this specific case, 1 pCPU equals 1 physical core of the processor (without considering Hyper-Threading). The number of vCPUs is summed across all defined resource pools (each of which may have its own consolidation ratio).
\nThe consolidation ratio for loaded systems is obtained empirically, based on the existing infrastructure or during pilot installations and load testing. For unloaded systems, 'best practices' are applied. In particular, VMware cites an average ratio of 8:1.<\/p>\n

Memory<\/b><\/p>\n

The total memory requirement is obtained by simple summation. Overcommitment of memory is not recommended.<\/p>\n

Storage Resources<\/b><\/p>\n

Storage resource requirements are obtained by summing all pools based on capacity and performance.
\nPerformance requirements are expressed in IOPS in conjunction with the average read\/write ratio and, if necessary, maximum response latency.
\nRequirements for quality of service (QoS) should be specified separately for specific pools or systems.<\/p>\n

Data Transmission Network Resources<\/b><\/p>\n

Data transmission network requirements are obtained by simply summing all bandwidth pools.
\nRequirements for quality of service (QoS) and latencies (RTT) should be specified separately for specific pools or systems.
\nWithin the requirements for data transmission network resources, requirements for isolation and\/or encryption of network traffic and preferred mechanisms (802.1q, IPSec, etc.) are also indicated.<\/p>\n

Architecture Selection<\/b><\/p>\n

This guide does not consider any choice other than x86 architecture and 100% server virtualization. Therefore, the choice of the computing subsystem architecture is reduced to selecting the server virtualization platform, server form factor, and overall server configuration requirements.<\/p>\n

A key point in the selection is clarity on using the classic approach with separation of data processing, storage, and transmission functions or a converged one.<\/p>\n

Classic Architecture<\/i> implies the use of intelligent external storage and data transmission subsystems, while servers contribute only processing power and RAM to the overall pool of physical resources. In extreme cases, servers become completely anonymous, lacking not only their own disks but even a system identifier. In this case, the operating system or hypervisor is booted from embedded flash drives or from an external storage system (boot from SAN).
\nWithin the classical architecture, the choice between blades and racks is primarily based on the following principles:
\n \u2014 Economic efficiency (on average, rack servers are cheaper);
\n \u2014 Computational density (blades have higher density);
\n \u2014 Power consumption and heat generation (blades have higher values per unit);
\n \u2014 Scalability and manageability (blades generally require less effort for large installations);
\n \u2014 Use of expansion cards (the choice for blades is very limited).
\nConverged architecture<\/i> (also known as hyper-converged<\/i>) combines processing and storage functions, leading to the use of local disks in servers and consequently abandoning the form factor of traditional blades. Converged systems use either rack servers or clustered systems that combine multiple blade servers and local disks within a single chassis.<\/p>\n

CPU \/ Memory <\/b><\/p>\n

To correctly calculate the configuration, one must understand the type of load for the environment or each of the independent clusters.
\nCPU bound<\/i> \u2013 an environment constrained by processing power. Adding RAM will not change performance (the number of VMs on the server).
\nMemory bound<\/i> \u2013 an environment constrained by RAM. More RAM on the server allows for a greater number of VMs to be run on the server.
\nGB \/ MHz (GB \/ pCPU) \u2013 the average ratio of RAM and processing power consumption for a particular load. This can be used to calculate the necessary amount of memory for a given performance and vice versa. <\/p>\n

Server configuration calculation<\/b><\/p>\n

\"Design
\n
\nFirst, it is necessary to identify all types of loads and decide whether to combine or separate different computing pools across various clusters.
\nNext, for each of the identified clusters, the ratio of GB \/ MHz is determined based on the known load in advance. If the load is not known in advance but there is a rough understanding of the level of CPU utilization, standard vCPU:pCPU ratios can be used to translate pool requirements into physical resources. <\/p>\n

For each cluster, the total vCPU requirements of the pools are divided by the ratio:
\nvCPU sum \/ vCPU:pCPU = pCPU sum \u2013 required number of physical cores
\npCPU sum \/ 1.25 = pCPU ht \u2013 number of cores adjusted for Hyper-Threading
\nAssume we need to calculate a cluster with 190 cores \/ 3.5 TB of RAM. In this case, we assume a target CPU utilization of 50% and 75% for RAM.<\/p>\n

pCPU<\/b>
\n190
\nCPU util<\/b>
\n50%<\/p>\n

Mem<\/b>
\n3500
\nMem util<\/b>
\n75%<\/p>\n

Socket<\/b>
\nCore<\/b>
\nSrv \/ CPU<\/b>
\nSrv Mem<\/b>
\nSrv \/ Mem<\/b><\/p>\n

2
\n6
\n25,3
\n128
\n36,5<\/p>\n

2
\n8
\n19,0
\n192
\n24,3<\/p>\n

2
\n10
\n15,2
\n256
\n18,2<\/p>\n

2
\n14
\n10,9
\n384
\n12,2<\/p>\n

2
\n18
\n8,4
\n512
\n9,1<\/p>\n

In this case, we always use rounding up to the nearest whole number (=ROUNDUP(A1;0)).
\nFrom the table, it becomes clear that several server configurations meet the target metrics:
\n \u2014 26 servers 2*6c \/ 192 GB
\n \u2014 19 servers 2*10c \/ 256 GB
\n \u2014 10 servers 2*18c \/ 512 GB <\/p>\n

The choice among these configurations must be made based on additional factors, such as thermal capacity and available cooling, existing servers, or cost.<\/p>\n

Considerations for choosing a server configuration<\/b><\/p>\n

Wide VMs. When placing wide VMs (comparable to 1 NUMA node and larger), it is recommended to choose a server configuration that allows such VMs to stay within a NUMA node if possible. With a large number of wide VMs, there is a risk of fragmenting cluster resources, and in this case, servers should be chosen that allow for the densest placement of wide VMs.<\/p>\n

Single point of failure domain size.<\/b> <\/p>\n

The choice of server size is also based on the principle of minimizing the single point of failure domain. For example, when choosing between:
\n \u2014 3 x 4*10c \/ 512 GB
\n \u2014 6 x 2*10c \/ 256 GB
\nWith all else being equal, the second option should be chosen, since if one server fails (or is under maintenance), not 33% of cluster resources are lost, but 17%. Similarly, the number of VMs and IS affected by the failure is halved.<\/p>\n

Calculating the performance of a classic storage system<\/b><\/p>\n

\"Design
\n
\nThe classic storage system is always calculated based on the worst-case scenario, excluding the influence of operational cache and optimization of operations.
\nWe will consider the mechanical performance from the disk (IOPSdisk) as the basic performance indicators:
\n \u2014 7.2k \u2013 75 IOPS
\n \u2014 10k \u2013 125 IOPS
\n \u2014 15k \u2013 175 IOPS<\/p>\n

Next, the number of disks in the disk pool is calculated using the following formula: = TotalIOPS * ( RW + (1 \u2013RW) * RAIDPen) \/ IOPSdisk<\/i>. Where:
\n \u2014 TotalIOPS<\/i> \u2013 total required performance in IOPS from the disk pool
\n \u2014 RW<\/i> \u2013 percentage of read operations
\n \u2014 RAIDpen<\/i> \u2013 RAID penalty for the selected RAID level<\/p>\n

Learn more about RAID devices and RAID Penalty here \u2014 Performance of storage systems. Part one.<\/a><\/noindex> and Performance of storage systems. Part two.<\/a><\/noindex> and Performance of storage systems. Part three.<\/a><\/noindex><\/p>\n

Based on the number of disks obtained, possible options are calculated that meet the storage capacity requirements, including those with tiered storage.
\nThe calculation of systems using SSD as a storage tier is considered separately.
\nFeatures of calculating systems with Flash Cache
\n<\/b><\/p>\n

Flash Cache<\/i> \u2013 a general term for all proprietary technologies that use flash memory as a second-level cache. When using flash cache, the storage system is usually calculated to meet the established load from magnetic disks, while peak loads are handled by the cache.
\nIt is important to understand the load profile and the degree of localization of accesses to the storage volume blocks. Flash cache is a technology for workloads with high request localization and is practically not applicable for evenly loaded volumes (as in analytical systems). <\/p>\n

Calculation of hybrid systems low-end \/ mid-range<\/b><\/p>\n

Hybrid systems of the lower and mid-range classes use tiered storage with data movement between levels on a schedule. The block size of tiered storage in the best models is 256 MB. These characteristics do not allow the tiered storage technology to be considered a performance-enhancing technology, as is mistakenly believed by many. Tiered storage in lower and mid-range systems is a cost optimization technology for systems with pronounced uneven load.<\/p>\n

For multi-tier storage, performance is primarily calculated at the upper level, while the lower storage layer is considered only to provide the missing storage capacity. For a hybrid multi-tier system, it is essential to use flash cache technology within the multi-tier pool to compensate for performance drops for suddenly heated data from the lower level.<\/p>\n

Using SSDs in a multi-tier disk pool<\/b><\/p>\n

\"Design
\n
\nUsing SSDs in a multi-tier disk pool varies depending on the specific implementation of flash cache algorithms by each manufacturer.
\nThe general practice for storage policy in an SSD-level disk pool is SSD first.
\nRead Only Flash Cache.<\/i> For read-only flash cache, the SSD storage level appears when write operations are significantly localized, regardless of the cache.
\nRead \/ Write Flash Cache.<\/i> In the case of write flash cache, the maximum cache size is first established, and the SSD storage level appears only when the cache size is insufficient to handle the entire localized load.
\nThe performance of SSDs and cache is calculated each time based on manufacturer recommendations, but always for the worst-case scenario.
\n
Source: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0418\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0441 \u0442\u043e\u0447\u043a\u0438 \u0437\u0440\u0435\u043d\u0438\u044f \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044f \u0445\u043e\u0440\u043e\u0448\u043e \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u044f\u0435\u0442\u0441\u044f \u0432 \u0413\u041e\u0421\u0422 \u0420\u0412 51987 \u2014 \u00ab\u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430, \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u043c \u0444\u0443\u043d\u043a\u0446\u0438\u043e\u043d\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043a\u043e\u0442\u043e\u0440\u043e\u0439 \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u0432\u044b\u0445\u043e\u0434\u043d\u043e\u0439 \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u0438 \u0434\u043b\u044f \u043f\u043e\u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0435\u0433\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f\u00bb. \u0415\u0441\u043b\u0438 \u0440\u0430\u0441\u0441\u043c\u0430\u0442\u0440\u0438\u0432\u0430\u0442\u044c \u0432\u043d\u0443\u0442\u0440\u0435\u043d\u043d\u044e\u044e \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0443, \u0442\u043e \u043f\u043e \u0441\u0443\u0442\u0438 \u043b\u044e\u0431\u0430\u044f \u0418\u0421 \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u043e\u0439 \u0440\u0435\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043d\u043d\u044b\u0445 \u0432 \u043a\u043e\u0434\u0435 \u0432\u0437\u0430\u0438\u043c\u043e\u0441\u0432\u044f\u0437\u0430\u043d\u043d\u044b\u0445 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c\u043e\u0432. \u0412 \u0448\u0438\u0440\u043e\u043a\u043e\u043c \u043f\u043e\u043d\u0438\u043c\u0430\u043d\u0438\u0438 \u0442\u0435\u0437\u0438\u0441\u0430 \u0422\u044c\u044e\u0440\u0438\u043d\u0433\u0430-\u0427\u0435\u0440\u0447\u0430 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c (\u0430 \u0441\u043b-\u043d\u043e \u0418\u0421) \u043e\u0441\u0443\u0449\u0435\u0441\u0442\u0432\u043b\u044f\u0435\u0442 \u0442\u0440\u0430\u043d\u0441\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044e \u043c\u043d\u043e\u0436\u0435\u0441\u0442\u0432\u0430 \u0432\u0445\u043e\u0434\u043d\u044b\u0445 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":26806,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-35800","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Design of Virtualized Data Centers | ProHoster","description":"Introduction to Information System Points.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/dizajn-virtualizovannogo-tsod","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0414\u0438\u0437\u0430\u0439\u043d \u0432\u0438\u0440\u0442\u0443\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043d\u043d\u043e\u0433\u043e \u0426\u041e\u0414 | ProHoster","og:description":"\u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0418\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0441 \u0442\u043e\u0447\u043a\u0438.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/dizajn-virtualizovannogo-tsod","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:06:24+00:00","article:modified_time":"2019-10-31T19:06:24+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"35800","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 00:48:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 18:06:24","updated":"2026-01-22 00:48:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/35800","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=35800"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/35800\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/26806"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=35800"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=35800"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=35800"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}