{"id":38966,"date":"2019-10-31T22:27:03","date_gmt":"2019-10-31T19:27:03","guid":{"rendered":"https:\/\/prohoster.info\/blog\/admin-bez-ruk-giperkonvergentsiya\/"},"modified":"2019-10-31T22:27:03","modified_gmt":"2019-10-31T19:27:03","slug":"admin-bez-ruk-giperkonvergentsiya","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/admin-bez-ruk-giperkonvergentsiya","title":{"rendered":"Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103?","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103?\" src=\"\/wp-content\/uploads\/2019\/10\/ea1763132185c97bb4edf0e67640d58f.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<img decoding=\"async\" alt=\"Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103?\" src=\"\/wp-content\/uploads\/2019\/10\/b6355c444c310570e1f2619cd931e1ca.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAcesta este un mit destul de r\u0103sp\u00e2ndit \u00een domeniul echipamentului server. \u00cen practic\u0103, solu\u021biile hiperconvergente (c\u00e2nd totul este integrat) sunt necesare din multe puncte de vedere. Istoric, primele arhitecturi au fost dezvoltate de Amazon \u0219i Google pentru serviciile lor. Atunci ideea era s\u0103 se creeze o ferm\u0103 de calcul din noduri identice, fiecare av\u00e2nd propriile discuri. Totul era integrat printr-un software sistemic (hypervisor) \u0219i fragmentat \u00een ma\u0219ini virtuale. Principala sarcin\u0103 era minimizarea eforturilor pentru \u00eentre\u021binerea unui nod \u0219i minimizarea problemelor la scalare: pur \u0219i simplu se mai ad\u0103ugau \u00eenc\u0103 o mie-dou\u0103 de servere asem\u0103n\u0103toare \u0219i se conectau al\u0103turi. \u00cen practic\u0103, acestea sunt cazuri rare, iar mai des se vorbe\u0219te despre un num\u0103r mai mic de noduri \u0219i o arhitectur\u0103 pu\u021bin diferit\u0103. <\/p>\n<p>Dar avantajul r\u0103m\u00e2ne acela\u0219i \u2013 o scalare \u0219i o gestionare incredibil de simpl\u0103. Dezavantajul \u2013 diferitele sarcini consum\u0103 resurse \u00een mod diferit \u0219i, \u00een unele locuri, pot fi multe discuri locale, \u00een altele, pu\u021bin\u0103 memorie RAM \u0219i a\u0219a mai departe, deci, \u00een func\u021bie de tipul sarcinii, utilizarea resurselor va sc\u0103dea. <\/p>\n<p>Se dovedea c\u0103 pl\u0103ti\u021bi cu 10\u201315% mai mult pentru confortul configur\u0103rii. Aceasta a generat mitul din titlu. Am c\u0103utat mult timp unde ar putea fi utilizat\u0103 tehnologia optim, \u0219i am g\u0103sit. Problema era c\u0103 Cisco nu avea propriile solu\u021bii de stocare, dar vroiau s\u0103 acopere \u00eentreaga pia\u021b\u0103 a serverelor. \u0218i au creat Cisco Hyperflex \u2013 o solu\u021bie cu stocare local\u0103 pe noduri. <\/p>\n<p>\u0218i din aceasta a rezultat o solu\u021bie foarte bun\u0103 pentru centrele de date de rezerv\u0103 (Disaster Recovery). De ce \u0219i cum \u2013 voi explica acum. \u0218i voi ar\u0103ta teste ale clusterului. <noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h3>Acolo unde este nevoie<\/h3>\n<p>\nHiperconvergen\u021ba este: <\/p>\n<ol>\n<li>Transferul discurilor \u00een nodurile de calcul.<\/li>\n<li>Integrarea complet\u0103 a subsistemului de stocare cu subsistemul de virtualizare.<\/li>\n<li>Transferul\/integrarea cu subsistemul de re\u021bea.<\/li>\n<\/ol>\n<p>\nAceast\u0103 leg\u0103tur\u0103 permite implementarea multor func\u021bii ale solu\u021biilor de stocare la nivelul virtualiz\u0103rii, totul dintr-o singur\u0103 interfa\u021b\u0103 de gestionare.<\/p>\n<p>\u00cen compania noastr\u0103, proiectele de proiectare a centrelor de date de rezerv\u0103 sunt foarte solicitate, iar solu\u021biile hiperconvergente sunt adesea alese datorit\u0103 numeroaselor op\u021biuni de replicare (p\u00e2n\u0103 la metrokluster) din cutie. <\/p>\n<p>\u00cen cazul centrelor de date de rezerv\u0103, de obicei, ne referim la o loca\u021bie \u00eendep\u0103rtat\u0103 situat\u0103 la marginea ora\u0219ului sau chiar \u00eentr-un alt ora\u0219. Aceasta permite restaurarea sistemelor critice \u00een cazul unei defec\u021biuni par\u021biale sau complete a centrului de date principal. Datele sunt repliocate constant de la v\u00e2nzare, iar aceast\u0103 replicare poate fi efectuat\u0103 la nivel de aplica\u021bie sau la nivel de dispozitiv de stocare (SDS).<\/p>\n<p>Prin urmare, voi discuta acum despre structura sistemului \u0219i teste, iar apoi \u2014 despre c\u00e2teva scenarii reale de aplicare cu date despre economii. <\/p>\n<h3>Teste<\/h3>\n<p>\nInstan\u021ba noastr\u0103 const\u0103 din patru servere, fiecare av\u00e2nd 10 SSD-uri de 960 GB. Exist\u0103 un disc dedicat pentru cache-ul opera\u021biunilor de scriere \u0219i stocarea unei ma\u0219ini virtuale de servicii. Solu\u021bia \u00een sine este a patra versiune. Prima a fost evident nefinisat\u0103 (conform recenziilor), a doua era \u00eenc\u0103 pu\u021bin lipsit\u0103 de rafinament, a treia este deja suficient de stabil\u0103, iar aceasta poate fi considerat\u0103 o versiune de lansare dup\u0103 \u00eencheierea test\u0103rii beta pe scar\u0103 larg\u0103. \u00cen timpul test\u0103rii, nu am \u00eent\u00e2mpinat probleme, totul func\u021bioneaz\u0103 ca un ceas.<\/p>\n<p><b class=\"spoiler_title\">Modific\u0103ri \u00een v4<\/b>O mul\u021bime de erori au fost corectate. <\/p>\n<p>Ini\u021bial, platforma putea lucra doar cu hypervisorul VMware ESXi \u0219i sus\u021binea un num\u0103r mic de noduri. De asemenea, procesul de implementare nu se \u00eencheia \u00eentotdeauna cu succes, trebuia s\u0103 repornesc anumite etape, erau probleme cu actualizarea de la versiuni mai vechi, datele din GUI nu se afi\u0219au \u00eentotdeauna corect (de\u0219i nici acum nu sunt pe deplin mul\u021bumit de afi\u0219area graficelor de performan\u021b\u0103), uneori ap\u0103reau probleme la interfa\u021ba cu virtualizarea.<\/p>\n<p>Acum toate problemele ini\u021biale au fost corectate, HyperFlex suport\u0103 at\u00e2t ESXi, c\u00e2t \u0219i Hyper-V, plus c\u0103 este posibil:<\/p>\n<ol>\n<li>Crearea unui cluster extins. <\/li>\n<li>Crearea unui cluster pentru birouri f\u0103r\u0103 utilizarea Fabric Interconnect, de la dou\u0103 la patru noduri (cump\u0103r\u0103m doar serverele).<\/li>\n<li>Posibilitatea de a lucra cu SDS externe.<\/li>\n<li>Suport pentru containere \u0219i Kubernetes.<\/li>\n<li>Crearea zonelor de disponibilitate.<\/li>\n<li>Integrarea cu VMware SRM, dac\u0103 func\u021bionalitatea \u00eencorporat\u0103 nu este satisf\u0103c\u0103toare.<\/li>\n<\/ol>\n<p>Arhitectura nu se deosebe\u0219te mult de solu\u021biile principalelor competitori, nu s-a reinventat roata. Toate acestea func\u021bioneaz\u0103 pe platforma de virtualizare VMware sau Hyper-V. Tehnologia este implementat\u0103 pe servere dezvoltate intern de Cisco UCS. Exist\u0103 persoane care ur\u0103sc platforma din cauza complexit\u0103\u021bii ini\u021biale a configur\u0103rii, a numeroaselor butoane, a sistemului de \u0219abloane \u0219i dependen\u021be neintuitive, dar exist\u0103 \u0219i cei care au \u00eenv\u0103\u021bat s\u0103 o aprecieze, s-au ata\u0219at de idee \u0219i nu mai doresc s\u0103 lucreze cu alte servere. <\/p>\n<p>Vom examina exact solu\u021bia pentru VMware, deoarece aceasta a fost dezvoltat\u0103 ini\u021bial pentru acesta \u0219i ofer\u0103 un func\u021bionalitate mai mare, \u00een timp ce Hyper-V a fost adaptat pe parcurs pentru a nu r\u0103m\u00e2ne \u00een urm\u0103 fa\u021b\u0103 de competitori \u0219i pentru a r\u0103spunde a\u0219tept\u0103rilor pie\u021bei.<\/p>\n<p>Exist\u0103 un cluster de servere echipate cu discuri. Exist\u0103 discuri pentru stocarea datelor (SSD sau HDD \u2014 alegerea este a dumneavoastr\u0103, \u00een func\u021bie de necesit\u0103\u021bi), plus un disc SSD pentru caching. Atunci c\u00e2nd se scriu date pe datastore, acestea sunt salvate pe stratul de caching (disc SSD dedicat \u0219i RAM-ul ma\u0219inii virtuale de serviciu). \u00cen paralel, un bloc de date este trimis c\u0103tre nodurile din cluster (num\u0103rul de noduri depinde de factorul de replicare al clusterului). Dup\u0103 confirmarea scrierii reu\u0219ite de c\u0103tre toate nodurile, confirmarea este trimis\u0103 hiper-vizorului \u0219i mai departe c\u0103tre ma\u0219ina virtual\u0103. Datele scrise sunt deduplicat, compresate \u0219i stocate pe discurile de stocare \u00een fundal. \u00cen acest mod, pe discurile de stocare se scrie \u00eentotdeauna un bloc mare de date, consecutiv, ceea ce reduce sarcina pe discurile de stocare.<\/p>\n<p>Deduplicarea \u0219i compresia sunt activate constant \u0219i nu pot fi dezactivate. Citirea datelor se face direct de pe discurile de stocare sau din cache-ul RAM. Dac\u0103 se folose\u0219te o configura\u021bie hibrid\u0103, citirea este, de asemenea, stocat\u0103 \u00een cache pe discul SSD.<\/p>\n<p>Datele nu sunt legate de loca\u021bia actual\u0103 a ma\u0219inii virtuale \u0219i sunt distribuite uniform \u00eentre noduri. Aceast\u0103 abordare permite o \u00eenc\u0103rcare uniform\u0103 a tuturor discurilor \u0219i interfe\u021belor de re\u021bea. Un dezavantaj evident este c\u0103 nu putem minimiza la maxim \u00eent\u00e2rzierea citirii, deoarece nu exist\u0103 nicio garan\u021bie c\u0103 datele sunt disponibile local. Dar consider c\u0103 este o renun\u021bare nesemnificativ\u0103 \u00een compara\u021bie cu avantajele ob\u021binute. Mai ales c\u0103 \u00eent\u00e2rzierea \u00een re\u021bea a atins valori astfel \u00eenc\u00e2t s\u0103 nu influen\u021beze semnificativ rezultatul total.<\/p>\n<p>Toat\u0103 logica de func\u021bionare a subsistemului de stocare este gestionat\u0103 de o ma\u0219in\u0103 virtual\u0103 de serviciu special\u0103, Cisco HyperFlex Data Platform controller, care este creat\u0103 pe fiecare nod de stocare. \u00cen configura\u021bia noastr\u0103, ma\u0219inii virtuale de serviciu i-au fost alocate opt vCPU \u0219i 72 GB RAM, ceea ce nu este deloc pu\u021bin. Amintesc c\u0103 hostul are disponibili 28 de nuclee fizice \u0219i 512 GB RAM.<\/p>\n<p>Ma\u0219ina virtual\u0103 de serviciu are acces direct la discurile fizice prin intermediul controller-ului SAS. Comunica\u021bia cu hypervisorul se realizeaz\u0103 printr-un modul special, IOVisor, care intercepta opera\u021biile de intrare-ie\u0219ire, \u0219i printr-un agent care permite transmiterea comenzilor c\u0103tre API-ul hypervisorului. Agentul se ocup\u0103 de gestionarea snapshot-urilor \u0219i clonelor HyperFlex.<\/p>\n<p>\u00cen hypervisor, resursele de disc sunt montate ca share-uri NFS sau SMB (\u00een func\u021bie de tipul hypervisorului, ghici\u021bi care este care). Sub capot\u0103, aceasta este un sistem de fi\u0219iere distribuit care permite ad\u0103ugarea de func\u021bionalit\u0103\u021bi pentru sisteme de stocare complete: rezervarea fin\u0103 a volumelor, comprimare \u0219i deduplicare, snapshot-uri prin tehnologia Redirect-on-Write, replicare sincron\u0103\/asincron\u0103.<\/p>\n<p>Ma\u0219ina virtual\u0103 de serviciu ofer\u0103 acces la interfa\u021ba WEB de gestionare a subsistemului HyperFlex. Exist\u0103 integrare cu vCenter, iar cea mai mare parte a sarcinilor de zi cu zi pot fi realizate din acesta, dar datastor-urile, de exemplu, sunt mai u\u0219or de gestionat dintr-o interfa\u021b\u0103 web separat\u0103, dac\u0103 a\u021bi trecut deja la interfa\u021ba rapid\u0103 HTML5, sau pute\u021bi folosi clientul Flash complet integrat. \u00cen interfa\u021ba web de serviciu pute\u021bi verifica performan\u021ba \u0219i starea detaliat\u0103 a sistemului.<\/p>\n<p><img decoding=\"async\" alt=\"Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103?\" src=\"\/wp-content\/uploads\/2019\/10\/4f603b38d6489836892bebc76663eb2f.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nExist\u0103 \u0219i un alt tip de noduri \u00een cluster \u2014 noduri de calcul. Acestea pot fi servere rack sau blade f\u0103r\u0103 discuri \u00eencorporate. Pe aceste servere pot fi rulate ma\u0219ini virtuale ale c\u0103ror date sunt stocate pe serverele cu discuri. Din punct de vedere al accesului la date, nu exist\u0103 nicio diferen\u021b\u0103 \u00eentre tipurile de noduri, deoarece arhitectura presupune abstractizarea de la loca\u021bia fizic\u0103 a datelor. Raportul maxim \u00eentre nodurile de calcul \u0219i nodurile de stocare este 2:1.<\/p>\n<p>Utilizarea nodurilor de calcul cre\u0219te flexibilitatea la scalarea resurselor cluster-ului: nu trebuie neap\u0103rat s\u0103 achizi\u021bion\u0103m noduri cu discuri dac\u0103 avem nevoie doar de CPU\/RAM. \u00cen plus, putem ad\u0103uga un chassis blade \u0219i ob\u021bine economii la amplasarea serverelor \u00een rack.<\/p>\n<p>Ca rezultat, avem o platform\u0103 hiperconvergent\u0103 cu urm\u0103toarele caracteristici:<\/p>\n<ul>\n<li>P\u00e2n\u0103 la 64 de noduri \u00een cluster (p\u00e2n\u0103 la 32 de noduri de stocare).<\/li>\n<li>Num\u0103rul minim de noduri \u00een cluster este de trei (dou\u0103 pentru clusterul Edge).<\/li>\n<li>Mecanism de redundan\u021b\u0103 a datelor: mirroring cu factor de replicare 2 \u0219i 3.<\/li>\n<li>Cluster Metro.<\/li>\n<li>Replicare asincron\u0103 a VM-urilor pe un alt cluster HyperFlex.<\/li>\n<li>Orchestrarea comut\u0103rii VM-urilor \u00een centrul de date remote.<\/li>\n<li>Instantanee native pe tehnologia Redirect-on-Write.<\/li>\n<li>P\u00e2n\u0103 la 1 PB de spa\u021biu util cu factor de replicare 3 \u0219i f\u0103r\u0103 a \u021bine cont de deduplicare. Factorul de replicare 2 nu este luat \u00een considerare, deoarece nu este o op\u021biune viabil\u0103 pentru v\u00e2nz\u0103ri serioase.<\/li>\n<\/ul>\n<p>\nUn alt avantaj major este u\u0219urin\u021ba de gestionare \u0219i implementare. Toat\u0103 complexitatea configur\u0103rii serverelor UCS este preluat\u0103 de o VM specializat\u0103, preg\u0103tit\u0103 de inginerii Cisco. <\/p>\n<h3>Configura\u021bia bancului de testare:<\/h3>\n<p><\/p>\n<ul>\n<li>2 x Cisco UCS Fabric Interconnect 6248UP ca cluster de control \u0219i componente de re\u021bea (48 porturi, func\u021bion\u00e2nd \u00een mod Ethernet 10G\/FC 16G).<\/li>\n<li>Patru servere Cisco UCS HXAF240 M4.<\/li>\n<\/ul>\n<p>\nSpecifica\u021biile serverelor:<\/p>\n<p><\/p>\n<p>CPU<\/p>\n<p>2 x Intel \u00ae Xeon \u00ae E5-2690 v4<\/p>\n<p>RAM<\/p>\n<p>16 x 32GB DDR4-2400-MHz RDIMM\/PC4-19200\/rang dublu\/x4\/1.2v<\/p>\n<p>Re\u021bea<\/p>\n<p>UCSC-MLOM-CSC-02 (VIC 1227). 2 porturi Ethernet 10G<\/p>\n<p>HBA de stocare<\/p>\n<p>Cisco 12G Modular SAS Pass through Controller<\/p>\n<p>Discuri de stocare<\/p>\n<p>1 x SSD Intel S3520 120 GB, 1 x SSD Samsung MZ-IES800D, 10 x SSD Samsung PM863a 960 GB<\/p>\n<p>\n<b class=\"spoiler_title\">Mai multe op\u021biuni de configura\u021bie<\/b>\u00cen afar\u0103 de hardware-ul ales, \u00een prezent sunt disponibile urm\u0103toarele op\u021biuni:<\/p>\n<ul>\n<li>HXAF240c M5.<\/li>\n<li>Unul sau dou\u0103 CPU \u00eencep\u00e2nd de la Intel Silver 4110 p\u00e2n\u0103 la Intel Platinum I8260Y. Este disponibil\u0103 a doua genera\u021bie.<\/li>\n<li>24 sloturi de memorie, module de la 16 GB RDIMM 2600 p\u00e2n\u0103 la 128 GB LRDIMM 2933.<\/li>\n<li>De la 6 la 23 de discuri pentru date, un disc de cache, un disc sistem \u0219i un disc de boot.<\/li>\n<\/ul>\n<p>\n<b>Discuri de capacitate<\/b><\/p>\n<ul>\n<li>HX-SD960G61X-EV 960GB 2.5 inch Enterprise Value 6G SATA SSD (1X endurance) SAS 960 GB.<\/li>\n<li>HX-SD38T61X-EV 3.8TB 2.5 inch Enterprise Value 6G SATA SSD (1X endurance) SAS 3.8 TB.<\/li>\n<li>Discuri de cache<\/li>\n<li>HX-NVMEXPB-I375 375GB 2.5 inch Intel Optane Drive, Extreme Perf &amp; Endurance.<\/li>\n<li>HX-NVMEHW-H1600* 1.6TB 2.5 inch Ent. Perf. NVMe SSD (3X endurance) NVMe 1.6 TB.<\/li>\n<li>HX-SD400G12TX-EP 400GB 2.5 inch Ent. Perf. 12G SAS SSD (10X endurance) SAS 400 GB.<\/li>\n<li>HX-SD800GBENK9** 800GB 2.5 inch Ent. Perf. 12G SAS SED SSD (10X endurance) SAS 800 GB.<\/li>\n<li>HX-SD16T123X-EP 1.6TB 2.5 inch Enterprise performance 12G SAS SSD (3X endurance).<\/li>\n<\/ul>\n<p>\n<b>Discuri de sistem \/ jurnal<\/b><\/p>\n<ul>\n<li>HX-SD240GM1X-EV 240GB 2.5 inch Enterprise Value 6G SATA SSD (Necesit\u0103 actualizare).<\/li>\n<\/ul>\n<p>\n<b>Discuri de boot<\/b><\/p>\n<ul>\n<li>HX-M2-240GB 240GB SATA M.2 SSD SATA 240 GB.<\/li>\n<\/ul>\n<p>Conectare la re\u021bea prin porturi Ethernet de 40G, 25G sau 10G. <\/p>\n<p>Ca FI, pot fi utilizate HX-FI-6332 (40G), HX-FI-6332-16UP (40G), HX-FI-6454 (40G\/100G).<\/p>\n<h3>Testul \u00een sine<\/h3>\n<p>\nPentru testarea subsistemului de discuri, am folosit HCIBench 2.2.1. Este o unealt\u0103 gratuit\u0103 care permite automatizarea gener\u0103rii de sarcini din mai multe ma\u0219ini virtuale. Sarcina este generat\u0103 de fio obi\u0219nuit. <\/p>\n<p>Clusterul nostru const\u0103 din patru noduri, factor de replicare 3, toate discurile Flash.<\/p>\n<p>Pentru testare, am creat patru datastore-uri \u0219i opt ma\u0219ini virtuale. Pentru teste de scriere, se consider\u0103 un scenariu \u00een care discul de cache nu este supra\u00eenc\u0103rcat.<\/p>\n<p>Rezultatele testelor sunt urm\u0103toarele:<\/p>\n<p>100 % Citire 100 % Aleatorie<\/p>\n<p>0 % Citire 100% Aleatorie<\/p>\n<p>Bloca\u021bi \/ ad\u00e2ncimea cozii<\/p>\n<p>128<\/p>\n<p>256<\/p>\n<p>512<\/p>\n<p>1024<\/p>\n<p>2048<\/p>\n<p>128<\/p>\n<p>256<\/p>\n<p>512<\/p>\n<p>1024<\/p>\n<p>2048<\/p>\n<p>4K<\/p>\n<p>0,59 ms 213804 IOPS<\/p>\n<p>0,84 ms 303540 IOPS<\/p>\n<p>1,36 ms 374348 IOPS<\/p>\n<p>2,47 ms 414116 IOPS<\/p>\n<p><b>4,86 ms 420180 IOPS<\/b><\/p>\n<p>2,22 ms 57408 IOPS<\/p>\n<p>3,09 ms 82744 IOPS<\/p>\n<p>5,02 ms 101824 IOPS<\/p>\n<p>8,75 ms 116912 IOPS<\/p>\n<p><b>17,2 ms 118592 IOPS<\/b><\/p>\n<p>8K<\/p>\n<p>0,67 ms 188416 IOPS<\/p>\n<p>0,93 ms 273280 IOPS<\/p>\n<p>1,7 ms 299932 IOPS<\/p>\n<p>2,72 ms 376484 IOPS<\/p>\n<p><b>5,47 ms 373176 IOPS<\/b><\/p>\n<p>3,1 ms 41148 IOPS<\/p>\n<p>4,7 ms 54396 IOPS<\/p>\n<p>7,09 ms 72192 IOPS<\/p>\n<p><b>12,77 ms 80132 IOPS<\/b><\/p>\n<p>16K<\/p>\n<p>0,77 ms 164116 IOPS<\/p>\n<p>1,12 ms 228328 IOPS<\/p>\n<p>1,9 ms 268140 IOPS<\/p>\n<p><b>3,96 ms 258480 IOPS<\/b><\/p>\n<p>3,8 ms 33640 IOPS<\/p>\n<p>6,97 ms 36696 IOPS<\/p>\n<p><b>11,35 ms 45060 IOPS<\/b><\/p>\n<p>32K<\/p>\n<p>1,07 ms 119292 IOPS<\/p>\n<p>1,79 ms 142888 IOPS<\/p>\n<p><b>3,56 ms 143760 IOPS<\/b><\/p>\n<p>7,17 ms 17810 IOPS<\/p>\n<p><b>11,96 ms 21396 IOPS<\/b><\/p>\n<p>64K<\/p>\n<p>1,84 ms 69440 IOPS<\/p>\n<p>3,6 ms 71008 IOPS<\/p>\n<p><b>7,26 ms 70404 IOPS<\/b><\/p>\n<p><b>11,37 ms 11248 IOPS<\/b><\/p>\n<p><i>Valorile eviden\u021biate cu caractere aldine sunt cele dup\u0103 care nu exist\u0103 cre\u0219teri de performan\u021b\u0103, uneori exist\u00e2nd chiar o degradare. Acest lucru se datoreaz\u0103 faptului c\u0103 suntem limita\u021bi de performan\u021ba re\u021belei \/ controlerelor \/ discurilor.<\/i><\/p>\n<ul>\n<li>Citire secven\u021bial\u0103 4432 MB\/s.<\/li>\n<li>Scriere secven\u021bial\u0103 804 MB\/s.<\/li>\n<li>\u00cen cazul defect\u0103rii unui controler (defectarea unei ma\u0219ini virtuale sau a gazdei), sc\u0103derea performan\u021bei este de dou\u0103 ori.<\/li>\n<li>\u00cen cazul defect\u0103rii discului de stocare, sc\u0103derea este de 1\/3. Rebuild-ul discului ocup\u0103 5% din resursele fiec\u0103rui controler.<\/li>\n<\/ul>\n<p>\nPe un bloc mic, ne confrunt\u0103m cu performan\u021ba controlerului (ma\u0219ina virtual\u0103), CPU-ul s\u0103u este \u00eenc\u0103rcat la 100%, iar la cre\u0219terea blocului ne atingem de l\u0103\u021bimea de band\u0103 a porturilor. 10 Gbps nu sunt suficiente pentru a descoperi poten\u021bialul unui sistem AllFlash. Din p\u0103cate, nu ne permit parametrii demo-stand-ului s\u0103 verific\u0103m func\u021bionarea la 40 Gbps.<\/p>\n<p>Din impresiile mele din teste \u0219i studiul arhitecturii, datorit\u0103 algoritmului care distribuie datele \u00eentre toate gazdele, ob\u021binem performan\u021b\u0103 previzibil\u0103 \u0219i scalabil\u0103, dar aceasta este \u0219i o limitare \u00een citire, deoarece de pe discurile locale s-ar putea extrage mai mult; aici ar putea ajuta o re\u021bea mai performant\u0103, de exemplu, FI disponibile la 40 Gbps.<\/p>\n<p>De asemenea, un singur disc pentru caching \u0219i deduplicare poate reprezenta o limitare; practic, \u00een acest stand putem scrie pe patru SSD-uri. Ar fi excelent s\u0103 avem posibilitatea de a cre\u0219te num\u0103rul de discuri de cache \u0219i de a observa diferen\u021ba.<\/p>\n<h3>Utilizare real\u0103<\/h3>\n<p>\nPentru organizarea unui centru de date de rezerv\u0103, se pot folosi dou\u0103 abord\u0103ri (nu lu\u0103m \u00een considerare plasarea backup-ului \u00eentr-o loca\u021bie extern\u0103):<\/p>\n<ol>\n<li>Active-Passive. Toate aplica\u021biile sunt g\u0103zduite \u00een principalul data center. Replicarea este sincron\u0103 sau asincron\u0103. \u00cen caz de c\u0103dere a principalului data center, trebuie s\u0103 activ\u0103m backup-ul. Acest lucru se poate face manual, prin scripturi sau aplica\u021bii de orchestrare. Aici vom ob\u021bine un RPO comparabil cu frecven\u021ba de replicare, iar RTO depinde de reac\u021bia \u0219i abilit\u0103\u021bile administratorului, precum \u0219i de calitatea planului de failover.<\/li>\n<li>Active-Active. \u00cen acest caz exist\u0103 doar replicare sincron\u0103, disponibilitatea data center-elor este determinat\u0103 de quorum\/arbitru, situat strict pe o a treia loca\u021bie. RPO = 0, iar RTO poate ajunge la 0 (dac\u0103 aplica\u021bia permite) sau este egal cu timpul necesar pentru gestionarea e\u0219ecului nodului \u00eentr-un cluster de virtualizare. La nivel de virtualizare, se creeaz\u0103 un cluster extins (Metro), care necesit\u0103 stocare Active-Active.<\/li>\n<\/ol>\n<p>\nDe obicei, vedem la clien\u021bi o arhitectur\u0103 deja implementat\u0103 cu stocare clasic\u0103 \u00een principalul data center, a\u0219a c\u0103 proiect\u0103m \u00eenc\u0103 unul pentru replicare. A\u0219a cum am men\u021bionat, Cisco HyperFlex ofer\u0103 replicare asincron\u0103 \u0219i crearea unui cluster de virtualizare extins. \u00cen acest caz, nu avem nevoie de un sistem de stocare de tip Midrange sau superior, cu func\u021bii costisitoare de replicare \u0219i acces Active-Active la date \u00een dou\u0103 sisteme de stocare.<\/p>\n<p><b>Scenariul 1:<\/b> Avem un data center principal \u0219i unul de rezerv\u0103, platforma de virtualizare fiind bazat\u0103 pe VMware vSphere. Toate sistemele productive se afl\u0103 \u00een principalul data center, iar replicarea ma\u0219inilor virtuale se face la nivelul hypervisor-ului, ceea ce va permite ca VM-urile s\u0103 nu fie men\u021binute active \u00een data center-ul de rezerv\u0103. Bazele de date \u0219i aplica\u021biile speciale sunt replicate prin instrumente integrate \u0219i men\u021binem VM-urile active. \u00cen cazul unei c\u0103deri a principalului data center, activ\u0103m sistemele \u00een data center-ul de rezerv\u0103. Consider\u0103m c\u0103 avem aproximativ 100 de ma\u0219ini virtuale. At\u00e2ta timp c\u00e2t principalul data center func\u021bioneaz\u0103, \u00een data center-ul de rezerv\u0103 pot fi pornite medii de testare \u0219i alte sisteme, care pot fi oprite \u00een cazul unei comut\u0103ri a principalului data center. De asemenea, exist\u0103 varianta \u00een care utiliz\u0103m replicare bidirec\u021bional\u0103. Din punct de vedere hardware, nu se va schimba nimic.<\/p>\n<p>\u00cen cazul arhitecturii clasice, vom instala \u00een fiecare centru de date un sistem de stocare hibrid cu acces prin FibreChannel, tiering, deduplicare \u0219i compresie (dar nu online), 8 servere pentru fiecare loca\u021bie, c\u00e2te 2 switch-uri FibreChannel \u0219i Ethernet 10G. Pentru replicare \u0219i gestionarea comut\u0103rii \u00een arhitectura clasic\u0103 putem folosi solu\u021bii VMware (Replication + SRM) sau solu\u021bii ter\u021be care vor fi pu\u021bin mai ieftine \u0219i uneori mai convenabile.<\/p>\n<p>Schema este prezentat\u0103 \u00een ilustratie.<\/p>\n<p><img decoding=\"async\" alt=\"Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103?\" src=\"\/wp-content\/uploads\/2019\/10\/e7a1ef66c0ae1a8cbb60d0411007d822.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n\u00cen cazul utiliz\u0103rii Cisco HyperFlex, ob\u021binem urm\u0103toarea arhitectur\u0103:<\/p>\n<p><img decoding=\"async\" alt=\"Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103?\" src=\"\/wp-content\/uploads\/2019\/10\/71d58163728f37a163ec914b81dc73ab.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPentru HyperFlex am folosit servere cu resurse mari CPU\/RAM, deoarece o parte din resurse va fi utilizat\u0103 pentru VM-ul controller-ului HyperFlex; pentru CPU \u0219i memorie am avut chiar \u0219i o rezerv\u0103 \u00een configura\u021bia HyperFlex, pentru a nu favoriza Cisco \u0219i a garanta resursele pentru celelalte VM-uri. Astfel, putem renun\u021ba la switch-urile FibreChannel, iar porturile Ethernet nu vor fi necesare pentru fiecare server, traficul local fiind comutat \u00een interiorul FI.<\/p>\n<p>\u00cen final, a rezultat urm\u0103toarea configura\u021bie pentru fiecare centru de date:<\/p>\n<p>Servere<\/p>\n<p>8 x Server 1U (384 GB RAM, 2 x Intel Gold 6132, FC HBA)<\/p>\n<p>8 x HX240C-M5L (512 GB RAM, 2 x Intel Gold 6150, 3,2 GB SSD, 10 x 6 TB NL-SAS)<\/p>\n<p>Sisteme de stocare<\/p>\n<p>Sistem de stocare hibrid cu FC Front-End (20TB SSD, 130 TB NL-SAS)<\/p>\n<p>\u2014<\/p>\n<p>LAN<\/p>\n<p>2 x switch Ethernet 10G cu 12 porturi<\/p>\n<p>\u2014<\/p>\n<p>SAN<\/p>\n<p>2 x switch FC 32\/16Gb cu 24 porturi<\/p>\n<p>2 x Cisco UCS FI 6332<\/p>\n<p>Licen\u021be<\/p>\n<p>VMware Ent Plus<\/p>\n<p><\/p>\n<p>Replicare \u0219i\/sau orchestrare a comut\u0103rii VM<\/p>\n<p>VMware Ent Plus<\/p>\n<p>Pentru HyperFlex nu am inclus licen\u021bele pentru software de replicare, deoarece este disponibil din start.<\/p>\n<p>Pentru arhitectura clasic\u0103 am ales un furnizor care s-a dovedit a fi un produc\u0103tor de calitate \u0219i accesibil. Pentru ambele op\u021biuni am aplicat discountul standard pentru solu\u021bia specific\u0103, la final am ob\u021binut pre\u021burile reale. <\/p>\n<p>Solu\u021bia pe Cisco HyperFlex a rezultat cu 13% mai ieftin\u0103.<\/p>\n<p><b>Scenariul 2:<\/b> crearea a dou\u0103 centre de date active. \u00cen acest scenariu, proiect\u0103m un cluster extins pe VMware. <\/p>\n<p>Arhitectura clasic\u0103 const\u0103 din servere de virtualizare, SAN (protocol FC) \u0219i dou\u0103 sisteme de stocare, care pot citi \u0219i scrie date \u00eentr-un mod extins \u00eentre ele. Pe fiecare sistem de stocare aloc\u0103m capacitate util\u0103 pentru loca\u021bie.<\/p>\n<p><img decoding=\"async\" alt=\"Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103?\" src=\"\/wp-content\/uploads\/2019\/10\/c3830f5bab128b724d94e727218b21b5.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n <br \/>\nPentru HyperFlex pur \u0219i simplu cre\u0103m un Stretch Cluster cu acela\u0219i num\u0103r de noduri pe ambele loca\u021bii. \u00cen acest caz, se utilizeaz\u0103 un factor de replicare de 2+2.<\/p>\n<p><img decoding=\"async\" alt=\"Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103?\" src=\"\/wp-content\/uploads\/2019\/10\/6351c830c3fba60dc4e71a9a4245ec6f.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n <br \/>\nRezultatul este urm\u0103toarea configura\u021bie:<\/p>\n<p>Arhitectura clasic\u0103<\/p>\n<p>HyperFlex<\/p>\n<p>Servere<\/p>\n<p>16 x Server 1U (384 GB RAM, 2 x Intel Gold 6132, FC HBA, 2 x 10G NIC)<\/p>\n<p>16 x HX240C-M5L (512 GB RAM, 2 x Intel Gold 6132, 1,6 TB NVMe, 12 x 3,8 TB SSD, VIC 1387)<\/p>\n<p>Sisteme de stocare<\/p>\n<p>2 x Sisteme de stocare AllFlash (150 TB SSD)<\/p>\n<p>\u2014<\/p>\n<p>LAN<\/p>\n<p>4 x switch Ethernet 10G cu 24 porturi<\/p>\n<p>\u2014<\/p>\n<p>SAN<\/p>\n<p>4 x switch FC 32\/16Gb cu 24 porturi<\/p>\n<p>4 x Cisco UCS FI 6332<\/p>\n<p>Licen\u021be<\/p>\n<p>VMware Ent Plus<\/p>\n<p>VMware Ent Plus<\/p>\n<p>\u00cen toate calculele, nu am luat \u00een considerare infrastructura de re\u021bea, costurile pentru centru de date etc.: acestea vor fi acelea\u0219i at\u00e2t pentru arhitectura clasic\u0103, c\u00e2t \u0219i pentru solu\u021bia HyperFlex.<\/p>\n<p>Solu\u021bia HyperFlex a fost cu 5% mai scump\u0103. Merit\u0103 men\u021bionat c\u0103, \u00een ceea ce prive\u0219te resursele CPU\/RAM, la Cisco am avut o dispropor\u021bie, deoarece \u00een configura\u021bie am umplut canalele controlerelor de memorie uniform. Costul este pu\u021bin mai mare, dar nu cu mult, ceea ce indic\u0103 clar c\u0103 hiperconvergen\u021ba nu este neap\u0103rat o \u00abjuc\u0103rie pentru boga\u021bi\u00bb, ci poate concura cu abordarea standard de construire a datacentrelor. De asemenea, aceasta poate fi interesant\u0103 celor care au deja servere Cisco UCS \u0219i infrastructura corespunz\u0103toare pentru acestea. <\/p>\n<p>Printre avantajele pe care le ob\u021binem se num\u0103r\u0103 absen\u021ba costurilor administrative pentru SAN \u0219i stocare, compresie online \u0219i deduplicare, un punct unic de contact pentru suport (virtualizare, servere, acestea sunt \u0219i solu\u021bia de stocare), economisirea de spa\u021biu (dar nu \u00een toate scenariile), simplificarea exploat\u0103rii.<\/p>\n<p>\u00cen ceea ce prive\u0219te suportul, \u00eel primi\u021bi de la un singur furnizor \u2014 Cisco. Dac\u0103 judec\u0103m dup\u0103 experien\u021ba avut\u0103 cu serverele Cisco UCS, \u00eemi place, la HyperFlex nu a fost nevoie s\u0103 deschid, totul a func\u021bionat bine. Inginerii r\u0103spund rapid \u0219i pot rezolva nu doar problemele standard, ci \u0219i cazuri complexe. Uneori, \u00eei \u00eentreb: \u00abSe poate face a\u0219a, s\u0103 se conecteze asta?\u00bb sau \u00abAm configurat ceva \u0219i nu vrea s\u0103 func\u021bioneze. Ajuta\u021bi-m\u0103!\u00bb \u2014 ei g\u0103sesc cu r\u0103bdare ghidul necesar \u0219i indic\u0103 ac\u021biunile corecte, nu vor r\u0103spunde: \u00abNoi rezolv\u0103m doar problemele hardware\u00bb.<\/p>\n<h3>Linkuri<\/h3>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.cisco.com\/c\/dam\/en\/us\/products\/collateral\/hyperconverged-infrastructure\/hyperflex-hx-series\/hxaf-240c-m5-specsheet.pdf\">Specifica\u021bii<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/croc\/blog\/146536\/\">Centru de date virtual<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/croc\/blog\/342820\/\">Centru de date \u00een cutia biroului<\/a><\/noindex><\/li>\n<li>Emailul meu \u2014 StGeneralov@croc.ru<\/li>\n<\/ul>\n<p>Sursa: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/croc\/blog\/471508\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u042d\u0442\u043e \u043c\u0438\u0444, \u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e \u0440\u0430\u0441\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0451\u043d\u043d\u044b\u0439 \u0432 \u0441\u0444\u0435\u0440\u0435 \u0441\u0435\u0440\u0432\u0435\u0440\u043d\u043e\u0433\u043e \u0436\u0435\u043b\u0435\u0437\u0430. \u041d\u0430 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u0435 \u0436\u0435 \u0433\u0438\u043f\u0435\u0440\u043a\u043e\u043d\u0432\u0435\u0440\u0433\u0435\u043d\u0442\u043d\u044b\u0435 \u0440\u0435\u0448\u0435\u043d\u0438\u044f (\u043a\u043e\u0433\u0434\u0430 \u0432\u0441\u0451 \u0432 \u043e\u0434\u043d\u043e\u043c) \u043d\u0443\u0436\u043d\u044b \u043c\u043d\u043e\u0433\u043e \u0434\u043b\u044f \u0447\u0435\u0433\u043e. \u0418\u0441\u0442\u043e\u0440\u0438\u0447\u0435\u0441\u043a\u0438 \u0441\u043b\u043e\u0436\u0438\u043b\u043e\u0441\u044c, \u0447\u0442\u043e \u043f\u0435\u0440\u0432\u044b\u0435 \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u044b \u0431\u044b\u043b\u0438 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0430\u043d\u044b Amazon \u0438 Google \u043f\u043e\u0434 \u0441\u0432\u043e\u0438 \u0441\u0435\u0440\u0432\u0438\u0441\u044b. \u0422\u043e\u0433\u0434\u0430 \u0438\u0434\u0435\u044f \u0431\u044b\u043b\u0430 \u0432 \u0442\u043e\u043c, \u0447\u0442\u043e\u0431\u044b \u0441\u0434\u0435\u043b\u0430\u0442\u044c \u0432\u044b\u0447\u0438\u0441\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u0443\u044e \u0444\u0435\u0440\u043c\u0443 \u0438\u0437 \u043e\u0434\u0438\u043d\u0430\u043a\u043e\u0432\u044b\u0445 \u0443\u0437\u043b\u043e\u0432, \u0443 \u043a\u0430\u0436\u0434\u043e\u0433\u043e \u0438\u0437 \u043a\u043e\u0442\u043e\u0440\u044b\u0445 \u0435\u0441\u0442\u044c \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u044b\u0435 \u0434\u0438\u0441\u043a\u0438. \u0412\u0441\u0451 \u044d\u0442\u043e [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":29233,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-38966","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2.1 - aioseo.com -->\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/admin-bez-ruk-giperkonvergentsiya\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"ro_RO\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0410\u0434\u043c\u0438\u043d \u0431\u0435\u0437 \u0440\u0443\u043a = \u0433\u0438\u043f\u0435\u0440\u043a\u043e\u043d\u0432\u0435\u0440\u0433\u0435\u043d\u0446\u0438\u044f? | ProHoster\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/admin-bez-ruk-giperkonvergentsiya\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T19:27:03+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T19:27:03+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Admin f\u0103r\u0103 m\u00e2ini = hiperconvergen\u021b\u0103? | ProHoster","description":"","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/admin-bez-ruk-giperkonvergentsiya","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0410\u0434\u043c\u0438\u043d \u0431\u0435\u0437 \u0440\u0443\u043a = \u0433\u0438\u043f\u0435\u0440\u043a\u043e\u043d\u0432\u0435\u0440\u0433\u0435\u043d\u0446\u0438\u044f? | ProHoster","og:url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/admin-bez-ruk-giperkonvergentsiya","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:27:03+00:00","article:modified_time":"2019-10-31T19:27:03+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"38966","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 00:12:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:59:27","updated":"2026-01-24 00:12:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/38966","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=38966"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/38966\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media\/29233"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=38966"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=38966"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=38966"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}