{"id":35800,"date":"2019-10-31T22:06:24","date_gmt":"2019-10-31T19:06:24","guid":{"rendered":"https:\/\/prohoster.info\/blog\/dizajn-virtualizovannogo-tsod\/"},"modified":"2019-10-31T22:06:24","modified_gmt":"2019-10-31T19:06:24","slug":"dizajn-virtualizovannogo-tsod","status":"publish","type":"post","link":"https:\/\/prohoster.info\/lv\/blog\/administrirovanie\/dizajn-virtualizovannogo-tsod","title":{"rendered":"Virtualiz\u0113to Datu centru dizains","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Virtualiz\u0113to Datu centru dizains\" src=\"\/wp-content\/uploads\/2019\/06\/37aadc70b5523acd959fc4c6c4372bfe.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<b>Ievads<\/b><\/p>\n<p>Inform\u0101cijas sist\u0113ma no lietot\u0101ja skatpunkta tiek labi defin\u0113ta GOST RV 51987 \u2014 \u201eautom\u0101tiz\u0113ta sist\u0113ma, kuras darb\u012bbas rezult\u0101ts ir izejas inform\u0101cijas snieg\u0161ana turpm\u0101kai izmanto\u0161anai\u201d. Ja apl\u016bkojam iek\u0161\u0113jo strukt\u016bru, tad b\u016bt\u012bb\u0101 jebkura IS ir sist\u0113ma, kur\u0101 ir realiz\u0113ti saist\u012bti algoritmi kod\u0101. Pla\u0161\u0101 noz\u012bm\u0113 Tj\u016bringa-\u010c\u0113r\u010da t\u0113zes ietvaros algoritms (un l\u012bdz ar to IS) veic daudzuma ievades datu transform\u0101ciju uz daudzumu izejas datu. <br \/>\nVar pat teikt, ka ievades datu transform\u0101cij\u0101 sl\u0113pjas inform\u0101cijas sist\u0113mas eksistences j\u0113ga. T\u0101d\u0113j\u0101di IS un visa IS kompleksa v\u0113rt\u012bba tiek noteikta caur ievades un izejas datu v\u0113rt\u012bbu.<br \/>\n\u0145emot to v\u0113r\u0101, projekt\u0113\u0161ana b\u016btu j\u0101s\u0101k ar datu izp\u0113ti, piel\u0101gojot arhitekt\u016bru un metodes atbilsto\u0161i datu strukt\u016brai un noz\u012bm\u012bgumam.<\/p>\n<p><b>Uzglab\u0101tie dati<\/b><br \/>\nGalvenais posms sagatavo\u0161an\u0101s projekt\u0113\u0161anai ir visu datu kopu, kuras pl\u0101nots apstr\u0101d\u0101t un uzglab\u0101t, raksturojuma ieg\u016b\u0161ana. \u0160ie raksturojumi ietver:<br \/>\n \u2014 Datu apjoms;<br \/>\n \u2014 Inform\u0101cija par datu dz\u012bves ciklu (jauno datu pieaugums, m\u016b\u017ea ilgums, novecoju\u0161o datu apstr\u0101de);<br \/>\n \u2014 Datu klasifik\u0101cija no uz\u0146\u0113muma pamatbiznesa ietekmes viedok\u013ca (trij\u0101s aspektos \u2014 konfidencialit\u0101te, integrit\u0101te, pieejam\u012bba) kop\u0101 ar finansi\u0101lajiem r\u0101d\u012bt\u0101jiem (piem\u0113ram, datu zaud\u0113\u0161anas izmaksas p\u0113d\u0113j\u0101 stund\u0101);<br \/>\n \u2014 Datu apstr\u0101des \u0123eogr\u0101fija (fizisk\u0101 sist\u0113mu atra\u0161an\u0101s vieta);<br \/>\n \u2014 Regul\u0113jo\u0161o iest\u0101\u017eu pras\u012bbas katram datu klasim (piem\u0113ram, FZ-152, PCI DSS).<\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p><b>Inform\u0101cijas sist\u0113mas<\/b><\/p>\n<p>Dati ne tikai tiek uzglab\u0101ti, bet ar\u012b tiek apstr\u0101d\u0101ti (transform\u0113ti) ar inform\u0101cijas sist\u0113m\u0101m. N\u0101kamais solis p\u0113c datu raksturojumu ieg\u016b\u0161anas ir maksim\u0101li pilna inform\u0101cijas sist\u0113mu inventariz\u0101cija, to arhitekt\u016bras \u012bpatn\u012bbu, savstarp\u0113j\u0101m saist\u012bb\u0101m un pras\u012bb\u0101m pret infrastrukt\u016bru par \u010detriem resursu veidiem:<br \/>\n \u2014 Procesoru skait\u013co\u0161anas jauda;<br \/>\n \u2014 Operat\u012bv\u0101s atmi\u0146as apjoms;<br \/>\n \u2014 Pras\u012bbas attiec\u012bb\u0101 uz uzglab\u0101\u0161anas sist\u0113mas apjomu un veiktsp\u0113ju;<br \/>\n \u2014 Datu p\u0101rs\u016bt\u012b\u0161anas t\u012bkla pras\u012bbas (\u0101r\u0113jie kan\u0101li, kan\u0101li starp IS komponent\u0113m).<br \/>\nPras\u012bbas tam ir j\u0101b\u016bt katram pakalpojumam \/ mikroservisam inform\u0101cijas sist\u0113m\u0101.<br \/>\nAtsevi\u0161\u0137i j\u0101nor\u0101da, ka, lai pareizi projekt\u0113tu, ir nepiecie\u0161ami dati par inform\u0101cijas sist\u0113mas ietekmi uz uz\u0146\u0113muma pamatbiznesu, piem\u0113ram, inform\u0101cija par d\u012bkst\u0101ves izmaks\u0101m (rub\u013cos stund\u0101).<\/p>\n<p><b>Draudu modelis<\/b><\/p>\n<p>Oblig\u0101ti j\u0101b\u016bt form\u0101lai apdraud\u0113juma modelim, no kura pl\u0101nots aizsarg\u0101t datus \/ pakalpojumus. \u0160is apdraud\u0113juma modelis ietver ne tikai konfidencialit\u0101tes aspektus, bet ar\u012b integrit\u0101ti un pieejam\u012bbu. Proti, piem\u0113ram:<br \/>\n \u2014 Fizi\u0161a servera boj\u0101jums;<br \/>\n \u2014 Top-of-the-rack komutatora boj\u0101jums;<br \/>\n \u2014 Optisk\u0101 sakaru kan\u0101la p\u0101rtraukums starp datu centriem;<br \/>\n \u2014 Darb\u012bbas trauc\u0113jumi operat\u012bvaj\u0101 glab\u0101\u0161anas sist\u0113m\u0101 kopum\u0101.<br \/>\nDa\u017eos gad\u012bjumos apdraud\u0113juma mode\u013ci tiek izstr\u0101d\u0101ti ne tikai infrastrukt\u016bras komponentiem, bet ar\u012b konkr\u0113t\u0101m inform\u0101cijas sist\u0113m\u0101m vai to komponentiem, piem\u0113ram, datu b\u0101zu vad\u012bbas sist\u0113mas (DVVS) atteice ar lo\u0123isku datu strukt\u016bras sabrukumu. <br \/>\nVisi l\u0113mumi projekta ietvaros par aizsardz\u012bbu pret neaprakst\u012btu apdraud\u0113jumu ir lieki.<\/p>\n<p><b>Regulatoru pras\u012bbas<\/b><\/p>\n<p>Ja apstr\u0101d\u0101jamie dati ietilpst \u012bpa\u0161u noteikumu darb\u012bbas jom\u0101, ko nosaka regulatori, noteikti nepiecie\u0161ama inform\u0101cija par datu kop\u0101m un apstr\u0101des \/ glab\u0101\u0161anas noteikumiem.<\/p>\n<p><b>M\u0113r\u0137r\u0101d\u012bt\u0101ji RPO \/ RTO<\/b><\/p>\n<p>Jebkura veida aizsardz\u012bbas projekt\u0113\u0161ana prasa m\u0113r\u0137r\u0101d\u012bt\u0101jus attiec\u012bb\u0101 uz m\u0113r\u0137a datu zudumu un m\u0113r\u0137a pakalpojuma atjauno\u0161anas laiku katram no aprakst\u012btajiem apdraud\u0113jumiem.<br \/>\nIde\u0101l\u0101 gad\u012bjum\u0101 RPO un RTO j\u0101b\u016bt saist\u012btiem ar datu zuduma un d\u012bkst\u0101ves izmaks\u0101m laika vien\u012bb\u0101.<\/p>\n<p><img decoding=\"async\" alt=\"Virtualiz\u0113to Datu centru dizains\" src=\"\/wp-content\/uploads\/2019\/06\/ff035eb7dda41bc8ea0f77e8641f89e0.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\n<b>Sadale resursu kop\u0101s.<\/b><\/p>\n<p>P\u0113c visu s\u0101kotn\u0113jo ievaddatu apkopo\u0161anas pirmais solis ir datu kopu un inform\u0101cijas sist\u0113mu grup\u0113\u0161ana krog\u0101 atbilsto\u0161i apdraud\u0113juma mode\u013ciem un regulatoru pras\u012bb\u0101m. Tiek noteikts resursu kopu sadal\u012bjuma veids \u2013 programmatically uz sist\u0113mas programmat\u016bras l\u012bme\u0146a vai fiziski.<br \/>\nPiem\u0113ri:<br \/>\n \u2014 Kont\u016brs, kas apstr\u0101d\u0101 personas datus, ir piln\u012bb\u0101 fiziski atdal\u012bts no p\u0101r\u0113j\u0101m sist\u0113m\u0101m;<br \/>\n \u2014 Dubl\u0113jumkopijas tiek glab\u0101tas atsevi\u0161\u0137\u0101 glab\u0101\u0161anas sist\u0113m\u0101.<\/p>\n<p>Tom\u0113r kop\u0101m var b\u016bt nepiln\u012bga neatkar\u012bba, piem\u0113ram, tiek noteikti divi skait\u013co\u0161anas resursu kopas (procesora jauda + operat\u012bv\u0101 atmi\u0146a), kas izmanto vienu datu glab\u0101\u0161anas un vienu datu p\u0101rs\u016bt\u012b\u0161anas resursu kopu.<\/p>\n<p><b>Procesora jauda<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Virtualiz\u0113to Datu centru dizains\" src=\"\/wp-content\/uploads\/2019\/06\/e2d4a0a18db83e5c87b2834773dd2316.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nAbstrakt\u0101s vajadz\u012bbas p\u0113c virtu\u0101l\u0101s datu centra procesora jaudas tiek m\u0113r\u012btas k\u0101 virtu\u0101lo procesoru (vCPU) skaits un to konsolid\u0101cijas koeficients uz fiziskajiem procesoriem (pCPU). \u0160aj\u0101 konkr\u0113taj\u0101 gad\u012bjum\u0101 1 pCPU = 1 fizisk\u0101 procesora kodols (ne\u0146emot v\u0113r\u0101 Hyper-Threading). vCPU skaits tiek summ\u0113ts visiem noteiktajiem resursu p\u016b\u013ciem (katram no kuriem var b\u016bt savs konsolid\u0101cijas koeficients).<br \/>\nKonsolid\u0101cijas koeficients noslogot\u0101m sist\u0113m\u0101m tiek ieg\u016bts emp\u012briski, balstoties uz jau eso\u0161o infrastrukt\u016bru, vai pilota instal\u0101cijas un slodzes test\u0113\u0161anas laik\u0101. Nenoslogot\u0101m sist\u0113m\u0101m tiek piem\u0113rotas \u00abbest practice\u00bb. Piem\u0113ram, VMware nor\u0101da, ka vid\u0113jais koeficients ir 8:1.<\/p>\n<p><b>Operat\u012bv\u0101 atmi\u0146a<\/b><\/p>\n<p>Kop\u0113j\u0101 piepras\u012bjuma p\u0113c operat\u012bv\u0101s atmi\u0146as ieg\u016b\u0161ana notiek, vienk\u0101r\u0161i summ\u0113jot. Operat\u012bv\u0101s atmi\u0146as p\u0101rsnieg\u0161anas izmanto\u0161ana netiek ieteikta.<\/p>\n<p><b>Glab\u0101\u0161anas resursi<\/b><\/p>\n<p>Glab\u0101\u0161anas resursu pras\u012bbas tiek ieg\u016btas, vienk\u0101r\u0161i summ\u0113jot visu p\u016b\u013cu apjomu un veiktsp\u0113ju.<br \/>\nVeiktsp\u0113jas pras\u012bbas izsaka k\u0101 IOPS kombin\u0101cij\u0101 ar vid\u0113jo las\u012b\u0161anas\/ierakst\u012b\u0161anas attiec\u012bbu un, ja nepiecie\u0161ams, maksim\u0101lo reakcijas laiku.<br \/>\nAtsevi\u0161\u0137i j\u0101nor\u0101da pras\u012bbas attiec\u012bb\u0101 uz pakalpojumu kvalit\u0101tes nodro\u0161in\u0101\u0161anu (QoS) konkr\u0113tiem p\u016b\u013ciem vai sist\u0113m\u0101m.<\/p>\n<p><b>Datu p\u0101rs\u016bt\u012b\u0161anas t\u012bkla resursi<\/b><\/p>\n<p>Datu p\u0101rs\u016bt\u012b\u0161anas t\u012bkla pras\u012bbas tiek ieg\u016btas, vienk\u0101r\u0161i summ\u0113jot visu p\u016b\u013cu caurlaidsp\u0113ju.<br \/>\nAtsevi\u0161\u0137i j\u0101nor\u0101da pras\u012bbas attiec\u012bb\u0101 uz pakalpojumu kvalit\u0101tes nodro\u0161in\u0101\u0161anu (QoS) un aizkaves (RTT) konkr\u0113tiem p\u016b\u013ciem vai sist\u0113m\u0101m. <br \/>\nResursu pras\u012bb\u0101s attiec\u012bb\u0101 uz datu p\u0101rs\u016bt\u012b\u0161anas t\u012bklu ar\u012b j\u0101nor\u0101da pras\u012bbas attiec\u012bb\u0101 uz izol\u0101ciju un\/vai t\u012bkla datu \u0161ifr\u0113\u0161anu un v\u0113lamiem meh\u0101nismiem (802.1q, IPSec utt.)<\/p>\n<p><b>Arhitekt\u016bras izv\u0113le<\/b><\/p>\n<p>\u0160aj\u0101 rokasgr\u0101mat\u0101 netiek apspriests cits variants, iz\u0146emot x86 arhitekt\u016bru un 100% serveru virtualiz\u0101ciju. T\u0101d\u0113\u013c datorsist\u0113mas arhitekt\u016bras izv\u0113le aprobe\u017eojas ar serveru virtualiz\u0101cijas platformas, serveru formfaktora un visp\u0101r\u0113j\u0101m serveru konfigur\u0101cijas pras\u012bb\u0101m.<\/p>\n<p>Svar\u012bg\u0101kais punkts izv\u0113l\u0113 ir skaidr\u012bba, vai izmantot klasisko pieeju ar funkciju sadal\u012b\u0161anu datu apstr\u0101d\u0113, glab\u0101\u0161an\u0101 un p\u0101rs\u016bt\u012b\u0161an\u0101 vai konver\u0123ento pieeju.<\/p>\n<p><i>Klasisk\u0101 arhitekt\u016bra<\/i> uz\u0146\u0113mums izmanto inteli\u0123entas \u0101r\u0113j\u0101s datu uzglab\u0101\u0161anas un p\u0101rs\u016bt\u012b\u0161anas sist\u0113mas, kam\u0113r serveri nodro\u0161ina tikai procesora jaudu un operat\u012bvo atmi\u0146u kop\u0113j\u0101 fizisko resursu apjom\u0101. Ekstrem\u0101l\u0101 gad\u012bjum\u0101 serveri k\u013c\u016bst piln\u012bgi anon\u012bmi, bez saviem diskiem vai pat sist\u0113mas identifikatoriem. \u0160\u0101d\u0101 gad\u012bjum\u0101 tiek izmantota oper\u0113t\u0101jsist\u0113mas vai hipervizora iel\u0101de no ieb\u016bv\u0113tiem flash datu nes\u0113jiem vai no \u0101r\u0113j\u0101s datu uzglab\u0101\u0161anas sist\u0113mas (boot from SAN).<br \/>\nKlasisk\u0101s arhitekt\u016bras ietvaros izv\u0113le starp blade un rack vispirms balst\u0101s uz \u0161\u0101diem principiem:<br \/>\n \u2014 Ekonomisk\u0101 efektivit\u0101te (vid\u0113ji rack serveri ir l\u0113t\u0101ki);<br \/>\n \u2014 Apr\u0113\u0137inu bl\u012bvums (blade ir liel\u0101ks);<br \/>\n \u2014 Ener\u0123ijas pat\u0113ri\u0146\u0161 un siltuma izlaidums (blade ir liel\u0101ki r\u0101d\u012bt\u0101ji uz vien\u012bbu);<br \/>\n \u2014 M\u0113r\u0137tiec\u012bba un p\u0101rvald\u0101m\u012bba (blade kopum\u0101 prasa maz\u0101k p\u016b\u013cu lielos uzst\u0101d\u012bjumos);<br \/>\n \u2014 Papla\u0161in\u0101mo kar\u0161u izmanto\u0161ana (blade ir \u013coti ierobe\u017eots izv\u0113les kl\u0101sts).<br \/>\n<i>Konver\u0123ent\u0101 arhitekt\u016bra<\/i> (paz\u012bstama ar\u012b k\u0101 <i>hiper-konver\u0123ent\u0101<\/i>) ietver apstr\u0101des un datu uzglab\u0101\u0161anas funkciju apvieno\u0161anu, kas noved pie serveru iek\u0161\u0113jo disku izmanto\u0161anas un k\u0101 rezult\u0101t\u0101 atsak\u0161an\u0101s no klasisko blade form\u0101ta. Konver\u0123entaj\u0101m sist\u0113m\u0101m tiek izmantoti vai nu rack serveri, vai klasteru sist\u0113mas, kas apvieno vair\u0101ku blade serveru un viet\u0113jo disku funkcionalit\u0101ti vien\u0101 korpus\u0101.<\/p>\n<p><b>CPU \/ Atmi\u0146a <\/b><\/p>\n<p>Lai pareizi apr\u0113\u0137in\u0101tu konfigur\u0101ciju, ir nepiecie\u0161ams saprast slodzes veidu videi vai katram neatkar\u012bgajam klasterim.<br \/>\n<i>CPU bound<\/i> \u2013 vide, kuras veiktsp\u0113ju ierobe\u017eo procesora jauda. Papildu operat\u012bv\u0101 atmi\u0146a neko main\u012bs veiktsp\u0113jas zi\u0146\u0101 (VM skaits uz servera).<br \/>\n<i>Memory bound<\/i> \u2013 vide, kuras veiktsp\u0113ja ir ierobe\u017eota ar operat\u012bvo atmi\u0146u. Liel\u0101ks operat\u012bv\u0101s atmi\u0146as daudzums server\u012b \u013cauj palaist liel\u0101ku VM skaitu.<br \/>\nGB \/ MHz (GB \/ pCPU) \u2013 vid\u0113j\u0101 attiec\u012bba starp konkr\u0113t\u0101 slodzes CPU un operat\u012bv\u0101s atmi\u0146as pat\u0113ri\u0146u. To var izmantot, lai apr\u0113\u0137in\u0101tu nepiecie\u0161amo atmi\u0146as apjomu, balstoties uz noteiktu veiktsp\u0113ju, un otr\u0101di. <\/p>\n<p><b>Servera konfigur\u0101cijas apr\u0113\u0137ins<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Virtualiz\u0113to Datu centru dizains\" src=\"\/wp-content\/uploads\/2019\/06\/fdacf6dfa1ee4957b116666b78ddb58b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nVispirms ir j\u0101saprot visi slod\u017eu veidi un j\u0101pie\u0146em l\u0113mums par da\u017e\u0101du apr\u0113\u0137inu recep\u0161u apvieno\u0161anu vai dal\u012b\u0161anu pa da\u017e\u0101diem klasteriem.<br \/>\nT\u0101l\u0101k katram no noteiktajiem klasteriem nosaka GB \/ MHz attiec\u012bbu, ja slodze ir iepriek\u0161 zin\u0101ma. Ja slodze nav iepriek\u0161 zin\u0101ma, tom\u0113r ir aptuvens izpratne par procesoru jaudas l\u012bmeni, var izmantot standarta vCPU:pCPU koeficientus, lai p\u0101rveidotu pras\u012bbas saim\u0113m fiziskaj\u0101s. <\/p>\n<p>Katram klasterim vCPU pras\u012bbu kopumu dala ar koeficientu:<br \/>\nvCPUkop\u0101 \/ vCPU:pCPU = pCPUkop\u0101 \u2013 nepiecie\u0161amais fizisko kodolu skaits<br \/>\npCPUkop\u0101 \/ 1.25 = pCPUHT \u2013 kodolu skaits, \u0146emot v\u0113r\u0101 Hyper-Threading<br \/>\nPie\u0146emsim, ka j\u0101saskaita klasteris ar 190 kodoliem \/ 3.5TB RAM. Pie\u0146emam m\u0113r\u0137a 50% procesora jaudas slodzi un 75% operat\u012bv\u0101s atmi\u0146as.<\/p>\n<p><b>pCPU<\/b><br \/>\n190<br \/>\n<b>CPU izmanto\u0161ana<\/b><br \/>\n50%<\/p>\n<p><b>Mem<\/b><br \/>\n3500<br \/>\n<b>Atmi\u0146as izmanto\u0161ana<\/b><br \/>\n75%<\/p>\n<p><b>Sokets<\/b><br \/>\n<b>Core<\/b><br \/>\n<b>Srv \/ CPU<\/b><br \/>\n<b>Srv Atmi\u0146a<\/b><br \/>\n<b>Srv \/ Atmi\u0146a<\/b><\/p>\n<p>2<br \/>\n6<br \/>\n25,3<br \/>\n128<br \/>\n36,5<\/p>\n<p>2<br \/>\n8<br \/>\n19,0<br \/>\n192<br \/>\n24,3<\/p>\n<p>2<br \/>\n10<br \/>\n15,2<br \/>\n256<br \/>\n18,2<\/p>\n<p>2<br \/>\n14<br \/>\n10,9<br \/>\n384<br \/>\n12,2<\/p>\n<p>2<br \/>\n18<br \/>\n8,4<br \/>\n512<br \/>\n9,1<\/p>\n<p>\u0160aj\u0101 gad\u012bjum\u0101 vienm\u0113r izmantosim k\u0101rtot uz aug\u0161u l\u012bdz tuv\u0101kajam veselajam skaitlim (=ROUNDUP(A1;0)).<br \/>\nNo tabulas k\u013c\u016bst ac\u012bmredzams, ka vair\u0101ki serveru konfigur\u0101cijas ir l\u012bdzsvarotas attiec\u012bb\u0101 uz m\u0113r\u0137auditoriju:<br \/>\n \u2014 26 serveri 2*6c \/ 192 GB<br \/>\n \u2014 19 serveri 2*10c \/ 256 GB<br \/>\n \u2014 10 serveri 2*18c \/ 512 GB <\/p>\n<p>Izv\u0113loties no \u0161\u012bm konfigur\u0101cij\u0101m, n\u0101kotn\u0113 j\u0101vad\u0101s p\u0113c papildu faktoriem, piem\u0113ram, siltuma paketes un pieejam\u0101 dzes\u0113\u0161anas, jau izmantotajiem serveriem vai izmaks\u0101m.<\/p>\n<p><b>Servera konfigur\u0101cijas izv\u0113les \u012bpatn\u012bbas<\/b><\/p>\n<p>Platas VM. Ja nepiecie\u0161ams izvietot platas VM (sal\u012bdzin\u0101mas ar 1 NUMA mezglu un vair\u0101k), ieteicams, kur vien iesp\u0113jams, izv\u0113l\u0113ties serveri ar konfigur\u0101ciju, kas \u013cauj \u0161\u0101d\u0101m VM palikt NUMA mezgla ietvaros. Liel\u0101 skait\u0101 pla\u0161u VM past\u0101v risks, ka klastera resursi fragment\u0113jas, un \u0161\u0101dos gad\u012bjumos izv\u0113las serverus, kas \u013cauj izvietot platas VM p\u0113c iesp\u0113jas bl\u012bv\u0101k.<\/p>\n<p><b>Vienas k\u013c\u016bmes dom\u0113na lielums.<\/b> <\/p>\n<p>Servera lieluma izv\u0113le ar\u012b tiek veikta, balstoties uz vienas k\u013c\u016bmes dom\u0113na minimiz\u0113\u0161anas principu. Piem\u0113ram, izv\u0113loties starp:<br \/>\n \u2014 3 x 4*10c \/ 512 GB<br \/>\n \u2014 6 x 2*10c \/ 256 GB<br \/>\nJa visi citi faktori ir vien\u0101di, j\u0101p\u0101rtrauc otr\u0101 varianta izv\u0113le, jo, ja k\u0101ds servers izg\u0101\u017eas (vai tiek apkalpots), klastera resursu zudums ir nevis 33%, bet tikai 17%. Tie\u0161i t\u0101pat dubultojas VM un IT skaits, uz kuriem ietekm\u0113jas av\u0101rija.<\/p>\n<p><b>Klasisk\u0101s uzglab\u0101\u0161anas sist\u0113mas veiktsp\u0113jas apr\u0113\u0137ins<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Virtualiz\u0113to Datu centru dizains\" src=\"\/wp-content\/uploads\/2019\/06\/36e096865a6fd3d6229b09e351f2f280.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nKlasisk\u0101 uzglab\u0101\u0161anas sist\u0113ma vienm\u0113r tiek apr\u0113\u0137in\u0101ta, pamatojoties uz slikt\u0101ko scen\u0101riju (worst case scenario), izsl\u0113dzot operat\u012bv\u0101 ke\u0161a ietekmi un oper\u0101ciju optimiz\u0101ciju.<br \/>\nK\u0101 pamatu veiktsp\u0113jas r\u0101d\u012bt\u0101jiem pie\u0146emam meh\u0101nisko veiktsp\u0113ju no diska (IOPSdisk):<br \/>\n \u2014 7.2k \u2013 75 IOPS<br \/>\n \u2014 10k \u2013 125 IOPS<br \/>\n \u2014 15k \u2013 175 IOPS<\/p>\n<p>T\u0101l\u0101k disku skaits disku grup\u0101 tiek apr\u0113\u0137in\u0101ts, izmantojot \u0161\u0101du formulu: <i>= TotalIOPS * ( RW + (1 \u2013RW) * RAIDPen) \/ IOPSdisk<\/i>. Kur:<br \/>\n \u2014 <i>TotalIOPS<\/i> \u2013 kop\u0113j\u0101 nepiecie\u0161am\u0101 veiktsp\u0113ja IOPS no disku grupas<br \/>\n \u2014 <i>RW<\/i> \u2013 las\u012b\u0161anas oper\u0101ciju procentu\u0101lais daudzums<br \/>\n \u2014 <i>RAIDpen<\/i> \u2013 RAID soda punkti izv\u0113l\u0113tajam RAID l\u012bmenim<\/p>\n<p>Pla\u0161\u0101ka inform\u0101cija par RAID ier\u012bc\u0113m un RAID sodu ir pieejama \u0161eit \u2014<noindex><a rel=\"nofollow\" href=\"http:\/\/blog.vadmin.ru\/2011\/08\/blog-post.html\"> Uzglab\u0101\u0161anas sist\u0113mu veiktsp\u0113ja. Pirmaj\u0101 da\u013c\u0101.<\/a><\/noindex> un <noindex><a rel=\"nofollow\" href=\"http:\/\/blog.vadmin.ru\/2011\/08\/blog-post_18.html\">Uzglab\u0101\u0161anas sist\u0113mu veiktsp\u0113ja. Otraj\u0101 da\u013c\u0101.<\/a><\/noindex> un <noindex><a rel=\"nofollow\" href=\"http:\/\/blog.vadmin.ru\/2011\/09\/blog-post.html\">Uzglab\u0101\u0161anas sist\u0113mu veiktsp\u0113ja. Tre\u0161aj\u0101 da\u013c\u0101.<\/a><\/noindex><\/p>\n<p>Pamatojoties uz ieg\u016bto disku skaitu, tiek apr\u0113\u0137in\u0101ti iesp\u0113jamie izso\u013cu varianti, kas atbilst uzglab\u0101\u0161anas jaudas pras\u012bb\u0101m, tostarp varianti ar vair\u0101ku l\u012bme\u0146u uzglab\u0101\u0161anu.<br \/>\nSist\u0113mu apr\u0113\u0137ins, kas izmanto SSD k\u0101 uzglab\u0101\u0161anas l\u012bmeni, izskat\u0101ms atsevi\u0161\u0137i.<br \/>\n<b>Hibr\u012bdo sist\u0113mu apr\u0113\u0137inu \u012bpatn\u012bbas ar Flash Cache<br \/>\n<\/b><\/p>\n<p><i>Flash Cache<\/i> \u2013 visp\u0101r\u012bgs nosaukums vis\u0101m patent\u0113taj\u0101m tehnolo\u0123ij\u0101m, kas izmanto zibatmi\u0146u k\u0101 otro l\u012bme\u0146a ke\u0161u. Izmantojot flash ke\u0161u, uzglab\u0101\u0161anas sist\u0113ma parasti tiek apr\u0113\u0137in\u0101ta, lai apkalpotu slodzi no magn\u0113tiskajiem diskiem, kam\u0113r augst\u0101ko slodzi apkalpo ke\u0161atmi\u0146a.<br \/>\n\u0160aj\u0101 gad\u012bjum\u0101 ir j\u0101izprot slodzes profils un piepras\u012bjumu lokaliz\u0101cijas pak\u0101pe uz uzglab\u0101\u0161anas apjoma bloku. Flash ke\u0161s ir tehnolo\u0123ija slodz\u0113m ar augstu piepras\u012bjumu lokaliz\u0101ciju un praktiski nav piem\u0113rojama vienm\u0113r\u012bgi noslogot\u0101m apjomiem (piem\u0113ram, anal\u012btikas sist\u0113m\u0101m). <\/p>\n<p><b>Zema l\u012bme\u0146a \/ vid\u0113ja l\u012bme\u0146a hibr\u012bdo sist\u0113mu apr\u0113\u0137ins<\/b><\/p>\n<p>Zema un vid\u0113ja l\u012bme\u0146a hibr\u012bd\u0101s sist\u0113mas izmanto vair\u0101ku l\u012bme\u0146u uzglab\u0101\u0161anu ar datu p\u0101rvieto\u0161anu starp l\u012bme\u0146iem p\u0113c grafika. \u0160aj\u0101 proces\u0101 lab\u0101ko mode\u013cu vair\u0101ku l\u012bme\u0146u uzglab\u0101\u0161anas bloka izm\u0113rs ir 256 MB. \u0160\u012bs \u012bpa\u0161\u012bbas ne\u013cauj uzskat\u012bt vair\u0101ku l\u012bme\u0146u uzglab\u0101\u0161anas tehnolo\u0123iju par veiktsp\u0113jas uzlabo\u0161anas tehnolo\u0123iju k\u0101 to daudzi mald\u012bgi dom\u0101. Vair\u0101ku l\u012bme\u0146u uzglab\u0101\u0161ana zema un vid\u0113ja l\u012bme\u0146a sist\u0113m\u0101s ir izmaksu optimiz\u0113\u0161anas tehnolo\u0123ija uzglab\u0101\u0161anai sist\u0113m\u0101m ar izteiktu neviendab\u012bgu slodzi.<\/p>\n<p>Lielapjoma kr\u0101\u0161anai vispirms tiek apr\u0113\u0137in\u0101ta veiktsp\u0113ja aug\u0161\u0113j\u0101 l\u012bmen\u012b, kam\u0113r apak\u0161\u0113j\u0101 kr\u0101\u0161anas l\u012bme\u0146a ietilp\u012bba tiek uzskat\u012bta tikai par papildu kr\u0101\u0161anas jaudu. Hibr\u012bd\u0101s daudzl\u012bme\u0146u sist\u0113m\u0101 ir oblig\u0101ta flash ke\u0161a tehnolo\u0123ijas izmanto\u0161ana daudzl\u012bme\u0146u basein\u0101, lai kompens\u0113tu veiktsp\u0113jas kritumu p\u0113k\u0161\u0146i akt\u012bvaj\u0101m dat\u0101m no apak\u0161\u0113j\u0101 l\u012bme\u0146a.<\/p>\n<p><b>SSD izmanto\u0161ana daudzl\u012bme\u0146u disku basein\u0101<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Virtualiz\u0113to Datu centru dizains\" src=\"\/wp-content\/uploads\/2019\/06\/823d19df41face236b3ca9e192da6290.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSSD izmanto\u0161ana daudzl\u012bme\u0146u disku basein\u0101 at\u0161\u0137iras atkar\u012bb\u0101 no konkr\u0113t\u0101 ra\u017eot\u0101ja flash ke\u0161a algoritmu \u012bsteno\u0161anas \u012bpatn\u012bb\u0101m.<br \/>\nVisp\u0101r\u0113j\u0101 prakse kr\u0101\u0161anas politik\u0101 SSD l\u012bme\u0146a disku basein\u0101 ir SSD first.<br \/>\n<i>Read Only Flash Cache.<\/i> Read only flash ke\u0161a l\u012bmenis uz SSD par\u0101d\u0101s, ja iev\u0113rojami lokaliz\u0113t\u0101s rakst\u012b\u0161anas oper\u0101cijas ir \u0101rpus ke\u0161a. <br \/>\n<i>Read \/ Write Flash Cache.<\/i> Attiec\u012bb\u0101 uz rakst\u012b\u0161anai dom\u0101tu flash ke\u0161u vispirms tiek noteikts maksim\u0101lais ke\u0161a apjoms, un SSD kr\u0101\u0161anas l\u012bmenis par\u0101d\u0101s tikai tad, ja ke\u0161a izm\u0113rs nav pietiekams, lai apkalpotu visu lokaliz\u0113to slodzi.<br \/>\nSSD un ke\u0161a veiktsp\u0113jas apr\u0113\u0137ins tiek veikts katru reizi balstoties uz ra\u017eot\u0101ja ieteikumiem, bet vienm\u0113r slikt\u0101kaj\u0101 scen\u0101rij\u0101.<br \/>\n<br \/>Avots: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/321178\/\">habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0418\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0441 \u0442\u043e\u0447\u043a\u0438 \u0437\u0440\u0435\u043d\u0438\u044f \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044f \u0445\u043e\u0440\u043e\u0448\u043e \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u044f\u0435\u0442\u0441\u044f \u0432 \u0413\u041e\u0421\u0422 \u0420\u0412 51987 \u2014 \u00ab\u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430, \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u043e\u043c \u0444\u0443\u043d\u043a\u0446\u0438\u043e\u043d\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043a\u043e\u0442\u043e\u0440\u043e\u0439 \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u0432\u044b\u0445\u043e\u0434\u043d\u043e\u0439 \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u0438 \u0434\u043b\u044f \u043f\u043e\u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0435\u0433\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f\u00bb. \u0415\u0441\u043b\u0438 \u0440\u0430\u0441\u0441\u043c\u0430\u0442\u0440\u0438\u0432\u0430\u0442\u044c \u0432\u043d\u0443\u0442\u0440\u0435\u043d\u043d\u044e\u044e \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0443, \u0442\u043e \u043f\u043e \u0441\u0443\u0442\u0438 \u043b\u044e\u0431\u0430\u044f \u0418\u0421 \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u043e\u0439 \u0440\u0435\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043d\u043d\u044b\u0445 \u0432 \u043a\u043e\u0434\u0435 \u0432\u0437\u0430\u0438\u043c\u043e\u0441\u0432\u044f\u0437\u0430\u043d\u043d\u044b\u0445 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c\u043e\u0432. \u0412 \u0448\u0438\u0440\u043e\u043a\u043e\u043c \u043f\u043e\u043d\u0438\u043c\u0430\u043d\u0438\u0438 \u0442\u0435\u0437\u0438\u0441\u0430 \u0422\u044c\u044e\u0440\u0438\u043d\u0433\u0430-\u0427\u0435\u0440\u0447\u0430 \u0430\u043b\u0433\u043e\u0440\u0438\u0442\u043c (\u0430 \u0441\u043b-\u043d\u043e \u0418\u0421) \u043e\u0441\u0443\u0449\u0435\u0441\u0442\u0432\u043b\u044f\u0435\u0442 \u0442\u0440\u0430\u043d\u0441\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044e \u043c\u043d\u043e\u0436\u0435\u0441\u0442\u0432\u0430 \u0432\u0445\u043e\u0434\u043d\u044b\u0445 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":26806,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-35800","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.5 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0418\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0441 \u0442\u043e\u0447\u043a\u0438.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/lv\/blog\/administrirovanie\/dizajn-virtualizovannogo-tsod\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.5\" \/>\n\t\t<meta property=\"og:locale\" content=\"lv_LV\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0414\u0438\u0437\u0430\u0439\u043d \u0432\u0438\u0440\u0442\u0443\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043d\u043d\u043e\u0433\u043e \u0426\u041e\u0414 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0418\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0441 \u0442\u043e\u0447\u043a\u0438.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/lv\/blog\/administrirovanie\/dizajn-virtualizovannogo-tsod\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2019-10-31T19:06:24+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2019-10-31T19:06:24+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Virtualiz\u0113t\u0101 datu centra dizains | ProHoster","description":"Ievads Inform\u0101cijas sist\u0113ma ar punktu.","canonical_url":"https:\/\/prohoster.info\/lv\/blog\/administrirovanie\/dizajn-virtualizovannogo-tsod","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"lv_LV","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0414\u0438\u0437\u0430\u0439\u043d \u0432\u0438\u0440\u0442\u0443\u0430\u043b\u0438\u0437\u043e\u0432\u0430\u043d\u043d\u043e\u0433\u043e \u0426\u041e\u0414 | ProHoster","og:description":"\u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0418\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u0430\u044f \u0441\u0438\u0441\u0442\u0435\u043c\u0430 \u0441 \u0442\u043e\u0447\u043a\u0438.","og:url":"https:\/\/prohoster.info\/lv\/blog\/administrirovanie\/dizajn-virtualizovannogo-tsod","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:06:24+00:00","article:modified_time":"2019-10-31T19:06:24+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"35800","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-22 00:48:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 18:06:24","updated":"2026-01-22 00:48:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/posts\/35800","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/comments?post=35800"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/posts\/35800\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/media\/26806"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/media?parent=35800"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/categories?post=35800"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/lv\/wp-json\/wp\/v2\/tags?post=35800"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}