Umthengi wayefuna i-VDI. Ndijonge ngokwenene kwiSimpliVity + VDI Citrix Virtual Desktop indibaniselwano. Kubo bonke abaqhubi, abasebenzi beofisi yedolophu, njalo njalo. Kukho amawaka amahlanu abasebenzisi kumaza okuqala okufuduka bodwa, kwaye ke ngoko banyanzelise uvavanyo lomthwalo. I-VDI inokuqalisa ukucotha, inokulala ngokuzolileyo - kwaye oku akusoloko kusenzeka ngenxa yeengxaki zetshaneli. Sithenge iphakheji yokuvavanya enamandla kakhulu ngokukodwa kwi-VDI kwaye silayishe iziseko ezingundoqo de kube nzima kakhulu kwiidiski kunye neprosesa.
Ke, siya kufuna ibhotile yeplastiki kunye nesoftware ye-LoginVSI yovavanyo oluntsonkothileyo lweVDI. Sinayo kunye neelayisensi kubasebenzisi abangama-300. Emva koko sathatha i-HPE SimpliVity 380 hardware kwiphakheji efanele umsebenzi wobuninzi bobuninzi bomsebenzisi kwiseva nganye, sasika oomatshini ababonakalayo ngobhaliso olulungileyo, isoftware yeofisi efakwe kwiWin10 kubo kwaye yaqala ukuvavanya.
Masihambe!
Inkqubo
Iinodi ezimbini ze-HPE ezilula 380 Gen10 (iiseva). Kwinye:
- 2 x Intel Xeon Platinum 8170 26c 2.1Ghz.
- RAM: 768GB, 12 x 64GB LRDIMMs DDR4 2666MHz.
- Umlawuli wedisk ephambili: HPE Smart Array P816i-a SR Gen10.
- Iidrayivu ezinzima: 9 x 1.92 TB SATA 6Gb / s SSD (kwi-RAID6 7 + 2 uqwalaselo, oko kukuthi lo mzekelo oPhakathi kwimigaqo ye-HPE SimpliVity).
- Amakhadi womnatha: 4 x 1Gb Eth (idatha yomsebenzisi), 2 x 10Gb Eth (SimpliVity kunye ne-vMotion backend).
- Amakhadi eFPGA akhethekileyo awakhelwe-ngaphakathi kwindawo nganye yokudityaniswa/yokucinezelwa.
Iinode zixhunywe omnye komnye nge-10Gb Ethernet ukudibanisa ngokuthe ngqo ngaphandle kokutshintsha kwangaphandle, esetyenziswa njenge-SimpliVity backend kunye nokudlulisa idatha yomshini we-virtual nge-NFS. Idatha yomatshini obonakalayo kwiqela isoloko ibonwa phakathi kweendawo ezimbini.
Iindibano zidityanisiwe zibe yi-Vmware vSphere cluster elawulwa yi-vCenter.
Ukuvavanya, umlawuli wesizinda kunye nomthengisi woqhagamshelo lweCitrix wasetyenziswa. Umlawuli wesizinda, umthengisi kunye ne-vCenter zibekwe kwiqela elahlukileyo.
Njengesiseko sovavanyo, i-300 iidesktops ezibonakalayo zifakwe kwi-Dedicated - Full Copy configuration, oko kukuthi, i-desktop nganye yikopi epheleleyo yomfanekiso wokuqala womatshini obonakalayo kwaye igcina zonke iinguqu ezenziwe ngabasebenzisi.
Umatshini ngamnye onenyani une-2vCPU kunye ne-4GB RAM:
Le software ilandelayo efunekayo kuvavanyo yafakelwa koomatshini benyani:
- Windows 10 (64-bit), inguqulo 1809.
- I-Adobe Reader XI.
- I-Citrix Virtual Delivery Agent 1811.1.
- Doro PDF 1.82.
- Uhlaziyo lweJava 7 13.
- Microsoft Office Professional Plus 2016.
Phakathi kwee-nodes - ukuphindaphinda okuhambelanayo. Ibhloko nganye yedatha kwiqela ineekopi ezimbini. Oko kukuthi, ngoku kukho isethi epheleleyo yedatha kwindawo nganye. Ngeqela leendawo ezintathu okanye ngaphezulu, iikopi zeebhloko zikwindawo ezimbini ezahlukeneyo. Xa udala i-VM entsha, ikopi eyongezelelweyo yenziwe kwenye yeendawo ze-cluster. Xa enye node isilela, zonke ii-VM ebezisebenza kuyo ngaphambili ziphinda ziqaliswe ngokuzenzekelayo kwezinye iindawo apho zineekopi. Ukuba i-node iyasilela ixesha elide, ngoko ke ukubuyiswa ngokuthe ngcembe kwe-redundancy kuqala, kwaye iqela libuyela kwi-N + 1 yokuphindaphinda.
Ukulinganisa idatha kunye nokugcinwa kwenzeka kwinqanaba lokugcinwa kwesoftware ye-SimpliVity ngokwayo.
Oomatshini benyani baqhuba iqela le-virtualization, elikwabeka kwindawo yokugcina isoftwe. Iidesksi ngokwazo zithathwe ngokwe template eqhelekileyo: iidesika zabaxhasi bezemali kunye namagosa okusebenza beza kuvavanyo (ezi ziitemplate ezimbini ezahlukeneyo).
Ukuvavanywa
Ukuvavanya, i-LoginVSI 4.1 software test suite yasetyenziswa. I-complex ye-LoginVSI, equka iseva yokulawula kunye nemishini ye-12 yokudibanisa ukuvavanya, ifakwe kwi-host host eyahlukileyo.
Uvavanyo lwenziwa ngeendlela ezintathu:
Imowudi yeBenchmark - amatyala okulayisha abasebenzi abangama-300 boLwazi kunye nabasebenzi abangama-300 boGcino.
Imowudi esemgangathweni - ityala lomthwalo we-300 Abasebenzi baMandla.
Ukuvumela abasebenzi baMandla ukuba basebenze kwaye bandise ukuhlukahluka komthwalo, ilayibrari yeefayile zeThala leencwadi ezongezelelweyo zongezwa kwi-LoginVSI complex. Ukuqinisekisa ukuphinda-phinda kweziphumo, zonke iisetingi zebhentshi zovavanyo zishiywe njengeMiyo.
Uvavanyo lwabasebenzi boLwazi kunye naMandla lulinganisa umthwalo wokwenyani wabasebenzisi abasebenza kwizitishi zokusebenza ezibonakalayo.
Uvavanyo lwabasebenzi beSitoreji lwenziwa ngokukodwa ukuvavanya iinkqubo zokugcina idatha; kukude kumthwalo wokwenene womsebenzi kwaye ikakhulu kubandakanya umsebenzisi osebenza kunye nenani elikhulu leefayile ezinobungakanani obahlukeneyo.
Ngexesha lovavanyo, abasebenzisi bangena kwiindawo zokusebenza imizuzu engama-48 ngesantya esimalunga nomsebenzisi omnye rhoqo ngemizuzwana eli-10.
Iziphumo
Isiphumo esiphambili sovavanyo lwe-LoginVSI yi-metric ye-VSImax, ehlanganiswe ukusuka kwixesha lokwenziwa kwemisebenzi eyahlukeneyo eqaliswe ngumsebenzisi. Umzekelo: ixesha lokuvula ifayile kwi-Notepad, ixesha lokucinezela ifayile kwi-7-Zip, njl.
Inkcazo eneenkcukacha zokubala kweemetrics iyafumaneka kumaxwebhu asemthethweni e .
Ngamanye amagama, i-LoginVSI iphinda ipateni yomthwalo oqhelekileyo, ukulinganisa izenzo zomsebenzisi kwi-ofisi yeofisi, ukufunda i-PDF, njalo njalo, kunye nokulinganisa i-latencies eyahlukeneyo. Kukho inqanaba elibalulekileyo lokulibaziseka "yonke into iyancipha, akunakwenzeka ukusebenza"), ngaphambi kokuba kuqwalaselwe ukuba inani eliphezulu labasebenzisi alifikanga. Ukuba ixesha lokuphendula liyi-1 ms ngokukhawuleza kunokuba le "yonke into icothayo", ngoko inkqubo ithathwa njengesebenza ngokuqhelekileyo, kwaye abasebenzisi abaninzi banokongezwa.
Nazi iimetriki eziphambili:
Iimetriki
Amanyathelo athathiweyo
Iinkcukacha inkcazo
Amacandelo alayishiwe
N.S.L.D.
Ixesha lokuvula umbhalo
ifayile enobunzima be-1 KB
I-Notepad iyavula kwaye
ivula i-random 1 KB uxwebhu olukhutshelwe equleni
zixhobo
CPU kunye I/O
NFO
Ixesha lokuvula incoko
iifestile kwincwadana
Ivula ifayile yeVSI-Notepad [Ctrl+O]
CPU, RAM kunye I/O
ZHC*
Ixesha lokudala ifayile yeZip ecinezeleke kakhulu
Uxinzelelo lwasekhaya
random 5MB .pst ifayile ekotshwe kuyo
uvimba wemithombo
CPU kunye I/O
ZLC*
Ixesha lokudala ifayile yeZip ecinezelekileyo
Uxinzelelo lwasekhaya
random 5MB .pst ifayile ekotshwe kuyo
uvimba wemithombo
I / O
ICPU
Ukubala okukhulu
uluhlu lwedatha random
Ukudala uluhlu olukhulu
idatha engacwangciswanga eya kusetyenziswa kwigalelo/isiphumo sesibali-xesha (I/O isibali-xesha)
ICPU
Xa uvavanyo lwenziwa, i-metric esisiseko ye-VSIbase ibalwa kuqala, ebonisa isantya apho imisebenzi eyenziwa khona ngaphandle komthwalo kwinkqubo. Ngokusekelwe kuyo, i-VSImax Threshold imiselwe, elingana ne-VSIbase + 1ms.
Izigqibo malunga nokusebenza kwenkqubo zenziwe ngokusekelwe kwiimetriki ezimbini: i-VSIbase, emisela isantya senkqubo, kunye ne-VSImax threshold, emisela inani eliphezulu labasebenzisi ukuba inkqubo inokusingatha ngaphandle kokuthotywa okukhulu.
300 abasebenzi boLwazi benchmark
Abasebenzi bolwazi ngabasebenzisi abalayisha rhoqo imemori, iprosesa kunye ne-IO kunye neencopho ezincinci ezahlukeneyo. Isoftware ilinganisa umthwalo womsebenzi wabasebenzisi beofisi abafunayo, ngokungathi bahlala bephonononga into ethile (PDF, Java, suite yeofisi, ukujonga ifoto, 7-Zip). Njengoko usongeza abasebenzisi ukusuka kwi-zero ukuya kwi-300, ukulibaziseka komntu ngamnye kunyuka kancinci.
Idatha yezibalo ze-VSImax:
I-VSIbase = 986ms, i-VSI Threshold ayizange ifikelelwe.
Iinkcukacha-manani zokulayisha inkqubo yogcino ukusuka kuhlolo olulula:
Ngolu hlobo lomthwalo, inkqubo inokumelana nomthwalo owandisiweyo ngaphandle kokuthotywa kokusebenza. Ixesha elithathayo ukugqiba imisebenzi yabasebenzisi landa ngokukhawuleza, ixesha lokuphendula kwenkqubo alitshintshi ngexesha lokuvavanya kwaye lifikelela kwi-3 ms yokubhala kunye nokuya kwi-1 ms yokufunda.
Isiphelo: Abasebenzisi bolwazi be-300 basebenza kwi-cluster yangoku ngaphandle kweengxaki kwaye bangaphazamisi omnye nomnye, ukufikelela kwi-pCPU / vCPU yokugqithiswa kwe-1 ukuya kwe-6. Ukulibaziseka ngokubanzi kukhula ngokulinganayo njengoko umthwalo unyuka, kodwa umda ochaziweyo awufikanga.
300 abasebenzi boGcino benchmark
Aba ngabasebenzisi abahlala bebhala kwaye befunda kwireyishini ye-30 ukuya kwi-70, ngokulandelanayo. Olu vavanyo lwenziwa ngakumbi ngenxa yovavanyo. Idatha yezibalo ze-VSImax:
I-VSIbase = 1673, i-VSI Threshold ifikelelwe kubasebenzisi be-240.
Iinkcukacha-manani zokulayisha inkqubo yogcino ukusuka kuhlolo olulula:
Olu hlobo lomthwalo luyimvavanyo yoxinzelelo lwenkqubo yokugcina. Xa iphunyeziwe, umsebenzisi ngamnye ubhala iifayile ezininzi ezingaqhelekanga zobukhulu obahlukeneyo kwidiski. Kule meko, kunokubonwa ukuba xa umlinganiselo othile womthwalo ugqithiswa kubasebenzisi abathile, ixesha elithathayo ukugqiba imisebenzi yokubhala iifayile landa. Ngexesha elifanayo, umthwalo kwinkqubo yokugcina, iprosesa kunye nememori yemikhosi ayitshintshi kakhulu, ngoko okwangoku akunakwenzeka ukugqiba ukuba yintoni ebangela ukulibaziseka.
Izigqibo malunga nokusebenza kwenkqubo usebenzisa olu vavanyo lunokwenziwa kuphela xa kuthelekiswa neziphumo zovavanyo kwezinye iinkqubo, ekubeni imithwalo enjalo yenziwe kwaye ayinangqiqo. Nangona kunjalo, uvavanyo lulonke luhambe kakuhle. Yonke into yahamba kakuhle kwaze kwaba ziiseshoni ze-210, kwaye emva koko iimpendulo ezingaqhelekanga zaqala, ezingazange zilandelelwe naphi na ngaphandle kwe-Ngena nge-VSI.
Abasebenzi baMandla abangama-300
Aba ngabasebenzisi abathanda i-CPU, imemori kunye ne-IO ephezulu. Aba "basebenzisi bamandla" bahlala beqhuba imisebenzi enzima ngokugqabhuka okude, njengokufakela isoftware entsha kunye nokukhupha oovimba abakhulu. Idatha yezibalo ze-VSImax:
I-VSIbase = 970, i-VSI Threshold ayizange ifikelelwe.
Iinkcukacha-manani zokulayisha inkqubo yogcino ukusuka kuhlolo olulula:
Ngexesha lokuvavanya, iprosesa yomthwalo weprosesa yafikelelwa kwenye yeendlela zenkqubo, kodwa oku akuzange kube nempembelelo ebalulekileyo ekusebenzeni kwayo:
Kule meko, inkqubo inokumelana nomthwalo owandisiweyo ngaphandle kokuthotywa okubalulekileyo kokusebenza. Ixesha elithathayo ukugqiba imisebenzi yabasebenzisi landa ngokukhawuleza, ixesha lokuphendula kwenkqubo alitshintshi ngexesha lokuvavanya kwaye lifikelela kwi-3 ms yokubhala kunye nokuya kwi-1 ms yokufunda.
Iimvavanyo eziqhelekileyo azizange zanele kumthengi, kwaye sahamba phambili: sandisa iimpawu ze-VM (inani le-vCPU ukuvavanya ukwanda kokubhaliselwa okungaphezulu kunye nobukhulu bediski) kwaye songeza umthwalo owongezelelweyo.
Xa kuqhutywa iimvavanyo ezongezelelweyo, olu qwalaselo lulandelayo lusetyenzisiwe:
Iidesktops eziyi-300 zenyani zafakwa kwi-4vCPU, i-4GB RAM, i-80GB HDD uqwalaselo.
Ulungelelwaniso lomnye woomatshini bovavanyo:
Oomatshini basasazwe kuZinikezelwe-iKopi epheleleyo ukhetho:
Ama-300 abasebenzi boLwazi benchmark ngomrhumo ongaphezulu 12
Idatha yezibalo ze-VSImax:
VSIbase = 921 ms, VSI Threshold ayizange ifikelelwe.
Iinkcukacha-manani zokulayisha inkqubo yogcino ukusuka kuhlolo olulula:
Iziphumo ezifunyenweyo ziyafana nokuvavanya ubumbeko be-VM yangaphambili.
300 Abasebenzi Power kunye 12 oversubscriptions
Idatha yezibalo ze-VSImax:
I-VSIbase = 933, i-VSI Threshold ayizange ifikelelwe.
Iinkcukacha-manani zokulayisha inkqubo yogcino ukusuka kuhlolo olulula:
Ngexesha lovavanyo, iprosesa yomthwalo weprosesa nayo yafikelelwa, kodwa oku akuzange kube nempembelelo ebalulekileyo ekusebenzeni:
Iziphumo ezifunyenweyo zifana nokuvavanya ubumbeko lwangaphambili.
Kwenzeka ntoni ukuba uqhuba umthwalo iiyure ezili-10?
Ngoku makhe sibone ukuba kuya kubakho "umphumo wokuqokelela" kwaye uqhube iimvavanyo kwiiyure ze-10 ngokulandelelana.
Iimvavanyo zexesha elide kunye nenkcazo yecandelo kufuneka zijolise kwinto yokuba sifuna ukujonga ukuba ngaba kukho naziphi na iingxaki ezinokuvela kunye ne-truss phantsi komthwalo omude kuwo.
300 abasebenzi Ulwazi benchmark + 10 iiyure
Ukongeza, ityala lomthwalo labasebenzi bolwazi abangama-300 lavavanywa, lilandelwa ngumsebenzi wabasebenzisi iiyure ezili-10.
Idatha yezibalo ze-VSImax:
VSIbase = 919 ms, VSI Threshold ayizange ifikelelwe.
VSImax Iinkcukacha-manani zedatha:
Igrafu ibonisa ukuba akukho kuthotywa kwentsebenzo kuqatshelwe kulo lonke uvavanyo.
Iinkcukacha-manani zokulayisha inkqubo yogcino ukusuka kuhlolo olulula:
Ukusebenza kwenkqubo yokugcina kuhlala kufana kulo lonke uvavanyo.
Uvavanyo olongezelelweyo kunye nokongezwa komthwalo wokwenziwa
Umthengi ucele ukongeza umthwalo wasendle kwidiski. Ukwenza oku, umsebenzi wongezwa kwisistim sokugcina kumatshini ngamnye womsebenzisi wenyani ukuqhuba umthwalo wokwenziwa kwidiski xa umsebenzisi engena kwinkqubo. Umthwalo unikezelwe ngumsebenzi we-fio, okuvumela ukuba unciphise umthwalo kwidiski ngenani le-IOPS. Kumatshini ngamnye, kwaqaliswa umsebenzi wokuqalisa umthwalo owongezelelweyo kwisixa se-22 IOPS 70%/30% Random Read/Write.
300 abasebenzi boLwazi benchmark + 22 IOPS ngomsebenzisi ngamnye
Kuvavanyo lokuqala, i-fio yafunyanwa ibeka i-CPU engaphezulu kumatshini obonakalayo. Oku kukhokelele ekulayisheni ngokukhawuleza kwe-CPU yemikhosi kwaye kuchaphazele kakhulu ukusebenza kwenkqubo iyonke.
Ibamba umthwalo weCPU:
Kwangaxeshanye, ukulibaziseka kwenkqubo yokugcina nako kwanda ngokwendalo:
Ukunqongophala kwamandla ekhompyuter kuye kwabaluleka malunga nabasebenzisi abangama-240:
Ngenxa yeziphumo ezifunyenweyo, kwagqitywa ekubeni kuqhutywe uvavanyo oluncinci lwe-CPU.
230 abasebenzi baseOfisi benchmark + 22 IOPS ngomsebenzisi ngamnye
Ukunciphisa umthwalo kwi-CPU, uhlobo lomthwalo wabasebenzi be-Ofisi lukhethiwe, kunye ne-22 IOPS yomthwalo owenziweyo wongezwa kwiseshoni nganye.
Uvavanyo lwalulinganiselwe kwiiseshoni ze-230 ukwenzela ukuba ungagqithisi umthwalo ophezulu we-CPU.
Uvavanyo luqhutywe kunye nabasebenzisi abaqhuba iiyure ze-10 ukujonga ukuzinza kwenkqubo ngexesha lokusebenza kwexesha elide kufuphi nomthwalo omkhulu.
Idatha yezibalo ze-VSImax:
VSIbase = 918 ms, VSI Threshold ayizange ifikelelwe.
VSImax Iinkcukacha-manani zedatha:
Igrafu ibonisa ukuba akukho kuthotywa kwentsebenzo kuqatshelwe kulo lonke uvavanyo.
Iinkcukacha-manani zomthwalo we-CPU:
Xa usenza olu vavanyo, umthwalo kwi-CPU yomkhosi wawuphantse ube mkhulu.
Iinkcukacha-manani zokulayisha inkqubo yogcino ukusuka kuhlolo olulula:
Ukusebenza kwenkqubo yokugcina kuhlala kufana kulo lonke uvavanyo.
Umthwalo kwinkqubo yokugcina ngexesha lovavanyo malunga ne-6 IOPS kwi-500/60 ratio (i-40 IOPS ifundwe, i-3 IOPS ibhala), malunga ne-900 IOPS nganye kwindawo yokusebenza.
Ixesha lokuphendula libe yi-avareji ye-3 ms ekubhaleni ukuya kuthi ga kwi-1 ms yokufunda.
Isiphumo
Xa ulinganisa imithwalo yokwenyani kwiziseko zophuhliso zeHPE ezilula, iziphumo zafunyanwa ziqinisekisa amandla enkqubo ukuxhasa iidesktops ezibonakalayo ubuncinane ezingama-300 oomatshini beClone abaPheleleyo kwisibini se-SimpliVity node. Ngexesha elifanayo, ixesha lokuphendula lenkqubo yokugcina ligcinwe kwinqanaba elifanelekileyo kulo lonke uvavanyo.
Sichukumiseke kakhulu yindlela yovavanyo olude kunye nokuthelekisa izisombululo ngaphambi kokuphunyezwa. Sinokuvavanya ukusebenza komthwalo wakho womsebenzi ukuba uyafuna. Kubandakanya ezinye izisombululo ezihlanganisiweyo. Umthengi okhankanywe ngoku ugqiba iimvavanyo kwesinye isisombululo ngokuhambelanayo. Iziseko zayo zangoku zizininzi nje zeePC, isizinda kunye nesoftware kwindawo nganye yokusebenza. Ukuya kwi-VDI ngaphandle kovavanyo, kunjalo, kunzima kakhulu. Ngokukodwa, kunzima ukuqonda ubuchule bokwenyani befama yeVDI ngaphandle kokufudukela kuyo abasebenzisi bokwenyani. Kwaye ezi mvavanyo zikuvumela ukuba uvavanye ngokukhawuleza ubunakho bokwenyani benkqubo ethile ngaphandle kwesidingo sokubandakanya abasebenzisi abaqhelekileyo. Kulapho esi sifundo sivela khona.
Indlela yesibini ebalulekileyo kukuba umthengi uzibophelele ngoko nangoko ekulinganiseni okufanelekileyo. Apha ungathenga iseva eyongezelelweyo kwaye ungeze ifama, umzekelo, kubasebenzisi be-100, yonke into iqikelelwa kwixabiso lomsebenzisi. Ngokomzekelo, xa kufuneka bongeze abasebenzisi be-300 ngaphezulu, baya kukwazi ukuba bafuna iiseva ezimbini kwi-configuration esele ichaziwe, kunokuba baphinde baqwalasele ukuphuculwa kweziseko zabo zonke.
Izinto ezinokwenzeka kwi-HPE SimpliVity federation zinomdla. Ishishini lahlulwe ngokwejografi, ngoko kuyavakala ukufaka eyakho ihardware yeVDI eyahlukileyo kwiofisi ekude. Kumdibaniso we-SimpliVity, umatshini ngamnye wenyani uphinda-phindwa ngokweshedyuli ngokukwazi ukuphinda-phinda phakathi kwamaqela akude ngokwejografi ngokukhawuleza kwaye ngaphandle komthwalo kwisitishi - oku kukugcinwa kwe-backup eyakhelwe-ngaphakathi kwinqanaba elihle kakhulu. Xa uphindaphinda ii-VM phakathi kweesayithi, umjelo usetyenziselwa ubuncinci ngokusemandleni, kwaye oku kwenza kube lula ukwakha i-architectures ye-DR enomdla kakhulu phambi kwendawo yokulawula enye kunye neqela leendawo zokugcina ezibekwe phantsi.
Konke oku kunye kwenza kube lula ukuvavanya icala lezemali ngokweenkcukacha ezinkulu, kunye nokugqithisa iindleko ze-VDI kwizicwangciso zokukhula kwenkampani, kunye nokuqonda ukuba isisombululo siya kuhlawula ngokukhawuleza kangakanani kwaye siya kusebenza njani. Ngenxa yokuba nayiphi na i-VDI isisombululo ekugqibeleni sigcina izibonelelo ezininzi, kodwa kwangaxeshanye, kusenokwenzeka, ngaphandle kwexabiso eliphantsi ithuba lokuyitshintsha phakathi kweminyaka emi-5-7 yokusetyenziswa.
Ngokubanzi, ukuba unemibuzo engeyiyo yezimvo, ndibhalele nge-imeyile [imeyile ikhuselwe].
umthombo: www.habr.com