Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Molweni nonke! Ngeli nqaku, i-AERODISK ivula iblogi kuHabré. Hey, maqabane!

Amanqaku angaphambili kuHabré axubushe ngemibuzo malunga noyilo kunye noqwalaselo olusisiseko lweenkqubo zokugcina. Kweli nqaku siza kuqwalasela umbuzo ongazange ugutyungelwe ngaphambili, kodwa uvame ukubuzwa - malunga nokunyamezela impazamo kwiinkqubo zokugcina i-AERODISK ENGINE. Iqela lethu liya kwenza konke ukuqinisekisa ukuba inkqubo yokugcina i-AERODISK iyeka ukusebenza, oko kukuthi. yaphule.

Kwenzeka ukuba amanqaku malunga nembali yenkampani yethu, malunga neemveliso zethu, kunye nomzekelo wokuphunyezwa ngempumelelo sele exhonywe kwi-Habré, Sibulela kakhulu kumaqabane ethu - iTS Solution kunye neenkampani zeSoftline.

Ke ngoko, andizukuqeqesha izakhono zolawulo lokukopa-uncamathisele apha, kodwa ndiya kubonelela ngokulula amakhonkco kwimvelaphi yala manqaku:

Ndikwafuna ukwabelana ngeendaba ezimnandi. Kodwa ndiza kuqala, ewe, ngengxaki. Thina, njengomthengisi oselula, phakathi kwezinye iindleko, sihlala sijongene nenyaniso yokuba iinjineli ezininzi kunye nabalawuli abazi nje indlela yokusebenzisa kakuhle inkqubo yethu yokugcina.
Kucacile ukuba ukulawula iinkqubo ezininzi zokugcina zibukeka zifana ngokufanayo kumbono womlawuli, kodwa umenzi ngamnye uneempawu zakhe. Kwaye nathi asingomkhethe apha.

Ngoko ke, ukuze wenziwe lula umsebenzi wokuqeqesha iingcali ze-IT, sigqibe kwelokuba sinikele kulo nyaka kwimfundo yasimahla. Ukwenza oku, kwizixeko ezininzi ezikhulu zaseRashiya sivula uthungelwano lwamaZiko oBuchule be-AERODIS, apho nayiphi na ingcali yezobugcisa enomdla inokuthatha ikhosi ngaphandle kwentlawulo kwaye ifumane isatifikethi sokulawula iinkqubo zokugcina i-AERODISK ENGINE.

KwiZiko loBuchule ngalinye siya kufakela idemo stand egcwele ngokupheleleyo esuka kwisistim ye-AERODISK yokugcina kunye ne-physical server, apho utitshala wethu aya kuqhuba uqeqesho lobuso ngobuso. Siza kupapasha ishedyuli yomsebenzi wamaZiko oBuchule ekubonakaleni kwawo, kodwa sele sivule iziko eNizhny Novgorod kwaye isixeko saseKrasnodar silandelayo. Ungabhalisela uqeqesho usebenzisa amakhonkco angezantsi. Nalu ulwazi lwangoku olwaziwayo malunga nezixeko kunye nemihla:

  • Nizhny Novgorod (SELUVULELWE- ungabhalisa apha https://aerodisk.promo/nn/);
    Kuze kube ngo-Ephreli 16, 2019, unokutyelela iziko nangaliphi na ixesha lokusebenza, kwaye ngo-Ephreli 16, i-2019, ikhosi enkulu yoqeqesho iya kuququzelelwa.
  • Krasnodar (KUVULA KUNGEKUDALA- ungabhalisa apha https://aerodisk.promo/krsnd/ );
    Ukususela ngo-Apreli 9 ukuya ku-Aprili 25, 2019, unokutyelela iziko nangaliphi na ixesha lokusebenza, kwaye ngo-Aprili 25, i-2019, ikhosi enkulu yoqeqesho iya kuququzelelwa.
  • Екатеринбург (KUVULA KUNGEKUDALA, landela ulwazi olukwiwebhusayithi yethu okanye kwiHabré);
    NgoMeyi-Juni 2019.
  • Новосибирск (landela ulwazi olukwiwebhusayithi yethu okanye kwiHabré);
    Okthobha 2019.
  • Красноярск (landela ulwazi olukwiwebhusayithi yethu okanye kwiHabré);
    EyeNkanga 2019.

Kwaye, ngokuqinisekileyo, ukuba iMoscow ayikude nawe, ngoko nangaliphi na ixesha unokutyelela iofisi yethu eMoscow kwaye ufumane uqeqesho olufanayo.

Konke. Sigqibile ngentengiso, masiqhubele phambili kwitekhnoloji!

Kwi-Habré siya kupapasha rhoqo amanqaku ezobugcisa malunga neemveliso zethu, iimvavanyo zomthwalo, uthelekiso, iimpawu zokusetyenziswa kunye nokuphunyezwa okuthakazelisayo.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Isilumkiso! Emva kokufunda inqaku, unokuthi: Ewe, ngokuqinisekileyo, umthengisi uya kuzijonga ukuze yonke into isebenze "nge-bang," iimeko ze-greenhouse, njl. Ndiya kuphendula: akukho nto injalo! Ngokungafani nabakhuphiswano bethu bangaphandle, sihlala apha, kufuphi nawe, kwaye unako ukuza kuthi (eMoscow okanye nayiphi na iKomiti Ephakathi) kwaye uvavanye inkqubo yethu yokugcina nayiphi na indlela. Ke, akukho ngqiqweni kakhulu kuthi ukulungelelanisa iziphumo kumfanekiso ofanelekileyo wehlabathi, kuba Silula kakhulu ukujonga. Kwabo bonqena ukuya kwaye abanaxesha, sinokulungiselela uvavanyo olukude. Sinelebhu ekhethekileyo yale nto. Qhagamshelana nathi.

ACHTUNG-2! Olu vavanyo alukho uvavanyo lomthwalo, kuba apha sikhathalela kuphela ukunyamezela iimpazamo. Kwiiveki ezimbalwa, siya kulungiselela ukuma okunamandla kwaye siqhube uvavanyo lomthwalo wenkqubo yokugcina, ukupapasha iziphumo apha (ngendlela, izicelo zovavanyo zamkelwe).

Ngoko, masihambe siyoyiqhawula.

ibhentshi yovavanyo

Indawo yethu iqulathe ezi hardware zilandelayo:

  • I-1 x Injini ye-Aerodisk ye-N2 yokugcina inkqubo (abalawuli be-2, i-64GB cache, i-8xFC port 8Gb / s, i-4xEthernet port 10Gb / s SFP +, i-4xEthernet izibuko 1Gb / s); Ezi diski zilandelayo zifakwe kwisixokelelwano sogcino:
  • 4 x SAS SSD disks 900 GB;
  • 12 x SAS 10k disks 1,2 TB;
  • I-1 x Iseva yomzimba kunye neWindows Server 2016 (2xXeon E5 2667 v3, 96GB RAM, 2xFC port 8Gb / s, 2xEthernet port 10Gb / s SFP +);
  • 2 x SAN 8G iswitshi;
  • 2 x LAN 10G switch;

Siqhagamshele umncedisi kwinkqubo yokugcina ngokutshintsha nge-FC kunye ne-10G Ethernet. Umzobo wokuma ungezantsi.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Amacandelo esiwafunayo, afana neMPIO kunye ne-iSCSI initiator, ifakwe kwi-Windows Server.
Imimandla iqwalaselwe kutshintsho lweFC, iiVLAN ezihambelanayo ziqwalaselwe kwizitshixo zeLAN, kwaye iMTU 9000 ifakwe kwizibuko zokugcina, iiswitshi, kunye nomkhosi (indlela yokwenza konke oku kuchazwe kuxwebhu lwethu, ngoko asiyi kuchaza. le nkqubo apha).

Uvavanyo Lwendlela

Isicwangciso sovavanyo lokuwa sihamba ngolu hlobo lulandelayo:

  • Ukujonga ukusilela kweFC kunye ne-Ethernet port.
  • Ukuhlolwa kokusilela kwamandla.
  • Ukuhlolwa kokusilela komlawuli.
  • Ukukhangela ukusilela kwedisk kwiqela/iphuli.

Zonke iimvavanyo ziya kwenziwa phantsi kweemeko zomthwalo wokwenziwa, esiya kuvelisa ngenkqubo ye-IOMETER. Ngokuhambelanayo, siya kwenza iimvavanyo ezifanayo, kodwa phantsi kweemeko zokukopisha iifayile ezinkulu kwinkqubo yokugcina.

Uqwalaselo lwe-IOmeter luloluhlobo lulandelayo:

  • Funda/Bhala – 70/30
  • Block - 128k (sagqiba ukuhlamba iinkqubo zokugcina kwiibhloko ezinkulu)
  • Inani leentambo - 128 (efana kakhulu nomthwalo ovelisayo)
  • Random ngokupheleleyo
  • Inani labasebenzi – 4 (2 for FC, 2 for iSCSI)

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla
Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Uvavanyo lunezi njongo zilandelayo:

  1. Qinisekisa ukuba umthwalo owenziweyo kunye nenkqubo yokukhuphela ayiyi kuphazamisa okanye ibangele iimpazamo phantsi kweemeko ezahlukeneyo zokungaphumeleli.
  2. Qinisekisa ukuba inkqubo yokutshintsha amachweba, abalawuli, njl., i-automated ngokwaneleyo kwaye ayifuni izenzo zomlawuli xa kukho ukungaphumeleli (oko kukuthi, ngexesha lokusilela, asithethi ngokuhluleka, ngokuqinisekileyo).
  3. Qinisekisa ukuba ulwazi olukwiilog luboniswe ngokuchanekileyo.

Ukulungiselela inginginya kunye nenkqubo yokugcina

Siqwalasele ukufikelela kwibhloko kwinkqubo yokugcina usebenzisa i-FC kunye ne-Ethernet port (i-FC kunye ne-iSCSI, ngokulandelanayo). Abafana abavela kwi-TS Solution bachaze ngokweenkcukacha indlela yokwenza oku kwinqaku elidlulileyo (https://habr.com/ru/company/tssolution/blog/432876/). Kwaye, ngokuqinisekileyo, akukho mntu urhoxise iincwadana kunye nezifundo.

Saseta iqela lemixube sisebenzisa zonke iidrive esinazo. Iidiski ze-2 SSD zongezwa kwi-cache, iidiski ze-2 SSD zongezwa njenge-tier yokugcina eyongezelelweyo (i-Online-tier). Sahlula i-12 SAS10k drives kwi-RAID-60P (i-triple parity) ukwenzela ukujonga ukungaphumeleli kwee-drive ezintathu kwiqela kanye. Idiski enye yayishiywe ukuze ifakwe endaweni yayo.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Siqhagamshele ii-LUN ezimbini (enye nge-FC, enye nge-iSCSI).

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Umnini wazo zombini ii-LUN ngumlawuli we-Engine-0

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Masiqale uvavanyo

Senza i-IOMETER ngoqwalaselo olungentla.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Sirekhoda i-throughput ye-1.8 GB / s kunye ne-latency ye-3 milliseconds. Akukho ziphoso (Inani lemposiso iyonke).

Ngexesha elifanayo, ukusuka kwi-drive yendawo "C" yomninimzi wethu, ngokufanayo siqala ukukopisha iifayile ezimbini ezinkulu ze-100GB kwi-FC kunye ne-iSCSI yokugcina i-LUNs (i-drives E kunye ne-G kwi-Windows), isebenzisa ezinye ii-interfaces.

Ngentla yinkqubo yokukopa kwi-LUN FC, ngezantsi kwi-iSCSI.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Uvavanyo #1: Ukukhubaza izibuko ze-I / O

Sisondela kwinkqubo yokugcina ngasemva))) kunye nokunyakaza okuncinci kwesandla sikhupha zonke iintambo ze-FC kunye ne-Ethernet 10G kwi-Engine-0 controller. Kufana nokuba inkosikazi yokucoca ene-mop yahamba kwaye yagqiba ekubeni ihlambe umgangatho apho i-snot ilala khona kwaye iintambo zilele (oko kukuthi umlawuli usasebenza, kodwa i-I / O izibuko zifile).

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Makhe sijonge kwi-IOMETER kunye nokukopa iifayile. Ukugqithisa kwehle kwi-0,5 GB / s, kodwa ngokukhawuleza kubuyele kwinqanaba langaphambili (malunga ne-4-5 imizuzwana). Akukho zimpazamo.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Ukukhuphela iifayile akuyekanga, kukho ukuhla kwesantya, kodwa akukho nto ibalulekileyo (ukusuka kwi-840 MB / s yehla ukuya kwi-720 MB / s). Ukukopa akuyekanga.

Sijonga iilogi zenkqubo yokugcina kwaye sibone umyalezo malunga nokungafumaneki kwamazibuko kunye nokufuduka ngokuzenzekelayo kweqela.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Iphaneli yolwazi ikwasixelela ukuba yonke into ayilunganga kakhulu kumazibuko eFC.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Inkqubo yokugcina yasinda ukusilela kwe-I/O izibuko ngempumelelo.

Uvavanyo lweNombolo ye-2. Ukukhubaza umlawuli wokugcina

Phantse kwangoko (emva kokuba siqhagamshele iintambo kwakhona kwinkqubo yokugcina) sagqiba kwelokuba sigqibezele inkqubo yokugcina ngokukhupha umlawuli kwi-chassis.

Kwakhona sisondela kwinkqubo yokugcina ngasemva (esiyithandayo))) kwaye ngeli xesha sikhupha umlawuli we-Engine-1, okwangoku ngumnini we-RDG (apho iqela lihamba khona).

Imeko kwi-IOmeter imi ngolu hlobo lulandelayo. I/O ndiyeke malunga nemizuzwana emi-5. Iimpazamo aziqokeleli.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Emva kwemizuzwana emi-5, i-I/O iphinde yaqala kwakhona malunga nokuphuma okufanayo, kodwa nge-latencies ye-35 millisecond (i-latencies yalungiswa emva kwemizuzu embalwa). Njengoko kunokubonwa kwi-screenshots, i-Total error count value yi-0, oko kukuthi, akukho mpazamo yokubhala okanye yokufunda.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Makhe sijonge ukukopa iifayile zethu. Njengoko ubona, ayizange iphazamiseke, kukho ukuhla okuncinci ekusebenzeni, kodwa yonke into ibuyele kwi-~ 800 MB / s efanayo.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Siya kwinkqubo yokugcina kwaye sibone isiqalekiso kwipaneli yolwazi ukuba umlawuli we-Injini-1 akafumaneki (ngokuqinisekileyo, siyibulele).

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Kwakhona sibona ukungena okufanayo kwiilogi.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Umlawuli wokugcina uphinde wasinda ukusilela ngempumelelo.

Uvavanyo lwe-3: Ukuqhawula unikezelo lwamandla.

Nje ukuba kunokwenzeka, siqale ukukopa iifayile kwakhona, kodwa asizange siyeke i-IOMETER.
Sitsala iyunithi yonikezelo lwamandla.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Esinye isilumkiso songeziwe kwisixokelelwano sokugcina kwindawo yolwazi.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Kwakhona kwimenyu yabenzi boluvo sibona ukuba abenzi boluvo abanxulumene nonikezelo lwamandla olukhutshiweyo bajike babomvu.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Inkqubo yokugcina iyaqhubeka nokusebenza. Ukungaphumeleli kweyunithi yonikezelo lwamandla akuchaphazeli nangayiphi na indlela ukusebenza kwenkqubo yokugcina; ukusuka kwindawo yokujonga i-host host, isantya sekopi kunye nezikhombisi ze-IOMETER zahlala zingatshintshi.

Uvavanyo lokusilela kwamandla luphumelele ngempumelelo.

Ngaphambi kovavanyo lokugqibela, sagqiba ekubeni sibuyisele inkqubo yogcino ebomini kancinci, sibuyisele isilawuli kunye neyunithi yonikezelo lwamandla, kwaye sibeke iintambo ngokulandelelana, apho inkqubo yokugcina isixelele ngovuyo ngee-icon eziluhlaza kwiphaneli yayo yezempilo. .

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Uvavanyo lweNombolo 4. Ukungaphumeleli kweediski ezintathu kwiqela

Ngaphambi kolu vavanyo, senze inyathelo elongezelelweyo lokulungiselela. Inyaniso kukuba inkqubo yokugcina i-ENGINE inikeza into eluncedo kakhulu - imigaqo-nkqubo eyahlukeneyo yokwakha kwakhona. I-TS Solution ibhale ngeli nqaku ngaphambili, kodwa masikhumbule undoqo wayo. Umlawuli wogcino unokucacisa okuphambili kulwabiwo lwezibonelelo ngexesha lokwakha kwakhona. Nokuba kwicala lentsebenzo ye-I / O, oko kukuthi, ukwakhiwa kwakhona kuthatha ixesha elide, kodwa akukho kwehla kokusebenza. Okanye kwicala lesantya sokwakha kwakhona, kodwa imveliso iya kuncitshiswa. Okanye ukhetho olulungeleleneyo. Ekubeni ukusebenza kokugcinwa ngexesha lokwakhiwa kweqela le-disk lihlala liyintloko ye-admin, siya kuvavanya umgaqo-nkqubo kunye ne-bias ngokubhekiselele ekusebenzeni kwe-I / O kunye neendleko zesantya sokuvuselela.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Ngoku makhe sijonge ukusilela kwidisk. Sikwavumela ukurekhoda kwii-LUNs (iifayile kunye ne-IOMETER). Ekubeni sineqela eline-triple parity (RAID-60P), oku kuthetha ukuba inkqubo kufuneka imelane nokungaphumeleli kweediski ezintathu, kwaye emva kokusilela, ukutshintshwa kwe-auto kufuneka kusebenze, idiski enye kufuneka ithathe indawo yenye yezo zihlulekile. kwi-RDG, kwaye ukwakha kwakhona kufuneka kuqale kuyo.

Qala. Okokuqala, ngokusebenzisa ujongano lokugcina, makhe sigxininise iidiski esifuna ukuzikhupha (ukuze singaphuthelwa kwaye sitsale idiski yokutshintsha ngokuzenzekelayo).

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Sijonga isalathisi kwi-hardware. Yonke into ilungile, sibona iidiski ezintathu eziphawulweyo.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Kwaye sikhupha ezi diski zintathu.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Makhe sijonge into ekumamkeli. Kwaye akukho nto ikhethekileyo eyenzekayo.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla
Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Iimpawu zokukopisha (ziphezulu kunezokuqala, kuba i-cache ifudumala) kwaye i-IOMETER ayitshintshi kakhulu xa isusa iidiski kwaye iqala ukwakhiwa kwakhona (ngaphakathi kwe-5-10%).

Makhe sijonge kwinto ekwinkqubo yokugcina.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Kwimeko yeqela, sibona ukuba inkqubo yohlengahlengiso iqalile kwaye isondele ekugqityweni.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Kwi-RDG skeleton unokubona ukuba iidiski ezi-2 zikwisimo esibomvu, kwaye enye sele itshintshiwe. I-autoreplacement disk ayisekho; ithathe indawo yediski yesi-3 engaphumelelanga. Ukwakhiwa kwakhona kuthathe imizuzu emininzi, ukubhala iifayile xa iidiski ze-3 zingaphumelelanga azizange ziphazamiseke, kwaye ukusebenza kwe-I / O akuzange kutshintshe kakhulu.

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Iimvavanyo zokuphazamiseka kwe-AERODISK ENGINE N2 inkqubo yokugcina, uvavanyo lwamandla

Uvavanyo lokungaphumeleli kwediski luphumelele ngokuqinisekileyo ngempumelelo.

isiphelo

Ngeli xesha, sagqiba ekubeni siyeke ubundlobongela kwiinkqubo zokugcina. Masishwankathele:

  • Ukuhlolwa kokusilela kwezibuko leFC - kuphumelele
  • Ukuhlolwa kwe-Ethernet port ukusilela-kuphumelele
  • Ukuhlolwa kokusilela komlawuli - kuphumelele
  • Uvavanyo lokungaphumeleli kwamandla - luphumelele
  • Ukukhangela ukungaphumeleli kwedisk kwi-grouppool-iphumelele

Akukho nanye kwiintsilelo eziyekile ukurekhoda okanye ezibangele iimpazamo kumthwalo wokwenziwa; ewe, bekukho ukubetha kwentsebenzo (kwaye siyayazi indlela yokuyoyisa, esiya kuyenza kungekudala), kodwa ngenxa yokuba le yimizuzwana, yamkelekile. Isiphelo: ukunyamezela kwephutha kuzo zonke iinxalenye zenkqubo yokugcina i-AERODISK esebenza kwinqanaba, akukho manqaku angaphumeleli.

Ngokucacileyo, kwelinye inqaku asinakuvavanya zonke iimeko zokungaphumeleli, kodwa sizame ukugubungela ezona zidumileyo. Ke ngoko, nceda uthumele izimvo zakho, iingcebiso malunga nokupapashwa kwexesha elizayo kwaye, ngokuqinisekileyo, ukugxeka okwaneleyo. Siya kukuvuyela ukuxoxa (okanye ngcono okwangoku, yiza kuqeqesho, ndiphindaphinda ishedyuli ukuba kunokwenzeka)! De iimvavanyo ezintsha!

  • Nizhny Novgorod (SELUVULELWE- ungabhalisa apha https://aerodisk.promo/nn/);
    Kuze kube ngo-Ephreli 16, 2019, unokutyelela iziko nangaliphi na ixesha lokusebenza, kwaye ngo-Ephreli 16, i-2019, ikhosi enkulu yoqeqesho iya kuququzelelwa.
  • Krasnodar (KUVULA KUNGEKUDALA- ungabhalisa apha https://aerodisk.promo/krsnd/ );
    Ukususela ngo-Apreli 9 ukuya ku-Aprili 25, 2019, unokutyelela iziko nangaliphi na ixesha lokusebenza, kwaye ngo-Aprili 25, i-2019, ikhosi enkulu yoqeqesho iya kuququzelelwa.
  • Екатеринбург (KUVULA KUNGEKUDALA, landela ulwazi olukwiwebhusayithi yethu okanye kwiHabré);
    NgoMeyi-Juni 2019.
  • Новосибирск (landela ulwazi olukwiwebhusayithi yethu okanye kwiHabré);
    Okthobha 2019.
  • Красноярск (landela ulwazi olukwiwebhusayithi yethu okanye kwiHabré);
    EyeNkanga 2019.

umthombo: www.habr.com

Yongeza izimvo