Yadda muka keta babbar bangon wuta ta kasar Sin (Sashe na 2)

Sannu!

Nikita yana tare da ku kuma, injiniyan tsarin daga kamfanin SEMrush. Kuma da wannan labarin na ci gaba da labarin game da yadda muka fito da hanyar warware matsalar Firewall na kasar Sin don sabis ɗin mu semrush.com.

В bangaren da ya gabata Na ce:

• matsalolin da suka taso bayan yanke shawara "Muna buƙatar yin aikin hidimarmu a China"
• Wadanne matsaloli ne Intanet na kasar Sin ke da shi?
• me yasa kuke buƙatar lasisin ICP?
• ta yaya kuma dalilin da yasa muka yanke shawarar gwada wuraren gwajin mu tare da Catchpoint
• menene sakamakon mafitarmu ta farko bisa hanyar sadarwar China Cloudflare
• Yadda muka sami kwaro a cikin Cloudflare DNS

Wannan bangare shine mafi ban sha'awa, a ganina, saboda yana mai da hankali kan takamaiman aiwatar da fasaha na tsarawa. Kuma za mu fara, ko kuma a ci gaba, da Alibaba Cloud.

Alibaba Cloud

Alibaba Cloud babban mai ba da girgije ne mai gaskiya, wanda ke da duk ayyukan da ke ba shi damar kiran kansa mai samar da girgije. Yana da kyau cewa suna da damar yin rajistar masu amfani da ƙasashen waje, kuma yawancin rukunin yanar gizon ana fassara su cikin Ingilishi (don China wannan abin alatu ne). A cikin wannan gajimare, zaku iya aiki tare da yankuna da yawa na duniya, babban yankin kasar Sin, da kuma Asiya ta teku (Hong Kong, Taiwan, da sauransu).

IPSEC

Mun fara da labarin kasa. Tunda rukunin yanar gizon mu yana kan Google Cloud, muna buƙatar "haɗa" Alibaba Cloud tare da GCP, don haka mun buɗe jerin wuraren da Google ke ciki. A wannan lokacin har yanzu ba su da nasu cibiyar bayanai a Hong Kong.
Yankin mafi kusa ya zama Asiya-gabas1 (Taiwan). Ali ya zama yanki mafi kusanci na babban yankin kasar Sin da Taiwan cn-shenzhen (Shenzhen).

Tare da taimakon terraform ya bayyana kuma ya haɓaka duk abubuwan more rayuwa a cikin GCP da Ali. Ramin mita 100 Mbit/s tsakanin gajimare ya haura kusan nan take. A gefen Shenzhen da Taiwan, an ɗaga na'urori masu ɗaukar hoto. A Shenzhen, an daina zirga-zirgar masu amfani, ana ba da izini ta hanyar rami zuwa Taiwan, kuma daga can yana zuwa kai tsaye zuwa IP na waje na sabis ɗinmu a ciki. mu-gabas (Amurka Gabas Coast). Ping tsakanin injunan kama-da-wane ta hanyar rami 24ms, wanda ba shi da kyau sosai.

A lokaci guda, mun sanya wurin gwaji a ciki Alibaba Cloud DNS. Bayan ƙaddamar da yankin zuwa NS Ali, lokacin ƙuduri ya ragu daga 470 ms zuwa 50 ms. Kafin wannan, yankin kuma yana kan Cloudlfare.

Daidai da rami zuwa Asiya-gabas1 Ya tayar da wani rami daga Shenzhen kai tsaye zuwa us-gabas4. A can sun ƙirƙiri ƙarin injunan kama-da-wane kuma sun fara gwada mafita guda biyu, sarrafa zirga-zirgar gwaji ta amfani da Kukis ko DNS. An kwatanta bencin gwajin da tsari a cikin adadi mai zuwa:

Latency don tunnels ya zama kamar haka:
Ali cn-shenzhen <-> GCP asia-east1 - 24ms
Ali cn-shenzhen <—> GCP us-east4 — 200ms

Gwajin bincike na Catchpoint ya ba da rahoton ingantaccen ci gaba.

Kwatanta sakamakon gwaji don mafita guda biyu:

yanke shawara
Kyau
Median
Kashi 75 cikin ɗari
Kashi 95 cikin ɗari

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

Wannan bayanai ne daga mafita da ke amfani da rami IPSEC ta hanyar Asiya-gabas1. Ta hanyar mu-east4 sakamakon ya kasance mafi muni, kuma akwai ƙarin kurakurai, don haka ba zan ba da sakamakon ba.

Dangane da sakamakon wannan gwajin na wasu ramuka guda biyu, daya daga cikinsu an dakatar da shi a yankin mafi kusa da kasar Sin, dayan kuma a wurin karshe, ya bayyana cewa yana da muhimmanci a "fito" daga karkashin katangar ta kasar Sin da sauri. zai yiwu, sannan yi amfani da cibiyoyin sadarwa masu sauri (Masu samar da CDN, masu samar da girgije, da sauransu). Babu buƙatar ƙoƙarin shiga ta hanyar Tacewar zaɓi don isa wurin da kuke tafiya cikin faɗuwa ɗaya. Wannan ba hanya ce mafi sauri ba.

Gabaɗaya, sakamakon ba su da kyau, duk da haka, semrush.com yana da matsakaicin matsakaici na 8.8s, da 75 Percentile 9.4s (a kan wannan gwajin).
Kuma kafin in ci gaba, Ina so in yi ɗan gajeren waƙar digression.

Cutar mace mai narkewa

Bayan mai amfani ya shiga shafin www.semrushchina.cn, wanda ke warwarewa ta hanyar "sauri" sabar DNS na kasar Sin, buƙatar HTTP ta shiga cikin hanyar mu cikin sauri. Ana mayar da martani ta hanya ɗaya, amma an ƙayyade yankin a cikin duk rubutun JS, shafukan HTML da sauran abubuwan shafin yanar gizon. semrush.com don ƙarin albarkatu waɗanda dole ne a loda su lokacin da aka sanya shafin. Wato, abokin ciniki yana warware rikodin "babban" A www.semrushchina.cn kuma ya shiga cikin rami mai sauri, da sauri ya karɓi amsa - shafin HTML wanda ke cewa:

• zazzage irin wannan da irin wannan js daga sso.semrush.com,
• Samo fayilolin CSS daga cdn.semrush.com,
• da kuma ɗaukar wasu hotuna daga dab.semrush.com
• da sauransu.

Mai binciken ya fara zuwa Intanet na "waje" don waɗannan albarkatun, duk lokacin da ya wuce ta hanyar wuta wanda ke cinye lokacin amsawa.

Amma gwajin da ya gabata yana nuna sakamakon lokacin da babu albarkatu akan shafin semrush.comkawai semrushchina.cn, da kuma * .semrushchina.cn yanke zuwa adireshin na'ura mai mahimmanci a Shenzhen don shiga cikin rami.

Ta wannan hanyar kawai, ta hanyar tura duk zirga-zirgar zirga-zirga zuwa iyakar ta hanyar hanyar ku don wucewa da sauri ta bangon wuta na kasar Sin, zaku iya samun saurin karɓuwa da alamun samin gidan yanar gizo, da kuma sakamakon gaskiya na gwaje-gwajen mafita.
Mun yi wannan ba tare da gyara lamba ɗaya ba a ɓangaren samfurin ƙungiyar.

Subfilter

An haifi maganin kusan nan da nan bayan wannan matsalar ta bulla. Muna bukata PoC (Tabbacin Ra'ayi) cewa hanyoyin shigar da mu ta wuta da gaske suna aiki da kyau. Don yin wannan, kuna buƙatar kunsa duk zirga-zirgar rukunin yanar gizon a cikin wannan bayani gwargwadon yiwuwa. Kuma mun nema subfilter ku nginx.

Subfilter wani tsari ne mai sauƙi mai sauƙi a cikin nginx wanda ke ba ku damar canza layi ɗaya a cikin jikin amsawa zuwa wani layi. Don haka mun canza duk abubuwan da suka faru semrush.com a kan semrushchina.cn a duk amsoshi.

Kuma ... bai yi aiki ba saboda mun karɓi abubuwan da aka matse daga bayanan baya, don haka subfilter bai sami layin da ake buƙata ba. Dole ne in ƙara wani uwar garken gida zuwa nginx, wanda ya rage amsa kuma ya mika shi zuwa uwar garken gida na gaba, wanda ya riga ya shagaltu da maye gurbin kirtani, matsawa, da aika shi zuwa uwar garken wakili na gaba a cikin sarkar.

A sakamakon haka, inda abokin ciniki zai karɓa .semrush.com, ya karba .semrushchina.cn kuma cikin biyayya mun bi shawararmu.

Duk da haka, bai isa kawai canza yankin hanya ɗaya ba, saboda masu goyon baya har yanzu suna tsammanin semrush.com a cikin buƙatun na gaba daga abokin ciniki. Dangane da haka, akan uwar garken guda ɗaya inda aka yi maye gurbin ta hanya ɗaya, ta amfani da magana mai sauƙi na yau da kullun muna samun yanki daga buƙatar, sannan mu yi. proxy_pass tare da m $ mai masaukin baki, an nuna a ciki $subdomain.semrush.com. Yana iya zama kamar mai ruɗani, amma yana aiki. Kuma yana aiki da kyau. Don kowane yanki waɗanda ke buƙatar dabaru daban-daban, kawai ƙirƙiri tubalan uwar garken ku kuma yi tsari daban. A ƙasa akwai gajerun saitunan nginx don tsabta da nunin wannan makirci.

Tsarin saitin mai zuwa yana aiwatar da duk buƙatun daga China zuwa .semrushchina.cn:

    listen 80;

    server_name ~^(?<subdomain>[w-]+).semrushchina.cn$;

    sub_filter '.semrush.com' '.semrushchina.cn';
    sub_filter_last_modified on;
    sub_filter_once off;
    sub_filter_types *;

    gzip on;
    gzip_proxied any;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;

    location / {
        proxy_pass http://127.0.0.1:8083;
        proxy_set_header Accept-Encoding "";
        proxy_set_header Host $subdomain.semrush.com;
        proxy_set_header X-Accept-Encoding $http_accept_encoding;
    }
}

Wannan saitin wakili ne zuwa Localhost zuwa tashar jiragen ruwa 83, kuma saitin yana jira a can:

    listen 127.0.0.1:8083;

    server_name *.semrush.com;

    location / {
        resolver 8.8.8.8 ipv6=off;
        gunzip on;
        proxy_pass https://$host;
        proxy_set_header Accept-Encoding gzip;
    }
}

Ina maimaitawa, waɗannan gyare-gyare ne da aka yanke.

Kamar haka. Yana iya kama da rikitarwa, amma yana cikin kalmomi. A zahiri, komai ya fi sauƙi fiye da turnips masu tururi :)

Ƙarshen digression

Na ɗan lokaci muna farin ciki saboda ba a tabbatar da labarin faɗuwar IPSEC ba. Amma sai ramukan suka fara fadowa. Sau da yawa a rana don 'yan mintuna kaɗan. Kadan, amma hakan bai dace da mu ba. Tun da an dakatar da ramukan biyu a gefen Ali a kan na'ura mai ba da hanya tsakanin hanyoyin sadarwa guda ɗaya, mun yanke shawarar cewa watakila wannan matsala ce ta yanki kuma muna buƙatar haɓaka yankin madadin.

Suka karba. Tunnels sun fara kasawa a lokuta daban-daban, amma gazawar ta yi mana kyau a matakin sama a nginx. Amma sai ramukan sun fara faɗuwa a kusan lokaci guda 🙂 Kuma 502 da 504 sun sake farawa. Uptime ya fara lalacewa, don haka muka fara aiki akan zaɓi tare da Alibaba CEN (Cloud Enterprise Network).

CEN

CEN - wannan shine haɗin kai na VPC guda biyu daga yankuna daban-daban a cikin Alibaba Cloud, wato, za ku iya haɗa cibiyoyin sadarwa masu zaman kansu na kowane yanki a cikin girgije tare da juna. Kuma mafi mahimmanci: wannan tashar tana da tsattsauran ra'ayi SLA. Yana da matukar kwanciyar hankali duka a cikin sauri da kuma lokacin aiki. Amma ba haka ba ne mai sauƙi:

• yana da matukar wahala a samu idan ba ƴan ƙasar Sin ba ne ko kuma ƙungiyar doka,
• Kuna buƙatar biyan kowane megabit na ƙarfin tashar.

Samun damar haɗi Mainland China и kasashen waje, mun kirkiro CEN tsakanin yankunan Ali guda biyu: cn-shenzhen и us-gabas-1 (mafi kusa da mu-gabas4). In Ali us-gabas-1 ya tayar da wani injin kama-da-wane don a sami ƙarin hop.

Ya kasance kamar haka:

Sakamakon gwajin burauzar yana ƙasa:

yanke shawara
Kyau
Median
Kashi 75 cikin ɗari
Kashi 95 cikin ɗari

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

CEN
99.75
16s
21s
27s

Ayyukan ya ɗan fi IPSEC kyau. Amma ta hanyar IPSEC kuna iya yuwuwar zazzagewa akan saurin 100 Mbit/s, kuma ta hanyar CEN kawai akan saurin 5 Mbit/s da ƙari.

Sauti kamar matasan, dama? Haɗa saurin IPSEC da kwanciyar hankali na CEN.

Wannan shine abin da muka yi, yana ba da izinin zirga-zirga ta hanyar IPSEC da CEN a yayin da aka sami gazawar IPSEC rami. Uptime ya zama mafi girma, amma saurin lodin rukunin yanar gizon yana barin abubuwa da yawa da ake so. Sai na zana duk da’irorin da muka riga muka yi amfani da su kuma muka gwada, na yanke shawarar ƙara ɗan ƙara GCP a wannan kewaye, wato. hula.

hula

hula Shin Global Load Balancer (ko Google Cloud Load Balancer). Yana da amfani mai mahimmanci a gare mu: a cikin mahallin CDN yana da shi kowane IP, wanda ke ba ku damar tafiyar da zirga-zirga zuwa cibiyar bayanai mafi kusa da abokin ciniki, don haka zirga-zirga da sauri ya shiga cikin hanyar sadarwar sauri ta Google kuma ƙasa ta wuce ta hanyar Intanet "na yau da kullun".

Ba tare da tunani sau biyu ba, mun taso HTTP/HTTPS LB Mun shigar da injunan mu tare da tacewa a cikin GCP kuma azaman baya.

Akwai tsare-tsare da yawa:

• Amfani Cloudflare China Network, amma wannan lokacin Asalin ya kamata ya ƙayyade duniya IP GLB.
• Kashe abokan ciniki a cn-shenzhen, kuma daga can wakili na zirga-zirga kai tsaye zuwa hula.
• Tafi kai tsaye daga China zuwa hula.
• Kashe abokan ciniki a cn-shenzhen, daga can wakili zuwa Asiya-gabas1 ta hanyar IPSEC (in us-gabas4 ta hanyar CEN), daga nan zuwa GLB (a hankali, za a yi hoto da bayani a ƙasa)

Mun gwada duk waɗannan zaɓuɓɓukan da ƙarin wasu matasan:

• Cloudflare + GLB

Wannan makirci bai dace da mu ba saboda lokacin aiki da kurakuran DNS. Amma an yi gwajin kafin a gyara kwaro a gefen CF, watakila ya fi kyau a yanzu (duk da haka, wannan baya cire lokacin lokacin HTTP).

• Ali + GLB

Hakanan wannan tsari bai dace da mu ba dangane da lokacin aiki, tunda GLB sau da yawa yana faɗuwa daga sama saboda rashin yuwuwar haɗawa a cikin lokacin karɓuwa ko lokacin da aka yarda, saboda uwar garken da ke cikin China, adireshin GLB yana nan a waje, sabili da haka a bayan bayanan. Tacewar zaɓi na kasar Sin. Sihiri bai faru ba.

• GLB kawai

Wani zaɓi mai kama da na baya, kawai bai yi amfani da sabobin ba a China kanta: zirga-zirgar zirga-zirgar ta tafi kai tsaye zuwa GLB (an canza bayanan DNS). Sakamakon haka, sakamakon bai gamsar da su ba, tun da talakawan Sinawa abokan ciniki da ke amfani da sabis na masu samar da Intanet suna da mummunan yanayi tare da wucewa ta wuta fiye da Ali Cloud.

• Shenzhen -> (CEN/IPSEC) -> Wakili -> GLB

Anan mun yanke shawarar amfani da mafi kyawun duk mafita:

• kwanciyar hankali da garantin SLA daga CEN
• babban gudun daga IPSEC
• Cibiyar sadarwa ta "sauri" ta Google da kowane watsa ta.

Tsarin yana kama da wani abu kamar haka: an ƙare zirga-zirgar mai amfani akan injin kama-da-wane a ciki ch-shen. Nginx upstreams ana saita su a can, wasu daga cikinsu suna nuna sabar IP masu zaman kansu da suke a wancan ƙarshen ramin IPSEC, wasu kuma daga sama suna nuna adiresoshin masu zaman kansu na sabar a wancan gefen CEN. An saita IPSEC zuwa yanki Asiya-gabas1 a GCP (shi ne yanki mafi kusanci da kasar Sin a lokacin da aka samar da mafita. GCP yanzu ma yana da kasancewa a Hong Kong). CEN - zuwa yanki us-gabas1 in Ali Cloud.

Sa'an nan aka nufi zirga-zirga daga bangarorin biyu zuwa IP GLB, wato, zuwa mafi kusa da kasancewar Google, kuma ya bi ta hanyar sadarwarsa zuwa yankin us-gabas4 a cikin GCP, wanda akwai injunan maye gurbinsu (tare da subfilter a nginx).

Wannan ƙayyadaddun bayani, kamar yadda muke tsammani, ya yi amfani da fa'idodin kowane fasaha. Gabaɗaya, zirga-zirga yana tafiya cikin sauri IPSEC, amma idan matsala ta fara, muna sauri kuma na ɗan mintuna kaɗan muna fitar da waɗannan sabobin daga sama kuma mu aika da zirga-zirga ta hanyar CEN kawai har sai ramin ya daidaita.

Ta hanyar aiwatar da mafita na 4 daga jerin da ke sama, mun sami abin da muke so da abin da kasuwancin ke buƙata a gare mu a wannan lokacin a lokacin.

Sakamakon gwajin Browser don sabon bayani idan aka kwatanta da waɗanda suka gabata:

yanke shawara
Kyau
Median
Kashi 75 cikin ɗari
Kashi 95 cikin ɗari

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

CEN
99.75
16s
21s
27s

CEN/IPsec + GLB
99.79
13s
16s
25s

CDN

Komai yana da kyau a cikin mafita da muka aiwatar, amma babu CDN wanda zai iya hanzarta zirga-zirga a matakin yanki da ma birni. A ka'idar, wannan ya kamata ya hanzarta shafin don masu amfani da ƙarshen ta hanyar amfani da hanyoyin sadarwa mai sauri na mai bada CDN. Kuma mun yi tunani akai akai. Kuma yanzu, lokaci ya yi don ƙaddamar da aikin na gaba: bincike da gwada masu samar da CDN a China.

Kuma zan baku labarin wannan a kashi na gaba, na ƙarshe :)

source: www.habr.com