🥇Otu anyị si mebie nnukwu ọkụ ọkụ China (akụkụ 2)

Ndewo!

Nikita nọnyeere gị ọzọ, onye injinia sistemụ sitere na ụlọ ọrụ ahụ SEMrush. Na isiokwu a, m na-aga n'ihu na akụkọ banyere otú anyị si bịa na ngwọta ngwọta Firewall ndị China maka ọrụ anyị semrush.com.

В akụkụ gara aga M kwuru:

nsogbu ndị na-ebilite mgbe emechara mkpebi ahụ "Anyị kwesịrị ime ka ọrụ anyị rụọ ọrụ na China"
Kedu nsogbu ịntanetị ndị China nwere?
kedu ihe mere i ji chọọ ikikere ICP?
otu na ihe kpatara anyị ji kpebie iji Catchpoint nwalee akwa ule anyị
Kedu ihe si na ngwọta mbụ anyị dabere na Cloudflare China Network
Otu anyị siri chọta ahụhụ na Cloudflare DNS

Nke a na akụkụ bụ ihe kasị akpali, n'uche nke m, n'ihi na ọ na-elekwasị anya kpọmkwem teknuzu mmejuputa iwu nke staging. Na anyị ga-amalite, ma ọ bụ kama ịga n'ihu, na Alibaba Igwe ojii.

Alibaba Igwe ojii

Alibaba Igwe ojii bụ nnukwu onye na-eweta igwe ojii, nke nwere ọrụ niile na-enye ya ohere ịkpọ onwe ya onye na-eweta igwe ojii. Ọ dị mma na ha nwere ohere ịdebanye aha maka ndị ọrụ mba ọzọ, nakwa na a na-asụgharị ọtụtụ saịtị n'asụsụ Bekee (maka China nke a bụ okomoko). Na igwe ojii a, ị nwere ike ịrụ ọrụ na ọtụtụ mpaghara ụwa, China China, yana Oceanic Asia (Hong Kong, Taiwan, wdg).

IPsec

Anyị malitere na ọdịdị ala. Ebe ọ bụ na saịtị nnwale anyị dị na Google Cloud, anyị kwesịrị ijikọ Alibaba Cloud na GCP, yabụ anyị mepere ndepụta ebe Google nọ na ya. N'oge ahụ, ha enwebeghị ebe data nke ha na Hong Kong.
Mpaghara kacha nso wee bụrụ Asia-ọwụwa anyanwụ1 (Taiwan). Ali tụgharịrị bụrụ mpaghara kacha nso nke ala China na Taiwan cn-shenzhen (Shenzhen).

Site n'enyemaka nke terraform kọwara ma bulie akụrụngwa niile na GCP na Ali. Ọwara 100 Mbit/s dị n'etiti igwe ojii gbagoro ihe fọrọ nke nta ka ọ bụrụ ozugbo. N'akụkụ Shenzhen na Taiwan, a zụlitere igwe mebere ndị na-ahụ maka ọrụ. Na Shenzhen, a na-akwụsị okporo ụzọ onye ọrụ, na-esi n'ọdụ ụgbọ mmiri gaa Taiwan, ma site n'ebe ahụ ọ na-aga ozugbo na IP mpụga nke ọrụ anyị. anyị-ọwụwa anyanwụ (US East Coast). Ping n'etiti igwe mebere site na ọwara 24ms, nke na-adịghị njọ.

N'otu oge ahụ, anyị debere ebe ule na Alibaba Cloud DNS. Mgbe inyefe mpaghara ahụ na NS Ali, oge mkpebi belatara site na 470 ms ruo 50 MS. Tupu nke a, mpaghara ahụ dịkwa na Cloudlfare.

Yiri na ọwara ka Asia-ọwụwa anyanwụ1 welitere ọwara ọzọ si Shenzhen ozugbo na us-ọwụwa anyanwụ4. N'ebe ahụ, ha mepụtara igwe mebere proxy ọzọ wee malite ịnwale ụzọ abụọ ahụ, na-emegharị okporo ụzọ ule site na iji kuki ma ọ bụ DNS. A kọwara bench ule ahụ n'usoro n'usoro n'ụdị a:

Latency maka tunnels wee bụrụ nke a:
Ali cn-shenzhen <—> GCP asia-east1 — 24ms
Ali cn-shenzhen <—> GCP us-east4 — 200ms

Nnwale ihe nchọgharị Catchpoint kọrọ ezigbo nkwalite.

Tulee nsonaazụ ule maka azịza abụọ:

mkpebi
Kwa oge
Median
Pasent 75
Pasent 95

Igwe ojii
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

Nke a bụ data sitere na ngwọta nke na-eji ọwara IPSEC site na Asia-ọwụwa anyanwụ1. Site na anyị-east4 nsonaazụ dị njọ, ma enwere ọtụtụ njehie, yabụ agaghị m enye nsonaazụ ya.

Dabere na nsonaazụ nke ule a nke tunnels abụọ, otu n'ime ha kwụsịrị na mpaghara kacha nso na China, na nke ọzọ na njedebe ikpeazụ, ọ bịara doo anya na ọ dị mkpa "ịpụta" n'okpuru ọkụ ọkụ China ngwa ngwa. enwere ike, wee jiri netwọk ngwa ngwa (ndị na-enye CDN, ndị na-eweta igwe ojii, wdg). Ọ dịghị mkpa ịgbalị isi na firewall wee ruo ebe ị na-aga n'otu ntabi anya. Nke a abụghị ụzọ kacha ọsọ.

N'ozuzu, nsonaazụ adịghị njọ, Otú ọ dị, semrush.com nwere median nke 8.8s, na 75 Percentile 9.4s (na otu ule).
Ma tupu m aga n'ihu, ọ ga-amasị m ịme obere egwu egwu egwu.

Igwu nzu

Mgbe onye ọrụ batara na saịtị ahụ www.semrushchina.cn, nke na-edozi site na sava DNS nke China "ngwa ngwa", arịrịọ HTTP na-aga site na ngwọta ngwa ngwa anyị. A na-eweghachi nzaghachi n'otu ụzọ ahụ, mana akọwapụtara ngalaba ahụ na edemede JS niile, ibe HTML na ihe ndị ọzọ nke ibe weebụ. semrush.com maka akụrụngwa agbakwunyere nke a ga-ebunye mgbe arụpụtara ibe ahụ. Ya bụ, onye ahịa na-edozi "isi" A-ndekọ www.semrushchina.cn wee banye n'ọwara ọsọ ọsọ, na-enweta nzaghachi ngwa ngwa - ibe HTML nke na-ekwu:

budata ụdị js dị otú ahụ na sso.semrush.com,
Nweta faịlụ CSS na cdn.semrush.com,
ma werekwa foto ụfọdụ na dab.semrush.com
na na.

Ihe nchọgharị ahụ na-amalite ịga na Ịntanetị "mpụga" maka ihe ndị a, oge ọ bụla na-agafe na firewall na-eri oge nzaghachi.

Mana ule gara aga na-egosi nsonaazụ mgbe enweghị akụrụngwa na ibe semrush.comnaanị semrushchina.cn, na * .semrushchina.cn kpebie na adreesị nke mebere igwe na Shenzhen iji wee banye ọwara.

Naanị n'ụzọ dị otú a, site n'ịkwanye okporo ụzọ niile ga-ekwe omume ruo n'ókè site na ngwọta gị maka ịgafe ngwa ngwa ọkụ ọkụ China, ị nwere ike nweta ọsọ ọsọ na-anabata na ebe nrụọrụ weebụ na-egosi, yana nsonaazụ n'eziokwu nke ule ngwọta.
Anyị mere nke a na-enweghị otu koodu ndezi n'akụkụ ngwaahịa otu.

Subfilter

A mụrụ ngwọta ya ihe fọrọ nke nta ka ọ bụrụ ozugbo nsogbu a pụtachara. Anyị chọrọ PoC (Ngosipụta nke echiche) na ngwọta ntinye ọkụ ọkụ anyị na-arụ ọrụ nke ọma. Iji mee nke a, ịkwesịrị ịkwanye okporo ụzọ saịtị niile na ngwọta a dị ka o kwere mee. Anyị tinyekwara akwụkwọ subfilter na nginx.

Subfilter bụ modul dị mfe na nginx nke na-enye gị ohere ịgbanwe otu ahịrị na ahụ nzaghachi gaa na ahịrị ọzọ. Ya mere, anyị gbanwere ihe niile merenụ semrush.com on semrushchina.cn na azịza niile.

Ma ... ọ naghị arụ ọrụ n'ihi na anyị natara ọdịnaya abịakọrọ site na azụ azụ, ya mere subfilter ahụghị ahịrị achọrọ. Ekwesịrị m itinye ihe nkesa mpaghara ọzọ na nginx, nke mebiri nzaghachi ma nyefee ya na sava mpaghara na-esote, bụ nke na-arụsi ọrụ ike iji dochie eriri ahụ, na-atụgharị ya, na-eziga ya na ihe nkesa proxy ọzọ na agbụ.

N'ihi ya, ebe onye ahịa ga-enweta .semrush.com, ọ natara .semrushchina.cn ma jiri nrube-isi jegharịa na mkpebi anyị.

Otú ọ dị, ezughị ezu ịgbanwe ngalaba ahụ n'otu ụzọ, n'ihi na azụ azụ ka na-atụ anya semrush.com na arịrịọ ndị ahịa na-esote. N'ihi ya, na otu ihe nkesa ebe a na-eme mgbanwe otu ụzọ, na-eji okwu dị mfe mgbe niile anyị na-enweta subdomain site na arịrịọ ahụ, mgbe ahụ, anyị na-eme ya. proxy_pass na mgbanwe $onye ọbịa, gosipụtara na $subdomain.semrush.com. O nwere ike iyi ihe mgbagwoju anya, mana ọ na-arụ ọrụ. Ọ na-arụkwa ọrụ nke ọma. Maka ngalaba nke ọ bụla chọrọ mgbagha dị iche iche, naanị mepụta ngọngọ nkesa nke gị wee mee nhazi dị iche. N'okpuru bụ nhazi nginx dị mkpụmkpụ maka idoanya na ngosipụta nke atụmatụ a.

Nhazi nhazi ndị a na-eme arịrịọ niile sitere na China gaa .semrushchina.cn:

    listen 80;

    server_name ~^(?<subdomain>[w-]+).semrushchina.cn$;

    sub_filter '.semrush.com' '.semrushchina.cn';
    sub_filter_last_modified on;
    sub_filter_once off;
    sub_filter_types *;

    gzip on;
    gzip_proxied any;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;

    location / {
        proxy_pass http://127.0.0.1:8083;
        proxy_set_header Accept-Encoding "";
        proxy_set_header Host $subdomain.semrush.com;
        proxy_set_header X-Accept-Encoding $http_accept_encoding;
    }
}

Nke a config proxies localhost ruo ọdụ ụgbọ mmiri 83, na nhazi ndị a na-echere ebe ahụ:

    listen 127.0.0.1:8083;

    server_name *.semrush.com;

    location / {
        resolver 8.8.8.8 ipv6=off;
        gunzip on;
        proxy_pass https://$host;
        proxy_set_header Accept-Encoding gzip;
    }
}

M na-ekwughachi, ndị a bụ configs gbuturu.

Dị ka nke ahụ. Ọ nwere ike ịdị mgbagwoju anya, mana ọ bụ n'okwu. N'ezie, ihe niile dị mfe karịa steamed turnips :)

Ọgwụgwụ nke digression

Ruo oge ụfọdụ, anyị nwere obi ụtọ n'ihi na ekwenyeghi akụkọ ifo banyere ọdịda IPSEC tunnels. Ma mgbe ahụ, ọwara mmiri malitere ịda. Ọtụtụ ugboro n'ụbọchị maka nkeji ole na ole. Obere, mana nke ahụ adabaghị anyị. Ebe ọ bụ na a kwụsịrị ụzọ abụọ ahụ n'akụkụ Ali n'otu rawụta ahụ, anyị kpebiri na ikekwe nke a bụ nsogbu mpaghara na anyị kwesịrị ibuli mpaghara ndabere.

Ha bulitere ya. Ọwara ndị ahụ malitere ịda n'oge dị iche iche, mana ọdịda ahụ rụrụ ọrụ nke ọma maka anyị na ọkwa elu na nginx. Ma mgbe ahụ, tunnels malitere ịda n'ihe dị ka otu oge 🙂 Na 502 na 504 malitere ọzọ. Uptime malitere ịka njọ, ya mere, anyị malitere ịrụ ọrụ na nhọrọ na. Alibaba CEN Cloud Enterprise Network.

OA

OA - nke a bụ njikọ nke VPC abụọ sitere na mpaghara dị iche iche n'ime Alibaba Cloud, ya bụ, ị nwere ike jikọọ netwọk nzuzo nke mpaghara ọ bụla n'ime igwe ojii. Na kacha mkpa: a ọwa nwere pụtara siri ike SLA. Ọ kwụsiri ike ma n'ọsọ na n'oge. Ma ọ dịghị mgbe ọ dị mfe:

Ọ na-esiri gị ike inweta ma ọ bụrụ na ị bụghị nwa amaala China ma ọ bụ ụlọ ọrụ iwu kwadoro,
Ịkwesịrị ịkwụ ụgwọ maka megabit ọ bụla nke ikike ọwa.

Inwe ohere ijikọ Mainland China и Overseas, anyị mepụtara CEN n'etiti mpaghara Ali abụọ: cn-shenzhen и us-ọwụwa anyanwụ-1 (ebe kacha nso anyị-ọwụwa anyanwụ4). Na Ali us-ọwụwa anyanwụ-1 welitere igwe mebere ọzọ ka enwere otu ọzọ hop.

Ọ tụgharịrị dị ka nke a:

Nsonaazụ nyocha ihe nchọgharị dị n'okpuru:

mkpebi
Kwa oge
Median
Pasent 75
Pasent 95

Igwe ojii
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

OA
99.75
16s
21s
27s

Arụmọrụ a dịtụ mma karịa IPSEC. Mana site na IPSEC ị nwere ike ibudata na ọsọ 100 Mbit / s, yana site na CEN naanị na ọsọ nke 5 Mbit / s na ndị ọzọ.

Ọ dị ka ngwakọ, nri? Jikọta ọsọ IPSEC na nkwụsi ike CEN.

Nke a bụ ihe anyị mere, na-enye ohere ka okporo ụzọ site na IPSEC na CEN ma ọ bụrụ na ọdịda nke ọwara IPSEC dara. Uptime abụrụla nke ukwuu, mana ọsọ saịtị saịtị ka na-ahapụ ọtụtụ ihe achọrọ. M wee dọkpụrụ sekit niile anyị jibu ma nwalee, wee kpebie ịgbalị itinyekwu GCP na sekit a, ya bụ. NJ.

NJ

NJ Ndi Global Load Balancer (ma ọ bụ Google Cloud Load Balancer). Ọ nwere uru dị mkpa maka anyị: na ọnọdụ nke CDN o nwere IP ọ bụla, nke na-enye gị ohere ịmegharị okporo ụzọ gaa na ebe data kacha nso onye ahịa ahụ, nke mere na okporo ụzọ na-abanye ngwa ngwa na netwọk ngwa ngwa Google na obere na-agafe na Ịntanetị "mgbe niile".

N'echeghị echiche ugboro abụọ, anyị welitere HTTP/HTTPS LB Anyị rụnyere igwe mebere anyị nwere subfilter na GCP yana dịka azụ azụ.

E nwere ọtụtụ atụmatụ:

Jiri Cloudflare China Network, ma oge a Mmalite kwesịrị ịkọwapụta zuru ụwa ọnụ IP GLB.
Kwụsị ndị ahịa na cn-shenzhen, ma si ebe ahụ proxy okporo ụzọ gaa ozugbo NJ.
Si China gaa ozugbo NJ.
Kwụsị ndị ahịa na cn-shenzhen, si ebe ahụ proxy gaa Asia-ọwụwa anyanwụ1 site na IPSEC (na us-ọwụwa anyanwụ4 site na CEN), si ebe ahụ gaa GLB (jiri nwayọọ, a ga-enwe foto na nkọwa n'okpuru)

Anyị nwalere nhọrọ ndị a niile yana ọtụtụ ngwakọ ndị ọzọ:

Cloudflare + GLB

Atụmatụ a adabaghị anyị n'ihi njehie oge na DNS. Mana emere ule ahụ tupu edozi ahụhụ ahụ n'akụkụ CF, ikekwe ọ ka mma ugbu a (agbanyeghị, nke a anaghị ewepu oge HTTP).

Ali + GLB

Atụmatụ a adabaghịkwa anyị n'ihe gbasara oge, ebe ọ bụ na GLB na-adakarị n'elu elu n'ihi enweghị ike ijikọ na oge ma ọ bụ oge nkwụsị, n'ihi na maka ihe nkesa n'ime China, adreesị GLB na-anọgide n'èzí, ya mere n'azụ. Ọkụ ọkụ ndị China. Ime anwansi emeghị.

GLB naanị

Nhọrọ dị ka nke gara aga, naanị na ọ naghị eji sava na China n'onwe ya: okporo ụzọ gara GLB ozugbo (a gbanwere ndekọ DNS). N'ihi ya, nsonaazụ ya adịghị afọ ojuju, ebe ọ bụ na ndị ahịa China nkịtị na-eji ọrụ nke ndị na-eweta Ịntanetị na-enwe ọnọdụ dị njọ karịa ịgafe firewall karịa Ali Cloud.

Shenzhen -> (CEN/IPSEC) -> Proxy -> GLB

N'ebe a, anyị kpebiri iji ngwọta kachasị mma:

kwụsie ike na SLA na-ekwe nkwa site na CEN
nnukwu ọsọ si IPSEC
Netwọk "ngwa ngwa" nke Google na ihe nkedo ya.

Atụmatụ a dị ka nke a: Akwụsịla okporo ụzọ onye ọrụ na igwe mebere ch-shenzhen. A na-ahazi nginx elu ebe ahụ, ụfọdụ n'ime ha na-arụtụ aka na sava IP nkeonwe dị na nsọtụ nke ọzọ nke ọwara IPSEC, na ụfọdụ elu na-arụtụ aka na adreesị nzuzo nke sava n'akụkụ nke ọzọ nke CEN. Ahaziri IPSEC ka ọ bụrụ mpaghara Asia-ọwụwa anyanwụ1 na GCP (bụ mpaghara kacha nso na China n'oge e mepụtara ngwọta ahụ. GCP ugbu a nwekwara ọnụnọ na Hong Kong). CEN - na mpaghara us-ọwụwa anyanwụ1 na Ali Cloud.

Mgbe ahụ, a na-eduzi okporo ụzọ si na nsọtụ abụọ ahụ gaa Onye ọ bụla nke IP GLB, ya bụ, n'ebe kacha nso nke ọnụnọ nke Google, wee site na netwọk ya na mpaghara us-ọwụwa anyanwụ4 na GCP, ebe enwere igwe mebere ngbanwe (ya na subfilter na nginx).

Ngwọta ngwakọ a, dị ka anyị tụrụ anya ya, were uru nke teknụzụ ọ bụla. N'ozuzu, okporo ụzọ na-aga ngwa ngwa IPSEC, ma ọ bụrụ na nsogbu amalite, anyị ngwa ngwa na nkeji ole na ole na-achụpụ sava ndị a site na elu ma ziga okporo ụzọ naanị site na CEN ruo mgbe ọwara ahụ kwụsiri ike.

Site na itinye ihe ngwọta nke 4 site na ndepụta dị n'elu, anyị nwetara ihe anyị chọrọ na ihe azụmahịa chọrọ n'aka anyị n'oge ahụ.

Nsonaazụ nyocha ihe nchọgharị maka ngwọta ọhụrụ a atụnyere ndị gara aga:

mkpebi
Kwa oge
Median
Pasent 75
Pasent 95

Igwe ojii
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

OA
99.75
16s
21s
27s

CEN/IPsec + GLB
99.79
13s
16s
25s

CDN

Ihe niile dị mma na ngwọta anyị mejuputa, ma ọ dịghị CDN nwere ike mee ka okporo ụzọ dị ngwa na mpaghara na ọbụna obodo. Na tiori, nke a kwesịrị ime ka saịtị ahụ dị ngwa maka ndị ọrụ njedebe site na iji ọwa nkwukọrịta ngwa ngwa nke onye na-eweta CDN. Anyị na-eche banyere ya mgbe niile. Ma ugbu a, oge eruola maka ntinye ọzọ nke ọrụ ahụ: ịchọ na ịnwale ndị na-enye CDN na China.

M ga-agwa gị maka nke a n'akụkụ ikpeazụ :)

isi: www.habr.com

Otu anyị si gbasaa na nnukwu ọkụ ọkụ nke China (Nkebi nke 2)

Alibaba Igwe ojii

IPsec

Igwu nzu

Subfilter

Ọgwụgwụ nke digression

OA

NJ

CDN

Tinye a comment Отменить ответ