Pehea mākou i haki ai i ka pā ahi nui o Kina (Mahele 2)

Alohaʻoe!

Aia hou ʻo Nikita me ʻoe, he ʻenekini ʻōnaehana mai ka hui SEMrush. A me kēia ʻatikala ke hoʻomau nei au i ka moʻolelo e pili ana i ke ʻano o kā mākou hana ʻana i kahi hoʻonā workaround Pāpā Pākē no kā mākou lawelawe semrush.com.

В hapa mua 'Ōlelo wau:

• he aha nā pilikia e kū mai ma hope o ka hoʻoholo ʻana "Pono mākou e hana i kā mākou lawelawe ma Kina"
• He aha nā pilikia o ka Pūnaewele Kina?
• no ke aha ʻoe e pono ai i kahi laikini ICP?
• pehea a no ke aha mākou i hoʻoholo ai e hoʻāʻo i kā mākou mau hoʻāʻo me Catchpoint
• he aha ka hopena o kā mākou hopena mua e pili ana i Cloudflare China Network
• Pehea mākou i ʻike ai i kahi pōpoki ma Cloudflare DNS

ʻO kēia ʻāpana ka mea hoihoi loa, i koʻu manaʻo, no ka mea e kālele ana i nā hoʻokō ʻenehana kikoʻī o ka staging. A e hoʻomaka mākou, a i ʻole e hoʻomau, me Alibaba Kapua.

Alibaba Kapua

Alibaba Kapua He mea hoʻolako kapua nui, nona nā lawelawe a pau e ʻae iā ia e kapa ʻoiaʻiʻo iā ia iho he mea hāʻawi kapua. He mea maikaʻi e loaʻa iā lākou ka manawa e hoʻopaʻa inoa no nā mea hoʻohana haole, a ʻo ka hapa nui o ka pūnaewele i unuhi ʻia i ka ʻōlelo Pelekania (no Kina he mea waiwai kēia). Ma kēia ao, hiki iā ʻoe ke hana me nā wahi he nui o ka honua, ka ʻāina nui o Kina, a me Oceanic Asia (Hong Kong, Taiwan, etc.).

IPSEC

Ua hoʻomaka mākou me ka palapala honua. No ka loaʻa ʻana o kā mākou kahua hoʻāʻo ma Google Cloud, pono mākou e "hoʻohui" iā Alibaba Cloud me GCP, no laila ua wehe mākou i kahi papa inoa o nā wahi e noho ai ʻo Google. I kēlā manawa ʻaʻole i loaʻa iā lākou kā lākou kikowaena data ma Hong Kong.
ʻO ka ʻāina kokoke loa ʻĀsia-hikina1 (Taiwan). Ua lilo ʻo Ali ʻo ia ka ʻāina kokoke loa o ka ʻāina nui o Kina iā Taiwan cn-shenzhen (Shenzhen).

Me ke kōkuaʻana o ʻeleʻele wehewehe a hoʻonui i ka ʻōnaehana holoʻokoʻa ma GCP a me Ali. He 100 Mbit/s tunnel ma waena o nā ao i piʻi kokoke koke. Ma ka ʻaoʻao o Shenzhen a me Taiwan, ua hoʻāla ʻia nā mīkini virtual proxying. Ma Shenzhen, hoʻopau ʻia nā mea hoʻohana, hoʻopili ʻia ma kahi tunnel i Taiwan, a mai laila e hele pololei i ka IP waho o kā mākou lawelawe ma us-hikina (USA East Coast). Ping ma waena o nā mīkini virtual ma o ka tunnel 24ms, aole ino loa.

I ka manawa like, ua kau mākou i kahi wahi hoʻāʻo i loko Alibaba Cloud DNS. Ma hope o ka hāʻawi ʻana i ka ʻāpana iā NS Ali, ua emi ka manawa hoʻonā mai 470 ms a 50 ms. Ma mua o kēia, aia nō ka ʻāpana ma Cloudlfare.

Kaulike me ka tunnel i ʻĀsia-hikina1 ua hoʻāla hou aʻe i kahi tunnel mai Shenzhen pololei i mākou-hikina4. Ma laila lākou i hana ai i nā mīkini virtual proxy hou aʻe a hoʻomaka e hoʻāʻo i nā hoʻonā ʻelua, e hoʻokele ana i nā kaʻa hoʻāʻo me ka hoʻohana ʻana i nā Kuki a i ʻole DNS. Ua wehewehe ʻia ka papa hoʻāʻo ma ke kiʻi ma lalo nei:

Ua ʻike ʻia ka latency no nā tunnels penei:
Ali cn-shenzhen <—> GCP asia-hikina1 — 24ms
Ali cn-shenzhen <—> GCP us-hikina4 — 200ms

Ua hōʻike ʻia nā hoʻokolohua ʻo Catchpoint i ka hoʻomaikaʻi maikaʻi ʻana.

E hoʻohālikelike i nā hopena hōʻike no nā haʻina ʻelua:

olelo hooholo
ʻO Uptime
Median
75 pakeneka
95 pakeneka

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

ʻO kēia ka ʻikepili mai kahi hopena e hoʻohana ana i kahi tunnel IPSEC ma o ʻĀsia-hikina1. Ma o mākou-east4 ua ʻoi aku ka maikaʻi o nā hopena, a ua nui nā hewa, no laila ʻaʻole wau e hāʻawi i nā hopena.

Ma muli o nā hopena o kēia hoʻāʻo ʻana o nā tunnels ʻelua, ua hoʻopau ʻia kekahi ma kahi kokoke loa i Kina, a ʻo kekahi ma ka hopena hope loa, ua ʻike ʻia he mea nui e "puka" mai lalo o ka pā ahi Kina me ka wikiwiki. hiki, a laila hoʻohana i nā pūnaewele wikiwiki (nā mea hoʻolako CDN, nā mea hāʻawi kapua, a me nā mea ʻē aʻe). ʻAʻohe pono e hoʻāʻo e komo i loko o ka pā ahi a hiki i kāu huakaʻi i hoʻokahi hāʻule. ʻAʻole kēia ke ala wikiwiki loa.

Ma ka laulā, ʻaʻole maikaʻi nā hopena, akā naʻe, loaʻa i ka semrush.com ka median o 8.8s, a me 75 Percentile 9.4s (ma ka hoʻāʻo like).
A ma mua o ka neʻe ʻana, makemake wau e hana i kahi digression lyrical pōkole.

Hoʻopau lyrical

Ma hope o ke komo ʻana o ka mea hoʻohana i ka pūnaewele www.semrushchina.cn, e hoʻoholo ana ma o nā kikowaena DNS "wikiwiki", hele ka noi HTTP ma kā mākou hopena wikiwiki. Hoʻihoʻi ʻia ka pane ma ke ala like, akā ua kuhikuhi ʻia ke kikowaena ma nā palapala JS āpau, nā ʻaoʻao HTML a me nā mea ʻē aʻe o ka ʻaoʻao pūnaewele. semrush.com no nā kumuwaiwai hou aʻe e pono ke hoʻouka ʻia ke hoʻopuka ʻia ka ʻaoʻao. ʻO ia hoʻi, hoʻoholo ka mea kūʻai aku i ka "main" A-record www.semrushchina.cn a komo i loko o ka pūnana wikiwiki, loaʻa koke kahi pane - he palapala HTML e ʻōlelo ana:

• hoʻoiho i kēlā a me nā js mai sso.semrush.com,
• E kiʻi i nā faila CSS mai cdn.semrush.com,
• a lawe pū i kekahi mau kiʻi mai dab.semrush.com
• a no laila, ma luna o.

Hoʻomaka ka polokalamu kele pūnaewele e hele i ka pūnaewele "waho" no kēia mau kumuwaiwai, i kēlā me kēia manawa e hele i kahi pā ahi e ʻai i ka manawa pane.

Akā ʻo ka hōʻike mua e hōʻike i nā hopena inā ʻaʻohe kumu waiwai ma ka ʻaoʻao semrush.comwale no semrushchina.cn, a ʻo *.semrushchina.cn hoʻoholo i ka helu wahi o ka mīkini virtual ma Shenzhen i mea e komo ai i loko o ka tunnel.

Ma kēia ala wale nō, ma ka hoʻokuʻu ʻana i nā kaʻa a pau i ka nui ma o kāu hoʻonā no ka holo wikiwiki ʻana i ka pā ahi Kina, hiki iā ʻoe ke loaʻa nā wikiwiki kūpono a me nā hōʻailona loaʻa pūnaewele, a me nā hopena pololei o nā hoʻokolohua hoʻonā.
Hana mākou i kēia me ka ʻole o ka hoʻoponopono code ma ka ʻaoʻao huahana o ka hui.

Kānāwai hoʻi

Ua hānau ʻia ka hopena ma hope koke iho o ka puka ʻana o kēia pilikia. Ua pono mākou PoC (Hōʻoiaʻiʻo o ka Manaʻo) e hana maikaʻi loa kā mākou mau hoʻonā ʻana i ke ahi. No ka hana ʻana i kēia, pono ʻoe e hoʻopili i nā kaʻa pūnaewele āpau i kēia hoʻonā e like me ka hiki. A noi mākou kānana sub i ka nginx.

Kānāwai hoʻi he modula maʻalahi ma ka nginx e hiki ai iā ʻoe ke hoʻololi i kahi laina i ke kino pane i kahi laina ʻē aʻe. No laila ua hoʻololi mākou i nā hanana āpau semrush.com maluna o semrushchina.cn i na pane a pau.

A ... ʻaʻole ia i hana no ka mea ua loaʻa iā mākou nā ʻike i hoʻopaʻa ʻia mai nā ʻaoʻao hope, no laila ʻaʻole i loaʻa i ka subfilter ka laina i koi ʻia. Pono wau e hoʻohui i kahi kikowaena kūloko ʻē aʻe i ka nginx, nāna i hoʻopau i ka pane a hāʻawi iā ia i kahi kikowaena kūloko e hiki mai ana, ua luhi nei i ka hoʻololi ʻana i ke kaula, kaomi ʻana, a hoʻouna ʻia i ka server proxy hou ma ke kaulahao.

ʻO ka hopena, ma hea kahi e loaʻa ai ka mea kūʻai aku .semrush.com, ua loaa iaia .semrushchina.cn a hele me ka hoʻolohe i kā mākou hoʻoholo.

Eia naʻe, ʻaʻole lawa ka hoʻololi wale ʻana i ka domain i hoʻokahi ala, no ka mea ke manaʻo mau nei nā backends i ka semrush.com i nā noi aʻe mai ka mea kūʻai aku. No laila, ma ka kikowaena hoʻokahi kahi i hana ʻia ai ke ala hoʻokahi, me ka hoʻohana ʻana i kahi ʻōlelo maʻamau maʻalahi loaʻa mākou i ka subdomain mai ka noi, a laila hana mākou. proxy_pass me ka loli $host, hōʻike ʻia ma $subdomain.semrush.com. Me he mea la e huikau ana, aka e hana ana. A hana maikaʻi. No nā kāʻei kapu e koi ana i nā loina like ʻole, e hana wale i kāu mau poloka kikowaena a hana i kahi hoʻonohonoho ʻokoʻa. Ma lalo iho nei ka pōkole nginx configs no ka maopopo a me ka hōʻike ʻana o kēia hoʻolālā.

Hoʻopili kēia config i nā noi āpau mai Kina a .semrushchina.cn:

    listen 80;

    server_name ~^(?<subdomain>[w-]+).semrushchina.cn$;

    sub_filter '.semrush.com' '.semrushchina.cn';
    sub_filter_last_modified on;
    sub_filter_once off;
    sub_filter_types *;

    gzip on;
    gzip_proxied any;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;

    location / {
        proxy_pass http://127.0.0.1:8083;
        proxy_set_header Accept-Encoding "";
        proxy_set_header Host $subdomain.semrush.com;
        proxy_set_header X-Accept-Encoding $http_accept_encoding;
    }
}

Hoʻopili kēia config i localhost i ke awa 83, a ke kali nei ka config ma laila:

    listen 127.0.0.1:8083;

    server_name *.semrush.com;

    location / {
        resolver 8.8.8.8 ipv6=off;
        gunzip on;
        proxy_pass https://$host;
        proxy_set_header Accept-Encoding gzip;
    }
}

Ke haʻi hou aku nei au, ʻo kēia nā configs cropped.

E like me kēlā. Hiki ke nānā aku i ka paʻakikī, akā ma nā huaʻōlelo. ʻO ka ʻoiaʻiʻo, ʻoi aku ka maʻalahi o nā mea āpau ma mua o nā turnips steamed :)

Ka pau ana o ka digression

No kekahi manawa ua hauʻoli mākou no ka mea ʻaʻole i hōʻoia ʻia ka moʻolelo e pili ana i ka hāʻule ʻana o nā tunnels IPSEC. Akā, ua hoʻomaka nā tunnels e hāʻule. He mau manawa i ka lā no kekahi mau minuke. He liʻiliʻi, akā ʻaʻole kūpono kēlā iā mākou. Ma muli o ka pau ʻana o nā tunnel ʻelua ma ka ʻaoʻao Ali ma ka ʻaoʻao hoʻokahi, ua hoʻoholo mākou he pilikia ʻāpana paha kēia a pono mākou e hoʻāla i ka ʻāina hoʻihoʻi.

Lawe lākou. Ua hoʻomaka nā tunnels e hāʻule i nā manawa like ʻole, akā ua hana maikaʻi ka failover iā mākou ma ka pae kiʻekiʻe ma nginx. Akā, ua hoʻomaka nā tunnels e hāʻule i ka manawa like 🙂 A hoʻomaka hou ʻo 502 a me 504. Ua hoʻomaka ka wā o ka wā e hoʻopau ai, no laila ua hoʻomaka mākou e hana i ke koho me Alibaba CEN (Cloud Enterprise Network).

CEN

CEN - ʻo kēia ka pilina o nā VPC ʻelua mai nā ʻāpana like ʻole i loko o Alibaba Cloud, ʻo ia hoʻi, hiki iā ʻoe ke hoʻohui i nā pūnaewele pilikino o nā wahi i loko o ke ao me kekahi. A ʻo ka mea nui loa: he koʻikoʻi ko kēia kahawai SLA. Paʻa loa ia i ka wikiwiki a me ka uptime. Akā ʻaʻole maʻalahi loa ia:

• paʻakikī loa ke loaʻa inā ʻaʻole ʻoe he kamaʻāina Kina a i ʻole he hui kānāwai,
• Pono ʻoe e uku no kēlā me kēia megabit o ka bandwidth channel.

Loaʻa ka manawa e hoʻopili ai Mainland China и huakaʻi i ke kai, ua hana mākou i CEN ma waena o ʻelua mau ʻāpana Ali: cn-shenzhen и us-hikina-1 (kahi kokoke loa iā mākou-hikina4). Ma Ali us-hikina-1 ua hoʻāla hou i kekahi mīkini virtual i loaʻa hoʻokahi mea hou aʻe hop.

Ua like me keia:

Aia ma lalo nā hualoaʻa hoʻāʻo polokalamu kele:

olelo hooholo
ʻO Uptime
Median
75 pakeneka
95 pakeneka

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

CEN
99.75
16s
21s
27s

ʻOi aku ka maikaʻi o ka hana ma mua o IPSEC. Akā ma o IPSEC hiki iā ʻoe ke hoʻoiho i ka wikiwiki o 100 Mbit/s, a ma o CEN wale nō ma ka wikiwiki o 5 Mbit/s a ʻoi aku.

Ke kani nei me he hybrid, ʻeā? Hoʻohui i ka wikiwiki IPSEC a me ka paʻa CEN.

ʻO kēia ka mea a mākou i hana ai, e ʻae ana i nā kaʻa ma waena o IPSEC a me CEN i ka wā o ka hāʻule ʻole o ka tunnel IPSEC. Ua ʻoi aku ka kiʻekiʻe o ka uptime, akā ke waiho nei ka wikiwiki o ka hoʻouka ʻana i ka pūnaewele i nā mea e makemake ʻia. A laila huki au i nā kaʻapuni āpau a mākou i hoʻohana ai a hoʻāʻo ai, a ua hoʻoholo wau e hoʻāʻo e hoʻohui i kahi GCP hou aʻe i kēia kaapuni, ʻo ia hoʻi. pāpale.

pāpale

pāpale Ua ʻO ka mea kaulike hoʻouka honua (a i ʻole Google Cloud Load Balancer). He waiwai koʻikoʻi ko mākou: ma ka pōʻaiapili o kahi CDN iā ia anycast IP, ka mea e hiki ai iā ʻoe ke hoʻokele kaʻa i ke kikowaena ʻikepili kokoke loa i ka mea kūʻai aku, no laila e komo koke ke kaʻa i loko o ka pūnaewele wikiwiki o Google a liʻiliʻi ka hele ʻana ma ka Pūnaewele "maʻamau".

Me ka noʻonoʻo ʻole, hāpai mākou HTTP/HTTPS LB Hoʻokomo mākou i kā mākou mau mīkini virtual me ka subfilter ma GCP a ma ke ʻano he hope.

Aia kekahi mau papahana:

• Hoʻohana Cloudflare Kina, akā i kēia manawa pono e wehewehe ʻo Origin i ka honua IP GLB.
• Hoʻopau i nā mea kūʻai ma cn-shenzhen, a mai laila e hoʻololi pololei i ke kaʻa i pāpale.
• Hele pololei mai Kina a pāpale.
• Hoʻopau i nā mea kūʻai ma cn-shenzhen, mai laila mea koho i ʻĀsia-hikina1 ma o IPSEC (i mākou-hikina4 ma o CEN), mai laila aku e hele i GLB (me ka mālie, aia ke kiʻi a me ka wehewehe ʻana ma lalo)

Ua hoʻāʻo mākou i kēia mau koho āpau a me nā mea hybrid hou aʻe:

• Cloudflare + GLB

ʻAʻole kūpono kēia papahana iā mākou ma muli o ka uptime a me nā hewa DNS. Akā ua hoʻokō ʻia ka hoʻāʻo ma mua o ka hoʻopaʻa ʻia ʻana o ka bug ma ka ʻaoʻao CF, ʻoi aku paha ka maikaʻi i kēia manawa (akā naʻe, ʻaʻole kēia e kāpae i nā manawa manawa HTTP).

• Ali + GLB

ʻAʻole kūpono kēia hoʻolālā iā mākou ma ke ʻano o ka uptime, no ka mea, hāʻule pinepine ʻo GLB i waho o ke kahawai ma muli o ka hiki ʻole ke hoʻohui i kahi manawa kūpono a i ʻole ka manawa, no ka mea, no kahi kikowaena i loko o Kina, aia ka helu GLB ma waho, a no laila ma hope o ka. pā ahi Pākē. ʻAʻole i hiki ke kilokilo.

• GLB wale nō

ʻO kahi koho e like me ka mea ma mua, ʻaʻole wale ia i hoʻohana i nā kikowaena ma Kina ponoʻī: hele pololei ke kaʻa i GLB (ua hoʻololi ʻia nā moʻolelo DNS). No laila, ʻaʻole ʻoluʻolu nā hopena, no ka mea, ʻoi aku ka maikaʻi o nā mea kūʻai aku Kina maʻamau e hoʻohana ana i nā lawelawe o nā mea hoʻolako pūnaewele maʻamau me ka hala ʻana i ka pā ahi ma mua o Ali Cloud.

• Shenzhen -> (CEN/IPSEC) -> Proxy -> GLB

Maʻaneʻi ua hoʻoholo mākou e hoʻohana i ka maikaʻi o nā hoʻonā āpau:

• kūpaʻa a me ka SLA i hoʻopaʻa ʻia mai CEN
• kiʻekiʻe wikiwiki mai IPSEC
• ʻO ka pūnaewele "wikiwiki" a Google a me kāna mau leka uila.

Ua like ke ʻano o ka hoʻolālā e like me kēia: ua hoʻopau ʻia nā mea hoʻohana ma kahi mīkini virtual ch-shenzhen. Hoʻonohonoho ʻia ʻo Nginx upstreams ma laila, ʻo kekahi o ia mau mea e kuhikuhi ana i nā kikowaena IP pilikino e kū ana ma kēlā ʻaoʻao o ka tunnel IPSEC, a kuhikuhi kekahi mau hiʻohiʻona i nā wahi pilikino o nā kikowaena ma kēlā ʻaoʻao o ka CEN. Hoʻonohonoho ʻia ʻo IPSEC i ka ʻāina ʻĀsia-hikina1 ma GCP (ʻo ia ka ʻāina kokoke loa iā Kina i ka manawa i hana ʻia ai ka hoʻonā. Aia pū kekahi GCP i Hong Kong i kēia manawa). CEN - i ka ʻāina mākou-hikina1 ma Ali Cloud.

A laila ua kuhikuhi ʻia ke kaʻa mai nā ʻaoʻao ʻelua anycast IP GLB, ʻo ia hoʻi, i kahi kokoke loa o Google, a hele ma kāna mau pūnaewele i ka ʻāina mākou-hikina4 ma GCP, kahi i hoʻololi ʻia ai nā mīkini virtual (me ka subfilter ma nginx).

ʻO kēia hoʻonā hybrid, e like me kā mākou i manaʻo ai, ua lawe pono i nā pono o kēlā me kēia ʻenehana. Ma keʻano laulā, hele wikiwiki nā kaʻa i ka IPSEC, akā inā hoʻomaka nā pilikia, wikiwiki mākou a no kekahi mau minuke kiki i kēia mau kikowaena mai ka uka a hoʻouna i nā kaʻa wale ma CEN a hiki i ka hoʻopaʻa ʻana o ka tunnel.

Ma ka hoʻokō ʻana i ka hopena 4 mai ka papa inoa ma luna, ua hoʻokō mākou i ka mea a mākou i makemake ai a me nā mea e pono ai ka ʻoihana iā mākou i kēlā manawa.

Nā hualoaʻa hoʻāʻo Browser no ka hoʻonā hou i hoʻohālikelike ʻia me nā mea ma mua:

olelo hooholo
ʻO Uptime
Median
75 pakeneka
95 pakeneka

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

CEN
99.75
16s
21s
27s

CEN/IPsec + GLB
99.79
13s
16s
25s

CDN

Maikaʻi nā mea a pau i ka hopena a mākou i hoʻokō ai, akā ʻaʻohe CDN e hiki ke hoʻolōʻihi i ke kaʻa ma ka pae a me ke kūlanakauhale. Ma ke kumumanaʻo, pono kēia e wikiwiki i ka pūnaewele no nā mea hoʻohana hope ma o ka hoʻohana ʻana i nā ala kamaʻilio wikiwiki o ka mea hoʻolako CDN. A noʻonoʻo mākou i nā manawa a pau. A i kēia manawa, ua hiki mai ka manawa no ka ʻike hou ʻana o ka papahana: ʻimi a hoʻāʻo i nā mea hoʻolako CDN ma Kina.

A e haʻi aku wau iā ʻoe e pili ana i kēia ma ka ʻaoʻao hope loa :)

Source: www.habr.com