Momwe tidadutsira pa Great Firewall of China (Gawo 2)

Привет!

Nikita ali ndi inu kachiwiri, injiniya wamakampani kuchokera kukampani SEMrush. Ndipo ndi nkhaniyi ndikupitiriza nkhani ya momwe tinapezera njira yothetsera vutoli Chinese Firewall pa ntchito yathu semrush.com.

В gawo lapitalo Ndinati:

• mavuto omwe amabwera chigamulocho chikapangidwa "Tiyenera kupanga ntchito yathu ku China"
• Kodi intaneti yaku China ili ndi mavuto otani?
• chifukwa chiyani mukufuna chilolezo cha ICP?
• bwanji ndi chifukwa chiyani tinaganiza zoyesa mateti athu ndi Catchpoint
• chotsatira cha yankho lathu loyamba kutengera Cloudflare China Network
• Momwe tidapezera cholakwika mu Cloudflare DNS

Gawoli ndilosangalatsa kwambiri, m'malingaliro mwanga, chifukwa limayang'ana kwambiri zaukadaulo waukadaulo. Ndipo tiyamba, kapena kani tipitirize, ndi Alibaba Cloud.

Alibaba Cloud

Alibaba Cloud ndiwopereka mtambo waukulu, womwe uli ndi mautumiki onse omwe amalola kuti adzitchule moona mtima kuti ndi wothandizira mtambo. Ndibwino kuti ali ndi mwayi wolembetsa kwa ogwiritsa ntchito akunja, komanso kuti malo ambiri amamasuliridwa m'Chingelezi (kwa China izi ndizopambana). Mumtambowu, mutha kugwira ntchito ndi madera ambiri padziko lapansi, China, komanso Oceanic Asia (Hong Kong, Taiwan, etc.).

Mtengo wa IPSEC

Tinayamba ndi geography. Popeza malo athu oyesera anali pa Google Cloud, tinkafunika "kulumikiza" Alibaba Cloud ndi GCP, kotero tinatsegula mndandanda wa malo omwe Google ilipo. Panthawiyo analibe malo awo a data ku Hong Kong.
Dera lapafupi kwambiri linali Asia-kummawa1 (Taiwan). Ali adakhala dera loyandikira kwambiri ku China ku Taiwan cn-shenzhen (Shenzhen).

Ndi chithandizo cha terraform adalongosola ndikukweza zida zonse za GCP ndi Ali. Msewu wa 100 Mbit/s pakati pa mitambo unakwera pafupifupi nthawi yomweyo. Kumbali ya Shenzhen ndi Taiwan, makina opanga ma proxying adakwezedwa. Ku Shenzhen, kuchuluka kwa anthu ogwiritsa ntchito kumathetsedwa, kuyendetsedwa kudzera mumsewu kupita ku Taiwan, ndipo kuchokera pamenepo kumapita ku IP yakunja ya ntchito yathu ife-kummawa (USA East Coast). Ping pakati pa makina enieni kudzera mumsewu 24ms, zomwe sizili zoipa kwambiri.

Nthawi yomweyo, tinayika malo oyesera Alibaba Cloud DNS. Pambuyo popereka zone ku NS Ali, nthawi yosankha idatsika kuchokera pa 470 ms mpaka 50 ms. Izi zisanachitike, zone inalinso pa Cloudlfare.

Kufanana ndi ngalandeyo Asia-kummawa1 anakweza mumphanga wina ku Shenzhen mwachindunji kuti ife-kummawa4. Kumeneko adapanga makina opangira ma proxy ambiri ndikuyamba kuyesa mayankho onse awiri, kuyendetsa magalimoto oyesa pogwiritsa ntchito Cookies kapena DNS. Benchi yoyeserera ikufotokozedwa mwadongosolo mu chithunzi chotsatirachi:

Kuchedwa kwa tunnel kudakhala motere:
Ali cn-shenzhen <—> GCP asia-east1 — 24ms
Ali cn-shenzhen <—> GCP us-east4 — 200ms

Mayeso a msakatuli wa Catchpoint adawonetsa kusintha kwabwino kwambiri.

Fananizani zotsatira za mayeso a mayankho awiri:

chisankho
Uptime
Zamkatikati
75 peresenti
95 peresenti

Cloudflare
86.6
18
30
60

IPsec
99.79
18
21
30

Izi ndizomwe zimachokera ku yankho lomwe limagwiritsa ntchito njira ya IPSEC kudzera Asia-kummawa1. Kupyolera mwa ife-kummawa4 zotsatira zinali zoipitsitsa, ndipo panali zolakwika zambiri, kotero sindidzapereka zotsatira.

Kutengera zotsatira za mayesowa a ngalande ziwiri, imodzi yomwe imathetsedwa kudera lapafupi kwambiri ndi China, ndipo inayo pomaliza, zidawonekeratu kuti ndikofunikira "kutuluka" pansi pa firewall yaku China mwachangu monga zotheka, ndiyeno gwiritsani ntchito maukonde othamanga (opereka ma CDN , opereka mtambo, ndi zina). Palibe chifukwa choyesera kudutsa pa firewall ndikufika komwe mukupita mwachangu. Iyi si njira yachangu.

Nthawi zambiri, zotsatira zake sizoyipa, komabe, semrush.com ili ndi pakati pa 8.8s, ndi 75 Percentile 9.4s (pa mayeso omwewo).
Ndipo ndisanapitirire, ndikufuna kuti ndichepetseko pang'ono.

Kukoka kwachikale

Wogwiritsa ntchito akalowa patsamba www.semrushchina.cn, yomwe imathetsa kudzera pa ma seva a DNS "ofulumira" achi China, pempho la HTTP limadutsa yankho lathu lachangu. Yankho limabwezeredwa m'njira yomweyo, koma dera limafotokozedwa m'malemba onse a JS, masamba a HTML ndi zinthu zina zatsamba lawebusayiti. semrush.com pazowonjezera zomwe ziyenera kukwezedwa tsambalo likaperekedwa. Ndiko kuti, kasitomala amathetsa "main" A-rekodi www.semrushchina.cn ndikulowa mumsewu wofulumira, amalandira yankho mwachangu - tsamba la HTML lomwe limati:

• tsitsani izi ndi zina kuchokera ku sso.semrush.com,
• Pezani mafayilo a CSS kuchokera ku cdn.semrush.com,
• komanso kutenga zithunzi kuchokera dab.semrush.com
• ndi zina zotero.

Msakatuli akuyamba kupita ku intaneti "yakunja" pazinthu izi, nthawi iliyonse akudutsa pa firewall yomwe imadya nthawi yoyankha.

Koma mayeso apitawa amasonyeza zotsatira pamene palibe zothandizira pa tsamba semrush.comzokha chima.cn, ndi *.semrushchina.cn atsimikiza ku adiresi ya makina pafupifupi Shenzhen kuti ndiye kulowa mumphangayo.

Pokhapokha, pokankhira magalimoto onse omwe angatheke mpaka kufika pamlingo waukulu kudzera mu yankho lanu kuti mudutse mwachangu firewall yaku China, mutha kupeza liwiro lovomerezeka ndi zizindikiritso za kupezeka kwa webusayiti, komanso zotsatira zowona za mayeso a mayankho.
Tidachita izi popanda kusintha kamodzi pagulu lamagulu.

Zosefera

Njira yothetsera vutoli idabadwa pafupifupi vuto ili litawonekera. Tinkafunika PoC (Umboni wa Concept) kuti mayankho athu olowera pachitetezo chamoto amagwira ntchito bwino. Kuti muchite izi, muyenera kukulunga zonse zomwe zili patsamba ili munjira iyi momwe mungathere. Ndipo tinafunsira sefa mu nginx.

Zosefera ndi gawo losavuta mu nginx lomwe limakupatsani mwayi wosintha mzere umodzi muzoyankha kupita ku mzere wina. Kenako tinasintha zochitika zonse semrush.com pa chima.cn mu mayankho onse.

Ndipo ... sizinagwire ntchito chifukwa tidalandira zopsinjidwa kuchokera ku backends, kotero subfilter sinapeze mzere wofunikira. Ndinayenera kuwonjezera seva ina yapafupi ku nginx, yomwe inasokoneza yankho ndikuyipereka ku seva yapafupi yapafupi, yomwe inali yotanganidwa kale m'malo mwa chingwe, kukakamiza, ndikuitumiza ku seva yotsatila yotsatira mu unyolo.

Zotsatira zake, kodi kasitomala adzalandira kuti .semrush.com, analandira .semrushchina.cn ndipo momvera anayenda mu chisankho chathu.

Komabe, sikokwanira kungosintha malowo mwanjira imodzi, chifukwa ma backends amayembekezerabe semrush.com pazopempha zotsatila kuchokera kwa kasitomala. Chifukwa chake, pa seva yomweyi pomwe kusinthira kwa njira imodzi kumachitika, pogwiritsa ntchito mawu osavuta nthawi zonse timapeza subdomain kuchokera ku pempho, kenako timachita. proxy_pass ndi kusintha $host, zowonetsedwa mu $subdomain.semrush.com. Zingawoneke zosokoneza, koma zimagwira ntchito. Ndipo zimagwira ntchito bwino. Kwa madambwe omwe amafunikira malingaliro osiyanasiyana, ingopangani midadada yanu ya seva ndikupanga kasinthidwe kosiyana. Pansipa pali zofupikitsidwa za nginx kuti zimveke bwino komanso ziwonetsedwe za dongosololi.

Kukonzekera kotsatiraku kumayendetsa zopempha zonse kuchokera ku China kupita .semrushchina.cn:

    listen 80;

    server_name ~^(?<subdomain>[w-]+).semrushchina.cn$;

    sub_filter '.semrush.com' '.semrushchina.cn';
    sub_filter_last_modified on;
    sub_filter_once off;
    sub_filter_types *;

    gzip on;
    gzip_proxied any;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;

    location / {
        proxy_pass http://127.0.0.1:8083;
        proxy_set_header Accept-Encoding "";
        proxy_set_header Host $subdomain.semrush.com;
        proxy_set_header X-Accept-Encoding $http_accept_encoding;
    }
}

Izi config proxies ku localhost ku doko 83, ndipo zosintha zotsatirazi zikudikirira pamenepo:

    listen 127.0.0.1:8083;

    server_name *.semrush.com;

    location / {
        resolver 8.8.8.8 ipv6=off;
        gunzip on;
        proxy_pass https://$host;
        proxy_set_header Accept-Encoding gzip;
    }
}

Ndikubwereza, awa ndi masinthidwe odulidwa.

Monga choncho. Zingawoneke zovuta, koma ndi mawu. M'malo mwake, chilichonse ndi chosavuta kuposa ma turnips otenthedwa :)

Kutha kwa kutsika

Kwa kanthawi tinali okondwa chifukwa nthano ya kugwa kwa tunnel za IPSEC sinatsimikizidwe. Koma kenako ngalandezo zinayamba kugwa. Kangapo patsiku kwa mphindi zingapo. Pang'ono, koma izo sizinali bwino ife. Popeza kuti ngalandezo zinathetsedwa kumbali ya Ali pa rauta yomweyo, tinaganiza kuti mwina ili ndi vuto lachigawo ndipo tifunika kukweza dera losunga zobwezeretsera.

Iwo anainyamula iyo. Misewuyo idayamba kulephera nthawi zosiyanasiyana, koma kulephera kunatithandizira pamlingo wakumtunda mu nginx. Koma ngalandezo zinayamba kugwa pafupifupi nthawi yomweyo 🙂 Ndipo 502 ndi 504 zinayambanso. Uptime inayamba kuwonongeka, kotero tinayamba kugwira ntchito ndi chisankho. Alibaba CEN (Cloud Enterprise Network).

CEN

CEN - uku ndikulumikizana kwa ma VPC awiri ochokera kumadera osiyanasiyana mkati mwa Alibaba Cloud, ndiko kuti, mutha kulumikiza maukonde achinsinsi a zigawo zilizonse mkati mwamtambo wina ndi mnzake. Ndipo chofunika kwambiri: njira iyi ili ndi malamulo okhwima SLA. Ndiwokhazikika kwambiri pa liwiro komanso nthawi yayitali. Koma sizophweka choncho:

• ndizovuta kwambiri kupeza ngati simuli nzika zaku China kapena bungwe lovomerezeka,
• Muyenera kulipira megabit iliyonse ya bandwidth ya chaneli.

Kukhala ndi mwayi wolumikizana China China и kunja, tidapanga CEN pakati pa zigawo ziwiri za Ali: cn-shenzhen и ife-kummawa-1 (malo oyandikira kwambiri kwa ife-kummawa4). Mu Ali ife-kummawa-1 adakweza makina ena enieni kuti pakhale winanso siimakupiza.

Zinakhala chonchi:

Zotsatira za msakatuli zili pansipa:

chisankho
Uptime
Zamkatikati
75 peresenti
95 peresenti

Cloudflare
86.6
18
30
60

IPsec
99.79
18
21
30

CEN
99.75
16
21
27

Kuchita kwake kuli bwinoko pang'ono kuposa IPSEC. Koma kudzera mu IPSEC mutha kutsitsa pa liwiro la 100 Mbit/s, ndipo kudzera mu CEN kokha pa liwiro la 5 Mbit/s ndi kupitilira apo.

Zikumveka ngati haibridi, sichoncho? Phatikizani liwiro la IPSEC ndi kukhazikika kwa CEN.

Izi ndi zomwe tidachita, kulola magalimoto kudutsa onse a IPSEC ndi CEN pakagwa kulephera kwa ngalande ya IPSEC. Uptime yakhala yokwera kwambiri, koma kuthamanga kwa tsamba kumasiyabe zofunika. Kenako ndinajambula madera onse omwe tidawagwiritsa ntchito kale ndikuyesa, ndipo ndinaganiza zoyesa kuwonjezera GCP kuderali, ndiko kuti. Gulani.

Gulani

Gulani Ndi Global Load Balancer (kapena Google Cloud Load Balancer). Ili ndi phindu lofunikira kwa ife: mu nkhani ya CDN yomwe ili nayo IP iliyonse, zomwe zimakulolani kuti muyendetse magalimoto kumalo osungira deta pafupi ndi kasitomala, kotero kuti magalimoto amalowa mofulumira mu intaneti yofulumira ya Google ndipo zochepa zimadutsa pa intaneti "yokhazikika".

Popanda kuganiza kawiri, tinakweza HTTP/HTTPS LB Tidayika makina athu enieni okhala ndi zosefera mu GCP komanso ngati kumbuyo.

Pali zifukwa zingapo:

• Gwiritsani ntchito Cloudflare China Network, koma nthawi ino Origin iyenera kufotokoza zapadziko lonse lapansi IP GLB.
• Kuthetsa makasitomala pa cn-shenzhen, ndipo kuchokera pamenepo tidzayimilireni traffic molunjika ku Gulani.
• Pitani molunjika kuchokera ku China kupita Gulani.
• Kuthetsa makasitomala pa cn-shenzhen, kuchokera pamenepo proxy kupita Asia-kummawa1 kudzera IPSEC (in ife-kummawa4 kudzera pa CEN), kuchokera pamenepo pitani ku GLB (modekha, padzakhala chithunzi ndi kufotokozera pansipa)

Tidayesa njira zonsezi ndi zina zingapo zosakanizidwa:

• Cloudflare + GLB

Chiwembuchi sichinatigwirizane ndi ife chifukwa cha uptime ndi zolakwika za DNS. Koma kuyesedwa kunachitika chisanachitike cholakwikacho kumbali ya CF, mwina ndi bwino tsopano (komabe, izi sizikupatula nthawi ya HTTP).

• Ali + GLB

Chiwembuchi sichinatigwirizanenso ndi nthawi yowonjezereka, chifukwa GLB nthawi zambiri inagwa kuchokera kumtunda chifukwa chosatheka kulumikiza nthawi yovomerezeka kapena nthawi yopuma, chifukwa kwa seva mkati mwa China, adiresi ya GLB imakhalabe kunja, choncho kumbuyo kwa seva. Firewall yaku China. Zamatsenga sizinachitike.

• GLB yokha

Njira yofanana ndi yapitayi, yokhayo sinagwiritse ntchito ma seva ku China palokha: magalimoto adapita molunjika ku GLB (zolemba za DNS zidasinthidwa). Chifukwa chake, zotsatira zake sizinali zokhutiritsa, popeza makasitomala wamba aku China omwe amagwiritsa ntchito ntchito za opereka intaneti wamba amakhala ndi vuto lalikulu podutsa firewall kuposa Ali Cloud.

• Shenzhen -> (CEN/IPSEC) -> Proxy -> GLB

Apa tinaganiza zogwiritsa ntchito njira zabwino koposa zonse:

• kukhazikika ndi SLA yotsimikizika kuchokera ku CEN
• kuthamanga kwambiri kuchokera ku IPSEC
• Netiweki "yachangu" ya Google ndi kuyimba kwake kulikonse.

Chiwembucho chikuwoneka motere: kuchuluka kwa ogwiritsa ntchito kumathetsedwa pamakina enieni ch-shenzhen. Mitsinje ya Nginx imakhazikitsidwa pamenepo, ena amalozera ku ma seva achinsinsi a IP omwe ali kumapeto kwa ngalande ya IPSEC, ndipo madera ena okwera amaloza maadiresi achinsinsi a ma seva kumbali ina ya CEN. IPSEC yosinthidwa kukhala dera Asia-kummawa1 ku GCP (linali dera loyandikira kwambiri ku China panthawi yomwe yankho linapangidwa. GCP tsopano ilinso ndi kupezeka ku Hong Kong). CEN - kupita kudera ife-kummawa1 mu Ali Cloud.

Kenako magalimoto ochokera mbali zonse ziwiri adalunjikitsidwa IP GLB iliyonse, ndiye kuti, mpaka pafupi ndi Google, ndipo adadutsa pamanetiweki kupita kuderali ife-kummawa4 mu GCP, momwe munali makina osinthira (okhala ndi subfilter mu nginx).

Njira yosakanizidwa iyi, monga momwe timayembekezera, idatengera mwayi paukadaulo uliwonse. Kawirikawiri, magalimoto amadutsa mofulumira IPSEC, koma ngati mavuto ayamba, ife mwamsanga ndi kwa mphindi zingapo timakankhira ma sevawa kuchokera kumtunda ndikutumiza magalimoto kupyolera mu CEN mpaka msewu utakhazikika.

Pokhazikitsa yankho la 4 kuchokera pamndandanda womwe uli pamwambapa, tapeza zomwe tinkafuna komanso zomwe bizinesi idafuna kwa ife panthawiyo.

Zotsatira za msakatuli za yankho latsopano poyerekeza ndi zam'mbuyo:

chisankho
Uptime
Zamkatikati
75 peresenti
95 peresenti

Cloudflare
86.6
18
30
60

IPsec
99.79
18
21
30

CEN
99.75
16
21
27

CEN/IPsec + GLB
99.79
13
16
25

CDN

Chilichonse ndichabwino mu yankho lomwe tidakhazikitsa, koma palibe CDN yomwe imatha kufulumizitsa kuchuluka kwa anthu m'chigawo komanso ngakhale mzinda. Mwachidziwitso, izi ziyenera kufulumizitsa malowa kwa ogwiritsa ntchito mapeto pogwiritsa ntchito njira zoyankhulirana zofulumira za wothandizira CDN. Ndipo tinkaganiza za izo nthawi zonse. Ndipo tsopano, nthawi yafika yobwerezanso pulojekitiyi: kufufuza ndi kuyesa opereka CDN ku China.

Ndipo ndikuuzani za izi mu gawo lotsatira, lomaliza :)

Source: www.habr.com