🥇Mmene tidathyola pa Great Firewall of China (part 1)

Hello aliyense!

Nikita akulankhula - injiniya wa kampaniyo SEMrush. Lero ndikuuzani za momwe tinayang'anizana ndi ntchito yowonetsetsa kukhazikika kwa ntchito yathu ya semrush.com ku China, ndi mavuto ati omwe tidakumana nawo pakukhazikitsidwa kwake (kutengera komwe kuli malo athu a data pagombe lakum'mawa kwa United States).

Iyi idzakhala nkhani yaikulu, yogawidwa m'nkhani zingapo. Ndikuuzani momwe zidachitikira kwa ife: kuchokera ku ntchito yosagwira ntchito kuchokera ku China, kupita ku zizindikiro za ntchitoyo pamlingo wa mtundu wake waku America kwa aku America. Ndikulonjeza kuti zikhala zosangalatsa komanso zothandiza. Kotero, tiyeni tizipita.

Mavuto a intaneti aku China

Ngakhale munthu wotalikirapo kwambiri pazachindunji za kayendetsedwe ka maukonde adamvapo Great Firewall yaku China. Wow, zikumveka bwino, chabwino? Koma chomwe chiri komanso momwe chimagwirira ntchito ndi funso lovuta. Mutha kupeza zolemba zambiri pa intaneti zoperekedwa kwa izi, koma kuchokera pamalingaliro aukadaulo, mawonekedwe a firewall iyi samafotokozedwa paliponse. Zomwe, komabe, sizodabwitsa. Ndikuvomereza nthawi yomweyo kuti malinga ndi zotsatira za chaka cha ntchito, sindingathe kunena ndendende momwe zimagwirira ntchito, koma ndikuuzeni za ndemanga zanga ndi mfundo zothandiza. Ndipo tiyamba ndi mphekesera za firewall iyi.

Pali mphekesera zambiri za firewall iyi. Tiyeni tisonkhanitse zazikulu komanso zosangalatsa kwambiri mumndandanda umodzi:

Google, Facebook, Twitter ndi ntchito zina zofananira ndizoletsedwa ndipo sizigwira ntchito ku China.
Magalimoto aliwonse omwe amapita KUNJA kwa China ndi ku China amagawidwa ndikuchepa pogwiritsa ntchito makina ophunzirira (ngati pali magalimoto okayikitsa), zomwe zimachedwetsa kwambiri (magalimoto) podutsa malire.
Mabungwe azamalamulo aku China adzawononga magalimoto onse obisika omwe amadutsa pa firewall yawo.
Ma tunnel a VPN, ngalande za IPSEC sizikhazikika, zimawonongeka ndipo zimatsekedwa nthawi zonse.
Kusavuta kubisa, mawu osavuta omwe amagwiritsidwa ntchito kutsimikizira / kubisa kuchuluka kwa magalimoto, ndiye kuti amadutsa mwachangu pa firewall yaku China.

Nazi zomwe tidapeza za mphekesera izi:

Google, Facebook, Twitter ndi ntchito zina zofananira ndizotsekedwa (KO yanu), koma madera ambiri aukadaulo a Google, mwachitsanzo, samaletsedwa ndikugwira ntchito (gstatic.com yomweyo). Mapeto akutsatira izi: simuyenera kudula mosasamala zonse za Google ndi zinthu zina zomwe zikuwoneka kuti zatsekedwa.
Magalimoto aliwonse odutsa malirewo amawonjezera kuchedwa kwambiri panthawi yake. Yang'anani zotsatira ziwiri. Tsamba limodzi, tsamba limodzi, GET yosavuta kupiringa'om. Kuyeza koyamba kunali kochokera ku China komweko (mzinda wokongola wa Shenzhen). Yachiwiri inayesedwa kuchokera kunja kuchokera ku Hong Kong (ili ndi ulamuliro, ndipo palibe firewall pakati pake ndi dziko lapansi). Mtunda pakati pa mizinda molunjika ndi pafupifupi 30-40 km.

nikita@china-shenzhen:~# curl -o /dev/null -w@curl_time "https://www.semrush.com/info/ebay.com"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  381k    0  381k    0     0  71824      0 --:--:--  0:00:05 --:--:-- 82832
time_namelookup:  0.004500
time_connect:  0.169342
time_appconnect:  0.723189
time_pretransfer:  0.723499
time_redirect:  0.000000
time_starttransfer:  1.532912
----------
time_total:  5.443407
----------
size_download:  390968 Bytes
speed_download:  71824.000B/s

nikita@china-hongkong:~# curl -o /dev/null -w@curl_time "https://www.semrush.com/info/ebay.com"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  319k    0  319k    0     0  2555k      0 --:--:-- --:--:-- --:--:-- 2573k
time_namelookup:  0.029366
time_connect:  0.030742
time_appconnect:  0.047310
time_pretransfer:  0.047388
time_redirect:  0.000000
time_starttransfer:  0.120793
----------
time_total:  0.124871
----------
size_download:  326755 Bytes
speed_download:  2616740.000B/s

Samalani nthawi_connect. Ndipo zambiri, mukuwona zotsatira zake: firewall imawonjezera masekondi ena 4, omwe ndiatali kwambiri.

VPN ndi IPSEC tunnels zimalephera nthawi zambiri. Ndilankhula za izi posachedwa komanso mwatsatanetsatane. Ma seva a VPN omwe amagwiritsidwa ntchito ndi ogwiritsa ntchito amatsekedwa pakapita nthawi (nthawi zambiri pasanathe tsiku limodzi atayamba kugwiritsa ntchito).
Pali malingaliro omwe amalandira kuchokera kwa anthu okhala ku China kuti kubisa kosavuta kwa magalimoto, kumadutsa mofulumira kumalire, chifukwa n'zosavuta kumvetsa kuti palibe choletsedwa pa izo. Ndipo mofananamo, magalimoto "oyera" amalandira bandwidth yowonjezereka ndi liwiro la njira, pamene magalimoto "odetsedwa", omwe palibe chomwe chingamvetsetsedwe, amalandira, mosiyana, ndimeyi yocheperapo. Mwachitsanzo, ndimagwiritsa ntchito curl ifconfig.co kudzera pa HTTPS ndi HTTP protocol.

curl -o /dev/null -w@curl_time "https://ifconfig.co/"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    13  100    13    0     0      2      0  0:00:06  0:00:05  0:00:01     3
time_namelookup:  0.004305
time_connect:  0.397465
time_appconnect:  5.149305
time_pretransfer:  5.149393
time_redirect:  0.000000
time_starttransfer:  5.568847
----------
time_total:  5.568893
----------
size_download:  13 Bytes
speed_download:  2.000B/s

curl -o /dev/null -w@curl_time "http://ifconfig.co/"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    13  100    13    0     0     28      0 --:--:-- --:--:-- --:--:--    28
time_namelookup:  0.004282
time_connect:  0.212457
time_appconnect:  0.000000
time_pretransfer:  0.212484
time_redirect:  0.000000
time_starttransfer:  0.450565
----------
time_total:  0.450620
----------
size_download:  13 Bytes
speed_download:  28.000B/s

Kusiyana kwa masekondi 5 pa nthawi yonse yotsitsa ya 13 byte. Komanso, mukayesa kangapo, mutha kuzindikira kuti GET pa HTTP imamalizidwa nthawi yomweyo nthawi iliyonse, pomwe pa HTTPS tsambalo nthawi zina limayankha mu 3, 5, 10 ngakhale masekondi 17. Nthawi zina zolakwika za SSL zimachitika:

Unknown SSL protocol error in connection to ifconfig.co:443.

Ndiye zomwe tili nazo:

Mavuto opangidwa ndi firewall aku China akufotokozedwa pamwambapa.
Ziphuphu kuzinthu zakunja ndi ma tunnel amkati nthawi ndi nthawi zimasowa.
Kuchedwa pakati pa mfundo ziwiri kumasintha nthawi zonse, ndipo nthawi zambiri zimakhala zosayembekezereka. Mukagwirizanitsa mizinda / madera osiyanasiyana, mukuyembekeza kuti, kutengera malo a madera, kuchedwa kudzakhala kochepa, koma mumapeza zosiyana.
Intaneti ndi njira zoyankhulirana zimakhala zachangu kapena zodekha. Pali kudalira pang'ono pa nthawi ya tsiku ndi tsiku la sabata, koma osati nthawi zonse.
Zopempha za DNS kumayiko akunja kuchokera ku China nthawi zina zimadutsa nthawi yololedwa.

Chithunzi chomwe chikuwonekera ndi "chabwino kwambiri".

Deta ya data, monga ndanenera kale, ili kum'mawa kwa United States, ndipo SEMrush yonse imakhala ndi zinthu zambiri zolumikizidwa, zobwerera kumbuyo, kutsogolo, zolemba, ndi zonsezi mu DC ndi mitambo. Ife, monga gulu la oyang'anira machitidwe, tinapatsidwa ntchito yofulumira kugwira ntchito ku China popanda khama lochepa.

Tinayenera kuyankha funso lofunika kwambiri: kodi ndizotheka kuthana ndi ndalama zochepa ndikuthetsa mavuto onse okhudzana ndi intaneti yaku China ndi firewall pamaneti / mtambo / seva?

Tinayamba ndi kulandira ICP- zilolezo.

Chilolezo cha ICP

Kuti muthe kuchititsa ntchito yanu mkati mwa China (Mainland China) ndikuyesa mayeso, muyenera kupeza chiphaso cha ICP cha domain.

Ngati kuchuluka kwa ogwiritsa ntchito patsamba lanu kutha mkati mwa Mainland China, ndipo ngati dera lanu lilibe laisensi ya ICP, kuchuluka kwa magalimoto anu kudzatsekedwa kumbali ya ISP/hosting. Chosangalatsa ndichakuti, layisensi ya ICP imaphatikizanso wothandizira, akhale Cloudflare kapena Alibaba Cloud. Chifukwa chake, ngati mutalandira chilolezo cha ICP cha Cloudflare ndikusunga tsamba lanu, simungathe "kusamuka" kupita ku Alibaba Cloud. Mufunika kuwonjezera kuchititsa kwina ku chilolezochi.

Titalandira laisensi ya ICP pa domain, tidatha kubwera ndi kukhazikitsa malingaliro ndi mayankho aukadaulo.

Kuyesa mayankho

Koma musanayambe kupanga zosankha zachindunji, tembenuzirani zingwe, kukhathamiritsa momwe tsambalo likugwirira ntchito komanso liwiro lake, muyenera kusankha chida choyesera kuti muwone zomwe tikuchita bwino kapena, m'malo mwake, kukulitsa magwiridwe antchito a tsambalo.

Chida chathu choyesera chinayenera kukwaniritsa zofunikira ziwiri:

iyenera kuyesa mayeso kuchokera ku China,
iyenera kukhala ndi mayeso a msakatuli.

Kotero ife tinapeza Zolemba! Iwo ali ndi chidziwitso chabwino kwambiri cha malo oyesera padziko lonse lapansi. Ku China, mayeso amathanso kuyendetsedwa kuchokera kuzigawo 100500 kudzera pa chida ichi. Aliyense ali ndi opereka angapo osiyanasiyana + kuthekera kochita Backbone-mayeso (chinachake ngati makina enieni mu data center) ndi Lastmile-mayeso (pafupi ndi momwe angagwiritsire ntchito, aka workstation). Mayeso amtundu wotsiriza ndi okwera mtengo.

Titamaliza mgwirizano wapachaka (zocheperapo zomwe sizingatheke), tinayamba kuphunzira chidacho. Kunena zowona, tinali odabwa ndi magwiridwe ake. Mutha kuthamanga:

DNS mayeso,
Mayeso a pa intaneti (mayeso a msakatuli, GET/POST yosavuta, kutsanzira kasitomala wam'manja, ndi zina zambiri),
Macheke a transaction (mwachitsanzo, kulowa),
Mayeso a API,
Ping, traceroute, NTP, etc.

Simungatchule chilichonse. Ndipo chofunikira kwambiri, mayeso aliwonse amatha kusinthidwa bwino powonjezera mitu yambiri ndi magawo ena. Linanena bungwe ndi yaikulu kuchuluka kwa chidziwitso kuti mokwanira amafotokoza mayeso anu. Ngati tilankhula za zinthu zosangalatsa kwambiri kwa ife (mayeso a msakatuli), zotsatira zake zikuphatikizapo:

Lumikizani, Dikirani, Katundu, SSL, DNS nthawi,
TTFB, TTLB, Document yatha, Nthawi yopereka, katundu wa DOM,
Yankho (chinachake chapafupi ndi Time To First Byte), Response Webusaiti (chinachake chapafupi ndi Time To Last Byte),
Maperesenti aliwonse, Avereji, nthawi yapakati
Ndi zina zotero.

Chifukwa chake, ma metric onsewa ndi abwino kuwona kusintha ndikumvetsetsa ngati zinthu zakhala bwino. Tinkayang'ana makamaka Mayankho, Mayankho a Tsamba la Webusaiti, Median, 75 ndi 95 Percentiles.

Funso lofunika kwambiri lomwe linali mlengalenga kuyambira pachiyambi: Kodi mungakhulupirire Catchpoint?? Kodi chida ichi chikuwonetsa kuthamanga kwa tsamba lenileni ku China kuchokera kumizinda yosiyana siyana, kapena ndi mtundu wina chabe wa mayeso opanda kanthu omwe alibe chochita ndi ogwiritsa ntchito enieni?
Ili ndi vuto lalikulu, chifukwa kukhala ku Russia ndizosatheka kudziwa momwe tsamba lochokera ku China limagwirira ntchito. Pochita socks-proxy kudzera mu makina enieni, mapeto ake ndi chakuti malowa amadzaza mkati mwa mphindi zingapo, zomwe ndizosavomerezeka kuyesedwa, kotero njira yokhayo yoyesera pamanja ndi yopiringa ndi yosavuta GET kuchokera ku console yokhala ndi timer. . Izi zimathandiza chifukwa mayesowa akuwonetsa bwino kuthamanga kwa intaneti, ndipo ngati palinso mayeso asakatuli, ndiye kuti ndiabwino kwambiri.

Pambuyo pake ifenso tinapita ku China ndipo tinakhutiritsidwa zimenezo Mutha kukhulupirira Catchpoint; imawonetsa zolondola zenizeni zenizeni.

Cloudflare China Network

Popeza timagwiritsa ntchito bwino Cloudflare pa domain main semrush.com, tidaganiza zoyesa nthawi yomweyo mawonekedwe awo otchedwa China Network. Njirayi imayatsidwa ndi masamba a Enterprise okhawo akafunsidwa mosiyana komanso kuti awonjezere ndalama. Imapezekanso kumasamba omwe ali ndi layisensi yoyenera ya ICP yomwe imatchula Cloudflare ngati opereka. Pambuyo poyambitsa, "Chinese CDN" yochokera ku Cloudflare imapezeka patsambali - magalimoto ochokera kumadera aku China amafika ku PoP (Points of Presence) CF yapafupi, kenako kudzera pamanetiweki kapena ma netiweki a othandizira/mabwenzi amaperekedwa kochokera. .

Chithunzi cha benchi yoyeserayi chikuwonetsedwa pansipa.

Iyi ndi njira yabwino kwa ife. Zikuoneka kuti dera lachiwiri lidzakhalanso la CF, lomwe silikuwonjezera pa chiwerengero cha mayankho omwe amagwiritsidwa ntchito pakampaniyo, komanso sizikusokoneza zomangamanga.

Tidayesa ma browser ndipo izi ndi zomwe zidachitika:

Ma diamondi ofiira ndi kulephera kwa mayeso. Mafayilo omwe ali pansipa ndi zolakwika za DNS (thetsani nthawi). Zolephera zomwe zili pamwamba ndi kutha kwa nthawi.

Kutalika: 86.6
Nthawi: 18s
75 peresenti: 29.3s
95 peresenti: 60s

Wapakati, pambuyo Mumakonda anachotsedwa recaptcha (Ntchito ya Google yoletsedwa ku China) idatsika kuchokera pa masekondi 28 mpaka 18. Koma izi zikadali zotsatira zoyipa, poganizira kuti mayeso omwewo a semrush.com (ochokera ku US) adapereka masekondi osakwana 10 kwa 95% ya ogwiritsa ntchito (kuchokera ku US) patsamba lomwelo (static + dynamic).

Mutha kulowa mu mayeso aliwonse ndikuyang'ana Mapiri ndi zina zambiri zatsatanetsatane. Tinayamba kufufuza zifukwa za zolakwikazo, ndipo ngati nthawi yatha nthawi zonse zimakhala zomveka bwino: intaneti ku China "imayenda ndi kutuluka", chifukwa cha izi kuthamanga kwa kugwirizana ndi kukweza chuma kuchokera kunja ndi kosakhazikika komanso kosagwirizana, ndiye zolakwika za DNS zidatidabwitsa kwambiri. Ife tinazipeza izo Po Cloudflare imapezeka ku China, adilesi yatsambalo imakhazikika ku IP iliyonse, koma ma seva a DNS ndi aku America, ndichifukwa chake zopempha za DNS zimakakamizika kudutsa malire, kotero nthawi zina zimalephera.

Nditafotokozera funso ili ndi CF, zidapezeka kuti Alibe ma seva awo a DNS ku China, ndipo kuti chidzachitika liti sichidziwikabe.

Chifukwa chake, tidaganiza zongoyesa Cloudflare DNS ndikusintha makina ogwiritsira ntchito a Cloudflare patsamba lathu kukhala "DNS yokha" Iyi ndi njira yomwe Cloudflare siigwiritsa ntchito proxy traffic yokha, zomwe zikutanthauza kuti sizipereka chitetezo cha DDoS, CDN ndi zina, ndipo zimagwira ntchito ngati seva ya DNS yokhazikika.

Maimidwe awa akuwonetsedwa m'chithunzi chotsatirachi. Chiwerengerocho chimaganizira zomwe zikubwera kuti ma seva a Cloudflare a DNS ali kumbuyo kwa firewall.

Ku Catchpoint tidayesa mayeso osavuta a GET (osati kuyesa osatsegula), omwe adawonetsa zolephera zambiri. Zinayambitsidwa ndi zolakwika zomwezo za DNS.

Tinayamba kukonza zolakwika izi pogwiritsa ntchito kukumba ndipo anapeza kuti pa pempho loyamba adiresi yatsimikiziridwa molondola, ndipo pa pempho mobwerezabwereza timalandira nthawi iliyonse SERVFAIL и sinapezeke. N’chifukwa chiyani zimenezi zikuchitika mwadzidzidzi?

root@iZwz97n2wgbp61qucbfrjsZ:~# host semrushchina.cn
semrushchina.cn has address 220.170.186.192
Host semrushchina.cn not found: 2(SERVFAIL)
root@iZwz97n2wgbp61qucbfrjsZ:~# host semrushchina.cn
semrushchina.cn has address 220.170.186.192
Host semrushchina.cn not found: 2(SERVFAIL)
root@iZwz97n2wgbp61qucbfrjsZ:~# host semrushchina.cn
semrushchina.cn has address 220.170.186.192
Host semrushchina.cn not found: 2(SERVFAIL)
root@iZwz97n2wgbp61qucbfrjsZ:~# host semrushchina.cn
semrushchina.cn has address 220.170.186.192
Host semrushchina.cn not found: 2(SERVFAIL)

Palibe zolakwika zotere mukafunsa ma seva a Cloudflare NS mwachindunji:

root@iZwz97n2wgbp61qucbfrjsZ:~# for i in `seq 1 2`; do host semrushchina.cn ray.ns.cloudflare.com.; done
Using domain server:
Name: ray.ns.cloudflare.com.
Address: 173.245.59.138#53
Aliases: 

semrushchina.cn has address 220.170.186.192
semrushchina.cn has address 220.170.186.192
Using domain server:
Name: ray.ns.cloudflare.com.
Address: 173.245.59.138#53
Aliases: 

semrushchina.cn has address 220.170.186.192
semrushchina.cn has address 220.170.186.192

Izi zikutanthauza kuti vuto liri kumbali ya "local" DNS seva kapena seva ya wothandizira.
Kufufuza kwina kunavumbula zimenezo SERVFAIL timapanga chisankho AAAA- zolemba.

Zinapezeka kuti popempha kuchokera ku Cloudflare AAAA-mbiri yomwe kulibe mu domain, Cloudflare adayankha А-kulowa komwe kuli kolakwika komanso kusagwirizana ndi RFC. Chifukwa chiyani woweruza wamba (xxx) Sindinakonde, ndipo anayankha SERVFAIL. Khalidweli likuwoneka bwino mu chipika chomwe chili pansipa:

root@iZwz97n2wgbp61qucbfrjsZ:~# dig -t AAAA semrushchina.cn @x.x.x.x

; <<>> DiG 9.10.3-P4-Ubuntu <<>> -t AAAA semrushchina.cn @x.x.x.x
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 55467
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;semrushchina.cn.               IN      AAAA

;; Query time: 334 msec
;; SERVER: x.x.x.x#53(x.x.x.x)
;; WHEN: Tue Aug 14 23:38:50 CST 2018
;; MSG SIZE  rcvd: 44

root@iZwz97n2wgbp61qucbfrjsZ:~# dig -t AAAA semrushchina.cn @dana.ns.cloudflare.com.

; <<>> DiG 9.10.3-P4-Ubuntu <<>> -t AAAA semrushchina.cn @dana.ns.cloudflare.com.
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63944
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;semrushchina.cn.               IN      AAAA

;; ANSWER SECTION:
semrushchina.cn.        300     IN      A       220.170.186.192

;; Query time: 185 msec
;; SERVER: 173.245.58.105#53(173.245.58.105)
;; WHEN: Tue Aug 14 23:43:03 CST 2018
;; MSG SIZE  rcvd: 60

Tinapereka lipoti la cholakwika ku Cloudflare, ndipo adakonza pakapita nthawi. Zinakhala zosangalatsa: pakadali pano kulibe chithandizo cha IPv6 ku China, kotero Cloudflare sinathe kupereka adilesi yake ya IPv6 pamenepo poyankha pempho. AAAA- zolemba. Pamapeto pake, zonse zidathetsedwa mwanjira yomwe Cloudflare idayamba kuyankha ku China NODATA ku zopempha zotere.

Chifukwa chake, zolakwika za DNS mu mayeso a Catchpoint zidachepa kwambiri, koma osati kwathunthu. Nthawi ikadali pano:

Ndipo tinayamba kufunafuna njira ina.

Mu gawo lotsatira ndikuwuzani momwe tidayesera mtambo waku China Alibaba Cloud, momwe, mothandizidwa ndi "matsenga" ang'onoang'ono a Nginx, tinatha kupanga mwamsanga mayankho a PoC (Proof of Concept), momwe tidapangira mayankho a Multi-Cloud, omwe pamapeto pake adathandizira kwambiri kufulumizitsa ntchito yautumiki. kuchokera ku China.

Dzimvetserani!