Saqhawuka njani iFirewall enkulu yaseTshayina (Icandelo 2)

Sawubona!

U-Nikita ukunye nawe kwakhona, injineli yenkqubo evela kwinkampani I-SEMrush. Kwaye ngeli nqaku ndiqhubeka nebali malunga nendlela esize ngayo nesisombululo sokusebenza I-Firewall yaseTshayina kwinkonzo yethu semrush.com.

В inxalenye yangaphambili Ndithe:

• zeziphi iingxaki ezivela emva kokuba isigqibo senziwe "Sifuna ukwenza inkonzo yethu isebenze eChina"
• Ziziphi iingxaki enazo i-Intanethi yaseTshayina?
• kutheni ufuna ilayisenisi ye-ICP?
• njani kwaye kutheni sigqibe kwelokuba sivavanye iibhedi zethu zovavanyo ngeCatchpoint
• saba yintoni umphumo wesisombululo sethu sokuqala esekwe kwi-Cloudflare China Network
• Sifumene njani ibug kwi-Cloudflare DNS

Le nxalenye yeyona nto inomdla kakhulu, ngokombono wam, kuba igxile ekuphunyezweni okuthe ngqo kwezobugcisa. Kwaye siza kuqala, okanye endaweni yoko siqhubeke, ngayo Alibaba Cloud.

Alibaba Cloud

Alibaba Cloud ngumboneleli wefu ngokufanelekileyo, onazo zonke iinkonzo ezivumela ukuba zibize ngokunyanisekileyo ukuba ngumnikezeli wefu. Kulungile ukuba banethuba lokubhalisa kubasebenzisi bangaphandle, kwaye ininzi yesayithi iguqulelwe kwisiNgesi (i-China le yinto yokunethezeka). Kweli lifu, unokusebenza kunye nemimandla emininzi yehlabathi, ilizwe laseTshayina, kunye ne-Oceanic Asia (Hong Kong, Taiwan, njl.).

IPSEC

Saqala ngejografi. Kuba indawo yethu yovavanyo ibikwiLifu likaGoogle, bekufuneka "sidibanise" i-Alibaba Cloud kunye ne-GCP, ke sivule uluhlu lweendawo apho uGoogle ekhona. Ngelo xesha babengekabinalo iziko labo ledatha eHong Kong.
Owona mmandla ukufutshane wajika waba iAsia-mpuma1 (eTaiwan). U-Ali waba ngowona mmandla ukufutshane welizwe laseTshayina ukuya eTaiwan cn-shenzhen (Shenzhen).

Ngo kunceda iterraform ichaze kwaye inyuse iziseko zophuhliso kwi-GCP kunye ne-Ali. Itonela ye-100 Mbit/s phakathi kwamafu yenyuka phantse ngoko nangoko. Kwicala leShenzhen kunye neTaiwan, oomatshini ababambekayo baphakanyiswa. E-Shenzhen, i-traffic traffic iphelile, i-proxied ngetonela ukuya eTaiwan, kwaye ukusuka apho iya ngqo kwi-IP yangaphandle yenkonzo yethu us-empuma (UNxweme oluseMpuma lwaseMelika). I-Ping phakathi koomatshini benyani ngetonela 24ms, engeyombi kangako.

Kwangaxeshanye, sabeka indawo yovavanyo kuyo Alibaba Cloud DNS. Emva kokunikezelwa kwendawo kwi-NS Ali, ixesha lesisombululo lehlile ukusuka kwi-470 ms ukuya 50 ms. Ngaphambi koku, lo mmandla wawukwa-Cloudlfare.

Ingqamene netonela ukuya iAsia-mpuma1 waphakamisa enye itonela ukusuka Shenzhen ngqo ukuya us-empuma4. Apho badale oomatshini abangakumbi be-proxy kwaye baqala ukuvavanya zombini izisombululo, ukuhambisa uvavanyo lwetrafikhi usebenzisa iiKuki okanye iDNS. Ibhentshi yovavanyo ichazwa ngokucwangcisekileyo kulo mzobo ulandelayo:

Ukubambezeleka kweetonela kuye kwahamba ngolu hlobo lulandelayo:
Ali cn-shenzhen <—> GCP asia-east1 — 24ms
Ali cn-shenzhen <—> GCP us-east4 — 200ms

Iimvavanyo zebrowser yeCatchpoint zichaze uphuculo olubalaseleyo.

Thelekisa iziphumo zovavanyo lwezisombululo ezibini:

Isisombululo
Ixesha lokuphumla
Median
75 Ipesenti
95 Ipesenti

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

Le datha evela kwisisombululo esisebenzisa itonela ye-IPSEC nge iAsia-mpuma1. Ngokusebenzisa us-east4 iziphumo bezimbi kakhulu, kwaye bekukho iimpazamo ezininzi, ke andizukunika iziphumo.

Ngokusekelwe kwiziphumo zolu vavanyo lweetonela ezimbini, enye yazo iphelile kwindawo ekufutshane neChina, kwaye enye kwindawo yokugqibela, kwacaca ukuba kubalulekile "ukuphuma" phantsi kwe-firewall yaseTshayina ngokukhawuleza njengoko. kunokwenzeka, kwaye ke usebenzise iinethiwekhi ezikhawulezayo (ababoneleli be-CDN , ababoneleli befu, njl.). Akukho mfuneko yokuba uzame ukungena kwi-firewall kwaye ufike kwindawo oya kuyo nge-swoop enye. Le asiyondlela ikhawulezayo.

Ngokubanzi, iziphumo azikho zimbi, nangona kunjalo, i-semrush.com ine-median ye-8.8s, kunye ne-75 Percentile 9.4s (kuvavanyo olufanayo).
Kwaye ngaphambi kokuba ndiqhubele phambili, ndingathanda ukwenza uphumlo olufutshane lweengoma.

Ukuqhawulwa kwelizwi

Emva kokuba umsebenzisi engena kwindawo www.semrushchina.cn, esisombulula ngeeseva ze-DNS zaseTshayina "ezikhawulezayo", isicelo se-HTTP sidlula kwisisombululo sethu esikhawulezayo. Impendulo ibuyiswa ngendlela efanayo, kodwa ithambeka licacisiwe kuzo zonke izikripthi ze-JS, amaphepha e-HTML kunye nezinye izinto zephepha lewebhu. semrush.com kwizibonelelo ezongezelelweyo ekufuneka zilayishwe xa iphepha linikezelwa. Oko kukuthi, umxhasi uxazulula "irekhodi eliphambili" le-A www.semrushchina.cn kwaye ingena kwitonela ekhawulezayo, ifumana ngokukhawuleza impendulo-iphepha le-HTML elithi:

• Khuphela ezinje nezinje js kwi-sso.semrush.com,
• Fumana iifayile zeCSS kwi-cdn.semrush.com,
• kwaye uthathe imifanekiso ethile kwi-dab.semrush.com
• kwaye njalo njalo.

Isikhangeli siqala ukuya kwi-Intanethi "yangaphandle" kwezi zixhobo, ixesha ngalinye lidlula kwi-firewall etya ixesha lokuphendula.

Kodwa uvavanyo lwangaphambili lubonisa iziphumo xa kungekho zixhobo kwiphepha semrush.comkuphela i-semrushchina.cn, kwaye *.semrushchina.cn isombulule kwidilesi yomatshini wenyani eShenzhen ukuze ke ungene kwitonela.

Kungale ndlela kuphela, ngokutyhala yonke i-traffic enokwenzeka ukuya kwinqanaba eliphezulu ngesisombululo sakho sokudlula ngokukhawuleza i-firewall yaseTshayina, unokufumana isantya esamkelekileyo kunye nezalathi zokufumaneka kwewebhusayithi, kunye neziphumo ezithembekileyo zovavanyo lwesisombululo.
Senze oku ngaphandle kokuhlelwa kwekhowudi enye kwicala lemveliso yeqela.

Isihluzi esisezantsi

Isisombululo sazalwa ngokukhawuleza emva kokuba le ngxaki ivele. Sasidinga I-PoC (Ubungqina beNgcaciso) yokuba izisombululo zethu zokungena kwi-firewall zisebenza kakuhle ngokwenene. Ukwenza oku, kufuneka usonge yonke indawo yokugcwala kwesi sisombululo kangangoko kunokwenzeka. Kwaye safaka isicelo isihluzi esisezantsi kwi nginx.

Isihluzi esisezantsi yimodyuli elula kwi nginx ekuvumela ukuba utshintshe umgca omnye kumzimba wempendulo komnye umgca. Ngoko ke satshintsha zonke iziganeko semrush.com phezu i-semrushchina.cn kuzo zonke iimpendulo.

Kwaye ... ayizange isebenze ngenxa yokuba sifumene umxholo ocinezelweyo ukusuka kwi-backends, ngoko ke isihluzi asifumananga umgca ofunekayo. Kwafuneka ndongeze enye iseva yendawo kwi-nginx, eyanciphisa impendulo kwaye yagqithisela kwi-server yasekhaya elandelayo, eyayisele ixakeke ngokubuyisela umtya, ukuyicinezela, kwaye uyithumele kumncedisi wommeleli olandelayo kwikhonkco.

Ngenxa yoko, umxhasi uya kufumana phi .semrush.com, wamkela .semrushchina.cn kwaye ngokuthobela wahamba kwisigqibo sethu.

Nangona kunjalo, akwanelanga ukutshintsha i-domain ngendlela enye, kuba i-backends isalindele i-semrush.com kwizicelo ezilandelayo ezivela kumxhasi. Ngokufanelekileyo, kumncedisi omnye apho utshintshiselwano lwendlela enye lwenziwayo, kusetyenziswa intetho eqhelekileyo eqhelekileyo sifumana isizinda esisezantsi kwisicelo, kwaye ke senza iproxy_pass ngokuguquguquka $umamkeli, iboniswe kwi $ subdomain.semrush.com. Isenokubonakala ibhidekile, kodwa iyasebenza. Kwaye isebenza kakuhle. Kwimimandla nganye efuna ingqiqo eyahlukileyo, yenza ngokulula iibhloko zakho zeseva kwaye wenze uqwalaselo olwahlukileyo. Apha ngezantsi kushunqulelwe inginx configs ukucaca kunye nokuboniswa kolu cwangciso.

Uqwalaselo olulandelayo luqhuba zonke izicelo ezivela eTshayina ukuya .semrushchina.cn:

    listen 80;

    server_name ~^(?<subdomain>[w-]+).semrushchina.cn$;

    sub_filter '.semrush.com' '.semrushchina.cn';
    sub_filter_last_modified on;
    sub_filter_once off;
    sub_filter_types *;

    gzip on;
    gzip_proxied any;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;

    location / {
        proxy_pass http://127.0.0.1:8083;
        proxy_set_header Accept-Encoding "";
        proxy_set_header Host $subdomain.semrush.com;
        proxy_set_header X-Accept-Encoding $http_accept_encoding;
    }
}

Olu qwalaselo lwe proxies ukuya localhost kwizibuko 83, kwaye uqwalaselo olulandelayo lulindile apho:

    listen 127.0.0.1:8083;

    server_name *.semrush.com;

    location / {
        resolver 8.8.8.8 ipv6=off;
        gunzip on;
        proxy_pass https://$host;
        proxy_set_header Accept-Encoding gzip;
    }
}

Ndiyaphinda, ezi ziziqwalaselo ezisikiweyo.

Kanjalo. Isenokubonakala intsonkothile, kodwa ibhalwe ngamagama. Ngapha koko, yonke into ilula kuneetheniphu ezishushu :)

Ukuphela kokuphambuka

Ngexesha elithile sasivuya kuba intsomi malunga nokuwa kweetonela ze-IPSEC ayizange iqinisekiswe. Kodwa ke amatonela aqalisa ukuwa. Amaxesha amaninzi ngemini imizuzu embalwa. Kancinci, kodwa oko akuzange kusilungele. Ekubeni zombini iitonela zipheliswe kwicala lika-Ali kwi-router efanayo, sagqiba ekubeni mhlawumbi le yingxaki yengingqi kwaye kufuneka siphakamise ummandla wokugcina.

Bayichola. Iitonela zaqala ukusilela ngamaxesha ahlukeneyo, kodwa i-failover yasisebenzela kakuhle kwinqanaba eliphezulu kwi-nginx. Kodwa ke iitonela zaqala ukuwa malunga nexesha elifanayo 🙂 Kwaye i-502 kunye ne-504 yaqala kwakhona. Ixesha lokuphumla laqala ukuwohloka, ngoko ke saqala ukusebenza kwinketho kunye Alibaba CEN (Cloud Enterprise Network).

CEN

CEN -oku kuqhagamshelo lwee-VPC ezimbini ezivela kwimimandla eyahlukeneyo ngaphakathi kwe-Alibaba Cloud, oko kukuthi, ungaqhagamshela uthungelwano lwabucala kuyo nayiphi na imimandla ngaphakathi kwelifu kunye nomnye. Kwaye okona kubaluleke kakhulu: eli jelo lingqongqo ngokufanelekileyo SLA. Izinzile kakhulu zombini kwisantya kunye nexesha. Kodwa akulula kangako:

• KUNZIMA kakhulu ukuyifumana ukuba awungobemi baseTshayina okanye umbutho osemthethweni,
• Kufuneka uhlawule i-megabit nganye ye-bandwidth yesitishi.

Ukuba nethuba lokudibanisa Mainland China и lwaphesheya, senze i-CEN phakathi kwemimandla emibini ye-Ali: cn-shenzhen и us-empuma-1 (eyona ndawo ikufutshane kuthi-empuma4). Ku-Ali us-empuma-1 waphakamisa omnye umatshini virtual ukuze kubekho omnye ngaphezulu hop.

Kwavela ngolu hlobo:

Iziphumo zovavanyo lwesikhangeli zingezantsi:

Isisombululo
Ixesha lokuphumla
Median
75 Ipesenti
95 Ipesenti

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

CEN
99.75
16s
21s
27s

Ukusebenza ngcono kancinci kune-IPSEC. Kodwa nge-IPSEC unako ukukhuphela ngesantya se-100 Mbit / s, kwaye nge-CEN kuphela ngesantya se-5 Mbit / s nangaphezulu.

Ivakala ngathi ngumxube, akunjalo? Hlanganisa isantya se-IPSEC kunye nokuzinza kwe-CEN.

Yile nto esiyenzileyo, sivumela i-traffic ngokusebenzisa zombini i-IPSEC kunye ne-CEN kwimeko yokungaphumeleli kwe-tunnel ye-IPSEC. I-Uptime iphezulu kakhulu, kodwa isantya sokulayisha indawo sisashiya okuninzi ekufuneka sinqwenelwe. Emva koko ndazoba zonke iisekethe esele sizisebenzisile kwaye sazivavanya, kwaye ndagqiba ekubeni ndizame ukongeza i-GCP encinci kule sekethe, oko kukuthi. I-GLB.

I-GLB

I-GLB - yi le IGlobal Load Balancer (okanye iGoogle Cloud Load Balancer). Inenzuzo ebalulekileyo kuthi: kumxholo we-CDN enayo nayiphi na i-IP, ekuvumela ukuba uhambise i-traffic kwiziko ledatha elisondeleyo kumxhasi, ukwenzela ukuba i-traffic ifike ngokukhawuleza kwinethiwekhi ye-Google ekhawulezayo kwaye incinci ihamba nge-Intanethi "eqhelekileyo".

Ngaphandle kokucinga kabini, saphakamisa I-HTTP/HTTPS LB Sifake oomatshini bethu benyani kunye nesihluzi esisezantsi kwi-GCP nanjenge-backend.

Kwakukho izicwangciso ezininzi:

• Sebenzisa Cloudflare China Network, kodwa ngeli xesha Imvelaphi kufuneka ichaze jikelele IP GLB.
• Ukuphelisa abathengi kwi cn-shenzhen, kwaye ukusuka apho ummeli wetrafikhi ngokuthe ngqo ukuya I-GLB.
• Yiya ngqo ukusuka eTshayina ukuya I-GLB.
• Ukuphelisa abathengi kwi cn-shenzhen, ukusuka apho ummeli ukuya iAsia-mpuma1 nge-IPSEC (nge us-empuma4 nge-CEN), ukusuka apho uye kwi-GLB (ngokuzola, kuya kubakho umfanekiso kunye nengcaciso engezantsi)

Sivavanye zonke ezi nketho kunye nezinye ezininzi ezixubileyo:

• Cloudflare + GLB

Esi sikimu asizange sisilungele ngenxa yexesha kunye neempazamo ze-DNS. Kodwa uvavanyo lwenziwa ngaphambi kokuba i-bug ilungiswe kwicala le-CF, mhlawumbi kungcono ngoku (nangona kunjalo, oku akubandakanyi ixesha lokuphuma kwe-HTTP).

• UAli + GLB

Esi sikimu asizange sivumelane nathi ngokwexesha lokuphumla, kuba i-GLB yayidla ngokuwa ngaphandle komlambo ngenxa yokungakwazi ukudibanisa ngexesha elamkelekileyo okanye ixesha lokuphuma, kuba iseva ngaphakathi kweTshayina, idilesi ye-GLB ihlala ngaphandle, kwaye ngoko emva I-firewall yaseTshayina. Umlingo awuzange wenzeke.

• I-GLB kuphela

Inketho efana neyangaphambili, kuphela ayizange isebenzise amaseva e-China ngokwayo: i-traffic yaya ngqo kwi-GLB (iirekhodi ze-DNS zatshintshwa). Ngokufanelekileyo, iziphumo azizange zanelise, kuba abathengi abaqhelekileyo baseTshayina abasebenzisa iinkonzo zababoneleli be-Intanethi abaqhelekileyo banemeko embi kakhulu ngokudlula i-firewall kune-Ali Cloud.

• I-Shenzhen -> (CEN / IPSEC) -> Ummeleli -> GLB

Apha sigqibe ekubeni sisebenzise ezona zisombululo zingcono:

• uzinzo kunye ne-SLA eqinisekisiweyo evela kwi-CEN
• isantya esiphezulu ukusuka IPSEC
• Inethiwekhi "ekhawulezayo" kaGoogle kunye nayo nayiphi na icast.

Iskimu sijongeka ngolu hlobo: itrafikhi yomsebenzisi iphelisiwe kumatshini wenyani ch-shenzhen. I-Nginx upstreams iqwalaselwe apho, ezinye zazo zikhomba kwiiseva ze-IP zabucala ezibekwe kwelinye icala letonela ye-IPSEC, kwaye ezinye iindawo eziphezulu zikhomba kwiidilesi zabucala zabancedisi kwelinye icala le-CEN. IPSEC iqwalaselwe kummandla iAsia-mpuma1 kwi-GCP (yayiyeyona ndawo ikufutshane ne-China ngexesha isisombululo senziwe. I-GCP ngoku nayo inobukho eHong Kong). CEN - ukuya kummandla us-empuma1 kwi Ali Cloud.

Emva koko i-traffic evela kumacala omabini yayalelwa nayiphi na i-IP GLB, oko kukuthi, ukuya kwindawo ekufutshane yobukho beGoogle, kwaye yahamba ngeenethiwekhi zayo ukuya kummandla us-empuma4 kwi-GCP, apho bekukho oomatshini abatshintshiweyo benyani (nge subfilter kwi nginx).

Esi sisombululo sixubileyo, njengoko besilindele, sasebenzisa inzuzo yetekhnoloji nganye. Ngokubanzi, i-traffic ihamba nge-IPSEC ekhawulezayo, kodwa ukuba iingxaki ziqala, ngokukhawuleza kunye nemizuzu embalwa sikhaba ezi seva ngaphandle komsinga kwaye sithumele i-traffic kuphela nge-CEN de kube itonela lizinzile.

Ngokuphumeza isisombululo se-4 kuluhlu olungentla, sifezekise into ebesiyifunayo kunye neyona nto ifunwa lishishini kuthi ngelo xesha.

Iziphumo zovavanyo lwebhrawuza yesisombululo esitsha xa kuthelekiswa nezidlulileyo:

Isisombululo
Ixesha lokuphumla
Median
75 Ipesenti
95 Ipesenti

Cloudflare
86.6
18s
30s
60s

IPsec
99.79
18s
21s
30s

CEN
99.75
16s
21s
27s

CEN/IPsec + GLB
99.79
13s
16s
25s

CDN

Yonke into ilungile kwisisombululo esisiphumezileyo, kodwa akukho CDN inokukhawulezisa i-traffic kwinqanaba lengingqi kunye nesixeko. Kwithiyori, oku kufuneka kukhawulezise indawo kubasebenzisi bokugqibela ngokusebenzisa amajelo onxibelelwano okhawulezayo womnikezeli weCDN. Kwaye sasicinga ngayo ngalo lonke ixesha. Kwaye ngoku, ixesha lifikile lokuphindaphinda okulandelayo kweprojekthi: ukukhangela kunye nokuvavanya ababoneleli be-CDN eChina.

Kwaye ndiza kukuxelela ngale nto kwilandelayo, inxalenye yokugqibela :)

umthombo: www.habr.com