I ka makahiki 2010 ka hui aia he 50 mau kikowaena a me kahi hoʻohālike pūnaewele maʻalahi: backend, frontend and firewall. Ua ulu ka nui o nā kikowaena, ua ʻoi aku ka paʻakikī o ke kumu hoʻohālike: ka hoʻonohonoho ʻana, nā VLAN kaʻawale me nā ACL, a laila nā VPN me nā VRF, nā VLAN me nā ACL ma L2, nā VRF me nā ACL ma L3. Ke wili nei ke poʻo? E ʻoi aku ka leʻaleʻa ma hope.
I ka loaʻa ʻana o 16 mau kikowaena, ua lilo ia i mea hiki ʻole ke hana me ka waimaka ʻole me ka nui o nā ʻāpana heterogeneous. No laila ua loaʻa iā mākou kahi hopena ʻē aʻe. Lawe mākou i ka waihona Netfilter, hoʻohui i ke Kanikela ma ke ʻano he kumu ʻikepili, a loaʻa iā mākou kahi pā ahi wikiwiki. Ua hoʻololi lākou i nā ACL ma nā ala ala a hoʻohana iā lākou ma ke ʻano he pā ahi waho a i loko. No ka hoʻokele ikaika ʻana i ka hāmeʻa, hoʻomohala mākou i ka ʻōnaehana BEFW, i hoʻohana ʻia ma nā wahi āpau: mai ka hoʻokele ʻana i ka mea hoʻohana i ka ʻoihana huahana i ka hoʻokaʻawale ʻana i nā ʻāpana pūnaewele mai kekahi i kekahi.
E haʻi ʻo ia iā ʻoe pehea e hana ai a no ke aha ʻoe e nānā pono ai i kēia ʻōnaehana. ʻO Ivan Agarkov () ʻo ia ke poʻo o ka pūʻulu palekana ʻoihana o ka mahele Maintenance ma ke kikowaena hoʻomohala Minsk o ka hui. He kanaka SELinux ʻo Ivan, aloha iā Perl, a kākau i ke code. Ma ke ʻano he poʻo o ka pūʻulu palekana ʻike, hana mau ʻo ia me nā lāʻau, backups a me R&D e pale iā Wargaming mai nā hackers a hōʻoia i ka hana o nā kikowaena pāʻani āpau i ka hui.
Ka Hanana Hōʻike
Ma mua o koʻu haʻi ʻana iā ʻoe pehea mākou i hana ai, e haʻi wau iā ʻoe pehea mākou i hiki ai i kēia ma kahi mua a me ke kumu e pono ai. No ka hana ʻana i kēia, e hoʻi kākou i 9 mau makahiki: 2010, ua ʻike ʻia ka World of Tanks. Loaʻa iā Wargaming ma kahi o 50 mau kikowaena.
ʻO ka pakuhi ulu o ke kikowaena ʻoihana.
Loaʻa iā mākou kahi hiʻohiʻona pūnaewele. ʻO ia ka manawa kūpono loa.
Hoʻohālike pūnaewele i ka makahiki 2010.
Aia nā poʻe ʻino ma ka ʻaoʻao mua e makemake e uhaʻi iā mākou, akā he pā ahi. ʻAʻohe pā ahi ma ka hope, akā aia nā kikowaena 50 ma laila, ʻike mākou iā lākou āpau. Hana maikaʻi nā mea a pau.
I loko o 4 mau makahiki, ua ulu ka ʻauwaʻa kikowaena i nā manawa 100, i 5000. Ua ʻike ʻia nā pūnaewele kaʻawale mua - hoʻokūkū: ʻaʻole hiki iā lākou ke hele i ka hana, a he mau mea e holo ana ma laila e hiki ke pilikia.
Hoʻohālike pūnaewele i ka makahiki 2014.
Ma ka inertia, ua hoʻohana mākou i nā ʻāpana like ʻole, a ua hana ʻia nā hana a pau ma nā VLAN kaʻawale: Ua kākau ʻia nā ACL i nā VLAN, e ʻae a hōʻole paha i kekahi ʻano pili.
I ka makahiki 2016, ua hiki ka heluna o nā kikowaena i ka 8000. Ua komo ʻo Wargaming i nā studio ʻē aʻe, a ua ʻike ʻia nā pūnaewele pili hou. Me he mea lā iā mākou, ʻaʻole naʻe: ʻaʻole hana pinepine ʻo VLAN no nā hoa, pono ʻoe e hoʻohana VPN me VRF, ʻoi aku ka paʻakikī o ka noho kaʻawale. Ua ulu ka hui insulation ACL.
Hoʻohālike pūnaewele i ka makahiki 2016.
Ma ka hoʻomakaʻana o 2018, ua ulu ka'auwaʻa o nā mīkini i 16. Aia nā'āpana 000, aʻaʻole mākou i helu i ke koena, me nā mea i paniʻia i mālamaʻia nāʻikepili kālā. ʻO nā pūnaewele pahu (Kubernetes), DevOps, nā pūnaewele kapuaʻi i hoʻopili ʻia ma VPN, no ka laʻana, mai kahi IVS, ua ʻike ʻia. Nui nā lula - he ʻeha.
ʻO ka hoʻohālike pūnaewele a me nā ʻano kaʻawale i 2018.
No ka hoʻokaʻawale ʻana ua hoʻohana mākou: VLAN me ACL ma L2, VRF me ACL ma L3, VPN a ʻoi aku. Nui loa.
Nā pilikia
Noho nā kānaka a pau me ACL a me VLAN. He aha ka pilikia? E pane ʻia kēia nīnau e Harold, e hūnā ana i ka ʻeha.
Nui nā pilikia, akā ʻelima mau pilikia nui.
- Hoʻonui kumu kūʻai Geometric no nā lula hou. Ua lōʻihi ka hoʻohui ʻana i kēlā me kēia lula hou ma mua o ka mea ma mua, no ka mea pono e ʻike mua inā aia kekahi lula.
- ʻAʻohe pā ahi i loko o nā ʻāpana. Ua hoʻokaʻawale ʻia nā ʻāpana mai kekahi i kekahi, a ʻaʻole lawa nā kumuwaiwai i loko.
- Ua hoʻohana ʻia nā lula no ka manawa lōʻihi. Hiki i nā mea hana ke kākau i hoʻokahi lula kūloko ma ka lima i hoʻokahi hola. Ua hala ka honua i kekahi mau lā.
- Paʻakikī me nā lula hoʻoponopono. ʻOi aku ka pololei, ʻaʻole hiki. Ua kākau ʻia nā lula mua i ka makahiki 2010, a ʻaʻole hana hou ka hapa nui o kā lākou mea kākau no ka hui.
- Haʻahaʻa haʻahaʻa o ka hoʻokele waiwai. ʻO kēia ka pilikia nui - ʻaʻole maopopo mākou i nā mea e hana nei ma ko mākou ʻāina.
ʻO kēia ke ʻano o kahi ʻenekini pūnaewele ma 2018 i kona lohe ʻana: "Pono i kahi ACL hou."
Pāʻoihana
I ka hoʻomaka ʻana o 2018, ua hoʻoholo ʻia e hana i kekahi mea e pili ana iā ia.
Ke ulu mau nei ke kumukūʻai o nā hoʻohui. ʻO ka hoʻomaka ʻana, ua kāpae nā kikowaena data nui i ke kākoʻo ʻana i nā VLAN kaʻawale a me nā ACL no ka pau ʻana o nā mea hoʻomanaʻo.
Pane: ua wehe mākou i ke kumu kanaka a hoʻomaʻamaʻa i ka hāʻawi ʻana i ke komo i ka palena kiʻekiʻe.
He lōʻihi ka hoʻohana ʻana i nā lula hou. ʻO ka hopena: e hoʻolalelale i ka hoʻohana ʻana i nā lula, e puʻunaue a like. Pono kēia i kahi ʻōnaehana puʻupuʻu i hāʻawi ʻia nā lula iā lākou iho, me ka ʻole o rsync a i ʻole SFTP i hoʻokahi tausani ʻōnaehana.
ʻAʻohe pā ahi i loko o nā ʻāpana. Ua hoʻomaka ka pā ahi i loko o nā ʻāpana iā mākou i ka wā i ʻike ʻia ai nā lawelawe like ʻole i loko o ka pūnaewele like. Pane: hoʻohana i kahi pā ahi ma ka pae hoʻokipa - nā pā ahi e pili ana i ka host. Ma kahi kokoke i nā wahi āpau i loaʻa iā mākou Linux, a ma nā wahi āpau i loaʻa iā mākou iptables, ʻaʻole kēia pilikia.
Paʻakikī me nā lula hoʻoponopono. Pane: E mālama i nā lula a pau ma kahi hoʻokahi no ka loiloi a me ka hoʻokele ʻana, no laila hiki iā mākou ke loiloi i nā mea āpau.
Haʻahaʻa haʻahaʻa o ka mana ma luna o nā ʻoihana. Pane: e lawe i kahi papa helu o nā lawelawe āpau a me ke komo ʻana ma waena o lākou.
ʻOi aku kēia ma ke kaʻina hoʻokele ma mua o ka ʻenehana. I kekahi manawa, loaʻa iā mākou he 200-300 mau mea hou i hoʻokahi pule, ʻoi aku hoʻi i ka wā hoʻolaha a me nā lā hoʻomaha. Eia kekahi, no ka hui hoʻokahi wale nō o kā mākou DevOps. Me ka nui o nā hoʻokuʻu, ʻaʻole hiki ke ʻike i nā awa, IP, a me nā hoʻohui e pono ai. No laila, makemake mākou i nā luna lawelawe i hoʻomaʻamaʻa kūikawā i nīnau i nā hui: "He aha ka mea a no ke aha ʻoe i hāpai ai?"
Ma hope o nā mea a pau a mākou i hoʻomaka ai, ua hoʻomaka ka ʻenekini pūnaewele ma 2019 e like me kēia.
Kanikela
Ua hoʻoholo mākou e hoʻokomo i nā mea a pau i loaʻa iā mākou me ke kōkua o nā luna lawelawe i ke Kanikela a mai laila mākou e kākau i nā lula iptables.
Pehea mākou i hoʻoholo ai e hana i kēia?
- E hōʻiliʻili mākou i nā lawelawe āpau, nā pūnaewele a me nā mea hoʻohana.
- E hana kākou i nā lula iptables e pili ana iā lākou.
- Hoʻokolo mākou i ka mana.
- ....
- Pomaikai.
ʻAʻole ʻo Consul kahi API mamao, hiki iā ia ke holo ma kēlā me kēia node a kākau i nā iptables. ʻO nā mea a pau e hiki mai ana me nā mana maʻalahi e hoʻomaʻemaʻe i nā mea pono ʻole, a e hoʻopau ʻia ka hapa nui o nā pilikia! E hana mākou i ke koena i ko mākou hele ʻana.
No ke aha ke Kanikela?
Ua hōʻoia maikaʻi iā ia iho. Ma 2014-15, ua hoʻohana mākou iā ia ma ke ʻano he hope no Vault, kahi e mālama ai mākou i nā ʻōlelo huna.
ʻAʻole nalowale ka ʻikepili. I ka wā o ka hoʻohana ʻana, ʻaʻole i nalowale ʻo Consul i ka ʻikepili i ka wā hoʻokahi ulia. He mea hoʻohui nui kēia no ka ʻōnaehana hoʻokele firewall.
Hoʻonui nā pilina P2P i ka laha o ka loli. Me P2P, hiki koke mai nā loli a pau, ʻaʻole pono e kali i nā hola.
API hoʻomaha maʻalahi. Ua noʻonoʻo pū mākou iā Apache ZooKeeper, akā ʻaʻohe ona REST API, no laila pono ʻoe e hoʻokomo i nā koʻokoʻo.
Hana ʻia ma ke ʻano he Key Vault (KV) a me kahi Papa kuhikuhi (Service Discovery). Hiki iā ʻoe ke mālama i nā lawelawe, catalogs, a me nā kikowaena data i ka manawa hoʻokahi. ʻAʻole maʻalahi kēia no mākou wale nō, akā no nā hui pili pū kekahi, no ka mea i ke kūkulu ʻana i kahi lawelawe honua, manaʻo nui mākou.
Kākau ʻia ma Go, ʻo ia kahi ʻāpana o ka waihona Wargaming. Aloha mākou i kēia ʻōlelo, nui nā mea hoʻomohala Go.
Pūnaehana ACL ikaika. Ma ke Kanikela, hiki iā ʻoe ke hoʻohana i nā ACL e hoʻomalu i ka mea nāna e kākau i ka mea. Ke hōʻoiaʻiʻo nei mākou ʻaʻole e uhi ʻia nā lula ahi me nā mea ʻē aʻe a ʻaʻole mākou e pilikia me kēia.
Aka, he mau hemahema no ke Kanikela.
- ʻAʻole i hoʻonui ʻia i loko o kahi kikowaena data ke ʻole ʻoe he mana ʻoihana. Hiki ke hoʻonui wale ʻia e ka hui.
- Pili loa i ka maikaʻi o ka pūnaewele a me ka ukana kikowaena. ʻAʻole e hana pono ke kanikela ma ke ʻano he kikowaena ma kahi kikowaena paʻa inā loaʻa kekahi mau lag i ka pūnaewele, no ka laʻana, ka wikiwiki ʻole. Ma muli o nā pilina P2P a me nā hiʻohiʻona hoʻohele hou.
- Loaʻa ka paʻakikī o ka nānā ʻana. Ma ke kūlana Kanikela hiki iā ia ke ʻōlelo ua maikaʻi nā mea a pau, akā ua make ʻo ia i kahi manawa lōʻihi.
Ua hoʻoponopono mākou i ka hapa nui o kēia mau pilikia me ka hoʻohana ʻana i ke Kanikela, ʻo ia ke kumu i koho ai mākou. He hoʻolālā ka hui no kahi hope ʻē aʻe, akā ua aʻo mākou e hoʻoponopono i nā pilikia a ke noho nei mākou me ke Kanikela.
Pehea e hana ai ke Kanikela
E hoʻouka mākou i ʻekolu a ʻelima mau kikowaena ma kahi kikowaena data conditional. ʻAʻole e hana hoʻokahi a ʻelua paha mau kikowaena: ʻaʻole hiki iā lākou ke hoʻonohonoho i kahi kōrum a hoʻoholo i ka mea pololei a ʻo wai ka hewa inā ʻaʻole like ka ʻikepili. ʻOi aku ma mua o ʻelima ʻaʻohe manaʻo, hāʻule ka huahana.
Hoʻopili nā mea kūʻai aku i nā kikowaena i kēlā me kēia kauoha: nā ʻelele like, me ka hae wale nō server = false.
Ma hope o kēia, loaʻa nā mea kūʻai aku i kahi papa inoa o nā pilina P2P a kūkulu i nā pilina ma waena o lākou iho.
Ma ka pae honua, hoʻohui mākou i kekahi mau kikowaena data. Hoʻopili pū lākou iā P2P a kamaʻilio.
Ke makemake mākou e kiʻi i ka ʻikepili mai kahi kikowaena ʻikepili ʻē aʻe, hele ka noi mai kahi kikowaena i kahi kikowaena. Kapa ʻia kēia papahana Serf protocol. ʻO ka Serf protocol, e like me Consul, hoʻomohala ʻia e HashiCorp.
ʻO kekahi mau mea nui e pili ana i ke Kanikela
Loaʻa iā Consul nā palapala e wehewehe ana i ka hana. E hāʻawi wale wau i nā mea i koho ʻia i kūpono ke ʻike.
Koho nā kikowaena kanikela i haku mai waena mai o ka poʻe koho pāloka. Koho ke Kanikela i haku mai ka papa inoa o nā kikowaena no kēlā me kēia kikowaena data, a ʻo nā noi a pau e hele wale iā ia, me ka nānā ʻole i ka helu o nā kikowaena. ʻAʻole alakaʻi i ke koho hou ʻana o ka haku hau. Inā ʻaʻole koho ʻia ka haku, ʻaʻole lawelawe ʻia nā noi e kekahi.
Makemake ʻoe i ka hoʻonui ʻia ʻana? Aloha ʻoe, ʻaʻole.
Ke noi nei i kahi kikowaena ʻikepili ʻē aʻe mai ka haku a i ka haku, me ka nānā ʻole i ke kikowaena i hele mai ai. Loaʻa i ka haku i koho ʻia he 100% o ka ukana, koe wale nō ka ukana ma nā noi i mua. Loaʻa i nā kikowaena a pau i ke kikowaena ʻikepili kahi kope hou o ka ʻikepili, akā hoʻokahi wale nō pane.
ʻO ke ala wale nō e hoʻonui ai, ʻo ia ka hoʻā ʻana i ke ʻano stale ma ka mea kūʻai aku.
Ma ke ʻano stale, hiki iā ʻoe ke pane me ka ʻole o kahi kōrum. He ʻano kēia e hāʻawi ai mākou i ka ʻikepili kūlike, akā heluhelu ʻoi aku ka wikiwiki ma mua o ka mea maʻamau, a pane kekahi kikowaena. ʻO ka mea maʻamau, hoʻopaʻa wale ma o ka haku.
ʻAʻole kope ke Kanikela i ka ʻikepili ma waena o nā kikowaena data. Ke hui ʻia kahi hui, e loaʻa i kēlā me kēia kikowaena kāna ʻikepili ponoʻī. No nā mea ʻē aʻe, huli mau ʻo ia i kekahi.
ʻAʻole hōʻoia ʻia ka Atomicity o nā hana ma waho o kahi kālepa. E hoʻomanaʻo ʻaʻole ʻoe wale nō ka mea hiki ke hoʻololi i nā mea. Inā makemake ʻoe i kahi ʻokoʻa, e hana i kahi kālepa me kahi laka.
ʻAʻole hōʻoia ka paʻa ʻana i ka laka. Hele ka noi mai ka haku i ka haku, ʻaʻole pololei, no laila, ʻaʻohe mea e hōʻoiaʻiʻo e hana ka pale i ka wā e poloka ai ʻoe, no ka laʻana, ma kahi kikowaena data ʻē aʻe.
ʻAʻole hōʻoia ʻo ACL i ke komo ʻana (i nā manawa he nui). ʻAʻole hiki ke hana ʻia ka ACL no ka mea ua mālama ʻia ia i loko o kahi kikowaena data federation - ma ke kikowaena data ACL (Primary DC). Inā ʻaʻole pane ka DC iā ʻoe, ʻaʻole e holo ka ACL.
Hoʻokahi haku hau e hoʻomaloʻo i ka hui holoʻokoʻa. No ka laʻana, aia he 10 mau kikowaena ʻikepili i loko o kahi hui, a he ʻupena ʻino kekahi, a hāʻule kekahi haku. ʻO nā mea a pau e kamaʻilio pū me ia e maloʻo i ka pōʻai: aia kahi noi, ʻaʻohe pane iā ia, maloʻo ke kaula. ʻAʻohe ala e ʻike ai i ka wā e hiki mai ai kēia, i hoʻokahi hola a ʻelua paha e hāʻule ka hui holoʻokoʻa. ʻAʻohe mea hiki iā ʻoe ke hana no ia mea.
Hoʻohana ʻia ke kūlana, koruma a me nā koho e kahi pae ʻokoʻa. ʻAʻole hiki ke koho hou, ʻaʻole e hōʻike ke kūlana i kekahi mea. Manaʻo ʻoe he Kanikela ola kāu, nīnau ʻoe, ʻaʻohe mea i hiki - ʻaʻohe pane. I ka manawa like, hōʻike ke kūlana ua maikaʻi nā mea a pau.
Ua loaʻa iā mākou kēia pilikia a pono mākou e kūkulu hou i nā ʻāpana kikoʻī o nā kikowaena data e pale aku ai.
ʻAʻole i loaʻa i ka mana pāʻoihana o Consul Enterprise kekahi o nā hemahema ma luna. Loaʻa iā ia nā hana pono he nui: ke koho ʻana i ka poʻe koho, hoʻohele, scaling. Hoʻokahi wale nō "akā" - he kumukūʻai loa ka ʻōnaehana laikini no kahi ʻōnaehana puʻupuʻu.
Ola hacking: rm -rf /var/lib/consul - he lāʻau lapaʻau no nā maʻi a pau o ka luna. Inā maikaʻi ʻole kekahi mea iā ʻoe, e holoi wale i kāu ʻikepili a hoʻoiho i ka ʻikepili mai kahi kope. Malia paha, e hana ana ke Kanikela.
BEFW
I kēia manawa, e kamaʻilio kākou e pili ana i nā mea a mākou i hoʻohui ai iā Kanikela.
he acronym no BackEndFnoreWpau loa. Pono wau e inoa i ka huahana i ka wā i hana ai au i ka waihona i mea e hoʻokomo ai i ka hoʻāʻo mua i loko. Ke waiho nei kēia inoa.
Nā laʻana lula
Ua kākau ʻia nā lula ma ka syntax iptables.
- -N BEFW
- -P INPUT DROP
- -A INPUT -m mokuʻāina—mokuʻāina PILI, KAU ʻO -j ʻAʻE
- -A INPUT -i lo -j ALOHA
- -A INPUT -j BEFW
Hele nā mea a pau i ke kaulahao BEFW, koe wale nō ESTABLISHED, RELATED a localhost. Hiki ke hoʻohālike i kekahi mea, he laʻana wale nō kēia.
Pehea ka pono o BEFW?
Nā lawelawe
Loaʻa iā mākou kahi lawelawe, loaʻa mau ke awa, kahi node e holo ai. Mai kā mākou node, hiki iā mākou ke nīnau i ka ʻelele a ʻike mākou he ʻano lawelawe. Hiki iā ʻoe ke kau i nā lepili.
ʻO kēlā me kēia lawelawe e holo nei a hoʻopaʻa inoa ʻia me ke Kanikela e lilo i lula iptables. Loaʻa iā mākou SSH - wehe i ke awa 22. He maʻalahi ka palapala Bash: curl a iptables, ʻaʻohe mea e pono ai.
Ka poʻe lawelawe
Pehea e wehe ai i ke komo ʻaʻole i nā mea a pau, akā koho? Hoʻohui i nā papa inoa IP i ka waihona KV ma ka inoa lawelawe.
No ka laʻana, makemake mākou i nā poʻe a pau ma ka pūnaewele ʻumi e hiki ke komo i ka lawelawe SSH_TCP_22. Hoʻohui i hoʻokahi kahua TTL liʻiliʻi? a i kēia manawa ua loaʻa iā mākou nā ʻae no ka manawa pōkole, no ka laʻana, no ka lā.
Loaʻa
Hoʻohui mākou i nā lawelawe a me nā mea kūʻai aku: loaʻa iā mākou kahi lawelawe, ua mākaukau ka waihona KV no kēlā me kēia. I kēia manawa hāʻawi mākou i ke komo ʻana ʻaʻole i nā mea a pau, akā koho.
Nā Pūʻulu
Inā kākau mākou i mau tausani IP no ke komo ʻana i kēlā me kēia manawa, e luhi mākou. E hui pū kākou - he ʻāpana ʻokoʻa ma KV. E kapa iā ia ʻo Alias (a i ʻole nā hui) a mālama i nā hui ma laila e like me ke kumu like.
E hoʻohui: i kēia manawa hiki iā mākou ke wehe iā SSH ʻaʻole kikoʻī no P2P, akā no kahi hui holoʻokoʻa a i ʻole nā hui. Ma ke ala like, aia kahi TTL - hiki iā ʻoe ke hoʻohui i kahi hui a wehe i ka hui no ka manawa pōkole.
Hoʻohuiʻia
ʻO kā mākou pilikia ke kumu kanaka a me ka automation. I kēia manawa ua hoʻoholo mākou i kēia ala.
Hana mākou me ka Puppet, a hoʻoili i nā mea a pau e pili ana i ka ʻōnaehana (code code) iā lākou. Mālama ʻo Puppetdb (PostgreSQL maʻamau) i kahi papa inoa o nā lawelawe e holo nei ma laila, hiki ke loaʻa iā lākou ma ke ʻano kumu. Ma laila ʻoe e ʻike ai i ka mea e noi nei ma hea. Loaʻa iā mākou kahi noi huki a hoʻohui i ka ʻōnaehana noi no kēia.
Ua kākau mākou i ka befw-sync, kahi hopena maʻalahi e kōkua i ka hoʻoili ʻana i ka ʻikepili. ʻO ka mua, loaʻa nā kuki sync e puppetdb. Hoʻonohonoho ʻia kahi HTTP API ma laila: noi mākou i nā lawelawe i loaʻa iā mākou, nā mea e pono ai ke hana. A laila noi lākou i ke Kanikela.
Aia ka hoʻohui? ʻAe: ua kākau lākou i nā lula a ua ʻae ʻia nā noi huki. Pono ʻoe i kekahi awa a hoʻohui i kahi hoʻokipa i kekahi hui? Huki Noi, loiloi - ʻaʻohe hou "E ʻimi i 200 mau ACL ʻē aʻe a hoʻāʻo e hana i kekahi mea e pili ana iā ia."
Hoʻomaikaʻi
ʻO ka pinging localhost me kahi kaulahao lula ʻole e lawe i 0,075 ms.
E hoʻohui kākou i 10 mau helu iptables i kēia kaulahao. ʻO ka hopena, e hoʻonui ʻia ka ping i 000 mau manawa: ʻo ka iptables he laina laina, ʻo ka hoʻoponopono ʻana i kēlā me kēia helu e lawe i kekahi manawa.
No kahi pā ahi kahi e neʻe ai mākou i nā tausani o nā ACL, he nui nā lula, a hoʻolauna kēia i ka latency. He hewa kēia no nā protocols pāʻani.
Akā inā mākou e kau 10 helu wahi ma ipset E emi ana ka ping.
'O ka mana'o, 'o ia ka "O" (algorithm complexity) no ka ipset i nā manawa a pau me ka 1, no ka nui o nā lula. ʻOiaʻiʻo, aia kahi palena - ʻaʻole hiki ke ʻoi aku ma mua o nā lula 65535. No kēia manawa ke ola nei mākou me kēia: hiki iā ʻoe ke hoʻohui iā lākou, hoʻonui iā lākou, hana i ʻelua ipset i hoʻokahi.
Pale
ʻO ka hoʻomau kūpono o ke kaʻina hana ʻike e mālama nei i ka ʻike e pili ana i nā mea kūʻai aku no ka lawelawe ma ipset.
I kēia manawa, loaʻa iā mākou ka SSH like, a ʻaʻole mākou e kākau i 100 IP i ka manawa hoʻokahi, akā hoʻonoho i ka inoa o ka ipset e pono ai mākou e kamaʻilio, a me ka lula aʻe. DROP. Hiki ke hoʻololi i hoʻokahi lula "ʻO wai ʻaʻole ma ʻaneʻi, DROP", akā ʻoi aku ka maopopo.
I kēia manawa ua loaʻa iā mākou nā lula a me nā hoʻonohonoho. ʻO ka hana nui ka hana ʻana i kahi hoʻonohonoho ma mua o ke kākau ʻana i ka lula, no ka mea inā ʻaʻole e kākau nā iptables i ka lula.
Nā kulekele nui
Ma ke ʻano o kahi kiʻi, ua like nā mea a pau aʻu i ʻōlelo ai.
Hoʻopaʻa mākou i ka Puppet, hoʻouna ʻia nā mea a pau i ka mea hoʻokipa, lawelawe ma ʻaneʻi, ipset ma laila, a ʻaʻole ʻae ʻia ka mea i kākau ʻole ʻia ma laila.
ʻAe & hōʻole
No ka ho'ōla wikiwiki i ka honua a i ʻole e hoʻopau koke i kekahi, i ka hoʻomaka ʻana o nā kaulahao a pau mākou i hana i ʻelua ipset: rules_allow и rules_deny. Pehea ia hana?
No ka laʻana, hana kekahi i kahi ukana ma kā mākou Pūnaewele me nā bots. Ma mua, pono ʻoe e ʻimi i kāna IP mai nā lāʻau, lawe iā ia i nā ʻenekini pūnaewele, i hiki iā lākou ke ʻike i ke kumu o ke kaʻa a pāpā iā ia. He ʻokoʻa kona ʻano i kēia manawa.
Hoʻouna mākou iā Kanikela, kali i 2,5 kekona, a ua pau. No ka mea e puunaue koke ana ke Kanikela ma o P2P, hana ia ma na wahi a pau, ma na wahi a pau o ka honua.
I kekahi manawa ua ho'ōki loa au i ka WOT ma muli o kahi kuhi me ka pā ahi. rules_allow - ʻo kā mākou ʻinikua kēia e kūʻē i ia mau hihia. Inā mākou i hana hewa ma kahi me ka pā ahi, ua ālai ʻia kekahi mea ma kahi, hiki iā mākou ke hoʻouna mau i kahi kūlana 0.0/0e kiʻi koke i nā mea a pau. Ma hope e hoʻoponopono mākou i nā mea a pau me ka lima.
Nā pūʻulu ʻē aʻe
Hiki iā ʻoe ke hoʻohui i nā pūʻulu ʻē aʻe ma ke ākea $IPSETS$.
He aha ke kumu? I kekahi manawa pono kekahi ipset, no ka laʻana, e hoʻohālike i ka pani ʻana o kekahi ʻāpana o ka hui. Hiki i nā mea a pau ke lawe mai i nā pūʻulu, inoa iā lākou, a e kiʻi ʻia lākou mai ke Kanikela. I ka manawa like, hiki i nā hoʻonohonoho ke komo i nā lula iptables a i ʻole e hana ma ke ʻano he hui NOOP: E mālama ʻia ke kūpaʻa e ka daemon.
mea hoʻohana
Ma mua, ua like kēia: pili ka mea hoʻohana i ka pūnaewele a loaʻa nā ʻāpana ma o ka domain. Ma mua o ka hiki ʻana mai o nā pā ahi o nā hanauna hou, ʻaʻole ʻike ʻo Cisco pehea e hoʻomaopopo ai i kahi o ka mea hoʻohana a i hea ka IP. No laila, ua hāʻawi ʻia ke komo ma o ka inoa inoa o ka mīkini.
He aha kā mākou i hana ai? Ua paʻa mākou i ka manawa i loaʻa iā mākou ka helu wahi. ʻO ka maʻamau kēia ʻo dot1x, Wi-Fi a i ʻole VPN - hele nā mea āpau ma RADIUS. No kēlā me kēia mea hoʻohana, hana mākou i kahi hui ma ka inoa inoa a kau i kahi IP i loko me kahi TTL e like me kāna dhcp.lease - i ka wā e pau ai, e nalowale ke kānāwai.
I kēia manawa hiki iā mākou ke wehe i ke komo i nā lawelawe, e like me nā hui ʻē aʻe, ma ka inoa inoa. Ua lawe mākou i ka ʻeha mai nā inoa inoa ke hoʻololi lākou, a ua lawe mākou i ke kaumaha mai nā ʻenekini pūnaewele no ka mea ʻaʻole pono lākou iā Cisco. I kēia manawa, hoʻopaʻa inoa nā ʻenekinia i ke komo ʻana ma kā lākou mau kikowaena.
ʻO ka hoʻonani
I ka manawa like, hoʻomaka mākou e wehe i ka insulation. Ua lawe nā luna lawelawe i kahi papa inoa, a nānā mākou i kā mākou pūnaewele āpau. E hoʻokaʻawale iā lākou i nā hui like, a ma nā kikowaena pono i hoʻohui ʻia nā hui, no ka laʻana, e hōʻole. I kēia manawa ua pau ka hoʻokaʻawale ʻana i nā lula_deny o ka hana, akā ʻaʻole i ka hana ponoʻī.
Hana wikiwiki a maʻalahi ka papahana: wehe mākou i nā ACL a pau mai nā kikowaena, wehe i ka lako lako, a hoʻemi i ka helu o nā VLAN kaʻawale.
Ka hoomalu pono
Ma mua, ua loaʻa iā mākou kahi hoʻomaka kūikawā i hōʻike ʻia i ka wā i hoʻololi lima ai kekahi i kahi lula ahi. Ke kākau nei au i kahi linter nui no ka nānā ʻana i nā lula firewall, paʻakikī. Hoʻomalu ʻia ka pono e BEFW. Hoʻoikaika ʻo ia i ka loli ʻole o nā lula āna e hana ai. Inā hoʻololi kekahi i nā lula o ka pā ahi, e hoʻololi ia i nā mea a pau. "Hoʻonohonoho koke wau i kahi koho i hiki iaʻu ke hana mai ka home" -ʻaʻohe koho hou.
Mālama ʻo BEFW i ka ipset mai nā lawelawe a me ka papa inoa ma befw.conf, nā lula o nā lawelawe ma ka kaulahao BEFW. Akā ʻaʻole ia e nānā i nā kaulahao ʻē aʻe a me nā lula a me nā ipset ʻē aʻe.
Ka pale pōʻino
Mālama mau ʻo BEFW i ka mokuʻāina maikaʻi hope loa ma ka state.bin binary structure. Inā hewa kekahi mea, e hoʻi mau i kēia state.bin.
He ʻinikua kēia e kūʻē i ka hana Consul unstable, inā ʻaʻole ia i hoʻouna i ka ʻikepili a i ʻole kekahi i hana hewa a hoʻohana i nā lula ʻaʻole hiki ke hoʻohana ʻia. No ka hōʻoia ʻana ʻaʻole i waiho ʻia me ka ʻole o kahi pā ahi, e ʻōwili hou ʻo BEFW i ka mokuʻāina hou inā loaʻa kahi hewa i kēlā me kēia pae.
I nā kūlana koʻikoʻi, he hōʻoia kēia e waiho ʻia mākou me kahi pā ahi hana. Wehe mākou i nā pūnaewele hina a pau me ka manaʻolana e hele mai ka luna e hoʻoponopono. I kekahi lā e hoʻokomo wau i kēia i loko o nā configs, akā i kēia manawa he ʻekolu mau pūnaewele hina: 10/8, 172/12 a me 192.168/16. I loko o ko mākou Kanikela, he mea nui kēia e kōkua iā mākou e hoʻomohala hou.
Demo: i ka wā o ka hōʻike, hōʻike ʻo Ivan i ke ʻano demo o BEFW. ʻOi aku ka maʻalahi o ka nānā ʻana i ka hōʻike . Loaʻa ke code kumu demo .
Pitfalls
E haʻi wau iā ʻoe e pili ana i nā pōpoki i loaʻa iā mākou.
ipset hoʻohui i kahi 0.0.0.0/0. He aha ka hopena inā hoʻohui ʻoe i ka 0.0.0.0/0 i ka ipset? E hoʻohui ʻia nā IP āpau? E loaʻa anei ka ʻike pūnaewele?
ʻAʻole, e loaʻa iā mākou kahi pōpoki e uku ana iā mākou i ʻelua hola o ka manawa hoʻomaha. Eia kekahi, ʻaʻole i hana ka bug mai ka makahiki 2016, aia ia ma RedHat Bugzilla ma lalo o ka helu #1297092, a loaʻa iā mākou ma ka pōʻino - mai ka hōʻike a ka mea hoʻomohala.
I kēia manawa he kānāwai koʻikoʻi ma BEFW kēlā 0.0.0.0/0 lilo i ʻelua helu wahi: 0.0.0.0/1 и 128.0.0.0/1.
ipset restore set < file. He aha ka hana a ipset ke haʻi aku ʻoe iā ia restore? Manaʻo ʻoe e hana like ia me nā iptables? E hoʻihoʻi hou i ka ʻikepili?
ʻAʻohe mea like - hana ia i kahi hui, a ʻaʻole hele nā ʻōlelo kahiko i nā wahi āpau, ʻaʻole ʻoe e ālai i ke komo.
Ua loaʻa iā mākou kahi pōʻino i ka wā e hoʻāʻo ai i kahi kaʻawale. I kēia manawa aia kahi ʻōnaehana paʻakikī - ma kahi o restore mālama ʻia create temp, laila restore flush temp и restore temp. I ka pau ʻana o ka swap: no ka atomicity, no ka mea inā ʻoe e hana mua flush a i kēia manawa hiki mai kekahi ʻeke, e hoʻolei ʻia a hele hewa kekahi mea. No laila, aia kekahi ʻano kilokilo ʻeleʻele ma laila.
consul kv loaa -datacenter=okoa. E like me kaʻu i ʻōlelo ai, manaʻo mākou e noi ana mākou i kekahi mau ʻikepili, akā e loaʻa iā mākou ka ʻikepili a i ʻole ka hewa. Hiki iā mākou ke hana i kēia ma o ke Kanikela ma ka ʻāina, akā i kēia hihia e maloʻo nā mea ʻelua.
ʻO ka mea kūʻai aku ke Kanikela kūloko he wīwī ma luna o ka HTTP API. Akā, kau wale ia a pane ʻole iā Ctrl+C, a i ʻole Ctrl+Z, a i ʻole kekahi mea, wale nō kill -9 ma ka console aʻe. Ua ʻike mākou i kēia i ke kūkulu ʻana i kahi hui nui. ʻAʻole naʻe i loaʻa iā mākou kahi hoʻonā; ke mākaukau nei mākou e hoʻoponopono i kēia hewa ma ke Kanikela.
ʻAʻole pane mai ke alakaʻi kanikela. ʻAʻole pane mai kā mākou haku i ka kikowaena data, manaʻo mākou: "E hana paha ka algorithm reelection i kēia manawa?"
ʻAʻole, ʻaʻole ia e hana, a ʻaʻole e hōʻike ʻia ka nānā ʻana i kekahi mea: E ʻōlelo ke Kanikela aia kahi papa kuhikuhi kūpaʻa, ua loaʻa kahi alakaʻi, maikaʻi nā mea a pau.
Pehea mākou e hana ai me kēia? service consul restart i ka cron i kēlā me kēia hola. Inā loaʻa iā ʻoe nā kikowaena 50, ʻaʻohe mea nui. Ke loaʻa iā 16 o lākou, e hoʻomaopopo ʻoe i ka hana ʻana.
hopena
ʻO ka hopena, ua loaʻa iā mākou nā pōmaikaʻi penei:
- 100% ka uhi ʻana o nā mīkini Linux āpau.
- Wikiwiki
- ʻOtometi.
- Ua hoʻokuʻu mākou i nā ʻenehana a me nā ʻenekini pūnaewele mai ka hoʻoluhi ʻana.
- Ua ʻike ʻia nā mea hiki ke hoʻohui ʻia i ka palena ʻole: ʻoiai me Kubernetes, ʻoiai me Ansible, a me Python.
Минусы: Kanikela, kahi e noho ai kakou i keia manawa, a me ke kumu kuai nui loa o ka hewa. ʻO kahi laʻana, hoʻokahi ma 6 pm (manawa nui ma Rūsia) ke hoʻoponopono nei au i kekahi mea ma ka papa inoa o nā pūnaewele. Ke kūkulu nei mākou i ka insulation ma BEFW i kēlā manawa. Ua hana hewa wau ma kahi, me he mea lā ua hōʻike wau i ka mask hewa, akā hāʻule nā mea āpau i ʻelua kekona. Hoʻomaka ka mākaʻikaʻi, hele mai ke kanaka kākoʻo ma ka hana: "Loaʻa iā mākou nā mea āpau!" Ua hina ke poʻo o ka ʻoihana i kona wehewehe ʻana i ka ʻoihana i ke kumu o kēia hana.
He kiʻekiʻe loa ke kumukūʻai o ka hewa i hiki mai ai mākou me kā mākou hana pale paʻakikī. Inā hoʻokō ʻoe i kēia ma kahi kahua hana nui, ʻaʻole pono ʻoe e hāʻawi i kahi master token ma luna o ke Consul i nā mea a pau. E pau ino ana keia.
Ka uku Ua kākau wau i ke code no 400 mau hola wale nō. Hoʻohana koʻu hui o 4 poʻe i 10 mau hola o ka mahina ma ke kākoʻo no kēlā me kēia. Ke hoʻohālikelike ʻia me ke kumukūʻai o nā pā ahi o nā hanauna hou, ʻaʻole ia.
Nā papahana. ʻO ka hoʻolālā lōʻihi ka loaʻa ʻana o nā kaʻa ʻē aʻe e pani ai a hoʻokō i ke Kanikela. Malia paha ʻo Kafka a i ʻole kekahi mea like. Akā i nā makahiki e hiki mai ana e noho mākou ma ke Kanikela.
Nā hoʻolālā koke: hoʻohui pū me Fail2ban, me ka nānā ʻana, me nā nftables, hiki paha me nā puʻupuʻu ʻē aʻe, metrics, nānā kiʻekiʻe, optimization. ʻO ke kākoʻo Kubernetes kekahi ma kahi o nā hoʻolālā, no ka mea i kēia manawa he nui nā pūʻulu a me ka makemake.
Nā mea hou aʻe mai nā papahana:
- ʻimi i nā anomalies i ke kaʻa;
- hooponopono palapala 'āina;
- Kākoʻo Kubernetes;
- ka hui ʻana i nā pūʻolo no nā ʻōnaehana āpau;
- Pūnaewele-UI.
Ke hana mau nei mākou i ka hoʻonui ʻana i ka hoʻonohonoho, hoʻonui i nā metric a me ka hoʻonui ʻana.
Hui pū i ka papahana. Ua ʻoluʻolu ka papahana, akā naʻe, ʻo ia ka papahana hoʻokahi kanaka. E hele mai i a hoʻāʻo e hana i kekahi mea: hoʻokō, hoʻāʻo, hōʻike i kekahi mea, hāʻawi i kāu loiloi.
I kēia manawa ke hoʻomākaukau nei mākou no , e hana ʻia ana ma ʻApelila 6 a me 7 ma St. Petersburg, a ke kono nei mākou i nā mea hoʻomohala o nā ʻōnaehana kiʻekiʻe. . Ua ʻike mua ka poʻe ʻōlelo akamai i ka mea e hana ai, akā no ka poʻe hou i ka haʻi ʻōlelo, paipai mākou ma ka liʻiliʻi . ʻO ke komo ʻana i ka ʻaha kūkā ma ke ʻano he haʻiʻōlelo he nui nā pono. Hiki iā ʻoe ke heluhelu i nā mea, no ka laʻana, ma ka hopena .
Source: www.habr.com