{"id":41051,"date":"2020-02-05T21:28:58","date_gmt":"2020-02-05T18:28:58","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/consul-iptables-3"},"modified":"2020-02-05T21:28:58","modified_gmt":"2020-02-05T18:28:58","slug":"consul-iptables-3","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/consul-iptables-3","title":{"rendered":"Consul + iptables = :3","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

In 2010, the company Wargaming<\/a><\/noindex> had 50 servers and a simple network model: backend, frontend, and firewall. The number of servers grew, the model became more complex: staging, isolated VLANs with ACLs, then VPNs with VRFs, VLANs with ACLs on L2, VRFs with ACLs on L3. Feeling overwhelmed? It gets more interesting.<\/p>\n

When the number of servers reached 16,000, it became impossible to work without tears with such a diverse array of segments. So, another solution was devised. They took the Netfilter stack, added Consul as a data source, resulting in a fast distributed firewall. This replaced ACLs on routers and was used as both an external and internal firewall. To dynamically manage the tool, they developed the BEFW system, which was applied everywhere: from managing user access in the product network to isolating network segments from one another.<\/p>\n

\"Consul
\n
\nHow all this works and why you should take a closer look at this system will be explained by Ivan Agarkov<\/strong> (annmuor<\/a><\/noindex>) \u2014 head of the infrastructure security group in the Maintenance division at the Minsk development center of the company. Ivan is a fan of SELinux, loves Perl, and writes code. As the head of the security group, he regularly works with logs, backups, and R&D to protect Wargaming from hackers and ensure the operation of all gaming servers in the company.
\n<\/a><\/noindex>
\n

\r\n \"Play\r\n
<\/div>\r\n <\/div><\/center><\/p>\n

Historical Background<\/h2>\n

\nBefore explaining how we did this, I will recount how we got here in the first place and why this was necessary. To do this, let\u2019s go back 9 years: 2010, when World of Tanks had just launched. Wargaming had about 50 servers.<\/p>\n

\"Consul
\nThe growth chart of the company\u2019s servers.<\/em><\/p>\n

We had a network model. For that time, it was optimal.<\/p>\n

\"Consul
\nThe network model in 2010.<\/em><\/p>\n

The bad guys who want to break us live on the frontend, but there\u2019s a firewall in place. There\u2019s no firewall on the backend, but there are 50 servers, and we know all of them. Everything works well.<\/p>\n

Within 4 years, the server park grew 100 times, to 5,000. The first isolated networks appeared \u2014 staging: they cannot access production, and they often ran potentially dangerous content.<\/p>\n

\"Consul
\nThe network model in 2014.<\/em><\/p>\n

By inertia, we continued to use the same hardware, and all work was conducted on isolated VLANs: ACLs were written to the VLANs that allow or deny certain connections.<\/p>\n

In 2016, the number of servers reached 8,000. Wargaming acquired other studios, and additional partner networks emerged. They seem to be ours, but not quite: the VLAN often doesn\u2019t work for partners, requiring the use of VPNs with VRF, complicating isolations. The mix of ACL isolations grew.<\/p>\n

\"Consul
\nThe network model in 2016.<\/em><\/p>\n

By early 2018, the fleet had grown to 16,000 machines. There were 6 segments, and we didn't count the others, including those holding financial data. Container networks (Kubernetes), DevOps, and cloud networks connected via VPN, like from IVS, appeared. There were too many rules\u2014it was painful.<\/p>\n

\"Consul
\nThe network model and isolation methods in 2018.<\/em><\/p>\n

For isolation, we used: VLANs with ACLs at L2, VRF with ACLs at L3, VPNs, and much more. Too much.<\/p>\n

Issues<\/h2>\n

\nEveryone operates with ACLs and VLANs. What\u2019s the problem? Harold, hiding the pain, will answer this question.<\/p>\n

\"Consul
\n
\nThere were many issues, but only five were widespread.<\/p>\n