α’αα»ααα½ααααΈ αααα»αα
ααα
αααααααααΆαα½αα’αααααΌααααα·ααααααααααααα»ααααα»αααΆαααα‘αΎα αα·αααααΎααααΆααααααΆαααα AWS EKS (Elastic Kubernetes Service) αααααΆαααα»αααΊααα Windows α¬ααΆααΆαα’αααΈααΆααα·αα’αΆα
αα
αα½α
ααααΆαααααΎααααΆααααΆ αα·αααα α»ααααααΆαααααΎααα
αααα»ααα»ααααααααα AWS αααααΆααα’αααααΆαααααα αααα
αΆααα’αΆααααααααΎααααΆααααααααααααΆααα§ααααααααα»αααΈαααΌ ααΌααα
αααααααααΆα
αααα»αααΉαααΆ Windows containers αα·ααααααΆαααααΆαααααααααα·αααα α αΎαααΆαααα»ααααα·α
ααΆαααααααααΎααΆ ααα»αααααααα»ααα
αααααααα
α
α·ααααααααα’αααααααα αααααΆαααΆαα’αααααααΈαααΈαα
ααΎ HabrΓ© αα
ααΎ kubernetes αα·α Windows α αΎααα
ααααΆαααα»αααααααααα
ΠΠ°ΡΠ°Π»ΠΎ
ααΆααΆααα’ααααΆαα αΆααααααΎααα ααααααααΆααααΌαααΆααααααα α α·αααααααΆααααααΌαααααΆαααααα αααα»ααααα»αα αα»αααααααΎααα kubernetes αααααΆ 70% Windows αα·α 30% Linux α α αααααααααααααα ααααΆααααααα AWS EKS ααααΌαααΆαα αΆαααα»αααΆααΆαααααΎααα½ααααα»αα αααααααααΎααααα’αΆα ααααΎααΆαα αα αΌααααααααααΈ 8 αααα»ααΆ ααααΆα 2019 AWS EKS Windows αααα·ααα αααα»αααΆαααΎαααΆααΆααΆααα αααα»αααΆαα αΆααααααΎαααΆαα½αααΆ αααα 1.11 α αΆαααααα kubernetes ααααΌαααΆαααααΎααααΆαααα ααΈααα ααα»αααααααα»αααΆααααααα α α·ααααα·αα·αααααΎαααΆ α αΎαααΎαααΆααΎααααΆαααααααααα·ααααα»αααααΆααααΆαααΆ ααΆααΎααΆααααΎαααΆαα¬α’αα ααΆααααααα ααΌα αααααΆααΆαααααααααΆα α’αααα ααΆααΆαααα α»αααΆαα½αααΉαααΆαααααααααΆααα pods α αα αααααααααα’αααα αΆααααΆααααααααΎαααααΆαααα ip ααΆααααα»αααΈ subnet ααΌα ααααΆαα ααΉα windows worker node α
ααΌα ααααααΆααααΌαααΆααααααα α α·αααααααααα ααααΆαααααΎααααΆαα AWS EKS αααα»αααΆααααα α·αααααα αααααααααΆαααααα½αααααααΎααα ααΎ kubernetes αα ααΎ EC2 ααΌα ααααΆ ααΆαααααΎααααααααααΌααααααΆα’αααΈαα»αααααΆαααΆααα’αα αα·α HA αααα½αααΎαααΆαααα CloudFormation α
Amazon EKS Windows Container Support α₯α‘αΌααααααΆαααΆααΌαα
ααα Martin Beeby | αα ααααααΈ 08 αααα»ααΆ ααααΆα 2019
αα»ααααααααααα»αααΆααααααααααααααΌαα½ααα
CloudFormation αααααΆααα
αααααααααΆαααααα½ααααααααα»α αααα»αααΆαααΎαααααααΆααααα
ααΆααΆααα·αααΆαα αααα»αααΆααα»αααΆαααΆαααΆααα’αααααααααα»ααα½αα‘αα α αΎαα
αΆααααααΎααα·ααααΆααΈα’αααΈααααα½αααααΆαααααΎαααααΆαα GA αα·ααααααααα’αααΈααααααααΆαααΆαααααΆααααααΌαααΆαα½αααΉαααΆαααΎαααΆααΆααΆαααα ααΆα AWS ααααΎααΆαααα’ ααααΎαα
αα
α»ααααααααΆαααΌαααΆααααααΆαα windows worker node αα
αααα 1.14 ααααΌα
ααΆ cluster αααα½αααΆ αααα 1.14 αα
αααα»α EKS α₯α‘αΌααααααΆαααα windows node α ααααααααα Public Preview αα
ααΆααα½ααααα αΌαα ααααα EKS αα αααα»α VPC αα·ααααααΆααααα αα α»αααααα
αα αααα»ααααααααΆααα’αα αα αααα»ααααααααΆααααΆαααΎαα ααΎααα ααααΈαααααΆα ααααΌα ααΆαα αααα»αα―αααΆα ααΆααααΌαααΆαααααΎα±ααααΆαααααααΆαα αααααααΆααααα§αααααααααΎααααΆαα eksctl αααααΆααααααα·αααα· α¬ααΆαααα CloudFormation + kubectl αααααΆααααΈααααΎαααααααΆαααααΆααΆααααα αααα»α Amazon ααααΌα ααΆααΆααααααΎα VPC ααΆα ααααα‘αααααααΆααα αααααααααΈα
αααααΎαααααα·ααααααα·αααααααΆααααα»αααααΆα αααΎααα ααΈαα½α VPC ααΆα ααααα‘ααααΆααααααΆααΆαα αααΆααααααααααααΆααααΆαα αααΆαααααααΆ + α ααΆα ααααααααΆαααα VPC αα αα α»ααααααααααα’αααα ααΎβα’αααβαααβααΆαβα αααααΆαα ααΆααααααααβαααβαααααβαα½α βααΆβααααα βαααα»α AWS ααΆαα½αβααΉαβααααΈ AWS α αααΎαβααααβαα½ααα VPC αααααΆαβαα ααΆααΆαβααααΌα α αααβα ααα αΌα αα·αβααααΎβα’αααΈβαααα? ααΆααΆααα·αααΆαα α’ααααα·αα ααααααα α¬ααααΎα‘αΎααα·αααΆααα’ααααααα α αΎαα’αααααααΌααααα αΌαα ααααα EKS ααααΈαα αααα»αα αααααΆαα ααΆαααααααααααααΆααα αα α»αααααα αααααααΎ VPC αααααΆαααααΆαα α αΎαααΆαα αααΎααααααΎααααααΆαααααααΈαααααΆααα αααααα
αααα»αααααΈαααααααα»α ααααΌααααααααΌαααΆαααααΎαααΎα αααα»αααΆαααααΎ VPC αααααΆαααααΆαα αααααααααααααΆαααααΆααΆααα 2 αα·ααααααΆαααα―ααα 2 αααααΆααα
αααααααααΈ ααΆααΆααα·α α
αααΆααααΆααα’ααααααΌαααΆααααααα·α
αΆαααΆααααα
ααΆαα―αααΆα
ααΆααααΆααααααααααα½ααααααα αα·αααΆαααααΆαααααααααα αααα»ααααααΆαααααΆααΆααααααααααΎ EIP ααα
eksctl αααααΉα CloudFormation
αααα»αααΉαααααΎααΆαααααα»αααααΆαα ααααααα»αααΆαααΆααααααα·ααΈααΆααααΈαααααΆαααΆαααααααΆαα ααααα αααα»αααααΈααΆααααΈαααΌαααΆαααΊααΌα ααααΆα
αααα»αααΉααααα αΆαα§ααΆα ααααααααααΎ eksctl ααΈααααααααααΌααα ααΈαααααΉαααααΈααΆαα αααααααΎ eksctl ααΆαααααααΆαα αααααααΆ 3 ααα αΆαα
1. ααΎααααααΎαα αααααααααααα½αααΆααααΆαα + ααααΆααααααααααΈαα»α ααααααααααααΉααααα αααααααααααα»αααΊααα αα·αα§ααααααααααΆ vpc-controller αααααΆαααΆαααΆααΌα ααααΆα
eksctl create cluster
--name yyy
--region www
--version 1.14
--vpc-private-subnets=subnet-xxxxx,subnet-xxxxx
--vpc-public-subnets=subnet-xxxxx,subnet-xxxxx
--asg-access
--nodegroup-name linux-workers
--node-type t3.small
--node-volume-size 20
--ssh-public-key wwwwwwww
--nodes 1
--nodes-min 1
--nodes-max 2
--node-ami auto
--node-private-networking
ααΎααααΈααΆαααααααΆααα VPC αααααΆαααααΆαα ααααΆαααααααααΆαα id αα subnets ααααα’ααα α αΎα eksctl ααΉαααααα VPC αααα½αα―αα
ααΎααααΈααΆααΆααΆααααΆααααααααααααα’αααααααΌαααΆαααΆααα±ααααααΎαααα αααααΆαααα―ααα α’αααααααΌααααααΆαα --node-private-networking αααααΆαα nodegroup α
2. ααΎαααα‘αΎα vpc-controller αα αααα»αα αααααααααααΎα αααααΉαααααΎαααΆαααααΆααααααααααααααΎα ααΆααα ααα½αα’αΆααααααΆα IP α₯ααα·ααααα ααααΌα ααΆα ααα½α ENIs αα ααΎ instance αααααα αα·αααααΆα ααα
eksctl utils install-vpc-controllers --name yyy --approve
3. αααααΆααααΈαα»αααααααααααααα’αααααΆαααααΎαααΆαααααααααααα ααΎααααΆααα’αααααααΎααΆαααΈαα»α ααααα’ααα αα½αααΆαα vpc-controller α’αααΈααΆααα’ααααααα αααααΊαααααΎααααα»ααα½ααααααΆαα½αααααααααΈαααΌα
eksctl create nodegroup
--region www
--cluster yyy
--version 1.14
--name windows-workers
--node-type t3.small
--ssh-public-key wwwwwwwwww
--nodes 1
--nodes-min 1
--nodes-max 2
--node-ami-family WindowsServer2019CoreContainer
--node-ami ami-0573336fc96252d05
--node-private-networking
αααααΆααααΈααααΆααααααα’αααααΆαααααΆααααααααααααα α αααααααααα’ααα α αΎαα’αααΈαα αΆααααΌα ααΆααα’ ααΆαααα·ααα αααα»αααααΆαααΆααα½α ααΆαα ααα»αααααα·ααααααα
ααα α»ααα αααα»α vpc-controller
ααααα·αααΎααΎαααααΆααΆαααααΎαααΆα pods αα ααΎ windows worker node ααΎαααΉαααα½αααΆαααα α»αα
NetworkPlugin cni failed to teardown pod "windows-server-iis-7dcfc7c79b-4z4v7_default" network: failed to parse Kubernetes args: pod does not have label vpc.amazonaws.com/PrivateIPv4Address]
ααααα·αααΎααΎαααΎαα±ααααΆαααααααΈααααα ααΎαααΎαααΆ instance ααααααΎααα αααα»α AWS ααΎααα ααΌα αααα
α αΎαααΆαα½αααααΌα
αααα
ααΈαααααΆα
αααΆααααΆααααΆ vpc-controller αα·αααΆαααααααααααααααααΆαααα ααα»αααα½αα
ααα½α α αΎααα·αα’αΆα
ααααααα’αΆααααααΆα IP ααααΈαα
instance ααΌα
αααα pods α’αΆα
ααααΎααΆααΆαα
αααααΎααααααα ααα»αααα vpc-controller pod α αΎααααααΆα’αααΈαααααΎαααΎαα
αααααα ααα» kubectl -n kube-system
I1011 06:32:03.910140 1 watcher.go:178] Node watcher processing node ip-10-xxx.ap-xxx.compute.internal.
I1011 06:32:03.910162 1 manager.go:109] Node manager adding node ip-10-xxx.ap-xxx.compute.internal with instanceID i-088xxxxx.
I1011 06:32:03.915238 1 watcher.go:238] Node watcher processing update on node ip-10-xxx.ap-xxx.compute.internal.
E1011 06:32:08.200423 1 manager.go:126] Node manager failed to get resource vpc.amazonaws.com/CIDRBlock pool on node ip-10-xxx.ap-xxx.compute.internal: failed to find the route table for subnet subnet-0xxxx
E1011 06:32:08.201211 1 watcher.go:183] Node watcher failed to add node ip-10-xxx.ap-xxx.compute.internal: failed to find the route table for subnet subnet-0xxx
I1011 06:32:08.201229 1 watcher.go:259] Node watcher adding key ip-10-xxx.ap-xxx.compute.internal (0): failed to find the route table for subnet subnet-0xxxx
I1011 06:32:08.201302 1 manager.go:173] Node manager updating node ip-10-xxx.ap-xxx.compute.internal.
E1011 06:32:08.201313 1 watcher.go:242] Node watcher failed to update node ip-10-xxx.ap-xxx.compute.internal: node manager: failed to find node ip-10-xxx.ap-xxx.compute.internal.
ααΆαααααααααα ααΎ Google αα·αααΆαααΆααα ααα’αααΈααααα αααααΆαααΆααααααα ααααΆαααααΆααααΆααααΆαα αΆααααα α»ααααααααα α‘αΎααα α¬αα·αααΆααααΆααααα αααααα αΆαα ααΎααΆ αααα»αααααΌααααα·αααΈαααααΎαααααααα½αα―αααΆαα»ααα·αα ααΏαααααΌααααααΉαααΎαααΊααΆ αααα ααααΆ vpc-controller αα·αα’αΆα αααααααΆα ip-10-xxx.ap-xxx.compute.internal α αΎαααΆααααααΆ ααΌα ααααα αΎαααΆαααα α»αααΎαα‘αΎαα
ααΆα/α αΆα ααΎαααααΎαααΆαααΈααα DNS ααααΆαααααα½ααα αααα»α VPC α αΎαααΆαααααΆαααααΎααα·αααααΎ Amazon αα ααΌα ααααααΌααααΈααααΆααααααΌααααααααα·αααααΌαααΆαααααααααααΆααααα ap-xxx.compute.internal ααααααα αααα»αααΆαααΆααααααααααΎαααα α αΎαααΆαα·αααΆαααααΌααααααααα αααα ααααΆααΆαααααΎαααααααααα·αααα’αΆααα ααΌα ααααα αΎα αα αααααααΆααααααααΆαα½ααααα½ααα αα ααααα αααα»αααΆαα α»αα αΆααααΉααααα·ααααααα½αααα
αααααΆααα·αααΆααααα·αα’αααΈααα αααα»αααααα·αα»αααΆααα’ααααααΌαααΆααααααΎαα‘αΎαααα eksctl αααα½αααΆααααΆαα ααΌα αααααα·αααΆαααΆαααααααα’αααΈααααααΆαααααααΆαααααααααα½ααααα ααΆααΆαααααΌαααααααΉαααααΌα ααΆα ααΈααα ααΆαα αΌαααααΎα’ααΈαααΊαα·αααΆαα½αααααΆααααααααααααΆααα ααΈααααααα
ααΎαααΈααααα ααα ααααα·αααΎα’αααααΆαααααααΆαααααΆαααααααααα αααααΆαααααΆααΆαααααααα·αααααΎ -node-private-networking ααααΆαααααααααΌαααΆαααααΎαα αα α»ααααααααΆαααααΆααααα vpc-controller α αΎαα’αααΈαααααΎαααΆαααΌα ααΆαααΆαααΆα‘α·ααΆα
ααΆααααααΎαααΈαα
- ααααααααΆ α αΎααααα αΆααα αΌααααααααΆααααΆαααααααΆα’αααΈααα α»αααααα αααα»α AWS α αΎααα½ααααα½ααα»αααΆ α αΎααααααΆααααα’αααα’αΆα ααααΎ AWS EKS Windows ααααα»ααααα·ααΆα ααααααα½αααααΎαααα ααααααΆααα αααα»α GA (8 ααααααΆαααααααα»ααα αα ααααααααα’αααααααα) ααα»αααααΆα αααΎααααα ααααΆααΉα ααΎαααΆαααααΌαααΌα ααααΆααΉααααα»αα
- ααααααα ααΆααααααααααα½α AWS α αΎαααααΆαααα½αααααΈααααΉαααΆααααααα αΆααΆαα½αααΉααααααα ααα»ααΆααααΌαααΈααααααα·αααΈ α αΎααααα αΆαα±αααα½αααααΉαααΆααααΆαααααααααα½ααααα·αααααΎαααΆααα αααααααΎ VPC αα·ααααααΆαααααααα’ααα ααΆαα·αααααααααΆααααααΆαα’αααΈαααααΎαααΆαααΆαααΆαααααααααα’αΆααΈααααααα α’ααααα½αααααααΎ αααΆαα αα ααΆαααααα :)
ααααΆααααααααΆαα½ααα·ααααα AWS
αααααΆααααααΎααααα»ααααα ααΎαα·ααααα αααα»αα ααα‘αααααΎαααΎαααααΎααααααααα»αααΆααααααα ααααα - α’ααΈααα α¬ααααααααααααα½α ααΆαααααααααΎαααα αα½αααα’αΆα ααααΎαα’ααααααααΆααααΈααΈαααΈααααααΆααα’αα αααααΈααΆααΆααα·αααΆαααα»ααααααααααα»αααΆαααΆαααααααααα - ααααααααα α»ααααααααααα ααα ααΆααααααΆααΆαααΆαααααΎααααααα»ααααααα <12 αααα α αΎαα αΆααααΆααααΈαααααΆααααα½αα’αΆααΈαααααααΆαααΆαααΆαααα 24/7 αααα»ααααααΉαααΆααα’αααα»α ααα»ααααααΆααΆαααααααααΆαααΌα ααΆααααα
αααα»ααααααααααα»αααααΌαααΆααα»αα ααααααα·αααΆααααααα αΆααααΈαααααα»ααααα αΌααααααααα αααα αααααΆαααααααα»αααΆααααααα α α·αααααααααα ααΆαααα½αααααααααα α αΎαααααΎαααΎααααααΎαααααΎαααααΆαααααα αααααΆααααΈαααα αΆαααααααααααΈ Harshad Madhav ααααΌαααΆααααααΆααα±αααααα½ααααα»α α αΎααααααΆααααααΆααΆαα αΆααααααΎα...
ααΎαααΆαααααΆααααα α»αααΆαα½αααΆααΆαα’ααΈαααΊαα·ααααααα 3 ααααααΆααααααΆ ααααααααααα ααα» ααΆαααΆαααααααΆαα αααααααΌα ααααΆαα αααα»ααααααΈααα·ααααα AWS ααΎααααΈααααΆααααΆααααα αΆ αααααΎαα αααααα‘αΎααα·ααα ααΎααααααααααααα»α α αΎαααΌα αααααα ααΎααα ααΏααααα½αααααααααΎαααΆααααααααΊααααΈ αααααα ααα»ααΆα αααΆααααΆααααΆ Resol αα·αααααΎαααΆαααααααααααΆααααα»ααααα AWS ααααααα»αααΆααααααα’αααΈααΆαααΎ α αΎα Harshad Madhav ααΆααα»αα±αααααα»ααααααΎαααΆααααααΌααααα αααα αααααααΆααααΆααΎαααααΎ DNS ααααΆαααααα½α α αΎααααα’αΆα ααΆαααα αΆα
αααααΌααααα
ap-xxx.compute.internal -> 10.x.x.2 (VPC CIDRBlock)
amazonaws.com -> 10.x.x.2 (VPC CIDRBlock)
αααα αΎαααΆα’αααΈαααααΆαααααΎ αα½αααααααΆαααααααα»ααα α αΎαα Harshad Madhav ααΆαααααααααα‘αααααα·αααΎααααΈαα·αα·αααααΎαααΆ α αΎαααΆαα½αααααααΎαααΆα ααα»αααααα αααααααααΆααα·αααΆααα½αα’αααΈααΆααα’ααα
αααααΆααααααΆαααΆαααααΆαααααααΆααααααΆαα½ααα·ααααα 2 ααΆααααα ααααΆααααααΆααααα ααααΈααΆααααα ααΆαααααααααΆααααααΆα ααααΈαααα»αααααΆα ααΈααΈαα αααΆαααααα½ααααααααααααα»ααααααααααΎααααααααααααααΆαααααΆααααα α»α ααααΎαααααα ααα» αααααΎαα αααααααΆααααααΆααα αααα»α α α»ααααα ααααΆααααααΆαααααα·ααΆαααα’ ααΆααααΎαααΆααααααΆαααααα»α αα ααΈααααααα»αααΊαααα»αααααΎα’αααΈααααααααΆαααΆααα αΆα α αα αααα»αα―αααΆαααααΌαααΆαα αΎαα’ααααα·αα’αααααΉαααα½αααΆαααααααα
ααααααα»αααΆααα»αα±ααααΆααα αΆαα ααααααα½ααα α αΎαα αΆααα±ααα’αααααααααα αααα»ααααααααααα»α ααααα·αααΎα’ααααα·αααΉαααΆααααΌαααααΎααααα αΆαα α―ααΆα
α α»αααααα
αα ααααααΈααΈ αα·αααααααααΈ Arun B. ααααΌαααΆαα αΆααα±αααααααα»α α αΎαααΆααααΈααααΌαααααΆαααααΆααααααααΆαα½αααΆαα ααΆα αααΆααααΆααααΆααααα·ααααααΆαα·ααααααα»ααααΆαα 3 ααΆαααααααα ααΆααααΆαα’αΆααααααααα·ααΆααααΌα α αΎαααΆαααααΎαα»αααααΆααα±αααααααΌααααααα ααα»αααααααΎααααααΈαααααΆαααααα½αααααααΆαααα ααΎ ps1 αααααΆααα ααΎ github ααααααΆααα αααααααΌαααΆαα’αα»ααααααΆαααααααααααααΆαααααΎα‘αΎααα·αααΆααα’ααααααΆααααααΎαα ααααα ααααααααΆααααααααΆ ααΆααααααΌααααααα ααα» ααα»αααα Arun B. αααα»αααααΎααααΎααααα»ααα·ααα ααααΉαααααΌα ααααα·αα·α ααααααααααα½αααααα½ααααααα»αα
ααΎαα αααααΆαααααΎαααΆααααα ααα»α ααααΆαααΎαααααΎαααΆα -stderrthreshold=debug αα αααα»α vpc-controller αααααα½ααα α αΎαααΎααΆαα’αααΈααΎαα‘αΎααααααΆααααα? ααΆβααΆαβαα·αβααΆααβααΆβαα·αβααααΎαβααΆαβ) pod ααααΆααβααβαα·αβα αΆααβααααΎβαβααΆβαα½αβααΉαβαααααΎαβαααβ, αα -stderrthreshold=info ααααΎαααΆαα
ααΎαααΆααααα αααα ααΈαααα αΎα Arun B. ααΆααα·ααΆαααΆααΆααααΉαααααΆααΆααααααΎαααα αΆααααααααα»αα‘αΎααα·αααΎααααΈααα½αααΆαααα α»αααΌα ααααΆα αα αααααααααΆαααααα»αααα½αααΆαααΆαααααΎαααααΈ Arun B. ααΆαααα·αααΆαααααααα ααααααΈααααα ααα»ααααααΆααααααααΌααααα½ααα·αα·ααα vpc-controller αααααα½ααα α αΎαααΆαααααΎααααααααααααΆαα α αΎαα ααα»α’αααΈααΆαααΆααΆαα·αααααΎαααΆαα
ααΌα
αααα ααααα·αααΎα’αααααααΎααΆααΆαααααΌαααααΆαααα
αααα»α VPC ααααα’ααα αααααΆαααααΆαααΎα ααΆαα·αααΆαααααΆααααααααΆαα½ααααααΆαααα
αΆαααΆα
ααα αααα
αΆαααΆα
ααααααΆαα vpc-controller αααα»αααααΈαααααααΆαααααΆααΆααα ααΆααΆαααΆααΆαααααΌαααααΆαααααα½αα αααααΆααααΆααα
αααααΆααααααααααΆααααααααααααΆααααΆααΆαααααΌαααααΆαα½ααααααΆαααα αΆαααΆα α αα·ααααααΎααααα»αα‘αΎααα·α α’αααΈαααααΎαααΆααααΆαααα’α₯αααα ααα
αααα»ααααααΉαααΆ Arun B. αα·αααΆααΉαααΆαααΆαααα’αααΈααα α»αααααα
ααΆααα’αααα’αα·αααααα EKS α αΎαααΎαααΉαααΎαααααααααΈαα vpc-controller αααα’αααΈαααΉαααααΎαααΆαα
ααααΈαααα’ααα αα
αα
α»ααααααααααα
α»αααααααααα»αααΊα 602401143452.dkr.ecr.ap-southeast-1.amazonaws.com/eks/vpc-resource-controller:0.2.1
ααΆααααα αΆαααα
ααΌαα’ααα»ααααα’αααααΆααα’ααααααΆαααααΆαα’αΆααααααΈαααα
αα ααΆαααααα’αααΈααααααααΆααααα’αααααΉαααααΎαααα»αααα·ααααααα»ααααα’αα»ααααα
ααααα: www.habr.com