ืื ืงืจืืกืื ืืื, ืงืืืืึทื ืฅ ืืขืื ืืื ืื ืฆื ืฆืืฉืืขืื ืึทืงืกืขืก ืฆื ืื Kubernetes ืงื ืืื ืฆื ืงืขื ืขื ืฆื ืึทืงืกืขืก ืกืขืจืืืืกืขืก ืืื ืืขื ืงื ืืื: ืฆื ืงืขื ืขื ืืืืึทื ืคืึทืจืืื ืื ืฆื ืขืืืขืืข ืืึทืืึทืืืืก ืึธืืขืจ ืกืขืจืืืืก, ืฆื ืคืึทืจืืื ืื ืึท ืืืืข ืึทืคึผืืึทืงืืืฉืึทื ืืื ืึทืคึผืืึทืงืืืฉืึทื ื ืืื ืืขื ืงื ืืื ...
ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืขืก ืืื ืึท ื ืืื ืฆื ืคืึทืจืืื ืื ืคืื ืืืื ืืืืข ืืึทืฉืื ืฆื ืึท ืกืขืจืืืืก memcached.staging.svc.cluster.local. ืืืจ ืฆืืฉืืขืื ืืขื ืคืืืืงืืื ื ืืฆื ืึท VPN ืืื ืืขื ืงื ืืื ืฆื ืืืึธืก ืืขืจ ืงืืืขื ื ืงืึทื ืขืงืฅ. ืฆื ืืึธื ืืึธืก, ืืืจ ืืขืืื ืกืืื ืขืฅ ืคืื ืคึผืึธืืก, ืืึทืืื ืื ืืก ืืื ืฉืืืคึผื ืงื ืืื ืื ืก ืฆื ืืขื ืงืืืขื ื. ืืืื, ืืืขื ืึท ืงืืืขื ื ืคืจืืืื ืฆื ืคืึทืจืืื ืื ืฆื ืื ืกืขืจืืืืก memcached.staging.svc.cluster.local, ืื ืืขืื ืืืื ืฆื ืื ืงื ืืื ืื ืก ืืื ืืื ืขื ืืคืขืจ ื ืขืื ืื ืึทืืจืขืก ืคืื ืืขื ืืื ืกื ืคืื ืื ืงื ืืื ืกืขืจืืืืก ื ืขืฅ ืึธืืขืจ ืื ืคึผืึธื ืึทืืจืขืก.
ืืืจ ืงืึทื ืคืืืืขืจ K8s ืงืืึทืกืืขืจื ื ืืฆื kubeadm, ืืื ืื ืคืขืืืงืืึทื ืกืขืจืืืืก ืกืืื ืขื ืืื 192.168.0.0/16, ืืื ืื ื ืขืฅ ืคืื ืคึผืึธืืก ืืื 10.244.0.0/16. ืืืืฉืึทืืืึทืื ืึทืืฅ ืึทืจืืขื ืืื, ืึธืืขืจ ืขืก ืืขื ืขื ืึท ืคึผืึธืจ ืคืื ืคืื ืงืื:
- ืกืืื ืขื
192.168.*.*ืึธืคื ืืขื ืืฆื ืืื ืงืืืขื ื ืึธืคืืก ื ืขืืืืึธืจืงืก, ืืื ืืคืืื ืืขืจ ืึธืคื ืืื ืืขืืืขืืึธืคึผืขืจืก ืืืื ื ืขืืืืึธืจืงืก. ืืื ืืขืืึธืื ืืืจ ืืึทืงืืืขื ืงืึธื ืคืืืงื: ืืืื ืจืึธืืืขืจืก ืึทืจืืขื ืืืืฃ ืืขื ืกืืื ืขื ืืื ืื ืืืคึผื ืคึผืืฉืื ืื ืกืืื ืขืฅ ืคืื ืืขื ืงื ืืื ืฆื ืืขื ืงืืืขื ื. - ืืืจ ืืึธืื ืขืืืขืืข ืงืืึทืกืืขืจื (ืคึผืจืึธืืืงืฆืืข, ืืื ืข ืืื / ืึธืืขืจ ืขืืืขืืข ืืขืื ืงืืึทืกืืขืจื). ืืขืจื ืึธื, ืืืจื ืคืขืืืงืืึทื, ืึทืืข ืคืื โโโโืืื ืืืขืื ืืึธืื ืื ืืขืืืข ืกืืื ืขืฅ ืคึฟืึทืจ ืคึผืึธืืก ืืื ืืึทืืื ืื ืืก, ืืืึธืก ืงืจืืืืฅ ืืจืืืก ืฉืืืขืจืืงืืืื ืคึฟืึทืจ ืกืืืืึทืืืืื ืืึทืก ืึทืจืืขื ืืื ืกืขืจืืืืกืขืก ืืื ืขืืืขืืข ืงืืึทืกืืขืจื.
ืืืจ ืืึธืื ืืึทื ื ืฆืืจืืง ืื ืืขื ืืืขื ืื ืคืืจ ืคืื ื ืืฆื ืคืึทืจืฉืืืขื ืข ืกืืื ืขืฅ ืคึฟืึทืจ ืืึทืืื ืื ืืก ืืื ืคึผืึธืืก ืืื ืืืื ืคึผืจืืืขืงื - ืืื ืึทืืืขืืืื, ืึทืืื ืึทื ืึทืืข ืงืืึทืกืืขืจื ืืึธืื ืคืึทืจืฉืืืขื ืข ื ืขืืืืึธืจืงืก. ืึธืืขืจ, ืขืก ืืขื ืขื ืึท ืืจืืืก ื ืืืขืจ ืคืื ืงืืึทืกืืขืจื ืืื ืึธืคึผืขืจืึทืฆืืข ืืืึธืก ืืื ืืืึธืื ื ืืฉื ืืืขืื ืฆื ืจืึธืืื ืืืืขืจ ืคึฟืื ืงืจืึทืฆื, ืืืืึทื ืืื ืืืืคื ืคืืืข ืืึทืืื ืื ืืก, ืกืืืืืคืื ืึทืคึผืืึทืงืืืฉืึทื ื, ืืื"ื ื.
ืืื ืืขืืึธืื ืืืจ ืืขืคืจืขืื ืืื: ืืื ืฆื ืืืืฉื ืื ืกืืื ืขื ืืื ืึท ืืืืืกืืื ื ืงื ืืื?
ืืืื ืคืื ืืืกืืืฉืึทื ื
ืื ืืขืจืกื ืคึผืจืึธืกื ืคืืจ ืืื ืฆื ืจืืงืจืืืื ืึทืืข ืกืขืจืืืืกืขืก ืืื ืืืคึผ ClusterIP. ืืื ืึทื ืึธืคึผืฆืืข, ืืื ืืืก:
ืื ืคืืืืขื ืืข ืคึผืจืึธืฆืขืก ืืื ืึท ืคึผืจืึธืืืขื: ื ืึธื ืึทืืฅ ืงืึทื ืคืืืืขืจื, ืื ืคึผืึธืืก ืงืืืขื ืึทืจืืืฃ ืืื ืื ืึทืื IP ืืื ืึท ืื ืก ื ืึทืืขืกืขืจืืืขืจ ืืื /etc/resolv.conf.
ืืื ื ืืื ื ืึธื ืืื ื ืืฉื ืืขืคึฟืื ืขื ืื ืืืืืื ื, ืืื ืืื ืฆื ืืึทืฉืืขืืืง ืื ืืื ืฆืข ืงื ืืื ืืื kubeadm ืืึทืฉืืขืืืง ืืื ืึธื ืืืืื ืขืก ืืืืืขืจ.
ืึธืืขืจ ืืึธืก ืืื ื ืืฉื ืคึผืึทืกืืง ืคึฟืึทืจ ืึทืืขืืขื ... ืืึธ ืืขื ืขื ืืขืจ ืืืืืืื ืืงืืื ืคึฟืึทืจ ืืื ืืืขืจ ืคืึทื:
- ืคืืึทื ืึทื ืืื ืืขื ืืฆื;
- ืขืก ืืขื ืขื ืงืืึทืกืืขืจื ืืืืืข ืืื ืื ืืืืืงื ืก ืืื ืืืืฃ ืืึทืื ืืืึทืจื;
- ืืื ืืืึธืื ืืื ืฆื ืืืกืืืืื ืฉืืึทืขื-ืืืคึผืืืืื ื ืึทืืข ืกืขืจืืืืกืขืก ืืื ืืขื ืงื ืืื;
- ืขืก ืืื ืึท ื ืืื ืฆื ืืึธื ืึทืืฅ ืืื ืึท ืืื ืืืื ื ืืืขืจ ืคืื ืคึผืจืึธืืืขืืก;
- Kubernetes ืืืขืจืกืืข ืืื 1.16.6 (ืึธืืขืจ, ืืืืึทืืขืจ ืกืืขืคึผืก ืืืขื ืืืื ืขื ืืขื ืคึฟืึทืจ ืื ืืขืจืข ืืืขืจืกืืขืก);
- ืื ืืืืคึผื ืึทืจืืขื ืืื ืฆื ืขื ืฉืืจ ืึทื ืืื ืึท ืงื ืืื ืืืคึผืืืื ื ืืฆื kubeadm ืืื ืึท ืกืขืจืืืืก ืกืืื ืขื
192.168.0.0/16, ืคืึทืจืืืึทืื ืขืก ืืื172.24.0.0/16.
ืืื ืขืก ืืื ืคึผืื ืงื ืึทืืื ืืขืฉืขื ืึทื ืืืจ ืืึธืื ืฉืืื ืืึทื ื ืืื ืืขืจืขืกืืจื ืฆื ืืขื ืืืึธืก ืืื ืืื ืืื ืงืืืขืจื ืขืืขืก ืืื ืกืืึธืจื ืืื ืืื', ืืืึธืก ืงืขื ืขื ืืืื ืืขืืื ืืื ืืื... ืึทืืื ืืืจ ืืขืืึทื ืง: "ืคืืจืืืืก ื ืืฉื ื ืึธืจ ืืขืจืืืึทื ืืืงื ืื ืืึทืื ืืื ืขืืง, ืจืืคึผืืืืกืื ื ืื ืึทืื IP ืึทืืจืขืกืขืก (ืกืืื ืขื) ืืื ื ืืึทืข? ยป
ืืืืื ืืขืืืื ืคืึทืจืืืง ืืืฉืืจืื ืคึฟืึทืจ ืืจืืขืื ืืื ืืึทืื ืืื ืขืืง, ืืืจ ืืึธืื ื ืื ืืขืคึฟืื ืขื ืขืคึผืขืก ืืืึธืก ืืึธืจ ืกืึทืืืื ืื ืคึผืจืึธืืืขื. (ืืื, ืืืื ืืืจ ืืืืกื ืืืขืื ืงืืื ืืืืืืึทืืื ืคึฟืึทืจ ืืจืืขืื ืืื ืืึทืื ืืืืึทื ืืื ืขืืง, ืืืจ ืืืึธืื ืึธืคึผืฉืึทืฆื ืื ืืื ืงืก.) ืึธืืขืจ, ืึท ืืื ืกืืึทืจืืื ื ืคืื ื ืืื (ืืึทื ืง ืฆื ืืืื ืืืืจืื!).
ืืขื ื ืืฆื ืงืขื ืขื ืคืึทืจืืื ืื ืฆื ืขืืง ื ืืฆื ืกืขืจืืืคืืงืึทืฅ ืืื ืืืืขื ืขื ืืึทืื ืคึฟืื ืืึธืจื ื ืืฆื ืงืึทืืึทื ืื ls, get, dump.
ืืืื ืขืืงืืืขืืคึผืขืจ
ืืขืจ ืืืืึทืืขืจ ืืขืืึทื ืง ืืื ืืึทืืืฉืืงืึทื: "ืืืึธืก ืกืืึธืคึผืคึผืื ื ืืืจ ืคืื ืึทืืื ื ืืขื ื ืืฆื ืืืจื ืึทืืื ื ืื ืคืืืืงืืื ืฆื ืฉืจืืึทืื ืืึทืื ืฆื ืขืืง?"
ืขืก ืืื ืืขืืืืจื ืึท ืืึทืืึทืคืืื ืืืขืจืกืืข ืคืื โโืขืืงืืืขืืคึผืขืจ ืืื ืฆืืืื ื ืืึทืข ืคืึทื ืืงืฉืึทื ื changeServiceCIDR ะธ changePodCIDR. ืืืืฃ ืืืจ ืืืจ ืงืขื ืขื ืืขื ืื ืงืึธื .
ืืืึธืก ืืึธื ืื ื ืืึทืข ืคึฟืขืึดืงืืืื? ืึทืืืขืจืืืึทื changeServiceCIDR:
- ืฉืึทืคึฟื ืึท ืืขืกืขืจืืึทืืืืขืจ;
- ืฆืื ืืืคื ืขืืขื ืึท ืจืขืืืืขืจ ืืืืกืืจืืง ืฆื ืคืึทืจืืืึทืื CIDR;
- ืืืจ ืืืื ืืืจื ืึทืืข ืกืขืจืืืืกืขืก ืืื ืื ClusterIP ืืืคึผ ืืื ืืขื ืงื ืืื:
- ืืขืงืึธืืข ืื ืืืขืจื ืคืื ืขืืง ืืื ืึท ืืืื ืืืืคืขืฅ;
- ื ืืฆื ืึท ืจืขืืืืขืจ ืืืืกืืจืืง ืืืจ ืคืึทืจืืืึทืื ืื ืขืจืฉืืขืจ ืฆืืืื ืืืืขืก ืคืื ืื ืึทืืจืขืก;
- ืืึทืฉืืืืขื ืื ืกืขืจืืืืก ืึทื IP ืึทืืจืขืก ืคืื ืื ื ืืึทืข ืกืืื ืขื;
- ืฉืึทืคึฟื ืึท ืกืืจืืึทืืืืขืจ, ืืขืจ ืื Go ืืืืคืขืฅ ืืื ืคึผืจืึธืืึธืืืฃ, ืฉืจืืึทืื ื ืืึทืข ืืึทืื ืฆื ืขืืง.
ืคืื ืงืฆืืึธื ืืจื changePodCIDR ืืกืขื ืฉืึทืื ืขื ืืขื changeServiceCIDR - ื ืึธืจ ืึทื ืฉืืึธื ืคืื ืขืืืืื ื ืื ืกืขืจืืืืก ืืึทืฉืจืืึทืืื ื, ืืืจ ืืึธื ืืึธืก ืคึฟืึทืจ ืื ื ืึธืืข ืืื ืืืืฉื .spec.PodCIDR ืฆื ืึท ื ืืึทืข ืกืืื ืขื.
ืคืืจ
ืืืืฉื ืืื ืกื CIDR
ืืขืจ ืคึผืืึทื ืคึฟืึทืจ ืืืคึผืืึทืืขื ืื ื ืื ืึทืจืืขื ืืื ืืืืขืจ ืคึผืฉืื, ืึธืืขืจ ืขืก ืื ืืืึทืืืื ืืึทืื ืืืื ืืฉืขืช ืึทืืข ืคึผืึธืืก ืืื ืืขื ืงื ืืื ืืขื ืขื ืจืืงืจืืืืืื. ื ืึธื ืืืกืงืจืืืืื ื ืื ืืืืคึผื ืกืืขืคึผืก, ืืืจ ืืืขืื ืืืื ืืืืื ืืขืืื ืงืขื ืืืขืื ืืื, ืืื ืืขืึธืจืืข, ืื ืืึทืื ืืืื ืงืขื ืขื ืืืื ืืื ืึทืืืืื.
ืคึผืจืืคึผืขืจืึทืืึธืจื ืกืืขืคึผืก:
- ืื ืกืืึธืืื ื ืื ื ืืืืืง ืืืืืืืืืจื ืืื ืึทืกืขืืืึทื ืื ืคึผืึทืืฉื ืขืืงืืืขืืคึผืขืจ;
- ืืึทืงืึทืคึผ ืขืืง ืืื
/etc/kubernetes.
ืงืืจืฅ ืงืึทืืฃ ืคึผืืึทื ืคึฟืึทืจ ืืฉืึทื ืืื ื ืกืขืจืืืืก ืกืืืจ:
- ืืฉืึทื ืืื ื ืื ืึทืคึผืืกืขืจืืืขืจ ืืื ืงืึธื ืืจืึธืืืขืจ-ืคืึทืจืืืึทืืืขืจ ืืึทื ืึทืคืขืกืฅ;
- ืจืขืืกืืืก ืคืื ืกืขืจืืืคืืงืึทืฅ;
- ืืฉืึทื ืืื ื ClusterIP ืืึทืืื ืื ืืก ืืื ืขืืง;
- ืจืืกืืึทืจื ืึทืืข ืคึผืึธืืก ืืื ืืขื ืงื ืืื.
ืื ืคืืืืขื ืืข ืืื ืึท ืืึทื ืฅ ืกืืงืืืึทื ืก ืคืื ืึทืงืฉืึทื ื ืืื ืืขืืึทื.
1. ืื ืกืืึทืืืจื ืขืืง-ืงืืืขื ื ืคึฟืึทืจ ืืึทืื ืืึทืืคึผ:
apt install etcd-client2. ืืืืขื ืขืืงืืืขืืคึผืขืจ:
- ืื ืกืืึทืืืจื ืืึธืืึทื ื:
GOPATH=/root/golang mkdir -p $GOPATH/local curl -sSL https://dl.google.com/go/go1.14.1.linux-amd64.tar.gz | tar -xzvC $GOPATH/local echo "export GOPATH="$GOPATH"" >> ~/.bashrc echo 'export GOROOT="$GOPATH/local/go"' >> ~/.bashrc echo 'export PATH="$PATH:$GOPATH/local/go/bin"' >> ~/.bashrc - ืืืจ ืจืึทืืขืืืขื ืคึฟืึทืจ ืืื
etcdhelper.go, ืืจืืคืงืืคืืข ืืืคึผืขื ืืึทื ืกืื, ืงืืืึทืื:wget https://raw.githubusercontent.com/flant/examples/master/2020/04-etcdhelper/etcdhelper.go go get go.etcd.io/etcd/clientv3 k8s.io/kubectl/pkg/scheme k8s.io/apimachinery/pkg/runtime go build -o etcdhelper etcdhelper.go
3. ืืึทืื ืึท ืืึทืงืึทืคึผ ืขืืง:
backup_dir=/root/backup
mkdir ${backup_dir}
cp -rL /etc/kubernetes ${backup_dir}
ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --key=/etc/kubernetes/pki/etcd/server.key --cert=/etc/kubernetes/pki/etcd/server.crt --endpoints https://192.168.199.100:2379 snapshot save ${backup_dir}/etcd.snapshot
4. ืืืืฉื ืื ืกืขืจืืืืก ืกืืื ืขื ืืื ืื ืงืืืขืจื ืขืืขืก ืงืึธื ืืจืึธื ืคืืึทื ืืึทื ืึทืคืขืกืฅ. ืืื ืืขืงืขืก /etc/kubernetes/manifests/kube-apiserver.yaml ะธ /etc/kubernetes/manifests/kube-controller-manager.yaml ืืืืฉื ืื ืคึผืึทืจืึทืืขืืขืจ --service-cluster-ip-range ืฆื ืึท ื ืืึทืข ืกืืื ืขื: 172.24.0.0/16 ืึทื ืฉืืึธื ืคืื 192.168.0.0/16.
5. ืืื ื ืืืจ ืืืืฉื ืื ืกืขืจืืืืก ืกืืื ืขื ืฆื ืืืึธืก kubeadm ืึทืจืืืกืืขืื ืกืขืจืืืคืืงืึทืฅ ืคึฟืึทืจ ืึทืคึผืืกืขืจืืืขืจ (ืึทืจืืึทื ืืขืจืขืื ื), ืืื ืืึทืจืคึฟื ืฆื ืืืื ืจืืกืืื:
- ืืึธืืืจ ืืขื ืืืึธืก ืืึธืืืืื ื ืืื IP ืึทืืจืขืกืขืก ืื ืงืจืึทื ื ืืึทืืืืึทืื ืืื ืืจืืืก ืคึฟืึทืจ:
openssl x509 -noout -ext subjectAltName </etc/kubernetes/pki/apiserver.crt X509v3 Subject Alternative Name: DNS:dev-1-master, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, DNS:apiserver, IP Address:192.168.0.1, IP Address:10.0.0.163, IP Address:192.168.199.100 - ืืึธืืืจ ืฆืืืจืืืื ืึท ืืื ืืืึทื ืงืึทื ืคืืืืขืจืืืฉืึทื ืคึฟืึทืจ kubeadm:
cat kubeadm-config.yaml apiVersion: kubeadm.k8s.io/v1beta1 kind: ClusterConfiguration networking: podSubnet: "10.244.0.0/16" serviceSubnet: "172.24.0.0/16" apiServer: certSANs: - "192.168.199.100" # IP-ะฐะดัะตั ะผะฐััะตั ัะทะปะฐ - ืืึธืืืจ ืืืกืืขืงื ืื ืึทืื ืงืจื ืืื ืฉืืืกื, ืืืืึทื ืึธื ืืขื ืื ื ืืึท ืืึทืืืืึทืื ืืืขื ื ืืฉื ืืืื ืืจืืืก:
rm /etc/kubernetes/pki/apiserver.{key,crt} - ืืึธืืืจ ืึทืจืืืกืืขืื ืกืขืจืืืคืืงืึทืฅ ืคึฟืึทืจ ืื ืึทืคึผื ืกืขืจืืืขืจ:
kubeadm init phase certs apiserver --config=kubeadm-config.yaml - ืืึธืืืจ ืงืึธื ืืจืึธืืืจื ืึทื ืื ืืึทืืืืึทืื ืืื ืืจืืืก ืคึฟืึทืจ ืื ื ืืึทืข ืกืืื ืขื:
openssl x509 -noout -ext subjectAltName </etc/kubernetes/pki/apiserver.crt X509v3 Subject Alternative Name: DNS:kube-2-master, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, IP Address:172.24.0.1, IP Address:10.0.0.163, IP Address:192.168.199.100 - ื ืึธื ืฉืืึทืขื-ืึทืจืืืกืืขืื ืื ืึทืคึผื ืกืขืจืืืขืจ ืืึทืืืืึทืื, ืจืืกืืึทืจื ืืืึทื ืงืึทื ืืืื ืขืจ:
docker ps | grep k8s_kube-apiserver | awk '{print $1}' | xargs docker restart - ืืื ืก ืจืขืืขื ืขืจืืจื ืื ืงืึธื ืคืืืืจืึทืืืึธื ืคึฟืึทืจ
admin.conf:kubeadm alpha certs renew admin.conf - ืืื ืก ืจืขืืึทืืืจื ืื ืืึทืื ืืื etcd:
./etcdhelper -cacert /etc/kubernetes/pki/etcd/ca.crt -cert /etc/kubernetes/pki/etcd/server.crt -key /etc/kubernetes/pki/etcd/server.key -endpoint https://127.0.0.1:2379 change-service-cidr 172.24.0.0/16ืืืืจืขื ืื ื! ืืื ืืขื ืืึธืืขื ื, ืคืขืื ืืึทืืืึธืืข ืกืืึทืคึผืก ืืจืืขืื ืืื ืืขื ืงื ืืื, ืืื ื ืืื ืืืืืกืืื ื ืคึผืึธืืก ืืื
/etc/resolv.confืื ืึทืื CoreDNS ืึทืืจืขืก (kube-dns) ืืื ืจืขืืืกืืจืืจื, ืืื kube-proxy ืขื ืืขืจืื ืืขื ืื iptables ืึผืืืื ืคืื ืื ืึทืื ืกืืื ืขื ืฆื ืื ื ืืึทืข. ืืืืึทืืขืจ ืืื ืืขื ืึทืจืืืงื ืขืก ืืื ืืขืฉืจืืื ืืืขืื ืืขืืืขื ืึธืคึผืฆืืขืก ืฆื ืืื ืึทืืืื ืืึทืื ืืืื. - ืืื ืก ืคืึทืจืจืืืื ConfigMap ืก ืืื ืื ื ืึทืืขืกืคึผืึทืกืข
kube-system:kubectl -n kube-system edit cm kubelet-config-1.16- ืคืึทืจืืืึทืื ืืึธ
clusterDNSืฆื ืื ื ืืึทืข IP ืึทืืจืขืก ืคืื ืื kube-dns ืกืขืจืืืืก:kubectl -n kube-system get svc kube-dns.kubectl -n kube-system edit cm kubeadm-config- ืืืจ ืืืขืื ืคืึทืจืจืืืื ืขืก
data.ClusterConfiguration.networking.serviceSubnetืฆื ืึท ื ืืึทืข ืกืืื ืขื. - ืืื ื ืื kube-dns ืึทืืจืขืก ืืื ืคืืจืขื ืืขืจื, ืขืก ืืื ื ืืืืืง ืฆื ืืขืจืืืึทื ืืืงื ืื ืงืืืขืืขื ืงืึทื ืคืืืืขืจืืืฉืึทื ืืืืฃ ืึทืืข ื ืึธืืื:
kubeadm upgrade node phase kubelet-config && systemctl restart kubelet - ืึทืืข ืืืึธืก ืืืืืื ืืื ืฆื ืจืืกืืึทืจื ืึทืืข ืคึผืึธืืก ืืื ืืขื ืงื ืืื:
kubectl get pods --no-headers=true --all-namespaces |sed -r 's/(S+)s+(S+).*/kubectl --namespace 1 delete pod 2/e'
ืืื ืึทืืืื ืืึทืื ืืืื
ืืขืืื ืงืขื ืืืขืื ืืื ืฆื ืืื ืึทืืืื ืืึทืื ืืืื:
- ื ืึธื ืืฉืึทื ืืื ื ืื ืงืึธื ืืจืึธื ืคืืึทื ืืึทื ืึทืคืขืกืฅ, ืฉืึทืคึฟื ืึท ื ืืึทืข kube-dns ืืื ืกื, ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืื ืืขื ื ืึธืืขื
kube-dns-tmpืืื ื ืืึท ืึทืืจืขืก172.24.0.10. - ืฆื ืืึทืื
ifืืื ืขืืงืืืขืืคึผืขืจ, ืืืึธืก ืืืขื ื ืืฉื ืืึธืืืคืืฆืืจื ืื kube-dns ืืื ืกื. - ืคืึทืจืืืึทืื ืื ืึทืืจืขืก ืืื ืึทืืข ืงืืืขืืขืฅ
ClusterDNSืฆื ืึท ื ืืึทืข, ืืฉืขืช ืื ืึทืื ืืื ืกื ืืืขื ืคืึธืจืืขืฆื ืฆื ืึทืจืืขืื ืกืืืืึทืืืืื ืืึทืกืื ืืื ืื ื ืืึทืข. - ืืืึทืจืื ืืื ืื ืคึผืึธืืก ืืื ืึทืคึผืืึทืงืืืฉืึทื ื ืจืึธืืื ืืืืขืจ ืืื ืคึฟืึทืจ ื ืึทืืืจืืขื ืกืืืืช ืึธืืขืจ ืืื ืึท ืืกืืื ืฆืืื.
- ืืืกืืขืงื ืืื ืกื
kube-dns-tmpืืื ืืืืฉืserviceSubnetCIDRืคึฟืึทืจ ืื ืงืืืข-ืื ืก ืืื ืกื.
ืืขืจ ืคึผืืึทื ืืืขื ืืึธืื ืืืจ ืฆื ืืื ืึทืืืื ืืึทืื ืืืื ืฆื ~ ืึท ืืื ืื - ืคึฟืึทืจ ืืขืจ ืืขืืืืขืจ ืคืื ืื ืืึทืืืึทืืืงืื ื ืคืื ืื ืกืขืจืืืืก kube-dns-tmp ืืื ืืฉืึทื ืืื ื ืื ืกืืื ืขื ืคึฟืึทืจ ืื ืืื ืกื kube-dns.
ืืึธืืืคืืงืึทืืืึธื ืคึผืึธืื ืขืืืืึธืจืง
ืืื ืืขืจ ืืขืืืืงืขืจ ืฆืืื, ืืืจ ืืึทืฉืืึธืกื ืฆื ืงืืงื ืืื ืฆื ืืึธืืืคืืฆืืจื ืคึผืึธืื ืขืืืืึธืจืง ืืื ืื ืจืืืึทืืืื ื ืขืืงืืืขืืคึผืขืจ. ืื ืกืืงืืืึทื ืก ืคืื ืึทืงืฉืึทื ื ืืื ืืื ืืืื:
- ืคืืงืกืืจ ืงืึทื ืคืืืืขืจืืืฉืึทื ื ืืื
kube-system; - ืคืืงืกืืจ ืื ืงืืืข-ืงืึธื ืืจืึธืืืขืจ-ืคืึทืจืืืึทืืืขืจ ืืึทืฉืืึทืืคึผืขืจืืขื;
- ืืืืฉื ืคึผืึธืืกืืืจ ืืืืึทื ืืื ืขืืง;
- ืจืขืืึธืึธื ืึทืืข ืงื ืืื ื ืึธืืื.
ืืืฆื ืืขืจ ืืืขืื ืื ืึทืงืฉืึทื ื:
1. ืืึธืืืคืืฆืืจื ืงืึธื ืคืืืืึทืคึผืก ืืื ืื ื ืึทืืขืกืคึผืึทืกืข kube-system:
kubectl -n kube-system edit cm kubeadm-config
- ืงืขืจืขืงืืื ื data.ClusterConfiguration.networking.podSubnet ืฆื ืึท ื ืืึทืข ืกืืื ืขื 10.55.0.0/16.
kubectl -n kube-system edit cm kube-proxy
- ืงืขืจืขืงืืื ื data.config.conf.clusterCIDR: 10.55.0.0/16.
2. ืืึธืืืคืืฆืืจื ืื ืงืึธื ืืจืึธืืืขืจ-ืคืึทืจืืืึทืืืขืจ ืืึทืฉืืึทืืคึผืขืจืืขื:
vim /etc/kubernetes/manifests/kube-controller-manager.yaml
- ืงืขืจืขืงืืื ื --cluster-cidr=10.55.0.0/16.
3. ืงืืง ืืื ืื ืงืจืึทื ื ืืืึทืืืขืก .spec.podCIDR, .spec.podCIDRs, .InternalIP, .status.addresses ืคึฟืึทืจ ืึทืืข ืงื ืืื ื ืึธืืื:
kubectl get no -o json | jq '[.items[] | {"name": .metadata.name, "podCIDR": .spec.podCIDR, "podCIDRs": .spec.podCIDRs, "InternalIP": (.status.addresses[] | select(.type == "InternalIP") | .address)}]'[
{
"name": "kube-2-master",
"podCIDR": "10.244.0.0/24",
"podCIDRs": [
"10.244.0.0/24"
],
"InternalIP": "192.168.199.2"
},
{
"name": "kube-2-master",
"podCIDR": "10.244.0.0/24",
"podCIDRs": [
"10.244.0.0/24"
],
"InternalIP": "10.0.1.239"
},
{
"name": "kube-2-worker-01f438cf-579f9fd987-5l657",
"podCIDR": "10.244.1.0/24",
"podCIDRs": [
"10.244.1.0/24"
],
"InternalIP": "192.168.199.222"
},
{
"name": "kube-2-worker-01f438cf-579f9fd987-5l657",
"podCIDR": "10.244.1.0/24",
"podCIDRs": [
"10.244.1.0/24"
],
"InternalIP": "10.0.4.73"
}
]4. ืคืึทืจืืืึทืื ืคึผืึธืืกืืืจ ืืืจื ืืึทืื ืขื ืืขืจืื ืืขื ืืืืึทื ืฆื ืขืืง:
./etcdhelper -cacert /etc/kubernetes/pki/etcd/ca.crt -cert /etc/kubernetes/pki/etcd/server.crt -key /etc/kubernetes/pki/etcd/server.key -endpoint https://127.0.0.1:2379 change-pod-cidr 10.55.0.0/165. ืืึธืืืจ ืงืึธื ืืจืึธืืืจื ืึทื ืคึผืึธืืกืืืจ ืืื ืืึทืงืข ืืขืืืื:
kubectl get no -o json | jq '[.items[] | {"name": .metadata.name, "podCIDR": .spec.podCIDR, "podCIDRs": .spec.podCIDRs, "InternalIP": (.status.addresses[] | select(.type == "InternalIP") | .address)}]'[
{
"name": "kube-2-master",
"podCIDR": "10.55.0.0/24",
"podCIDRs": [
"10.55.0.0/24"
],
"InternalIP": "192.168.199.2"
},
{
"name": "kube-2-master",
"podCIDR": "10.55.0.0/24",
"podCIDRs": [
"10.55.0.0/24"
],
"InternalIP": "10.0.1.239"
},
{
"name": "kube-2-worker-01f438cf-579f9fd987-5l657",
"podCIDR": "10.55.1.0/24",
"podCIDRs": [
"10.55.1.0/24"
],
"InternalIP": "192.168.199.222"
},
{
"name": "kube-2-worker-01f438cf-579f9fd987-5l657",
"podCIDR": "10.55.1.0/24",
"podCIDRs": [
"10.55.1.0/24"
],
"InternalIP": "10.0.4.73"
}
]6. ืืึธืืืจ ืจืขืืึธืึธื ืึทืืข ืงื ืืื ื ืึธืืื ืืืื ืขืจ ืืืจื ืืืื ืขืจ.
7. ืืืื ืืืจ ืืึธืื ืืืึท ืืื ืืกืืขืจ ืืืื ื ืึธืืข ืึทืื ืคึผืึธืืกืืืจ, ืืขืืึธืื kube-controller-manager ืืืขื ื ืืฉื ืงืขื ืขื ืฆื ืึธื ืืืืื, ืืื ืคึผืึธืืก ืืื ืืขื ืงื ืืื ืืืขื ื ืืฉื ืืืื ืกืงืขืืืฉืืื.
ืืื ืคืึทืงื, ืืฉืึทื ืืื ื ืคึผืึธืืกืืืจ ืงืขื ืขื ืืืื ืืขืืื ืืคืืื ืกืืืคึผืืขืจ (ืืืฉื, ). ืึธืืขืจ ืืืจ ืืขืืืืื ืฆื ืืขืจื ืขื ืืื ืฆื ืึทืจืืขืื ืืื ืขืืง ืืืืื, ืืืืึทื ืขืก ืืขื ืขื ืงืึทืกืขืก ืืืขื ืขืืืืื ื Kubernetes ืึทืืืืฉืขืงืฅ ืืื ืขืืง - ืืขืจ ืืืืื ืืขืืืขื ืืืึทืจืืึทื ื. (ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืืจ ืงืขื ืขื ื ืืฉื ื ืึธืจ ืืืืฉื ืื ืกืขืจืืืืก ืคืขืื ืึธื ืืึทืื ืืืื spec.clusterIP.)
ืืึทื ืฅ
ืืขืจ ืึทืจืืืงื ืืืกืงืืืืจื ืื ืืขืืืขืืงืืื ืคืื ืืจืืขืื ืืื ืืึทืื ืืื ืขืืง ืืืืึทื, ื.ื. ืืืืคึผืึทืกืื ื ืื Kubernetes API. ืืื ืืขื ืฆืืืึทื ื ืึทืืึทืื ืืืจ ืฆื ืืึธื "ืืจืืงื ืืื ืื." ืืืจ ืืขืกืืขื ืื ืึทืคึผืขืจืืืฉืึทื ื ืืขืืขืื ืืื ืืขื ืืขืงืกื ืืืืฃ ืคืึทืงืืืฉ ืง8ืก ืงืืึทืกืืขืจื. ืึธืืขืจ, ืืืืขืจ ืกืืึทืืืก ืคืื ืืจืืืืงืืึทื ืคึฟืึทืจ ืืืืืืกืคึผืจืขื ื ืืฆื ืืื PoC (ืืขืจืืืืึทื ืคืื ืืึทืืจืืฃ). ืืขืจืืืขืจ, ืืืื ืืืจ ืืืืื ืฆื ื ืืฆื ืึท ืืึทืืึทืคืืื ืืืขืจืกืืข ืคืื โโืื ืขืืงืืืขืืคึผืขืจ ื ืืฆื ืืืืฃ ืืืื ืงืืึทืกืืขืจื, ืืึธื ืืึธืก ืืืืฃ ืืืื ืืืืืขื ืข ืจืืืืงืืจื.
ืคึผืก
ืืืืขื ืขื ืืืื ืืืืฃ ืืื ืืืขืจ ืืืึธื:
- ยซ";
- ยซ";
- ยซ";
- ยซ'.
ืืงืืจ: www.habr.com