ืื ืื ืงืืื - ืืชืจื ืืืืช ืื ืืืืฆื? ืืชืืื ืื ืืืืจื ืืืืืจ ืขื Infrastructure-as-Code, ืื?
ืืื ืืืฆื?
ืืจืื, Infrastructure-as-Code (IaC) ืืื ืืจื ืืฆืืจืชืืช ืืืืฆืื ืชืฉืชืืช. ืื ืื ื ืืชืืจืื ืืช ืืืฆื ืฉืื ื ืจืืฆืื ืืืฉืื, ืืื ืืืืง ืืืืืจื ืืืื ืืชืฆืืจืช ืืชืืื ื. ืืื IaC ืืฉืืฉ ืขืืืจ:
- ืืชื ืืฉืืืื. ืืื ืื VMs, S3, VPC ืืื'. ืืืื ืืกืืกืืื ืืขืืืื:
Terraform ะธCloudFormation . ืชืฆืืจืช ืชืืื ื . ืืืื ืืกืืกืืื:ืืืชื ืืคืฉืจื ,ืฉืฃ ืืื'.
ืื ืงืื ื ืืฆื ื-git repositories. ืืืืืงืื ืื ืืืืืืจ ืจืืฉ ืืฆืืืช ืืืืื ืฉืฆืจืื ืืขืฉืืช ืืื ืกืืจ. ืืืื ืืฉืืืจ. ืืื ืืฆืืจ ืืืื ืืื ื. ืืืื ืืจืื ืฉืื ืืื.
ืืื ืื ืฉืื ืืืจ ืงืืื
ืืืืคื ืืืืฆื?
ืื ืื ืื ื ืืชืงืจืืื ืืืืจืื ืืฉืืื ืืืจืืืืช.
ืงืืื ืื, ืืชื ืฆืจืื ืืืชืืื ืขื ืืืืจ ืฉืืชืืจ ืืช ืืืื ื ืฉื ืืืืจืื ืืืจืื, ืืืื ืขืฆืื. ืืืืืื, ืืืืง ื-GitOps, ืขืืื ืืืืกืืฃ CI ืื ืฉืืฉืื ืืืื ืืืืฆืขื ืืืืคื ืืืืืืื.
ืื Git ืขืืืื ืื ื ืืฆืจ?
- ืืื ืืืืกื ืืืชื ื-Git?
- ืืืฆื ืืืชืงืื CI?
- ืื ื ืคืจืืก ืื ืืช Gitlab ืืืืฆืขืืช IaC, ืืืคืืื ื-Kubernetes?
- ืืื GitLab Runner ื-Kubernetes?
- ืื ืืืื Kubernetes ืืกืคืง ืืขื ื?
ืื ืื ืงืืื: GitLab ืฉืื ืื ื ืืขืื ืืช ืืงืื ืฉืื, ืื ืืงืื ืฉืืชืืจ ืืืื ืกืื ืฉื GitLab ืื ื ืฆืจืื?
ืขืืฃ ืขื ืืืฆืื
ยซืืืืืงืืืื 3 ืขื ืืื ืืืืืจ" [src ]
ืืืื ื ื ืกื ืืืฉื ืื ื ืืืืฆืขืืช ืืกืคืง ืขื ื
TL; DR
ืืื ื ืืชื ืืืฆืืจืฃ ืืฆืืืช ืืื ืืืช ืืืช?
$ export MY_SELECTEL_TOKEN=<token>
$ curl https://gitlab.com/chicken-or-egg/mks/make/-/snippets/2002106/raw | bash
ืืฆืจืืื:
- ืืฉืืื ื-my.selectel.ru;
- ืืกืืืื ืืฉืืื;
- ืืืฉืืจื Kubernetes;
- ืืืฉืืจื ืืื;
- ืืืืื ืืืืช Terraform;
- ืชืจืฉืื ืืื GitLab;
- ืชืจืฉืื ืืื GitLab Runner.
ืืชืืื:
- ืงืื ืืช MY_SELECTEL_TOKEN ืืืืืื ืืช my.selectel.ru.
- ืฆืืจ ืืฉืืื Kubernetes ืขื ืืื ืืขืืจืช ืืกืืืื ืืฉืืื ืืืื.
- ืงืื KUBECONFIG ืืืืฉืืื ืฉื ืืฆืจ.
- ืืชืงื ืืช GitLab ืขื Kubernetes.
- ืงืื ืืช GitLab-token ื-GitLab ืฉื ืืฆืจ ืขืืืจ ืืืฉืชืืฉ ืฉืืจืฉ.
- ืฆืืจ ืืื ื ืคืจืืืงื ื-GitLab ืืืืฆืขืืช GitLab-token.
- ืืืืฃ ืืช ืืงืื ืืงืืื ื-GitLab.
- ???
- ืจืืื!
ืฉืื 1. ืืช ืืืกืืืื ื ืืชื ืืืฉืื ืืืืืจ
ืฉืื 2. ืื ื ืืืื ืื ืืช ื- Terraform ืฉืื ื ื"ืืคืืืช" ืืฉืืื ืฉื 2 ืฆืืชืื. ืื ืืชื ืืืื ืฉืืฉ ืื ืืกืคืืง ืืฉืืืื ืืื ืืืจ, ืืชื ืืืื ืืืคืขืื ืืืกืืช ืืืืืืืืืช:
provider "selectel" {
token = var.my_selectel_token
}
variable "my_selectel_token" {}
variable "username" {}
variable "region" {}
resource "selectel_vpc_project_v2" "my-k8s" {
name = "my-k8s-cluster"
theme = {
color = "269926"
}
quotas {
resource_name = "compute_cores"
resource_quotas {
region = var.region
zone = "${var.region}a"
value = 16
}
}
quotas {
resource_name = "network_floatingips"
resource_quotas {
region = var.region
value = 1
}
}
quotas {
resource_name = "load_balancers"
resource_quotas {
region = var.region
value = 1
}
}
quotas {
resource_name = "compute_ram"
resource_quotas {
region = var.region
zone = "${var.region}a"
value = 32768
}
}
quotas {
resource_name = "volume_gigabytes_fast"
resource_quotas {
region = var.region
zone = "${var.region}a"
# (20 * 2) + 50 + (8 * 3 + 10)
value = 130
}
}
}
resource "selectel_mks_cluster_v1" "k8s-cluster" {
name = "k8s-cluster"
project_id = selectel_vpc_project_v2.my-k8s.id
region = var.region
kube_version = "1.17.9"
}
resource "selectel_mks_nodegroup_v1" "nodegroup_1" {
cluster_id = selectel_mks_cluster_v1.k8s-cluster.id
project_id = selectel_mks_cluster_v1.k8s-cluster.project_id
region = selectel_mks_cluster_v1.k8s-cluster.region
availability_zone = "${var.region}a"
nodes_count = 2
cpus = 8
ram_mb = 16384
volume_gb = 15
volume_type = "fast.${var.region}a"
labels = {
"project": "my",
}
}
ืืืกืฃ ืืฉืชืืฉ ืืคืจืืืงื:
resource "random_password" "my-k8s-user-pass" {
length = 16
special = true
override_special = "_%@"
}
resource "selectel_vpc_user_v2" "my-k8s-user" {
password = random_password.my-k8s-user-pass.result
name = var.username
enabled = true
}
resource "selectel_vpc_keypair_v2" "my-k8s-user-ssh" {
public_key = file("~/.ssh/id_rsa.pub")
user_id = selectel_vpc_user_v2.my-k8s-user.id
name = var.username
}
resource "selectel_vpc_role_v2" "my-k8s-role" {
project_id = selectel_vpc_project_v2.my-k8s.id
user_id = selectel_vpc_user_v2.my-k8s-user.id
}
ื ืชืื ื ืคืื:
output "project_id" {
value = selectel_vpc_project_v2.my-k8s.id
}
output "k8s_id" {
value = selectel_mks_cluster_v1.k8s-cluster.id
}
output "user_name" {
value = selectel_vpc_user_v2.my-k8s-user.name
}
output "user_pass" {
value = selectel_vpc_user_v2.my-k8s-user.password
}
ืืืื ื ืฉืืง:
$ env
TF_VAR_region=ru-3
TF_VAR_username=diamon
TF_VAR_my_selectel_token=<token>
terraform plan -out planfile
$ terraform apply -input=false -auto-approve planfile
ืฉืื 3. ืื ืื ื ืืงืืืื ืืช ื-cubeconfig.
ืืื ืืืืจืื ืืช KUBECONFIG ืืืืคื ืชืืื ืืชื, ืืชื ืฆืจืื ืืงืื ืืกืืืื ื-OpenStack:
openstack token issue -c id -f value > token
ืืขื ืืืกืืืื ืืื ืฉืื ืืงืฉื ื-Managed Kubernetes Selectel API. k8s_id ื ืืชื ืืืืฆื ืืจืคื:
curl -XGET -H "x-auth-token: $(cat token)" "https://ru-3.mks.selcloud.ru/v1/clusters/$(cat k8s_id)/kubeconfig" -o kubeConfig.yaml
ื ืืชื ืืืฉืช ื-Cupconfig ืื ืืจื ืืคืื ื.
ืฉืื 4. ืืืืจ ืฉืืืฉืืื ื ืืคื ืืืฉ ืื ื ืืืฉื ืืืื, ื ืืื ืืืืกืืฃ ืืื ืืขื ืืคื ืืืขื.
ืื ื ืืขืืืฃ ืืืืกืืฃ:
- ืืจืื ืฉืืืช,
- ืืืืงืช ืืืกืื
- ืืืื ืืืช ืืืืืช ืชืจืืืืื ืืื ืืืื.
ืืื ืืชืืืื ืืืจืชื ืืฉืืื ืืืืืจ ru-3a, ืื ืื ื ืฆืจืื ืืช ืืืืงืช ืืืืกืื ืืืืืืจ ืืื.
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast.ru-3a
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: cinder.csi.openstack.org
parameters:
type: fast.ru-3a
availability: ru-3a
allowVolumeExpansion: true
ืฉืื 5. ืืชืงื ืืืื ืขืืืกืื.
ืื ื ื ืฉืชืืฉ ืืกืื ืืจืื ืขืืืจ ืจืืื nginx-ingress. ืืฉ ืืืจ ืืจืื ืืืจืืืช ืืืชืงื ืชื, ืื ืื ื ืชืขืื ืขื ืื.
$ helm repo add nginx-stable https://helm.nginx.com/stable
$ helm upgrade nginx-ingress nginx-stable/nginx-ingress -n ingress --install -f ../internal/K8S-cluster/ingress/values.yml
ืื ื ืืืืื ืฉืืื ืืงืื IP ืืืฆืื ื ืืืฉื ื-3-4 ืืงืืช:
IP ืืืฆืื ื ืฉืืชืงืื:
ืฉืื 6. ืืชืงื ืืช GitLab.
$ helm repo add gitlab https://charts.gitlab.io
$ helm upgrade gitlab gitlab/gitlab -n gitlab --install -f gitlab/values.yml --set "global.hosts.domain=gitlab.$EXTERNAL_IP.nip.io"
ืฉืื ืื ืื ื ืืืืื ืฉืื ืืชืจืืืืื ืืขืื.
kubectl get po -n gitlab
NAME READY STATUS RESTARTS AGE
gitlab-gitaly-0 0/1 Pending 0 0s
gitlab-gitlab-exporter-88f6cc8c4-fl52d 0/1 Pending 0 0s
gitlab-gitlab-runner-6b6867c5cf-hd9dp 0/1 Pending 0 0s
gitlab-gitlab-shell-55cb6ccdb-h5g8x 0/1 Init:0/2 0 0s
gitlab-migrations.1-2cg6n 0/1 Pending 0 0s
gitlab-minio-6dd7d96ddb-zd9j6 0/1 Pending 0 0s
gitlab-minio-create-buckets.1-bncdp 0/1 Pending 0 0s
gitlab-postgresql-0 0/2 Pending 0 0s
gitlab-prometheus-server-6cfb57f575-v8k6j 0/2 Pending 0 0s
gitlab-redis-master-0 0/2 Pending 0 0s
gitlab-registry-6bd77b4b8c-pb9v9 0/1 Pending 0 0s
gitlab-registry-6bd77b4b8c-zgb6r 0/1 Init:0/2 0 0s
gitlab-shared-secrets.1-pc7-5jgq4 0/1 Completed 0 20s
gitlab-sidekiq-all-in-1-v1-54dbcf7f5f-qbq67 0/1 Pending 0 0s
gitlab-task-runner-6fd6857db7-9x567 0/1 Pending 0 0s
gitlab-webservice-d9d4fcff8-hp8wl 0/2 Pending 0 0s
Waiting gitlab
./wait_gitlab.sh ../internal/gitlab/gitlab/.pods
waiting for pod...
waiting for pod...
waiting for pod...
ืืชืจืืืืื ืขืื:
ืฉืื 7. ืื ืื ื ืืงืืืื ืืช GitLab-token.
ืจืืฉืืช, ืืื ืืช ืกืืกืืช ืืื ืืกื:
kubectl get secret -n gitlab gitlab-gitlab-initial-root-password -o jsonpath='{.data.password}' | base64 --decode
ืขืืฉืื ืืืื ื ืืื ืก ืื ืงืื ืืกืืืื:
python3 get_gitlab_token.py root $GITLAB_PASSWORD http://gitlab.gitlab.$EXTERNAL_IP.nip.io
ืฉืื 8. ืืืืช ืืืืจื Git ืืืืจืจืืื ืื ืืื ื ืืืืฆืขืืช ืกืคืง Gitlab.
cd ../internal/gitlab/hierarchy && terraform apply -input=false -auto-approve planfile
ืืจืืข ืืืื, ืืกืคืง terraform GitLab ืืฉ ืฆืฃ
ืืจืง . ืืืืจ ืืื ืชืฆืืจื ืืืืืง ืืช ืืคืจืืืงืืื ืืืชื ืืฉืื ืืืืคื ืืื ื ืขื ืื ืช ืฉ-tf.state ืืชืืงื. ืืืืจ ืืื ืืคืขื ืืืืฉ ืืช ืืคืงืืื `$make all`
ืฉืื 9. ืื ื ืืขืืืจืื ืืืืจืื ืืงืืืืื ืืฉืจืช.
$ make push
[master (root-commit) b61d977] Initial commit
3 files changed, 46 insertions(+)
create mode 100644 .gitignore
create mode 100644 values.yml
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 770 bytes | 770.00 KiB/s, done.
Total 5 (delta 0), reused 0 (delta 0)
ืืืฆืข:
ืืกืงื ื
ืืฉืื ื ืฉืื ืื ื ืืืืืื ืื ืื ืืื ืืฆืืจื ืืฆืืจืชืืช ืืืืืื ื ืืืงืืืืช ืฉืื ื. ืขืืฉืื ืื ื ืจืืฆื ืืืขืืืจ ืืช ืื ืืืฉืืืืช ืืืื ื-CI ืืคืฉืื ืืืืืฅ ืขื ืืคืชืืจืื. ืืฉื ืื, ืขืืื ื ืืืขืืืจ ืืช ืืืืื ืืช ืืืงืืืืืช ืฉืื ื (ืืืื ืช Terraform) ื-CI. ืืื ืืขืฉืืช ืืืช ืืืืง ืืื.
ืืืจืฉื ืืื ืื ืฉืื ื
ืืืื ืืื ืื ืืคืกืคืก ืืช ืคืจืกืื ืืืืืจืื ืืืืฉืื!
ืืงืืจ: www.habr.com