Hello! Dhawaan, qalab badan oo otomaatig ah ayaa la siidaayay labadaba si loogu dhiso sawirada Docker iyo geynta Kubernetes. Marka la eego arrintan, waxaan go'aansaday inaan la ciyaaro GitLab, si fiican u darso awoodeeda iyo, dabcan, waxaan dejiyaa dhuumaha.

Shaqadan waxaa dhiiri galiyay mareegaha kubernetes.io, kaas oo ka soo jeeda codes isha si toos ah, iyo codsi kasta oo barkad la soo diro, robotku wuxuu si toos ah u soo saarayaa nooca horudhaca ee goobta oo leh isbeddelladaada wuxuuna bixiyaa isku xirka daawashada.

Waxaan isku dayay inaan ka dhiso nidaam la mid ah xoqan, laakiin gebi ahaanba lagu dhisay Gitlab CI iyo qalabyada bilaashka ah ee aan isticmaalo si aan u geeyo codsiyada Kubernetes. Maanta waxaan ugu dambeyntii kuu sheegi doonaa wax badan oo iyaga ku saabsan.

Maqaalku wuxuu ka hadli doonaa qalabka sida:
Hugo, qbec, kaniko, git-crypto и GitLab CI iyadoo la abuurayo jawi firfircoon.

Nuxurka

1. La kulan Hugo
2. Diyaarinta Dockerfile
3. Barashada kaniko
4. Barashada qbec
5. Isku dayga Gitlab-orodiyaha leh Kubernetes-fuliyaha
6. Gelinaya jaantusyada Helm ee leh qbec
7. Soo bandhigida git-crypt
8. Abuuritaanka sawirka sanduuqa qalabka
9. Dhuumahayaga ugu horreeya iyo isu-ururinta sawirada tags
10. Automation geynta
11. Farshaxanka iyo isu-ururinta marka la riixayo si loo maareeyo
12. Deegaan firfircoon
13. Dib u eeg Apps

1. Barashada Hugo

Tusaale ahaan mashruucayaga, waxaan isku dayi doonaa inaan abuurno goobta daabacaadda dukumentiyada oo lagu dhisay Hugo. Hugo waa dhaliye nuxur ah oo taagan.

Kuwa aan aqoon u lahayn koronto-dhaliyeyaasha taagan, waxaan wax yar kaaga sheegi doonaa iyaga. Si ka duwan matoorada mareegaha caadiga ah ee leh database-ka iyo qaar ka mid ah PHP, kuwaas oo, marka uu codsado isticmaaluhu, soo saaro boggaga duullimaadka, koronto-dhaliyeyaasha joogtada ah ayaa loo qaabeeyey si ka duwan. Waxay kuu oggolaanayaan inaad qaadato ilo, badiyaa faylal ah oo ku jira calaamadaynta Markdown iyo mawduucyada mawduucyada, ka dibna ku ururiso shabakad gebi ahaanba dhammaatay.

Taasi waa, natiijadu, waxaad heli doontaa qaab-dhismeedka hagaha iyo faylal HTML ah oo la soo saaray, kuwaas oo aad si fudud ugu dhejin karto martigelin kasta oo jaban oo aad hesho degel shaqo.

Waxaad ku rakibi kartaa Hugo gudaha oo isku day:

Bilaabida goob cusub:

hugo new site docs.example.org

Isla mar ahaantaana kaydka git:

cd docs.example.org
git init

Ilaa hadda, goobtayadu waa mid nadiif ah oo si ay wax uga soo baxaan, waxaan marka hore u baahannahay inaan ku xirno mawduuc; mawduuca waa uun jaantusyo iyo xeerar cayiman oo goobtayada lagu soo saaray.

Dulucda waxaan u isticmaali doonaa Baro, kaas oo, fikradayda, si fiican ugu habboon goobta dukumeentigu.

Waxaan jeclaan lahaa inaan fiiro gaar ah u yeelo xaqiiqda ah inaanan u baahnayn inaan kaydino faylalka mawduuca ee ku jira kaydka mashruuca; taa beddelkeeda, waxaan si fudud ugu xidhi karnaa annaga oo adeegsanayna git submodule:

git submodule add https://github.com/matcornic/hugo-theme-learn themes/learn

Sidan, kaydkayagu wuxuu ka koobnaan doonaa oo keliya faylal toos ah oo la xidhiidha mashruucayaga, mawduuca ku xidhanna waxa uu ahaan doonaa xidhiidhiye kayd gaar ah iyo ballan, taas oo ah, had iyo jeer waa laga soo jiidan karaa isha asalka ah oo aan ka baqin. ee isbeddellada aan isqaban karin.

Aan saxno qaabeynta config.toml:

baseURL = "http://docs.example.org/"
languageCode = "en-us"
title = "My Docs Site"
theme = "learn"

Durba marxaladan waxaad ku orod kartaa:

hugo server

Iyo cinwaanka http://localhost:1313/ hubi boggayaga cusub ee la abuuray, dhammaan isbeddellada lagu sameeyay hagaha si toos ah u cusbooneysii bogga furan ee browserka, aad u habboon!

Aan isku dayno inaan abuurno bog dabool ah gudaha nuxurka/_index.md:

# My docs site

## Welcome to the docs!

You will be very smart :-)

Sawir-qaadista bogga cusub ee la sameeyay

Si aad u abuurto goob, kaliya orod:

hugo

Waxa ku jira hagaha dadweyne/ oo noqon doona mareegtaada.
Haa, jidka, aan isla markiiba ku darno .gitigoore:

echo /public > .gitignore

Ha iloobin inaad sameyso isbedeladayada:

git add .
git commit -m "New site created"

2. Diyaarinta Dockerfile-ka

Waa waqtigii la qeexi lahaa qaab-dhismeedka kaydkayaga. Caadi ahaan waxaan isticmaalaa wax sida:

.
├── deploy
│   ├── app1
│   └── app2
└── dockerfiles
    ├── image1
    └── image2

• dockerfiles/ - waxaa ku jira hagayaal wata Dockerfiles iyo wax kasta oo lagama maarmaanka u ah dhisidda sawirada Docker-keena.
• geyn/ - waxaa ku jira hagayaal loogu diro codsiyadayada Kubernetes

Markaa, waxaan ku abuuri doonaa Dockerfile-kayaga ugu horreeya ee jidka dockerfiles/website/Dockerfile

FROM alpine:3.11 as builder
ARG HUGO_VERSION=0.62.0
RUN wget -O- https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_linux-64bit.tar.gz | tar -xz -C /usr/local/bin
ADD . /src
RUN hugo -s /src

FROM alpine:3.11
RUN apk add --no-cache darkhttpd
COPY --from=builder /src/public /var/www
ENTRYPOINT [ "/usr/bin/darkhttpd" ]
CMD [ "/var/www" ]

Sida aad arki karto, Dockerfile wuxuu ka kooban yahay laba KA, sifadan waxaa loo yaqaan dhismo heerar badan leh waxayna kuu ogolaaneysaa inaad ka saarto wax kasta oo aan loo baahnayn sawirka docker-ka ugu dambeeya.
Sidaa darteed, sawirka kama dambaysta ah wuxuu ka koobnaan doonaa oo keliya madowhttpd (server HTTP fudud) iyo dadweyne/ - waxa ku jira shabakadeena si habaysan loo soo saaray.

Ha iloobin inaad sameyso isbedeladayada:

git add dockerfiles/website
git commit -m "Add Dockerfile for website"

3. Barashada kaniko

Sawir dhise ahaan, waxaan go'aansaday inaan isticmaalo kaniko,maadaama hawl-galkeedu aanu u baahnayn daemon docker-ka,dhismaha laftiisana waxa lagu fulin karaa mishiin kasta oo kaydka si toos ah loogu kaydin karaa diiwaanka,taas oo meesha ka saaraysa baahida loo qabo kayd joogto ah oo dhamaystiran.

Si aad u dhisto sawirka, kaliya ku orod weelka kaniko fuliye oo u gudbi macnaha dhismaha hadda jira; tan sidoo kale waxaa lagu samayn karaa gudaha, iyada oo loo marayo docker:

docker run -ti --rm 
  -v $PWD:/workspace 
  -v ~/.docker/config.json:/kaniko/.docker/config.json:ro 
  gcr.io/kaniko-project/executor:v0.15.0 
  --cache 
  --dockerfile=dockerfiles/website/Dockerfile 
  --destination=registry.gitlab.com/kvaps/docs.example.org/website:v0.0.1

Xagee registry.gitlab.com/kvaps/docs.example.org/website - magaca sawirka docker-ka; ka dib marka la dhiso, waxaa si toos ah loo bilaabi doonaa diiwaanka docker-ka.

Xildhibaan --cache wuxuu kuu ogolaanayaa inaad kaydiso lakabyada diiwaanka docker; tusaale ahaan, waxay ku kaydsan doonaan gudaha registry.gitlab.com/kvaps/docs.example.org/website/cache, laakiin waxaad cayimi kartaa waddo kale adoo isticmaalaya cabbirka --cache-repo.

Screenshot ee docker-registry

4. Barashada qbec

Qbec waa qalab geyn kuu ogolaanaya inaad si cad u qeexdo muujinta codsigaaga oo aad geyso Kubernetes. Isticmaalka Jsonnet sida syntax-ka ugu weyn wuxuu kuu ogolaanayaa inaad si weyn u fududayso sharaxaadda kala duwanaanshaha deegaan badan, iyo sidoo kale ku dhawaad ​​gebi ahaanba tirtirto ku celcelinta koodka.

Tani waxay si gaar ah run u noqon kartaa xaaladaha aad u baahan tahay inaad geyso arji dhowr rucubood oo leh cabbiro kala duwan oo aad rabto inaad si cad ugu qeexdo Git.

Qbec waxay sidoo kale kuu ogolaaneysaa inaad bixiso jaantusyada Helm adigoo u gudbinaya xuduudaha lagama maarmaanka ah ka dibna u shaqeysi si la mid ah calaamadaha caadiga ah, oo ay ku jiraan inaad u adeegsan karto isbeddello kala duwan iyaga, tanina waxay kuu ogolaaneysaa inaad ka takhalusto baahida Isticmaal ChartMuseum. Taasi waa, waxaad ku kaydin kartaa oo aad si toos ah uga dhigi kartaa jaantusyada git, halka ay ka tirsan yihiin.

Sidaan horey u sheegay, waxaan ku kaydin doonaa dhammaan meelaynta meel tusaha ah geyn/:

mkdir deploy
cd deploy

Aan bilowno codsigeena koowaad:

qbec init website
cd website

Hadda qaab dhismeedka codsigeenu wuxuu u eg yahay sidan:

.
├── components
├── environments
│   ├── base.libsonnet
│   └── default.libsonnet
├── params.libsonnet
└── qbec.yaml

bal aan eegno faylka qbec.yaml:

apiVersion: qbec.io/v1alpha1
kind: App
metadata:
  name: website
spec:
  environments:
    default:
      defaultNamespace: docs
      server: https://kubernetes.example.org:8443
  vars: {}

Halkan waxaan ugu horrayn xiisaynayaa spec.deegaan, qbec waxay horey noogu abuurtay jawi aan caadi ahayn oo waxay ka qaadatay cinwaanka server-ka, iyo sidoo kale magaca kubeconfig-kayaga hadda.
Hadda marka la geynayo Default deegaanka, qbec had iyo jeer waxay geyn doontaa kaliya kooxda Kubernetes ee la cayimay iyo meesha magaca la cayimay, taas oo ah, uma baahnid inaad mar dambe u kala beddesho macnaha guud iyo meelaha magacyada si aad u samayso hawlgelin.
Haddii loo baahdo, waxaad mar walba cusbooneysiin kartaa dejinta faylkan.

Dhammaan deegaankaaga ayaa lagu sifeeyay qbec.yaml, iyo in faylka params.libsonnet, halkaas oo ay ku qoran tahay meesha laga helo halbeegyada iyaga.

Marka xigta waxaan aragnaa laba hage:

• qaybaha / - Dhammaan caddaymaha codsigayagu halkan ayaa lagu kaydin doonaa; waxaa lagu qeexi karaa labadaba jsonnet iyo faylasha yaml ee caadiga ah
• deegaan/ - Halkan waxaan ku qeexi doonaa dhammaan doorsoomayaasha (parameters) ee deegaankeena.

Sida caadiga ah waxaan haynaa laba fayl:

• deegaanka/base.libsonnet - waxay ka koobnaan doontaa halbeegyo guud oo deegaan oo dhan ah
• deegaanka/default.libsonnet - waxay ka kooban tahay cabbirro lagu xad-gudbay deegaanka Default

aan furno deegaanka/base.libsonnet oo ku dar halbeegyada qaybtayada ugu horreysa halkaas:

{
  components: {
    website: {
      name: 'example-docs',
      image: 'registry.gitlab.com/kvaps/docs.example.org/website:v0.0.1',
      replicas: 1,
      containerPort: 80,
      servicePort: 80,
      nodeSelector: {},
      tolerations: [],
      ingressClass: 'nginx',
      domain: 'docs.example.org',
    },
  },
}

Aynu sidoo kale abuurno qaybteena koowaad qaybaha/website.jsonnet:

local env = {
  name: std.extVar('qbec.io/env'),
  namespace: std.extVar('qbec.io/defaultNs'),
};
local p = import '../params.libsonnet';
local params = p.components.website;

[
  {
    apiVersion: 'apps/v1',
    kind: 'Deployment',
    metadata: {
      labels: { app: params.name },
      name: params.name,
    },
    spec: {
      replicas: params.replicas,
      selector: {
        matchLabels: {
          app: params.name,
        },
      },
      template: {
        metadata: {
          labels: { app: params.name },
        },
        spec: {
          containers: [
            {
              name: 'darkhttpd',
              image: params.image,
              ports: [
                {
                  containerPort: params.containerPort,
                },
              ],
            },
          ],
          nodeSelector: params.nodeSelector,
          tolerations: params.tolerations,
          imagePullSecrets: [{ name: 'regsecret' }],
        },
      },
    },
  },
  {
    apiVersion: 'v1',
    kind: 'Service',
    metadata: {
      labels: { app: params.name },
      name: params.name,
    },
    spec: {
      selector: {
        app: params.name,
      },
      ports: [
        {
          port: params.servicePort,
          targetPort: params.containerPort,
        },
      ],
    },
  },
  {
    apiVersion: 'extensions/v1beta1',
    kind: 'Ingress',
    metadata: {
      annotations: {
        'kubernetes.io/ingress.class': params.ingressClass,
      },
      labels: { app: params.name },
      name: params.name,
    },
    spec: {
      rules: [
        {
          host: params.domain,
          http: {
            paths: [
              {
                backend: {
                  serviceName: params.name,
                  servicePort: params.servicePort,
                },
              },
            ],
          },
        },
      ],
    },
  },
]

Faylkan waxaanu ku sharaxnay saddex hay'adood oo Kubernetes ah hal mar, kuwanu waa: Bixitaan, Service и Ingress. Haddii aan rabno, waxaan ku dhejin karnaa qaybo kala duwan, laakiin marxaladdan mid ayaa nagu filan.

Saan jsonnet wuxuu aad ugu eg yahay json caadiga ah, mabda'a ahaan, json caadiga ah waa horeba ansax yahay jsonnet, marka marka hore waxaa laga yaabaa inay kuu fududaato inaad isticmaasho adeegyada khadka ah sida yaml2json si aad u bedesho yamlkaaga caadiga ah json, ama, haddii qaybahaagu aanay ku jirin wax doorsoomayaal ah, markaas waxa lagu sifayn karaa qaab yamleed caadi ah.

Markaad la shaqaynayso jsonnet Waxaan aad ugu talinayaa inaad ku rakibto plugin tafatirahaaga

Tusaale ahaan, waxaa jira plugin loogu talagalay vim vim-jsonnet, kaas oo daara muujinta syntax oo si toos ah u fuliya jsonnet fmt mar kasta oo aad kaydsato (waxay u baahan tahay in la rakibo jsonnet).

Wax walba waa diyaar, hadda waxaan bilaabi karnaa geynta:

Si aan u aragno waxa aan helnay, aan orodno:

qbec show default

Marka la soo saaro, waxaad arki doontaa muujinta yaml ee la soo bandhigay kuwaas oo lagu dabaqi doono kutlada caadiga ah.

Wacan, hadda codso:

qbec apply default

Soo saarida waxaad had iyo jeer arki doontaa waxa laga qaban doono kutladaada, qbec waxay ku waydiin doontaa inaad ogolaato isbedelada adoo ku qoraya y waxaad awoodi doontaa inaad xaqiijiso waxa aad damacsan tahay.

Codsigayagu waa diyaar oo waa la geeyay!

Haddii aad isbedel samayso, waxaad had iyo jeer samayn kartaa:

qbec diff default

si aan u aragno sida isbeddelladani u saameeyaan hawlgelinta hadda jirta

Ha iloobin inaad sameyso isbedeladayada:

cd ../..
git add deploy/website
git commit -m "Add deploy for website"

5. Isku dayga Gitlab-orodeeyaha Kubernetes-fuliyaha

Ilaa dhawaan waxaan kaliya isticmaali jiray si caadi ah gitlab-orodyahan Mashiin horay loo sii diyaariyay ( weel LXC ah) oo leh qolof ama docker-fuliye. Markii hore, waxaan haysanay dhowr orodyahan oo si caalami ah loogu qeexay gitlabkeena. Waxay u ururiyeen sawirada docker ee dhammaan mashaariicda.

Laakiin sida ficilku muujiyay, doorashadani maaha mid ugu habboon, labadaba marka la eego waxqabadka iyo badbaadada. Aad bay u fiican tahay oo fikir ahaan aad uga saxsan tahay in orodyahanno gaar ah loo diro mashruuc kasta, ama xitaa deegaan kasta.

Nasiib wanaag, tani maahan wax dhibaato ah, maadaama hadda aan hawlgelin doono gitlab-orodyahan si toos ah qayb ka mid ah mashruucayaga saxda ah ee Kubernetes.

Gitlab wuxuu bixiyaa jaantus koofiyad diyaarsan oo loogu dirayo gitlab-orodiyaha Kubernetes. Markaa waxa kaliya ee aad u baahan tahay inaad ogaato calaamadda diiwaangelinta mashruuceena in Settings -> CI / CD -> Runners una gudbi madaxa:

helm repo add gitlab https://charts.gitlab.io

helm install gitlab-runner 
  --set gitlabUrl=https://gitlab.com 
  --set runnerRegistrationToken=yga8y-jdCusVDn_t4Wxc 
  --set rbac.create=true 
  gitlab/gitlab-runner

Xagee:

• https://gitlab.com - ciwaanka serverkaaga Gitlab
• yga8y-jdCusVDn_t4Wxc - calaamada diiwaangelinta mashruucaaga.
• rbac.create=run - waxay siisaa orodyahanka qadarka lagama maarmaanka ah ee mudnaanta si uu u awoodo inuu abuuro pods si uu u guto hawlaheenna iyadoo la adeegsanayo kubernetes-executor.

Haddii wax walba si sax ah loo sameeyo, waa inaad qaybta ku aragto orodyahan diiwaangashan orodyahano, ee goobaha mashruucaaga.

Sawir-qaadista orodyahanka lagu daray

Ma sidaas baa fudud? - Haa, way fududahay! Wax dhib ah oo dambe ma leh in orodyahannada gacanta lagu diwaangeliyo, hadda wixii ka dambeeya orodyahannada waa la abuuri doonaa oo si toos ah ayaa loo burburin doonaa.

6. Geli jaantusyada Helm ee leh QBEC

Tan iyo markii aan go'aansanay inaan tixgelinno gitlab-orodyahan qayb ka mid ah mashruucayaga, waa waqtigii lagu qeexi lahaa kaydka Git.

Waxaan ku tilmaami karnaa inay tahay qayb gaar ah website, laakiin mustaqbalka waxaan qorsheyneynaa inaan geyno nuqullo kala duwan website marar badan, ka duwan gitlab-orodyahan, kaas oo la hawlgelin doono hal mar oo keliya koox kasta oo Kubernetes ah. Haddaba aan u bilowno codsi gooni ah:

cd deploy
qbec init gitlab-runner
cd gitlab-runner

Markan kuma sifayn doono hay'adaha Kubernetes gacanta, laakiin waxaan qaadan doonaa jaantus diyaarsan oo Helm ah. Mid ka mid ah faa'iidooyinka qbec waa awoodda in si toos ah loo soo saaro jaantusyada Helm ee kaydka Git.

Aynu ku xidhno git submodule:

git submodule add https://gitlab.com/gitlab-org/charts/gitlab-runner vendor/gitlab-runner

Hadda tusaha iibiyaha/gitlab-orodeeyaha Waxaan haynaa meel kayd ah oo leh jaantus loogu talagalay orodyahanka gitlab.

Si la mid ah, waxaad ku xidhi kartaa kaydka kale, tusaale ahaan, kaydka oo dhan oo leh jaantusyo rasmi ah https://github.com/helm/charts

Aynu qeexno qaybta qaybaha/gitlab-runner.jsonnet:

local env = {
  name: std.extVar('qbec.io/env'),
  namespace: std.extVar('qbec.io/defaultNs'),
};
local p = import '../params.libsonnet';
local params = p.components.gitlabRunner;

std.native('expandHelmTemplate')(
  '../vendor/gitlab-runner',
  params.values,
  {
    nameTemplate: params.name,
    namespace: env.namespace,
    thisFile: std.thisFile,
    verbose: true,
  }
)

Doodda ugu horreysa ee ku socota ballaarintaHelmTemplate waxaan u mareynaa dariiqa shaxda, markaa params.qiimaha, kaas oo aan ka soo qaadanno xuduudaha deegaanka, ka dibna la yimaado shayga

• magacaTemplate - Magaca sii daynta
• magacooda - booska magaca oo loo wareejiyay madaxa
• Faylkan - halbeeg loo baahan yahay oo ka gudbaya dariiqa faylka hadda jira
• hadal ahaan - waxay muujinaysaa amarka template koofiyadda oo leh dhammaan doodaha marka la bixinayo jaantuska

Hadda aynu qeexno cabbirrada qaybtayada deegaanka/base.libsonnet:

local secrets = import '../secrets/base.libsonnet';

{
  components: {
    gitlabRunner: {
      name: 'gitlab-runner',
      values: {
        gitlabUrl: 'https://gitlab.com/',
        rbac: {
          create: true,
        },
        runnerRegistrationToken: secrets.runnerRegistrationToken,
      },
    },
  },
}

Feejignow runnerRegistrationToken waxaan kasoo qaadanaa fayl dibadeed siraha/base.libsonnet, aynu abuurno:

{
  runnerRegistrationToken: 'yga8y-jdCusVDn_t4Wxc',
}

Aynu eegno haddii wax walba ay shaqeeyaan:

qbec show default

Haddii wax walba ay hagaagsan yihiin, markaa waxaan tirtiri karnaa sii-deyntayadii hore ee Helm:

helm uninstall gitlab-runner

oo u dhig si la mid ah, laakiin iyada oo loo marayo qbec:

qbec apply default

7. Hordhac git-crypt

Git-crypt waa qalab kuu ogolaanaya inaad u dejiso sirta hufan ee kaydkaaga.

Waqtigan xaadirka ah, qaab dhismeedka hagaha ee gitlab-runner wuxuu u eg yahay sidan:

.
├── components
│   ├── gitlab-runner.jsonnet
├── environments
│   ├── base.libsonnet
│   └── default.libsonnet
├── params.libsonnet
├── qbec.yaml
├── secrets
│   └── base.libsonnet
└── vendor
    └── gitlab-runner (submodule)

Laakin kaydinta siraha gudaha Git maahan badbaado, miyaa? Markaa waxaan u baahanahay inaan si sax ah u sirno.

Caadi ahaan, hal doorsoome aawadood, tani mar walba macno ma samaynayso. Waxaad u wareejin kartaa siraha qbec iyo iyada oo loo marayo doorsoomayaasha deegaanka ee nidaamkaaga CI.
Laakin waxaa xusid mudan in sidoo kale ay jiraan mashruucyo kakan oo ka koobnaan kara siro badan, ku wareejinta dhamaan doorsoomayaasha deegaanka waxay noqon doontaa mid aad u adag.

Waxaa intaa dheer, kiiskan ma awoodi doono inaan kuu sheego qalab cajiib ah sida git-crypto.

git-crypto Waxa kale oo ay ku habboon tahay in ay kuu ogolaato inaad kaydiso taariikhda siraha oo dhan, iyo sidoo kale isbarbardhigga, isku darka iyo xalinta khilaafaadka si la mid ah sida aan u isticmaalno inaan ku samayno kiiska Git.

Waxa ugu horreeya ka dib rakibidda git-crypto waxaan u baahanahay inaan abuurno furayaasha kaydkayaga:

git crypt init

Haddii aad leedahay furaha PGP, markaa waxaad isla markiiba ku dari kartaa naftaada iskaashi ahaan mashruucan:

git-crypt add-gpg-user [email protected]

Sidan ayaad had iyo jeer furahaaga u qarin kartaa kaydkan adoo isticmaalaya furahaaga gaarka ah.

Haddii aadan haysan furaha PGP oo aadan filaynin, markaa waxaad mari kartaa dariiqa kale oo aad dhoofin kartaa furaha mashruuca:

git crypt export-key /path/to/keyfile

Sidaa darteed, qof kasta oo haysta dhoofin keyfile waxay awoodi doontaa inay furto kaydkaaga

Waa markii aan dejin lahayn sirtayada koowaad.
Aan ku xasuusiyo in aan wali ku jirno buugga hagaha geyn/gitlab-orodeeyaha/, halkaas oo aan ku hayno hagaha siraha/, aynu sir u dhigno dhammaan faylasha ku jira, taas awgeed waxaan u abuuri doonaa fayl siraha/.gitattributes oo leh nuxurka soo socda:

* filter=git-crypt diff=git-crypt
.gitattributes !filter !diff

Sida ka muuqata nuxurka, dhammaan faylasha waa la daboolay * waa la dhex mari doonaa git-crypto, marka laga reebo inta ugu badan .gitattributes

Waxaan ku hubin karnaa tan annagoo ordaya:

git crypt status -e

Wax-soo-saarku wuxuu noqon doonaa liiska dhammaan faylasha ku jira kaydka kaas oo sirtu awood u leedahay

Taasi waa dhan, hadda waxaan si badbaado leh u samayn karnaa isbeddelladayada:

cd ../..
git add .
git commit -m "Add deploy for gitlab-runner"

Si aad u xannibto kaydka, kaliya orod:

git crypt lock

isla markaaba dhammaan faylasha sir ah waxay isu rogi doonaan wax binary, ma noqon doonto wax aan macquul aheyn in la akhriyo.
Si aad u furto kaydka, orod:

git crypt unlock

8. Samee sawirka sanduuqa qalabka

Sawirka sanduuqa qalabku waa sawir leh dhammaan agabyada aan u isticmaali doono si aan u geyno mashruucayaga. Waxaa isticmaali doona orodyahanka Gitlab si uu u qabto hawlo geyn caadi ah.

Wax walba waa ku fudud yihiin halkan, aan abuurno mid cusub dockerfiles/toolbox/Dockerfile oo leh nuxurka soo socda:

FROM alpine:3.11

RUN apk add --no-cache git git-crypt

RUN QBEC_VER=0.10.3 
 && wget -O- https://github.com/splunk/qbec/releases/download/v${QBEC_VER}/qbec-linux-amd64.tar.gz 
     | tar -C /tmp -xzf - 
 && mv /tmp/qbec /tmp/jsonnet-qbec /usr/local/bin/

RUN KUBECTL_VER=1.17.0 
 && wget -O /usr/local/bin/kubectl 
      https://storage.googleapis.com/kubernetes-release/release/v${KUBECTL_VER}/bin/linux/amd64/kubectl 
 && chmod +x /usr/local/bin/kubectl

RUN HELM_VER=3.0.2 
 && wget -O- https://get.helm.sh/helm-v${HELM_VER}-linux-amd64.tar.gz 
     | tar -C /tmp -zxf - 
 && mv /tmp/linux-amd64/helm /usr/local/bin/helm

Sida aad arki karto, sawirkan waxaan ku rakibnay dhammaan adeegyada aan adeegsan jirnay codsigayaga. Uma baahnid halkan ilaa kubectl, laakiin waxaa laga yaabaa inaad rabto inaad la ciyaarto inta lagu jiro marxaladda habaynta dhuumaha.

Sidoo kale, si aan awood ugu yeelanno inaan la xiriirno Kubernetes oo aan u dirno, waxaan u baahanahay inaan u habeyno doorka pods-ka uu sameeyay gitlab-runner.

Si tan loo sameeyo, aan tagno hagaha gitlab-runner:

cd deploy/gitlab-runner

oo ku dar qayb cusub qaybaha/rbac.jsonnet:

local env = {
  name: std.extVar('qbec.io/env'),
  namespace: std.extVar('qbec.io/defaultNs'),
};
local p = import '../params.libsonnet';
local params = p.components.rbac;

[
  {
    apiVersion: 'v1',
    kind: 'ServiceAccount',
    metadata: {
      labels: {
        app: params.name,
      },
      name: params.name,
    },
  },
  {
    apiVersion: 'rbac.authorization.k8s.io/v1',
    kind: 'Role',
    metadata: {
      labels: {
        app: params.name,
      },
      name: params.name,
    },
    rules: [
      {
        apiGroups: [
          '*',
        ],
        resources: [
          '*',
        ],
        verbs: [
          '*',
        ],
      },
    ],
  },
  {
    apiVersion: 'rbac.authorization.k8s.io/v1',
    kind: 'RoleBinding',
    metadata: {
      labels: {
        app: params.name,
      },
      name: params.name,
    },
    roleRef: {
      apiGroup: 'rbac.authorization.k8s.io',
      kind: 'Role',
      name: params.name,
    },
    subjects: [
      {
        kind: 'ServiceAccount',
        name: params.name,
        namespace: env.namespace,
      },
    ],
  },
]

Waxaan sidoo kale ku tilmaami doonaa xuduudaha cusub gudaha deegaanka/base.libsonnet, oo hadda u eg sidan:

local secrets = import '../secrets/base.libsonnet';

{
  components: {
    gitlabRunner: {
      name: 'gitlab-runner',
      values: {
        gitlabUrl: 'https://gitlab.com/',
        rbac: {
          create: true,
        },
        runnerRegistrationToken: secrets.runnerRegistrationToken,
        runners: {
          serviceAccountName: $.components.rbac.name,
          image: 'registry.gitlab.com/kvaps/docs.example.org/toolbox:v0.0.1',
        },
      },
    },
    rbac: {
      name: 'gitlab-runner-deploy',
    },
  },
}

Feejignow $.qaybaha.rbac.magaca waxaa loola jeedaa magaca qayb ahaan rbac

Aynu eegno waxa isbedelay:

qbec diff default

oo ku dabaq isbedeladayada Kubernetes:

qbec apply default

Sidoo kale, ha ilaawin inaad sameyso isbedeladayada git:

cd ../..
git add dockerfiles/toolbox
git commit -m "Add Dockerfile for toolbox"
git add deploy/gitlab-runner
git commit -m "Configure gitlab-runner to use toolbox"

9. Dhuumahayaga ugu horreeya iyo isu-ururinta sawirada tags

Asalka mashruuca waxaan ku abuuri doonaa .gitlab-ci.yml oo leh nuxurka soo socda:

.build_docker_image:
  stage: build
  image:
    name: gcr.io/kaniko-project/executor:debug-v0.15.0
    entrypoint: [""]
  before_script:
    - echo "{"auths":{"$CI_REGISTRY":{"username":"$CI_REGISTRY_USER","password":"$CI_REGISTRY_PASSWORD"}}}" > /kaniko/.docker/config.json

build_toolbox:
  extends: .build_docker_image
  script:
    - /kaniko/executor --cache --context $CI_PROJECT_DIR/dockerfiles/toolbox --dockerfile $CI_PROJECT_DIR/dockerfiles/toolbox/Dockerfile --destination $CI_REGISTRY_IMAGE/toolbox:$CI_COMMIT_TAG
  only:
    refs:
      - tags

build_website:
  extends: .build_docker_image
  variables:
    GIT_SUBMODULE_STRATEGY: normal
  script:
    - /kaniko/executor --cache --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/dockerfiles/website/Dockerfile --destination $CI_REGISTRY_IMAGE/website:$CI_COMMIT_TAG
  only:
    refs:
      - tags

Fadlan ogsoonow waan isticmaalnaa GIT_SUBMODULE_STRATEGY: caadi shaqooyinkaas meesha aad u baahan tahay inaad si cad u bilawdo submodules ka hor inta aan la fulin.

Ha iloobin inaad sameyso isbedeladayada:

git add .gitlab-ci.yml
git commit -m "Automate docker build"

Waxaan u maleynayaa inaan si badbaado leh ugu yeeri karno kan nooc v0.0.1 kuna dar summada:

git tag v0.0.1

Waxaan ku dari doonaa tags mar kasta oo aan u baahanahay inaan sii daayo version cusub. Tagsyada sawirada Docker waxa lagu xidhi doonaa tags Git. Riix kasta oo leh sumad cusub ayaa bilaabi doonta dhismaha sawirada sumaddan.

Aan samayno git riix --tags, oo aan eegno dhuumahayaga ugu horreeya:

Sawirka dhuumaha koowaad

Waxaa habboon in la soo jiito dareenkaaga xaqiiqda ah in isu imaatinka calaamaduhu ay ku habboon yihiin dhisidda sawirada docker, laakiin kuma habboona in la geeyo codsiga Kubernetes. Maadaama calaamado cusub loo qoondeyn karo ballanqaadyadii hore, kiiskan, bilaabista dhuumaha iyaga waxay u horseedi doontaa in la geeyo noocii hore.

Si loo xalliyo dhibaatadan, sida caadiga ah dhismaha sawirada dockerka waxay ku xidhan yihiin tags, iyo geynta codsiga laanta Master, kuwaas oo noocyada sawirada la ururiyey ay yihiin kuwo adag. Tani waa meesha aad ka bilaabi karto dib-u-soo-celinta iyadoo si fudud loo celinayo Master-laamo.

10. Automation ee meelaynta

Si Gitlab-orodeeyaha u furfuro sirahayaga, waxaan u baahan doonaa inaan dhoofino furaha kaydka oo aan ku darno doorsoomayaasha deegaanka CI:

git crypt export-key /tmp/docs-repo.key
base64 -w0 /tmp/docs-repo.key; echo

Waxaan keydin doonaa xariiqda natiijada ee Gitlab; si tan loo sameeyo, aan tagno goobaha mashruuca:
Settings -> CI / CD -> Variables

Oo aynu abuurno doorsoome cusub:

nooca
Key
Qiimaha
Difaacay
Masked
Baaxadda

File
GITCRYPT_KEY
<your string>
true (inta lagu jiro tababarka waxaad awoodi kartaa false)
true
All environments

Sawirka doorsoomaha lagu daray

Hadda aynu cusbooneysiinno our .gitlab-ci.yml isagoo ku daray:

.deploy_qbec_app:
  stage: deploy
  only:
    refs:
      - master

deploy_gitlab_runner:
  extends: .deploy_qbec_app
  variables:
    GIT_SUBMODULE_STRATEGY: normal
  before_script:
    - base64 -d "$GITCRYPT_KEY" | git-crypt unlock -
  script:
    - qbec apply default --root deploy/gitlab-runner --force:k8s-context __incluster__ --wait --yes

deploy_website:
  extends: .deploy_qbec_app
  script:
    - qbec apply default --root deploy/website --force:k8s-context __incluster__ --wait --yes

Halkan waxaan awood u siinnay dhowr ikhtiyaar oo cusub oo qbec ah:

• --xidid qaar/app - Waxay kuu ogolaaneysaa inaad go'aamiso hagaha codsi gaar ah
• --force:k8s-context __incluster__ - Kani waa doorsoome sixir ah oo sheegaya in diritaanku ku dhici doono isla kooxda uu gtilab-runner ku socdo. Tani waa lagama maarmaan sababtoo ah haddii kale qbec waxay isku dayi doontaa inay ka hesho kubernetes server ku habboon kubeconfig kaaga
• --sug - waxay ku qasbeysaa qbec in ay sugaan ilaa kheyraadka ay abuureyso ay galaan gobolka Diyaarsan oo kaliya ka baxitaanka si guul leh koodka bixista.
• -haa - waxay si fudud u baabi'isaa qolofka is-dhexgalka Ma hubtaa? marka la geeyo.

Ha iloobin inaad sameyso isbedeladayada:

git add .gitlab-ci.yml
git commit -m "Automate deploy"

Kadibna riixid waxaan arki doonaa sida codsiyadayada loo geeyay:

Sawirka dhuumaha labaad

11. Farshaxanka iyo isu-ururinta marka la riixayo si loo maareeyo

Caadi ahaan, tillaabooyinka kor lagu sharraxay waa ku filan yihiin in la dhiso oo la keeno ku dhawaad ​​adeeg kasta oo yar yar, laakiin ma doonayno inaan ku darno sumad mar kasta oo aan u baahanahay inaan cusbooneysiino goobta. Sidaa darteed, waxaanu qaadi doonaa dariiq aad u firfircoon oo waxaanu dejin doonaa qaybinta dheefshiidka ee laanta sayidkiisa.

Fikraddu waa mid fudud: hadda sawirkayaga website dib ayaa loo dhisi doonaa mar kasta oo aad riixdo Master, ka dibna si toos ah u geyn Kubernetes.

Aynu cusboonaysiiyo labadan shaqo ee our .gitlab-ci.yml:

build_website:
  extends: .build_docker_image
  variables:
    GIT_SUBMODULE_STRATEGY: normal
  script:
    - mkdir -p $CI_PROJECT_DIR/artifacts
    - /kaniko/executor --cache --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/dockerfiles/website/Dockerfile --destination $CI_REGISTRY_IMAGE/website:$CI_COMMIT_REF_NAME --digest-file $CI_PROJECT_DIR/artifacts/website.digest
  artifacts:
    paths:
      - artifacts/
  only:
    refs:
      - master
      - tags

deploy_website:
  extends: .deploy_qbec_app
  script:
    - DIGEST="$(cat artifacts/website.digest)"
    - qbec apply default --root deploy/website --force:k8s-context __incluster__ --wait --yes --vm:ext-str digest="$DIGEST"

Fadlan ogow inaan ku darnay dun Master к Refs shaqooyinka dhiso_website oo aan hadda isticmaalno $CI_COMMIT_REF_NAME halkii $CI_COMMIT_TAG, taas oo ah, waxaa nalaga furay tags Git oo hadda waxaan riixi doonaa sawir leh magaca laanta ballanqaad ee bilawday dhuumaha. Waxaa xusid mudan in tani ay sidoo kale la shaqeyn doonto tags, taas oo noo ogolaan doonta inaan badbaadino sawirada goobta oo leh nooc gaar ah oo ku yaal diiwaanka-docker-ka.

Marka magaca docker-ka ee nooca cusub ee goobta uu isbedeli karo, wali waa inaan ku qeexnaa isbedelada Kubernetes, haddii kale si fudud dib uma dhigi doonto codsiga sawirka cusub, maadaama aysan dareemi doonin wax isbedel ah muujinta keenista.

Ikhtiyaar —vm:ext-str digest=”$ DIGEST” for qbec - waxay kuu ogolaanaysaa inaad u gudbiso doorsoome dibadeed jsonnet. Waxaan rabnaa in dib loogu habeeyo kooxda iyadoo la sii daayo codsi kasta oo naga mid ah. Mar dambe ma isticmaali karno magaca sumadda, kaas oo hadda noqon kara mid aan la beddeli karin, maadaama aan u baahanahay in lagu xidho nooc gaar ah oo sawirka ah oo aan kicino geynta marka uu isbeddelo.

Halkan waxaa nagu caawin doona Kaniko awooda uu ku kaydinayo sawirka dheefshiidka faylka (ikhtiyaarka --dijeest-faylka)
Kadibna waan wareejin doonaa faylkan oo aan akhrin doonaa wakhtiga la dirayo.

Aynu cusboonaysiinno halbeegyadayada deploy/website/environments/base.libsonnet oo hadda u ekaan doona sidan:

{
  components: {
    website: {
      name: 'example-docs',
      image: 'registry.gitlab.com/kvaps/docs.example.org/website@' + std.extVar('digest'),
      replicas: 1,
      containerPort: 80,
      servicePort: 80,
      nodeSelector: {},
      tolerations: [],
      ingressClass: 'nginx',
      domain: 'docs.example.org',
    },
  },
}

La dhammeeyay, hadda mid kasta ballan qaado Master u bilaabaya dhismaha sawirka docker ee website, ka dibna u dir Kubernetes.

Ha iloobin inaad sameyso isbedeladayada:

git add .
git commit -m "Configure dynamic build"

Waanu hubin doonaa hadhow riixid waa inaan aragnaa wax sidan oo kale ah:

Sawir-qaadista dhuumaha ee master-ka

Mabda 'ahaan, uma baahnid inaan dib u habeyn ku sameyno gitlab-runner riix kasta, ilaa, dabcan, waxba isma bedelin qaabeyntiisa, aan ku hagaajino .gitlab-ci.yml:

deploy_gitlab_runner:
  extends: .deploy_qbec_app
  variables:
    GIT_SUBMODULE_STRATEGY: normal
  before_script:
    - base64 -d "$GITCRYPT_KEY" | git-crypt unlock -
  script:
    - qbec apply default --root deploy/gitlab-runner --force:k8s-context __incluster__ --wait --yes
  only:
    changes:
      - deploy/gitlab-runner/**/*

isbedel wuxuu kuu ogolaanayaa inaad la socoto isbedelada gudaha geyn/gitlab-orodeeyaha/ oo kicin doona shaqadeena kaliya haddii ay jiraan

Ha iloobin inaad sameyso isbedeladayada:

git add .gitlab-ci.yml
git commit -m "Reduce gitlab-runner deploy"

riixid, taas ayaa ka fiican:

Sawirka dhuumaha la cusboonaysiiyay

12. Deegaan firfircoon

Waxaa la joogaa waqtigii lagu kala duwanayn lahaa dhuumahayaga jawi firfircoon.

Marka hore, aan cusboonaysiinno shaqada dhiso_website in our .gitlab-ci.yml, oo ka saara baloogga oo keliya, kaas oo ku qasbi doona Gitlab inuu kiciyo go'aan kasta oo laan kasta:

build_website:
  extends: .build_docker_image
  variables:
    GIT_SUBMODULE_STRATEGY: normal
  script:
    - mkdir -p $CI_PROJECT_DIR/artifacts
    - /kaniko/executor --cache --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/dockerfiles/website/Dockerfile --destination $CI_REGISTRY_IMAGE/website:$CI_COMMIT_REF_NAME --digest-file $CI_PROJECT_DIR/artifacts/website.digest
  artifacts:
    paths:
      - artifacts/

Kadibna cusboonaysii shaqada deploy_website, ku dar block halkaas deegaanka:

deploy_website:
  extends: .deploy_qbec_app
  environment:
    name: prod
    url: https://docs.example.org
  script:
    - DIGEST="$(cat artifacts/website.digest)"
    - qbec apply default --root deploy/website --force:k8s-context __incluster__ --wait --yes --vm:ext-str digest="$DIGEST"

Tani waxay u oggolaan doontaa Gitlab inuu ku xidho shaqada soo saar deegaanka oo muuji isku xirka saxda ah ee iyada.

Hadda aan ku darno laba shaqo oo kale:

deploy_website:
  extends: .deploy_qbec_app
  environment:
    name: prod
    url: https://docs.example.org
  script:
    - DIGEST="$(cat artifacts/website.digest)"
    - qbec apply default --root deploy/website --force:k8s-context __incluster__ --wait --yes --vm:ext-str digest="$DIGEST"

deploy_review:
  extends: .deploy_qbec_app
  environment:
    name: review/$CI_COMMIT_REF_NAME
    url: http://$CI_ENVIRONMENT_SLUG.docs.example.org
    on_stop: stop_review
  script:
    - DIGEST="$(cat artifacts/website.digest)"
    - qbec apply review --root deploy/website --force:k8s-context __incluster__ --wait --yes --vm:ext-str digest="$DIGEST" --vm:ext-str subdomain="$CI_ENVIRONMENT_SLUG" --app-tag "$CI_ENVIRONMENT_SLUG"
  only:
    refs:
    - branches
  except:
    refs:
      - master

stop_review:
  extends: .deploy_qbec_app
  environment:
    name: review/$CI_COMMIT_REF_NAME
    action: stop
  stage: deploy
  before_script:
    - git clone "$CI_REPOSITORY_URL" master
    - cd master
  script:
    - qbec delete review --root deploy/website --force:k8s-context __incluster__ --yes --vm:ext-str digest="$DIGEST" --vm:ext-str subdomain="$CI_ENVIRONMENT_SLUG" --app-tag "$CI_ENVIRONMENT_SLUG"
  variables:
    GIT_STRATEGY: none
  only:
    refs:
    - branches
  except:
    refs:
      - master
  when: manual

Waxaa la bilaabi doonaa marka la riixo laamo kasta marka laga reebo master-ka waxayna geyn doonaan nooca horudhaca ee goobta.

Waxaan u aragnaa ikhtiyaar cusub qbec: --app-tag - waxay kuu ogolaanaysaa inaad calaamadiso noocyada arjiga la diray oo aad ka shaqeyso summadan oo keliya; marka la abuurayo oo la burburinayo ilaha Kubernetes, qbec waxay la shaqayn doontaa oo keliya.
Sidan ma abuuri karno jawi gaar ah dib u eegis kasta, laakiin si fudud dib u isticmaal mid la mid ah.

Halkan waxaan sidoo kale isticmaalnaa qbec codsan dib u eegis, Halkii qbec codso default - waa wakhtiga aan isku dayi doono inaan qeexno kala duwanaanshaha deegaankeena (dib u eegis iyo dib u eegis):

Aan ku daro dib u eegis deegaanka in geyn/website/qbec.yaml

spec:
  environments:
    review:
      defaultNamespace: docs
      server: https://kubernetes.example.org:8443

Markaas ayaanu ku dhawaaqi doonnaa deploy/website/params.libsonnet:

local env = std.extVar('qbec.io/env');
local paramsMap = {
  _: import './environments/base.libsonnet',
  default: import './environments/default.libsonnet',
  review: import './environments/review.libsonnet',
};

if std.objectHas(paramsMap, env) then paramsMap[env] else error 'environment ' + env + ' not defined in ' + std.thisFile

Oo u qor halbeegyada gaarka ah ee gudaha deploy/website/environments/review.libsonnet:

// this file has the param overrides for the default environment
local base = import './base.libsonnet';
local slug = std.extVar('qbec.io/tag');
local subdomain = std.extVar('subdomain');

base {
  components+: {
    website+: {
      name: 'example-docs-' + slug,
      domain: subdomain + '.docs.example.org',
    },
  },
}

Aynu sidoo kale si dhow u eegno jobu stop_view, waxa la kicin doona marka laanta la tirtiro si aanu gitlab isku dayin in uu hubiyo waxa loo isticmaalaa GIT_STRATEGY: midna, ka dib waanu xidhi doonaa Master- laan oo tirtir dib u eegista iyada oo loo marayo.
Waa wax yar oo jahawareer ah, laakiin weli ma helin hab ka qurux badan.
Ikhtiyaar kale ayaa noqon doona in dib u eegis kasta la geeyo goobta magaca huteelka, kaas oo had iyo jeer la dumin karo gebi ahaanba.

Ha iloobin inaad sameyso isbedeladayada:

git add .
git commit -m "Enable automatic review"

riixid, git checkout -b test, git riix asalka imtixaanka, hubi:

Sawirka deegaanka laga sameeyay Gitlab

Wax walba way shaqaynayaan? - weyn, tirtir laantayada imtixaanka: Git master master, git riix asalka:tijaabada, waxaanu hubinaa in shaqooyinka tirtirka deegaanku ay u shaqeeyeen khaladaad la'aan.

Halkan waxaan jeclaan lahaa inaan isla markiiba caddeeyo in horumariye kasta oo mashruuc ah uu abuuri karo laamo, sidoo kale wuu bedeli karaa .gitlab-ci.yml fayl gareeya oo geli doorsoomayaal sir ah.
Sidaa darteed, waxaa aad loogu talinayaa in loo ogolaado isticmaalkooda oo keliya laamaha la ilaaliyo, tusaale ahaan gudaha Master, ama abuur doorsoomayaal u gaar ah deegaan kasta.

13. Dib u eegis Apps

Dib u eeg Apps Tani waa qaabka GitLab kaas oo kuu ogolaanaya inaad ku darto badhan fayl kasta oo ku jira kaydka si aad si dhakhso ah ugu aragto jawiga la diray.

Si ay badhamadani u soo baxaan, waxaad u baahan tahay inaad abuurto fayl .gitlab/route-map.yml oo sharax dhammaan isbeddellada dariiqyada ku jira; xaaladdeenna waxay noqon doontaa mid aad u fudud:

# Indices
- source: /content/(.+?)_index.(md|html)/ 
  public: '1'

# Pages
- source: /content/(.+?).(md|html)/ 
  public: '1/'

Ha iloobin inaad sameyso isbedeladayada:

git add .gitlab/
git commit -m "Enable review apps"

riixid, oo hubi:

Sawirka badhanka App-ka ee dib u eegista

Shaqada waa la dhammeeyey!

Ilaha mashruuca:

• ee Gitlab: https://gitlab.com/kvaps/docs.example.org
• on GitHub: https://github.com/kvaps/docs.example.org

Waad ku mahadsan tahay dareenkaaga, waxaan rajeynayaa inaad ka heshay

Source: www.habr.com