Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

Ushbu maqola GitLab-dan foydalangan holda o'z mahsulotlarini sonatip va/yoki maven markaziy omborlarida tezda nashr etishga muhtoj bo'lgan java dasturchilari uchun mo'ljallangan. Ushbu maqolada men ushbu muammoni hal qilish uchun gitlab-runner, gitlab-ci va maven-pluginni o'rnatish haqida gapiraman.

Old shartlar:

  • mvn va GPG kalitlarini xavfsiz saqlash.
  • Ommaviy CI vazifalarini xavfsiz bajarish.
  • Ommaviy omborlarga artefaktlarni yuklash (reliz/snapshot).
  • Maven markazida chop etish uchun versiya versiyalarini avtomatik tekshirish.
  • Bir nechta loyihalar uchun repozitoriyga artefaktlarni yuklash uchun umumiy yechim.
  • Oddiylik va foydalanish qulayligi.

Mundarija

umumiy ma'lumot

  • Maven Central-da Sonatype OSS Repository Hosting xizmati orqali artefaktlarni nashr qilish mexanizmining batafsil tavsifi allaqachon tasvirlangan. Ushbu maqola foydalanuvchi Googolplex, shuning uchun men ushbu maqolaga kerakli joylarda murojaat qilaman.
  • Oldindan ro'yxatdan o'ting JIRA sonatipi va omborni ochish uchun chiptani oching (batafsil ma'lumot uchun bo'limni o'qing Sonatype JIRA-da chipta yarating). Repozitoriy ochilgandan so'ng JIRA login/parol juftligi (keyingi o'rinlarda Sonatype hisobi deb yuritiladi) Sonatype nexus-ga artefaktlarni yuklash uchun ishlatiladi.
  • Keyinchalik, GPG kalitini yaratish jarayoni juda quruq tasvirlangan. Batafsil ma'lumot uchun bo'limga qarang Artefaktlarni imzolash uchun GnuPG sozlanmoqda
  • Agar siz GPG kalitini (gnupg/gnupg2) yaratish uchun Linux konsolidan foydalansangiz, o'rnatishingiz kerak. rng-vositalari entropiya hosil qilish uchun. Aks holda, kalit yaratish juda uzoq vaqt talab qilishi mumkin.
  • Saqlash xizmatlari ommaviy GPG kalitlari

Mundarija jadvaliga qaytish

GitLab-da joylashtirish loyihasini sozlash

  • Avvalo, siz artefaktlarni joylashtirish uchun quvur liniyasi saqlanadigan loyihani yaratishingiz va sozlashingiz kerak. Men o'z loyihamni oddiy va sodda tarzda nomladim - tarqatish
  • Omborni yaratgandan so'ng, omborni o'zgartirish uchun kirishni cheklashingiz kerak.
    Loyiha -> Sozlamalar -> Repository -> Himoyalangan filiallarga o'ting. Biz barcha qoidalarni o'chirib tashlaymiz va faqat Maintainers roliga ega foydalanuvchilar uchun surish va birlashtirish huquqiga ega Wildcard * bilan bitta qoida qo'shamiz. Ushbu qoida ushbu loyihaning barcha foydalanuvchilari va ushbu loyiha tegishli bo'lgan guruh uchun ishlaydi.
    Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda
  • Agar bir nechta texnik xizmat ko'rsatuvchi bo'lsa, unda eng yaxshi yechim printsipial jihatdan loyihaga kirishni cheklash bo'ladi.
    Loyiha -> Sozlamalar -> Umumiy -> Koʻrinish, loyiha xususiyatlari, ruxsatnomalar boʻlimiga oʻting va Loyiha koʻrinishini oʻrnating shaxsiy.
    Menda hammaga ochiq loyiha bor, chunki men o'zimning GitLab Runner-dan foydalanaman va faqat men omborni o'zgartirish huquqiga egaman. Xo'sh, aslida, jamoat quvurlari jurnallarida shaxsiy ma'lumotlarni ko'rsatish mening manfaatlarimga mos kelmaydi.
  • Omborni o'zgartirish qoidalarini kuchaytirish
    Loyiha -> Sozlamalar -> Repository -> Push Rules-ga o'ting va Committer cheklovini o'rnating, muallif GitLab foydalanuvchi bayroqlari ekanligini tekshiring. Men ham sozlashni tavsiya qilaman imzo qo'yish, va "Reject unsigned commits" belgisini o'rnating.
  • Keyin vazifalarni ishga tushirish uchun triggerni sozlashingiz kerak
    Loyiha -> Sozlamalar -> CI / CD -> Quvur liniyasi triggerlariga o'ting va yangi trigger-token yarating.
    Ushbu token darhol loyihalar guruhi uchun o'zgaruvchilarning umumiy konfiguratsiyasiga qo'shilishi mumkin.
    Guruhga o'ting -> Sozlamalar -> CI / CD -> O'zgaruvchilar va o'zgaruvchini qo'shing DEPLOY_TOKEN qiymatida trigger-token bilan.

Mundarija jadvaliga qaytish

GitLab Runner

Ushbu bo'limda o'zingizning (Maxsus) va umumiy (Umumiy) yuguruvchingiz yordamida tarqatish bo'yicha vazifalarni bajarish konfiguratsiyasi tasvirlangan.

Maxsus yuguruvchi

Men o'zimning yuguruvchilarmdan foydalanaman, chunki, birinchi navbatda, bu qulay, tez va arzon.
Yuguruvchi uchun men 1 protsessor, 2 GB RAM, 20 GB HDD bilan Linux VDS ni tavsiya qilaman. Chiqarilish narxi yiliga ~3000₽.

Mening yuguruvchim

Yuguruvchi uchun men VDS 4 protsessor, 4 GB operativ xotira, 50 GB SSD oldim. Narxi ~11000 XNUMX₽ va hech qachon afsuslanmadi.
Menda jami 7 ta mashina bor. Arubada 5 ta va Ihorda 2 ta.

Shunday qilib, bizda yuguruvchi bor. Endi biz uni sozlaymiz.
Biz SSH orqali mashinaga o'tamiz va java, git, maven, gnupg2 ni o'rnatamiz.

Mundarija jadvaliga qaytish

Gitlab runner o'rnatilmoqda

  • Yangi guruh yarating runner
    sudo groupadd runner
  • Maven keshi uchun katalog yarating va guruh ruxsatlarini tayinlang runner
    Agar siz bir mashinada bir nechta yuguruvchilarni yugurishni rejalashtirmasangiz, bu nuqtani o'tkazib yuborishingiz mumkin.

    mkdir -p /usr/cache/.m2/repository
    chown -R :runner /usr/cache
    chmod -R 770 /usr/cache
  • Foydalanuvchi yarating gitlab-deployer va guruhga qo'shing runner
    useradd -m -d /home/gitlab-deployer gitlab-deployer
    usermod -a -G runner gitlab-deployer
  • Faylga qo'shish /etc/ssh/sshd_config keyingi qator
    AllowUsers root@* [email protected]
  • Qayta yuklash sshd
    systemctl restart sshd
  • Foydalanuvchi uchun parol o'rnatish gitlab-deployer (oddiy bo'lishi mumkin, chunki localhost uchun cheklov mavjud)
    passwd gitlab-deployer
  • GitLab Runner o'rnating (Linux x86-64)
    sudo wget -O /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
    sudo chmod +x /usr/local/bin/gitlab-runner
    ln -s /usr/local/bin/gitlab-runner /etc/alternatives/gitlab-runner
    ln -s /etc/alternatives/gitlab-runner /usr/bin/gitlab-runner
  • Gitlab.com veb-saytiga o'ting -> deploy-project -> Settings -> CI/CD -> Runners -> Specific Runners va ro'yxatdan o'tish tokenini nusxalash.

Ekran

Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

  • Yuguruvchini ro'yxatdan o'tkazish
    gitlab-runner register --config /etc/gitlab-runner/gitlab-deployer-config.toml

jarayon

Runtime platform arch=amd64 os=linux pid=17594 revision=3001a600 version=11.10.0
Running in system-mode.
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://gitlab.com/
Please enter the gitlab-ci token for this runner:
REGISTRATION_TOKEN
Please enter the gitlab-ci description for this runner:
[ih1174328.vds.myihor.ru]: Deploy Runner
Please enter the gitlab-ci tags for this runner (comma separated):
deploy
Registering runner... succeeded                     runner=ZvKdjJhx
Please enter the executor: docker-ssh, parallels, virtualbox, docker-ssh+machine, kubernetes, docker, ssh, docker+machine, shell:
shell
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
  • Yuguruvchining ro'yxatdan o'tganligini tekshiramiz. Gitlab.com veb-saytiga o'ting -> deploy-project -> Sozlamalar -> CI/CD -> Yuguruvchilar -> Maxsus yuguruvchilar -> Ushbu loyiha uchun faollashtirilgan yuguruvchilar

Ekran

Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

  • Qo'shing alohida xizmat ko'rsatish /etc/systemd/system/gitlab-deployer.service
    [Unit]
    Description=GitLab Deploy Runner
    After=syslog.target network.target
    ConditionFileIsExecutable=/usr/local/bin/gitlab-runner
    [Service]
    StartLimitInterval=5
    StartLimitBurst=10
    ExecStart=/usr/local/bin/gitlab-runner "run" "--working-directory" "/home/gitlab-deployer" "--config" "/etc/gitlab-runner/gitlab-deployer-config.toml" "--service" "gitlab-deployer" "--syslog" "--user" "gitlab-deployer"
    Restart=always
    RestartSec=120
    [Install]
    WantedBy=multi-user.target
  • Keling, xizmatni boshlaylik.
    systemctl enable gitlab-deployer.service
    systemctl start gitlab-deployer.service
    systemctl status gitlab-deployer.service
  • Yuguruvchi yugurayotganini tekshiramiz.

misol

Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

Mundarija jadvaliga qaytish

GPG kalitlarini yaratish

  • Xuddi shu mashinadan foydalanuvchi ostidagi ssh orqali tizimga kiramiz gitlab-deployer (bu GPG kalitini yaratish uchun muhim)
    ssh [email protected]
  • Savollarga javob berish orqali kalitni yaratamiz. Men o'z ismim va elektron pochtamdan foydalandim.
    Kalit uchun parolni ko'rsatganingizga ishonch hosil qiling. Artefaktlar ushbu kalit bilan imzolanadi.

    gpg --gen-key 
  • Tekshirish
    gpg --list-keys -a
    /home/gitlab-deployer/.gnupg/pubring.gpg
    ----------------------------------------
    pub   4096R/00000000 2019-04-19
    uid                  Petruha Petrov <[email protected]>
    sub   4096R/11111111 2019-04-19
  • Ochiq kalitimiz kalit serverga yuklanmoqda
    gpg --keyserver keys.gnupg.net --send-key 00000000
    gpg: sending key 00000000 to hkp server keys.gnupg.net

Mundarija jadvaliga qaytish

Maven o'rnatilmoqda

  • Foydalanuvchi sifatida tizimga kiring gitlab-deployer
    su gitlab-deployer 
  • maven katalogini yarating omborxona va keshga havola (xato qilmang)
    Agar siz bir mashinada bir nechta yuguruvchilarni yugurishni rejalashtirmasangiz, bu nuqtani o'tkazib yuborishingiz mumkin.

    mkdir -p ~/.m2/repository
    ln -s /usr/cache/.m2/repository /home/gitlab-deployer/.m2/repository
  • Asosiy kalit yarating
    mvn --encrypt-master-password password
    {hnkle5BJ9HUHUMP+CXfGBl8dScfFci/mpsur/73tR2I=}
  • ~/.m2/settings-security.xml faylini yarating
    <settingsSecurity>
    <master>{hnkle5BJ9HUHUMP+CXfGBl8dScfFci/mpsur/73tR2I=}</master>
    </settingsSecurity>
  • Sonatype hisobi uchun parolni shifrlash
    mvn --encrypt-password SONATYPE_PASSWORD
    {98Wv5+u+Tn0HX2z5G/kR4R8Z0WBgcDBgi7d12S/un+SCU7uxzaZGGmJ8Cu9pAZ2J}
  • ~/.m2/settings.xml faylini yarating
    <settings>  
    <profiles>
        <profile>
            <id>env</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <properties>
                <gpg.passphrase>GPG_SECRET_KEY_PASSPHRASE</gpg.passphrase>
            </properties>
        </profile>
    </profiles>
    <servers>
        <server>
            <id>sonatype</id>
            <username>SONATYPE_USERNAME</username>
            <password>{98Wv5+u+Tn0HX2z5G/kR4R8Z0WBgcDBgi7d12S/un+SCU7uxzaZGGmJ8Cu9pAZ2J}</password>
        </server>
    </servers>
    </settings>

qayerda,
GPG_SECRET_KEY_PASSPHRASE - GPG kaliti uchun parol
SONATYPE_USERNAME — hisob qaydnomasiga kirish

Bu yuguruvchini sozlashni yakunlaydi, siz bo'limga o'tishingiz mumkin GitLab CI

Mundarija jadvaliga qaytish

Umumiy yuguruvchi

GPG kalitlarini yaratish

  • Avvalo, siz GPG kalitini yaratishingiz kerak. Buning uchun gnupg-ni o'rnating.
    yum install -y gnupg
  • Savollarga javob berish orqali kalitni yaratamiz. Men o'z ismim va elektron pochtamdan foydalandim. Kalit uchun parolni ko'rsatganingizga ishonch hosil qiling.
    gpg --gen-key 
  • Kalitdagi ma'lumotlarni ko'rsatish
    gpg --list-keys -a
    pub   rsa3072 2019-04-24 [SC] [expires: 2021-04-23]
      2D0D1706366FC4AEF79669E24D09C55BBA3FD728
    uid           [ultimate] tttemp <[email protected]>
    sub   rsa3072 2019-04-24 [E] [expires: none]
  • Ochiq kalitimiz kalit serverga yuklanmoqda
    gpg --keyserver keys.gnupg.net --send-key 2D0D1706366FC4AEF79669E24D09C55BBA3FD728
    gpg: sending key 2D0D1706366FC4AEF79669E24D09C55BBA3FD728 to hkp server keys.gnupg.net
  • Biz shaxsiy kalitni olamiz
    gpg --export-secret-keys --armor 2D0D1706366FC4AEF79669E24D09C55BBA3FD728
    -----BEGIN PGP PRIVATE KEY BLOCK-----
    lQWGBFzAqp8BDADN41CPwJ/gQwiKEbyA902DKw/WSB1AvZQvV/ZFV77xGeG4K7k5
    ...
    =2Wd2
    -----END PGP PRIVATE KEY BLOCK-----
  • Loyiha sozlamalari -> Sozlamalar -> CI / CD -> O'zgaruvchilar bo'limiga o'ting va shaxsiy kalitni o'zgaruvchida saqlang GPG_SECRET_KEY
    Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

Mundarija jadvaliga qaytish

Maven o'rnatilmoqda

  • Asosiy kalit yarating
    mvn --encrypt-master-password password
    {hnkle5BJ9HUHUMP+CXfGBl8dScfFci/mpsur/73tR2I=}
  • Loyiha sozlamalari -> Sozlamalar -> CI / CD -> O'zgaruvchilar bo'limiga o'ting va o'zgaruvchida saqlang SETTINGS_SECURITY_XML quyidagi qatorlar:
    <settingsSecurity>
    <master>{hnkle5BJ9HUHUMP+CXfGBl8dScfFci/mpsur/73tR2I=}</master>
    </settingsSecurity>
  • Sonatype hisobi uchun parolni shifrlash
    mvn --encrypt-password SONATYPE_PASSWORD
    {98Wv5+u+Tn0HX2z5G/kR4R8Z0WBgcDBgi7d12S/un+SCU7uxzaZGGmJ8Cu9pAZ2J}
  • Loyiha sozlamalari -> Sozlamalar -> CI / CD -> O'zgaruvchilar bo'limiga o'ting va o'zgaruvchida saqlang SETTINGS_XML quyidagi qatorlar:
    <settings>  
    <profiles>
        <profile>
            <id>env</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <properties>
                <gpg.passphrase>GPG_SECRET_KEY_PASSPHRASE</gpg.passphrase>
            </properties>
        </profile>
    </profiles>
    <servers>
        <server>
            <id>sonatype</id>
            <username>sonatype_username</username>
            <password>{98Wv5+u+Tn0HX2z5G/kR4R8Z0WBgcDBgi7d12S/un+SCU7uxzaZGGmJ8Cu9pAZ2J}</password>
        </server>
    </servers>
    </settings>

qayerda,
GPG_SECRET_KEY_PASSPHRASE - GPG kaliti uchun parol
SONATYPE_USERNAME — hisob qaydnomasiga kirish

Mundarija jadvaliga qaytish

Docker tasvirini o'rnatish

  • Biz Java-ning kerakli versiyasi bilan joylashtirish vazifalarini bajarish uchun juda oddiy Dockerfile yaratamiz. Quyida alp tog'lari uchun misol keltirilgan.
    FROM java:8u111-jdk-alpine
    RUN apk add gnupg maven git --update-cache 
    --repository http://dl-4.alpinelinux.org/alpine/edge/community/ --allow-untrusted && 
    mkdir ~/.m2/
  • Loyihangiz uchun konteyner yig'ish
    docker build -t registry.gitlab.com/group/deploy .
  • Biz konteynerni autentifikatsiya qilamiz va registrga yuklaymiz.
    docker login -u USER -p PASSWORD registry.gitlab.com
    docker push registry.gitlab.com/group/deploy

Mundarija jadvaliga qaytish

GitLab CI

Loyihani tarqatish

.gitlab-ci.yml faylini joylashtirish loyihasining ildiziga qo'shing
Skript ikkita o'zaro eksklyuziv joylashtirish vazifasini taqdim etadi. Maxsus yuguruvchi yoki umumiy yuguruvchi.

.gitlab-ci.yml

stages:
  - deploy

Specific Runner:
  extends: .java_deploy_template
  # Задача будет выполняться на вашем shell-раннере
  tags:
    - deploy

Shared Runner:
  extends: .java_deploy_template
  # Задача будет выполняться на публичном docker-раннере
  tags:
    - docker
  # Образ из раздела GitLab Runner -> Shared Runner -> Docker
  image: registry.gitlab.com/group/deploy-project:latest
  before_script:
    # Импортируем GPG ключ
    - printf "${GPG_SECRET_KEY}" | gpg --batch --import
    # Сохраняем maven конфигурацию
    - printf "${SETTINGS_SECURITY_XML}" > ~/.m2/settings-security.xml
    - printf "${SETTINGS_XML}" > ~/.m2/settings.xml

.java_deploy_template:
  stage: deploy
  # Задача сработает по триггеру, если передана переменная DEPLOY со значением java
  only:
    variables:
    - $DEPLOY == "java"
  variables:
    # отключаем клонирование текущего проекта
    GIT_STRATEGY: none
  script:
    # Предоставляем возможность хранения пароля в незашифрованном виде
    - git config --global credential.helper store
    # Сохраняем временные креды пользователя gitlab-ci-token
    # Токен работает для всех публичных проектов gitlab.com и для проектов группы
    - echo "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com" >> ~/.git-credentials
    # Полностью чистим текущую директорию
    - rm -rf .* *
    # Клонируем проект который, будем деплоить в Sonatype Nexus
    - git clone ${DEPLOY_CI_REPOSITORY_URL} .
    # Переключаемся на нужный коммит
    - git checkout ${DEPLOY_CI_COMMIT_SHA} -f
    # Если хоть один pom.xml содержит параметр autoReleaseAfterClose валим сборку.
    # В противном случае есть риск залить сырые артефакты в maven central
    - >
      for pom in $(find . -name pom.xml); do
        if [[ $(grep -q autoReleaseAfterClose "$pom" && echo $?) == 0 ]]; then
          echo "File $pom contains prohibited setting: <autoReleaseAfterClose>";
          exit 1;
        fi;
      done
    # Если параметр DEPLOY_CI_COMMIT_TAG пустой, то принудительно ставим SNAPSHOT-версию
    - >
      if [[ "${DEPLOY_CI_COMMIT_TAG}" != "" ]]; then
        mvn versions:set -DnewVersion=${DEPLOY_CI_COMMIT_TAG}
      else
        VERSION=$(mvn -q -Dexec.executable=echo -Dexec.args='${project.version}' --non-recursive exec:exec)
        if [[ "${VERSION}" == *-SNAPSHOT ]]; then
          mvn versions:set -DnewVersion=${VERSION}
        else
          mvn versions:set -DnewVersion=${VERSION}-SNAPSHOT
        fi
      fi
    # Запускаем задачу на сборку и деплой артефактов
    - mvn clean deploy -DskipTests=true

Mundarija jadvaliga qaytish

Java loyihasi

Ommaviy omborlarga yuklanishi kerak bo'lgan java loyihalarida siz Release va Snapshot versiyalarini yuklab olish uchun 2 qadam qo'shishingiz kerak.

.gitlab-ci.yml

stages:
  - build
  - test
  - verify
  - deploy

<...>

Release:
  extends: .trigger_deploy
  # Запускать задачу только пo тегу.
  only:
    - tags

Snapshot:
  extends: .trigger_deploy
  # Запускаем задачу на публикацию SNAPSHOT версии вручную
  when: manual
  # Не запускать задачу, если проставлен тег.
  except:
    - tags

.trigger_deploy:
  stage: deploy
  variables:
    # Отключаем клонирование текущего проекта
    GIT_STRATEGY: none
    # Ссылка на триггер deploy-задачи
    URL: "https://gitlab.com/api/v4/projects/<deploy project ID>/trigger/pipeline"
    # Переменные deploy-задачи
    POST_DATA: "
      token=${DEPLOY_TOKEN}&
      ref=master&
      variables[DEPLOY]=${DEPLOY}&
      variables[DEPLOY_CI_REPOSITORY_URL]=${CI_REPOSITORY_URL}&
      variables[DEPLOY_CI_PROJECT_NAME]=${CI_PROJECT_NAME}&
      variables[DEPLOY_CI_COMMIT_SHA]=${CI_COMMIT_SHA}&
      variables[DEPLOY_CI_COMMIT_TAG]=${CI_COMMIT_TAG}
      "
  script:
    # Не использую cURL, так как с флагами --fail --show-error
    # он не выводит тело ответа, если HTTP код 400 и более 
    - wget --content-on-error -qO- ${URL} --post-data ${POST_DATA}

Ushbu yechimda men biroz oldinga bordim va java loyihalari uchun bitta CI shablonidan foydalanishga qaror qildim.

Batafsil ma'lumot

Men alohida loyiha yaratdim gitlab-ci unda men java loyihalari uchun CI shablonini joylashtirdim umumiy.yml.

umumiy.yml

stages:
  - build
  - test
  - verify
  - deploy

variables:
  SONAR_ARGS: "
  -Dsonar.gitlab.commit_sha=${CI_COMMIT_SHA} 
  -Dsonar.gitlab.ref_name=${CI_COMMIT_REF_NAME} 
  "

.build_java_project:
  stage: build
  tags:
    - touchbit-shell
  variables:
    SKIP_TEST: "false"
  script:
    - mvn clean
    - mvn package -DskipTests=${SKIP_TEST}
  artifacts:
    when: always
    expire_in: 30 day
    paths:
      - "*/target/reports"

.build_sphinx_doc:
  stage: build
  tags:
    - touchbit-shell
  variables:
    DOCKERFILE: .indirect/docs/Dockerfile
  script:
    - docker build --no-cache -t ${CI_PROJECT_NAME}/doc -f ${DOCKERFILE} .

.junit_module_test_run:
  stage: test
  tags:
    - touchbit-shell
  variables:
    MODULE: ""
  script:
    - cd ${MODULE}
    - mvn test
  artifacts:
    when: always
    expire_in: 30 day
    paths:
      - "*/target/reports"

.junit_test_run:
  stage: test
  tags:
    - touchbit-shell
  script:
    - mvn test
  artifacts:
    when: always
    expire_in: 30 day
    paths:
    - "*/target/reports"

.sonar_review:
  stage: verify
  tags:
    - touchbit-shell
  dependencies: []
  script:
    - >
      if [ "$CI_BUILD_REF_NAME" == "master" ]; then
        mvn compile sonar:sonar -Dsonar.login=$SONAR_LOGIN $SONAR_ARGS
      else
        mvn compile sonar:sonar -Dsonar.login=$SONAR_LOGIN $SONAR_ARGS -Dsonar.analysis.mode=preview
      fi

.trigger_deploy:
  stage: deploy
  tags:
    - touchbit-shell
  variables:
    URL: "https://gitlab.com/api/v4/projects/10345765/trigger/pipeline"
    POST_DATA: "
      token=${DEPLOY_TOKEN}&
      ref=master&
      variables[DEPLOY]=${DEPLOY}&
      variables[DEPLOY_CI_REPOSITORY_URL]=${CI_REPOSITORY_URL}&
      variables[DEPLOY_CI_PROJECT_NAME]=${CI_PROJECT_NAME}&
      variables[DEPLOY_CI_COMMIT_SHA]=${CI_COMMIT_SHA}&
      variables[DEPLOY_CI_COMMIT_TAG]=${CI_COMMIT_TAG}
      "
  script:
  - wget --content-on-error -qO- ${URL} --post-data ${POST_DATA}

.trigger_release_deploy:
  extends: .trigger_deploy
  only:
    - tags

.trigger_snapshot_deploy:
  extends: .trigger_deploy
  when: manual
  except:
    - tags

Natijada, java loyihalarida .gitlab-ci.yml juda ixcham ko'rinadi va batafsil emas

.gitlab-ci.yml

include: https://gitlab.com/TouchBIT/gitlab-ci/raw/master/common.yml

Shields4J:
  extends: .build_java_project

Sphinx doc:
  extends: .build_sphinx_doc
  variables:
    DOCKERFILE: .docs/Dockerfile

Sonar review:
  extends: .sonar_review
  dependencies:
    - Shields4J

Release:
  extends: .trigger_release_deploy

Snapshot:
  extends: .trigger_snapshot_deploy

Mundarija jadvaliga qaytish

Pom.xml konfiguratsiyasi

Ushbu mavzu juda batafsil tavsiflangan. Googolplex в Maven-ni avtomatik ravishda imzolash va artefaktlarni suratga olish va sahnalashtirish omborlariga yuklash uchun sozlash, shuning uchun men plaginlardan foydalanishning ba'zi nuanslarini tasvirlab beraman. Bundan tashqari, siz qanchalik oson va qulay foydalanishingiz mumkinligini tasvirlab beraman nexus-staging-maven-pluginagar siz loyihangiz uchun ota-ona sifatida org.sonatype.oss:oss-parent dan foydalanmasangiz yoki foydalana olmasangiz.

maven-o'rnatish-plagin

Modullarni mahalliy omborga o'rnatadi.
Boshqa loyihalardagi yechimlarni mahalliy tekshirish, shuningdek nazorat summasi uchun juda foydali.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-install-plugin</artifactId>
  <executions>
    <execution>
      <id>install-project</id>
      <!-- Если у вас многомодульный проект с деплоем родительского помика -->
      <phase>install</phase>
      <!-- Явно указываем файлы для локальной установки -->
      <configuration>
        <file>target/${project.artifactId}-${project.version}.jar</file>
```target/${project.artifactId}-${project.version}-sources.jar</sources>
        <pomFile>dependency-reduced-pom.xml</pomFile>
        <!-- Принудительное обновление метаданных проекта -->
        <updateReleaseInfo>true</updateReleaseInfo>
        <!-- Контрольные суммы для проверки целостности -->
        <createChecksum>true</createChecksum>
      </configuration>
    </execution>
  </executions>
</plugin>

Mundarija jadvaliga qaytish

maven-javadoc-plagin

Loyiha uchun javadoc yaratilmoqda.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-javadoc-plugin</artifactId>
  <executions>
    <execution>
      <goals>
        <goal>jar</goal>
      </goals>
      <!-- Генерация javadoc должна быть после фазы генерации ресурсов -->
      <phase>prepare-package</phase>
      <configuration>
        <!-- Очень помогает в публичных проектах -->
        <failOnError>true</failOnError>
        <failOnWarnings>true</failOnWarnings>
        <!-- Убирает ошибку поиска документации в target директории -->
        <detectOfflineLinks>false</detectOfflineLinks>
      </configuration>
    </execution>
  </executions>
</plugin>

Agar sizda java bo'lmagan modul bo'lsa (masalan, faqat resurslar)
Yoki printsipial ravishda javadoc yaratishni xohlamaysiz, keyin yordam bering maven-jar-plugin

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-jar-plugin</artifactId>
  <executions>
    <execution>
      <id>empty-javadoc-jar</id>
      <phase>generate-resources</phase>
      <goals>
        <goal>jar</goal>
      </goals>
      <configuration>
        <classifier>javadoc</classifier>
        <classesDirectory>${basedir}/javadoc</classesDirectory>
      </configuration>
    </execution>
  </executions>
</plugin>

Mundarija jadvaliga qaytish

maven-gpg-plagini

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-gpg-plugin</artifactId>
  <executions>
    <execution>
      <id>sign-artifacts</id>
      <!-- Сборка будет падать, если отсутствует GPG ключ -->
      <!-- Подписываем артефакты только на фазе deploy -->
      <phase>deploy</phase>
      <goals>
        <goal>sign</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Mundarija jadvaliga qaytish

nexus-staging-maven-plugini

Konfiguratsiya:

<project>
  <!-- ... -->
  <build>
    <plugins>
      <!-- ... -->
      <plugin>
        <groupId>org.sonatype.plugins</groupId>
        <artifactId>nexus-staging-maven-plugin</artifactId>
      </plugin>
    </plugins>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.sonatype.plugins</groupId>
          <artifactId>nexus-staging-maven-plugin</artifactId>
          <extensions>true</extensions>
          <configuration>
            <serverId>sonatype</serverId>
            <nexusUrl>https://oss.sonatype.org/</nexusUrl>
            <!-- Обновляем метаданные, чтобы пометить артефакт как release -->
            <!-- Не влияет на snapshot версии -->
            <updateReleaseInfo>true</updateReleaseInfo>
          </configuration>
        </plugin>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-deploy-plugin</artifactId>
          <configuration>
            <!-- Отключаем плагин -->
            <skip>true</skip>
          </configuration>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
  <distributionManagement>
    <snapshotRepository>
      <id>sonatype</id>
      <name>Nexus Snapshot Repository</name>
      <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
    </snapshotRepository>
    <repository>
      <id>sonatype</id>
      <name>Nexus Release Repository</name>
      <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
    </repository>
  </distributionManagement>
</project>

Agar sizda ko'p modulli loyihangiz bo'lsa va siz ma'lum modulni omborga yuklashingiz shart bo'lmasa, unda siz qo'shishingiz kerak. nexus-staging-maven-plugin bayroq bilan skipNexusStagingDeployMojo

<build>
  <plugins>
    <plugin>
      <groupId>org.sonatype.plugins</groupId>
      <artifactId>nexus-staging-maven-plugin</artifactId>
      <configuration>
        <skipNexusStagingDeployMojo>true</skipNexusStagingDeployMojo>
      </configuration>
    </plugin>
  </plugins>
</build>

Yuklab olingandan so'ng, oniy rasm/reliz versiyalari mavjud saqlash omborlari

<repositories>
  <repository>
    <id>SonatypeNexus</id>
    <url>https://oss.sonatype.org/content/groups/staging/</url>
    <!-- Не надо указывать флаги snapshot/release для репозитория -->
  </repository>
</repositories>

Ko'proq ortiqcha

  • Nexus repository bilan ishlash uchun juda boy maqsadlar ro'yxati (mvn help:describe -Dplugin=org.sonatype.plugins:nexus-staging-maven-plugin).
  • Maven markaziga yuklash uchun avtomatik reliz tekshiruvi

Mundarija jadvaliga qaytish

natija

SNAPSHOT versiyasi nashr etilmoqda

Loyihani yaratishda SNAPSHOT versiyasini nexus-ga yuklab olish uchun vazifani qo'lda ishga tushirish mumkin

Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

Ushbu vazifa ishga tushirilganda, joylashtirish loyihasidagi tegishli vazifa ishga tushiriladi (misol).

Kesilgan jurnal

Running with gitlab-runner 11.10.0 (3001a600)
  on Deploy runner JSKWyxUw
Using Shell executor...
Running on ih1174328.vds.myihor.ru...
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
$ rm -rf .* *
$ git config --global credential.helper store
$ echo "https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com" >> ~/.git-credentials
$ git clone ${DEPLOY_CI_REPOSITORY_URL} .
Cloning into 'shields4j'...
$ git checkout ${DEPLOY_CI_COMMIT_SHA}
Note: checking out '850f86aa317194395c5387790da1350e437125a7'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
  git checkout -b new_branch_name
HEAD is now at 850f86a... skip deploy test-core
$ for pom in $(find . -name pom.xml); do # collapsed multi-line command
$ if [[ "${DEPLOY_CI_COMMIT_TAG}" != "" ]]; then # collapsed multi-line command
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 4 modules...
[INFO] Installing Nexus Staging features:
[INFO]   ... total of 4 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Shields4J                                                          [pom]
[INFO] test-core                                                          [jar]
[INFO] Shields4J client                                                   [jar]
[INFO] TestNG listener                                                    [jar]
[INFO] 
[INFO] --------------< org.touchbit.shields4j:shields4j-parent >---------------
[INFO] Building Shields4J 1.0.0                                           [1/4]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- versions-maven-plugin:2.5:set (default-cli) @ shields4j-parent ---
[INFO] Searching for local aggregator root...
[INFO] Local aggregation root: /home/gitlab-deployer/JSKWyxUw/0/TouchBIT/deploy/shields4j
[INFO] Processing change of org.touchbit.shields4j:shields4j-parent:1.0.0 -> 1.0.0-SNAPSHOT
[INFO] Processing org.touchbit.shields4j:shields4j-parent
[INFO]     Updating project org.touchbit.shields4j:shields4j-parent
[INFO]         from version 1.0.0 to 1.0.0-SNAPSHOT
[INFO] 
[INFO] Processing org.touchbit.shields4j:client
[INFO]     Updating parent org.touchbit.shields4j:shields4j-parent
[INFO]         from version 1.0.0 to 1.0.0-SNAPSHOT
[INFO]     Updating dependency org.touchbit.shields4j:test-core
[INFO]         from version 1.0.0 to 1.0.0-SNAPSHOT
[INFO] 
[INFO] Processing org.touchbit.shields4j:test-core
[INFO]     Updating parent org.touchbit.shields4j:shields4j-parent
[INFO]         from version 1.0.0 to 1.0.0-SNAPSHOT
[INFO] 
[INFO] Processing org.touchbit.shields4j:testng
[INFO]     Updating parent org.touchbit.shields4j:shields4j-parent
[INFO]         from version 1.0.0 to 1.0.0-SNAPSHOT
[INFO]     Updating dependency org.touchbit.shields4j:client
[INFO]         from version 1.0.0 to 1.0.0-SNAPSHOT
[INFO]     Updating dependency org.touchbit.shields4j:test-core
[INFO]         from version 1.0.0 to 1.0.0-SNAPSHOT
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Shields4J 1.0.0 .................................... SUCCESS [  0.992 s]
[INFO] test-core .......................................... SKIPPED
[INFO] Shields4J client ................................... SKIPPED
[INFO] TestNG listener 1.0.0 .............................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.483 s
[INFO] Finished at: 2019-04-21T02:40:42+03:00
[INFO] ------------------------------------------------------------------------
$ mvn clean deploy -DskipTests=${SKIP_TESTS}
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 4 modules...
[INFO] Installing Nexus Staging features:
[INFO]   ... total of 4 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Shields4J                                                          [pom]
[INFO] test-core                                                          [jar]
[INFO] Shields4J client                                                   [jar]
[INFO] TestNG listener                                                    [jar]
[INFO] 
[INFO] --------------< org.touchbit.shields4j:shields4j-parent >---------------
[INFO] Building Shields4J 1.0.0-SNAPSHOT                                  [1/4]
[INFO] --------------------------------[ pom ]---------------------------------
...
DELETED
...
[INFO]  * Bulk deploy of locally gathered snapshot artifacts finished.
[INFO] Remote deploy finished with success.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Shields4J 1.0.0-SNAPSHOT ........................... SUCCESS [  2.375 s]
[INFO] test-core .......................................... SUCCESS [  3.929 s]
[INFO] Shields4J client ................................... SUCCESS [  3.815 s]
[INFO] TestNG listener 1.0.0-SNAPSHOT ..................... SUCCESS [ 36.134 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 47.629 s
[INFO] Finished at: 2019-04-21T02:41:32+03:00
[INFO] ------------------------------------------------------------------------

Natijada, versiya nexusga yuklanadi 1.0.0-SNAPSHOT.

Barcha snapshot versiyalari veb-saytdagi ombordan o'chirilishi mumkin oss.sonatype.org hisobingiz ostida.

Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

Mundarija jadvaliga qaytish

Chiqarish versiyasini nashr qilish

Teg o'rnatilganda, tarqatish loyihasidagi tegishli vazifa avtomatik ravishda nexus-ga reliz versiyasini yuklab olish uchun ishga tushiriladi (misol).

Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

Eng yaxshi tomoni shundaki, yaqin chiqarish avtomatik ravishda nexusda ishga tushadi.

[INFO] Performing remote staging...
[INFO] 
[INFO]  * Remote staging into staging profile ID "9043b43f77dcc9"
[INFO]  * Created staging repository with ID "orgtouchbit-1037".
[INFO]  * Staging repository at https://oss.sonatype.org:443/service/local/staging/deployByRepositoryId/orgtouchbit-1037
[INFO]  * Uploading locally staged artifacts to profile org.touchbit
[INFO]  * Upload of locally staged artifacts finished.
[INFO]  * Closing staging repository with ID "orgtouchbit-1037".
Waiting for operation to complete...
.........
[INFO] Remote staged 1 repositories, finished with success.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Shields4J 1.0.0 .................................... SUCCESS [  9.603 s]
[INFO] test-core .......................................... SUCCESS [  3.419 s]
[INFO] Shields4J client ................................... SUCCESS [  9.793 s]
[INFO] TestNG listener 1.0.0 .............................. SUCCESS [01:23 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:47 min
[INFO] Finished at: 2019-04-21T04:05:46+03:00
[INFO] ------------------------------------------------------------------------

Va agar biror narsa noto'g'ri bo'lsa, vazifa, albatta, muvaffaqiyatsiz bo'ladi

[INFO] Performing remote staging...
[INFO] 
[INFO]  * Remote staging into staging profile ID "9043b43f77dcc9"
[INFO]  * Created staging repository with ID "orgtouchbit-1038".
[INFO]  * Staging repository at https://oss.sonatype.org:443/service/local/staging/deployByRepositoryId/orgtouchbit-1038
[INFO]  * Uploading locally staged artifacts to profile org.touchbit
[INFO]  * Upload of locally staged artifacts finished.
[INFO]  * Closing staging repository with ID "orgtouchbit-1038".
Waiting for operation to complete...
.......
[ERROR] Rule failure while trying to close staging repository with ID "orgtouchbit-1039".
[ERROR] 
[ERROR] Nexus Staging Rules Failure Report
[ERROR] ==================================
[ERROR] 
[ERROR] Repository "orgtouchbit-1039" failures
[ERROR]   Rule "signature-staging" failures
[ERROR]     * No public key: Key with id: (1f42b618d1cbe1b5) was not able to be located on &lt;a href=http://keys.gnupg.net:11371/&gt;http://keys.gnupg.net:11371/&lt;/a&gt;. Upload your public key and try the operation again.
...
[ERROR] Cleaning up local stage directory after a Rule failure during close of staging repositories: [orgtouchbit-1039]
[ERROR]  * Deleting context 9043b43f77dcc9.properties
[ERROR] Cleaning up remote stage repositories after a Rule failure during close of staging repositories: [orgtouchbit-1039]
[ERROR]  * Dropping failed staging repository with ID "orgtouchbit-1039" (Rule failure during close of staging repositories: [orgtouchbit-1039]).
[ERROR] Remote staging finished with a failure: Staging rules failure!
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Shields4J 1.0.0 .................................... SUCCESS [  4.073 s]
[INFO] test-core .......................................... SUCCESS [  2.788 s]
[INFO] Shields4J client ................................... SUCCESS [  3.962 s]
[INFO] TestNG listener 1.0.0 .............................. FAILURE [01:07 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

Natijada bizda faqat bitta tanlov qoladi. Ushbu versiyani o'chiring yoki nashr qiling.

Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

Chiqarilganidan keyin, bir muncha vaqt o'tgach, artefaktlar ichkariga kiradi Maven centerga java loyihasini yuklash uchun GitLab CI o'rnatilmoqda

mavzudan tashqari

Maven boshqa ommaviy omborlarni indekslashi men uchun kashfiyot bo'ldi.
Men robots.txt ni qo'shishim kerak edi, chunki u mening eski omborimni indeksladi.

Mundarija jadvaliga qaytish

xulosa

Bizda nima bor

  • Alohida joylashtirish loyihasi bo'lib, unda siz turli xil tillar uchun ommaviy omborlarga artefaktlarni yuklash uchun bir nechta CI vazifalarini bajarishingiz mumkin.
  • Deploy loyihasi tashqi aralashuvdan ajratilgan va uni faqat Egasi va Ta'minotchi roliga ega foydalanuvchilar o'zgartirishi mumkin.
  • Faqat tarqatish vazifalarini bajarish uchun "issiq" keshga ega alohida maxsus yuguruvchi.
  • Snapshot/reliz versiyalarini ommaviy omborda nashr qilish.
  • Maven markazida nashrga tayyorligi uchun reliz versiyasini avtomatik tekshirish.
  • Maven markazida "xom" versiyalarning avtomatik nashr etilishidan himoya.
  • Snapshot versiyalarini “bosish orqali” yarating va nashr eting.
  • Snapshot/reliz versiyalarini olish uchun yagona ombor.
  • Java loyihasini qurish/sinov/nashr qilish uchun umumiy quvur liniyasi.

GitLab CI-ni sozlash birinchi qarashda ko'rinadigan darajada murakkab mavzu emas. CI ni bir necha marta kalit asosida o'rnatish kifoya va endi siz bu masalada havaskorlikdan yiroqsiz. Bundan tashqari, GitLab hujjatlari juda ortiqcha. Birinchi qadamni qo'yishdan qo'rqmang. Yo'l yurgan odamning zinapoyasi ostida paydo bo'ladi (kim aytgani esimda yo'q :)

Fikr-mulohazalarni qabul qilishdan xursand bo'laman.

Keyingi maqolada men GitLab CI-ni integratsiya testlari bilan vazifalarni raqobatdosh tarzda bajarish uchun qanday sozlash haqida gapiraman (docker-compose yordamida sinov ostidagi xizmatlarni ishga tushirish), agar sizda faqat bitta qobiq bo'lsa.

Mundarija jadvaliga qaytish

Manba: www.habr.com