🥇Buildbot u primjerima

Trebao sam postaviti proces sastavljanja i isporuke softverskih paketa iz Git repozitorija na stranicu. I kada sam, ne tako davno, ovdje na Habréu vidio članak o buildbotu (link na kraju), odlučio sam ga isprobati i primijeniti.

Budući da je buildbot distribuirani sustav, bilo bi logično stvoriti zaseban host za izgradnju za svaku arhitekturu i operativni sustav. U našem slučaju to će biti LXC spremnici (u slučaju Linuxa) i qemu (u slučaju Windowsa):

vm-srv-build1 - centos 7, bit će buildbot master i jedan od radnika
vm-srv-build2 - debian 10, za izgradnju DEB paketa
vm-srv-build3 - windows 10, za montažu, znate čemu

Skupit ćemo Rac GUI — grafičko sučelje za 1C rac za upravljanje klasterom poslužitelja. Za Linux će se koristiti standardni alati za svaki OS; za izradu exe datoteke za Windows iz tcl skripte koristite besplatni omot.

Instalacija

GNU / Linux

Na internetu ima dovoljno dokumentacije za instalaciju 1,2. Da, i to ne uzrokuje nikakve posebne probleme:
Za majstora:

pip3 install buildbot
pip3 install twisted
pip3 install autobahn
pip3 install pysqlite3
pip3 install sqlalchemy sqlalchemy-migrate
pip3 install buildbot-www buildbot-grid-view buildbot-console-view buildbot-waterfall-view
pip3 install python-dateutil

Za "radnike" je dovoljno ovo:

pip3 install buildbot-worker

Naravno, bilo bi ispravnije prikupljati pakete za svaki OS, ali to je izvan okvira članka. Također ćemo izostaviti opis postavljanja spremnika za rad, samo ću napomenuti da koristim ProxMox VE. Također ćete morati instalirati pakete za svaku os potrebnu za montažu (centos: rpmdevtools itd.; debian: build-essential, dh-make, pbuilder itd.)

Projekti izgradnje i usluge buildbota pokrenut će se kao neprivilegirani korisnik, tako da morate izraditi jedan na svim hostovima koji sudjeluju u procesu:


adduser buildbot

Zatim ćemo konfigurirati automatsko pokretanje usluga, odnosno, na svakom od hostova (spremnika):

Systemd jedinica za pokretanje čarobnjaka:

touch /etc/systemd/buildbot-master.service 

[Unit]
Description=BuildBot master service
After=network.target

[Service]
User=buildbot
Group=buildbot
WorkingDirectory=/home/buildbot/master
ExecStart=/usr/local/bin/buildbot start --nodaemon
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=multi-user.target

i "radnik"

touch /etc/systemd/buildbot-worker.service 

[Unit]
Description=BuildBot worker service
After=network.target

[Service]master
User=buildbot
Group=buildbot
WorkingDirectory=/home/buildbot/worker
ExecStart=/usr/local/bin/buildbot-worker start --nodaemon

[Install]
WantedBy=buildbot-master.service

Budući da se sve skripte (u našem slučaju) nalaze u /usr/local/, trebali biste postaviti put do njih u varijablama okruženja:

nano /root/.bash_profile

PATH=$PATH:$HOME/.local/bin:$HOME/bin:/usr/local/bin

Nakon ovoga, možete stvoriti infrastrukturu imenika za "radnike" (na svim hostovima), da biste to učinili, registrirajte se pod buildbot korisnikom i pokrenite sljedeće naredbe:

Na prvom hostu vm-srv-build1:

su - buildbot
mkdir /home/buildbot/worker
cd ~
buildbot-worker create-worker --umask=0o22 --keepalive=60 worker vm-srv-build1:4000 CentOS 123456

Na drugom hostu vm-srv-build2:

su - buildbot
mkdir /home/buildbot/worker
cd ~
buildbot-worker create-worker --umask=0o22 --keepalive=60 worker vm-srv-build1:4000 Debian-10 123456

Na radnim hostovima može se pokrenuti usluga buildbot-worker

systemctl start buildbot-worker

MS Windows

Kao "radnik" za montažu pod Windowsom koristit će se virtualni stroj s najnovijim izdanjem Win10.
Za rad trebate:

Piton
PyWIN
ići
Da biste instalirali razne stvari u Python, morali ste preuzeti Alati za izradu za Visual Studio 2019
7Zip

Nakon što je sve gore navedeno instalirano, možete instalirati sam buildbot:

pip install buildbot-worker

Kreirajmo radni imenik

md c:worker

I lansirajmo

buildbot-worker start c:worker

Ako sve radi (pogledajte zapisnik c:workertwistd.log), tada možete registrirati našeg "radnika" kao uslugu dodavanjem stavke s radnim imenikom u registar (naredbe se izvršavaju u powershell-u koji radi kao administrator):

buildbot_worker_windows_service.exe --user VM-SRV-BUILD3buildbot --password 123456 --startup auto install
New-ItemProperty -path Registry::HKEY_LOCAL_MACHINESYSTEMCurrentControlSetservicesBuildBotParameters -Name directories -PropertyType String -Value c:worker

I možete pokrenuti uslugu

Start-Service buildbot

To je sve s "radnicima", ne morate ih više dirati, sva kontrola dolazi od gospodara.

Postavljanje čarobnjaka

Za početak, stvorimo infrastrukturu za master (na glavnom hostu), da biste to učinili, registrirajte se pod buildbot korisnikom i pokrenite sljedeće naredbe:

su - buildbot
mkdir /home/buildbot/master
cd ~
buildbot create-master master

Za gotove pakete stvorite direktorij za izgradnju

mkdir /home/buildbot/builds

Datoteka master.cfg stvorena je u direktoriju /home/buildbot/master/. Ova datoteka je python kod i sadrži opis svih mehanizama sustava; s njom ćemo raditi u budućnosti.

nano /home/buildbot/master/master.cfg

import os, re
from buildbot.plugins import steps, util, schedulers, worker, changes, reporters

c= BuildmasterConfig ={}

# Описание наших рабочих.
c['workers'] = [ worker.Worker('CentOS', '123456'), worker.Worker('Debian-10', '123456'), worker.Worker('Windows-10', '123456')]
c['protocols'] = {'pb': {'port': 4000}} 

# Указание мастеру какой репозиторий отслеживать
c['change_source'] = []
c['change_source'].append(changes.GitPoller(
       repourl = 'https://bitbucket.org/svk28/rac-gui.git',
       project = 'Rac-GUI',
       branches = True,
       pollInterval = 60
      )
)

# служба запуска сборки
c['schedulers'] = []
c['schedulers'].append(schedulers.SingleBranchScheduler(
       name="Rac-GUI-schedulers",
       change_filter=util.ChangeFilter(branch='master'),
       builderNames=["Rac-GUI-RPM-builder", "Rac-GUI-DEB-builder", "Rac-GUI-WIN-builder"],
       properties = {'owner': 'admin'}
       )
)
@util.renderer

######################################3
# Сборка RPM-пакета
 rac_gui_build_RPM = util.BuildFactory()

rac_gui_build_RPM.addStep(steps.Git(
       repourl = 'https://bitbucket.org/svk28/rac-gui.git',
       workdir = 'rac-gui',
       haltOnFailure = True,
       submodules = True,
       mode='full',
       progress = True)
)

Za automatizaciju sastavljanja paketa različitih verzija, kako ne biste morali kopati u kod master.cfg datoteke, u glavnoj skripti programa rac_gui.tcl, u zaglavlju su dodani redovi s trenutnom verzijom i izdanjem :

######################################################
#        Rac GUI
...
# version: 1.0.3
# release: 1

I na temelju ovih redaka, buildbot će numerirati pakete. Za izdvajanje podataka upotrijebite grep poziv konzole. U buildbotu jednostavno ne možete definirati varijable za "radnike" (barem ja nisam pronašao kako). Za to se koriste svojstva. Oni. U procesu sklapanja dodajemo korake za određivanje verzije i izdanja te, sukladno tome, postavljamo svojstva verzije i izdanja. Svojstva se mogu postaviti na različite načine, u ovom slučaju pozivanjem konzolne naredbe:

# Добавим определение версии из основного файла
rac_gui_build_RPM.addStep(
       steps.SetPropertyFromCommand(
       command="grep version ../rac-gui/rac_gui.tcl | grep -oE 'b[0-9]{1,2}.[0-9]{1,2}.[0-9]{1,2}b'", property="version"
       )
)
# Добавим определение релиза из основного файла
rac_gui_build_RPM.addStep(
       steps.SetPropertyFromCommand(
       command="grep release ../rac-gui/rac_gui.tcl | grep -oE 'b[0-9]{1,3}b'", property="release"
       )
)

Dobivene vrijednosti možete zamijeniti pozivom util.Interpolate().

# Запакуем исходники
rac_gui_build_RPM.addStep(
       steps.ShellCommand(
          command=["tar", "czf", util.Interpolate("/home/buildbot/rpmbuild/SOURCES/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"), "../rac-gui"]
       )
)

Ovdje treba napomenuti da budući da se host također koristi za ručno sastavljanje paketa, sastavljanje će se odvijati duž standardnih putanja.

# Копируем spec
rac_gui_build_RPM.addStep(steps.ShellCommand(
       command=["cp", "../rac-gui/rac_gui.spec", "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"]))

Za postavljanje ispravnog izdanja i brojeva verzija upotrijebite standardni sed poziv, tj. naredba zamjenjuje vrijednosti unutar spec datoteke s potrebnima

# меняем версию
rac_gui_build_RPM.addStep(steps.ShellCommand(
       command=["sed", "-i", util.Interpolate("s/.*Version:.*/Version:t%(prop:version)s/"), "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"]))
# меняем релиз
rac_gui_build_RPM.addStep(steps.ShellCommand(
       command=["sed", "-i", util.Interpolate("s/.*Release:.*/Release:t%(prop:release)s/"), "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"]))

# запускаем процесс сборки
rac_gui_build_RPM.addStep(steps.RpmBuild(
       specfile="/home/buildbot/rpmbuild/SPECS/rac_gui.spec",
       dist='.el5',
       topdir='/home/buildbot/rpmbuild',
       builddir='/home/buildbot/rpmbuild/build',
       rpmdir='/home/buildbot/rpmbuild/RPMS',
       sourcedir='/home/buildbot/rpmbuild/SOURCES'
      )
)

Kopiramo gotov sastavljeni paket i arhivu s izvorima na master. Ali možete odmah kopirati datoteke iz svoje radne datoteke u svoje spremište ili web mjesto.

# Скопируем файл на мастер
rac_gui_build_RPM.addStep(
       steps.FileUpload(
          workersrc=util.Interpolate("/home/buildbot/rpmbuild/RPMS/noarch/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm"),
          masterdest=util.Interpolate("/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm")
      )
)

rac_gui_build_RPM.addStep(
       steps.FileUpload(
       workersrc=util.Interpolate("/home/buildbot/rpmbuild/SOURCES/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"),
          masterdest=util.Interpolate("/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz")
       )
)

Pokrenimo proces kopiranja prikupljenih paketa na hosting putem FTP-a na masteru. U tu svrhu koristi se skripta na tcl.

rac_gui_build_RPM.addStep(
        steps.MasterShellCommand(
            command=["/usr/local/bin/deploy-ftp.tcl",
                util.Interpolate("--local-file=/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm"),
                util.Interpolate("--remote-file=uploads/rac-gui/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm")]
        )
)

rac_gui_build_RPM.addStep(
        steps.MasterShellCommand(
            command=["/usr/local/bin/deploy-ftp.tcl",
                util.Interpolate("--local-file=/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"),
                util.Interpolate("--remote-file=uploads/rac-gui/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz")]
        )
)

To je to s RPM-om. Sada počnimo opisivati algoritam za izgradnju DEB paketa. Budući da su procesi za izgradnju paketa za različite sustave neovisni jedan o drugome, mnogi će se koraci ponavljati.


rac_gui_build_DEB = util.BuildFactory()

rac_gui_build_DEB.addStep(steps.Git(
       repourl = 'https://bitbucket.org/svk28/rac-gui.git',
       haltOnFailure = True,
       submodules = True,
       mode='full',
       workdir='build',
       progress = True)
)

# Добавим определение версии из основного файла
rac_gui_build_DEB.addStep(
       steps.SetPropertyFromCommand(
       command="grep version rac_gui.tcl | grep -oE 'b[0-9]{1,2}.[0-9]{1,2}.[0-9]{1,2}b'", property="version"
       )
)

# Добавим определение релиза из основного файла
rac_gui_build_DEB.addStep(
       steps.SetPropertyFromCommand(
       command="grep release rac_gui.tcl | grep -oE 'b[0-9]{1,3}b'", property="release"
       )
)

# Переименуем запускаемый файл
    rac_gui_build_DEB.addStep(steps.ShellCommand(
       command=["mv", "rac_gui.tcl", "racgui"]))

Za RPM paket, neke od sljedećih postupaka obavlja sam rpm tijekom sastavljanja i opisani su unutar specifikacije; za Debian to morate učiniti ovdje:

# Поменяем пути к библиотекам
rac_gui_build_DEB.addStep(steps.ShellCommand(
       command=["sed", "-i", "s+^set dir(lib)+set dir(lib) /usr/share/rac-gui/lib ;#+g", "racgui"]))

# Поменяем пути к файлам
rac_gui_build_DEB.addStep(steps.ShellCommand(
       command=["sed", "-i", "s+[pwd]+/usr/share/rac-gui+g", "racgui"]))

# заархивируем исходники
rac_gui_build_DEB.addStep(steps.ShellCommand(
       command=["tar", "czf", util.Interpolate("../rac-gui_%(prop:version)s.orig.tar.gz"), "."]))

# Соберём пакет
rac_gui_build_DEB.addStep(steps.ShellCommand(
       command=["dpkg-buildpackage"]))

# Скопируем файл на мастер
rac_gui_build_DEB.addStep(
       steps.FileUpload(
       workersrc=util.Interpolate("../rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb"),
           masterdest=util.Interpolate("/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb")
       )
)
rac_gui_build_DEB.addStep(
        steps.MasterShellCommand(
            command=["/usr/local/bin/deploy-ftp.tcl",
             util.Interpolate("--local-file=/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb"),
             util.Interpolate("--remote-file=uploads/rac-gui/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb")]
        )
)

I gotovo s DEB-om, sada Windows!

rac_gui_build_WIN = util.BuildFactory()

rac_gui_build_WIN.addStep(steps.Git(
       repourl = 'https://bitbucket.org/svk28/rac-gui.git',
       haltOnFailure = True,
       submodules = True,
       mode='full',
       workdir='build',
       progress = True)
)

Budući da grep i sed nisu standardni u Windowsima (ili postoje?), koristit ćemo powershell

# Добавим определение версии из основного файла
rac_gui_build_WIN.addStep(
       steps.SetPropertyFromCommand(
       command="powershell -command "((Get-Content .rac_gui.tcl | Select-String -Pattern 'version:') -split 's')[2]",
       property="version"
       )
    )

# Добавим определение релиза из основного файла
rac_gui_build_WIN.addStep(
       steps.SetPropertyFromCommand(
       command="powershell -command "((Get-Content .rac_gui.tcl | Select-String -Pattern 'release:') -split 's')[2]",
       property="release"
       )
)

# Создадим запускаемый файл
rac_gui_build_WIN.addStep(steps.ShellCommand(
       command=["c:binfreewrap.exe", "rac_gui.tcl"]))

# запакуем то, что получилось
rac_gui_build_WIN.addStep(steps.ShellCommand(
       command=["c:Program Files7-zip7z.exe", "a", "-r", util.Interpolate("..rac-gui_%(prop:version)s-%(prop:release)s.win.zip"), "..build"]))

# скопируем на мастер
rac_gui_build_WIN.addStep(
       steps.FileUpload(
       workersrc=util.Interpolate("..rac-gui_%(prop:version)s-%(prop:release)s.win.zip"),
           masterdest=util.Interpolate("/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s.win.zip")
       )
)
# Скопируем файл на хостинг
rac_gui_build_WIN.addStep(
        steps.MasterShellCommand(
            command=["/usr/local/bin/deploy-ftp.tcl",
            util.Interpolate("--local-file=/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s.win.zip"),
            util.Interpolate("--remote-file=uploads/rac-gui/rac-gui_%(prop:version)s-%(prop:release)s.win.zip")]
       )
)

# Тут определяем какие сборщики у нас есть
c['builders'] = [
        util.BuilderConfig(name="Rac-GUI-RPM-builder", workername='CentOS', factory=rac_gui_build_RPM),
        util.BuilderConfig(name="Rac-GUI-DEB-builder", workername='Debian-10', factory=rac_gui_build_DEB),
        util.BuilderConfig(name="Rac-GUI-WIN-builder", workername='Windows-10', factory=rac_gui_build_WIN),
]

E-poštom ćemo vas obavijestiti o statusu procesa izgradnje.


c['services'] = []

template=u'''
    <h4>Build status: {{ summary }}</h4>
    <p> Worker used: {{ workername }}</p>
    {% for step in build['steps'] %}
    <p> {{ step['name'] }}: {{ step['result'] }}</p>
    {% endfor %}
    <p><b> -- The Buildbot</b></p>
    '''

mailNotifier = reporters.MailNotifier(fromaddr="[email protected]",
             sendToInterestedUsers=False,
             mode=('all'),
             extraRecipients=["[email protected]"],
             relayhost="mail.domain.ru",
             smtpPort=587,
             smtpUser="[email protected]",
             smtpPassword="******",
             messageFormatter=reporters.MessageFormatter(
                                 template=template, template_type='html',
                                 wantProperties=True, wantSteps=True))

c['services'].append(mailNotifier)

# Основные настройки мастера
c['title'] = "The process of bulding"
c['titleURL'] = "http://vm-srv-build1:80/"
c['buildbotURL'] = "http://vm-srv-build1/"
c['www'] = dict(port=80,
                plugins=dict(waterfall_view={}, console_view={}, grid_view={}))
c['db'] = {
       'db_url' : "sqlite:///state.sqlite"
}

Spremite datoteku i možete pokušati pokrenuti uslugu čarobnjaka:

systemctl restart buildbot-master

U zapisniku ćemo provjeriti je li sve u redu s konfiguracijom i radi li sve normalno. Svi naši radnici sada bi se trebali povezati, što će biti sretno prijavljeno u dnevniku »»'/home/buildbot/master/twistd.log»»':

 2019-07-24 16:50:35+0300 [-] Loading buildbot.tac...
 2019-07-24 16:50:35+0300 [-] Loaded.
 2019-07-24 16:50:35+0300 [-] twistd 19.2.1 (/usr/bin/python3.6 3.6.8) starting up.
 2019-07-24 16:50:35+0300 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
 2019-07-24 16:50:35+0300 [-] Starting BuildMaster -- buildbot.version: 2.3.1
 2019-07-24 16:50:35+0300 [-] Loading configuration from '/home/buildbot/master/master.cfg'
 2019-07-24 16:50:36+0300 [-] /usr/local/lib/python3.6/site-packages/buildbot/config.py:90: buildbot.config.ConfigWarning: [0.9.0 and later] `buildbotNetUsageData` is not configured and defaults to basic.
        This parameter helps the buildbot development team to understand the installation base.
        No personal information is collected.
        Only installation software version info and plugin usage is sent.
        You can `opt-out` by setting this variable to None.
        Or `opt-in` for more information by setting it to "full".

 2019-07-24 16:50:36+0300 [-] Setting up database with URL 'sqlite:/state.sqlite'
 2019-07-24 16:50:36+0300 [-] setting database journal mode to 'wal'
 2019-07-24 16:50:36+0300 [-] adding 1 new services, removing 0
 2019-07-24 16:50:36+0300 [-] adding 1 new change_sources, removing 0
 2019-07-24 16:50:36+0300 [-] gitpoller: using workdir '/home/buildbot/master/gitpoller-work'
 2019-07-24 16:50:36+0300 [-] adding 3 new builders, removing 0
 2019-07-24 16:50:36+0300 [-] adding 1 new schedulers, removing 0
 2019-07-24 16:50:36+0300 [-] initializing www plugin 'waterfall_view'
 2019-07-24 16:50:36+0300 [-] initializing www plugin 'console_view'
 2019-07-24 16:50:36+0300 [-] initializing www plugin 'grid_view'
 2019-07-24 16:50:36+0300 [-] NOTE: www plugin 'sitenav' is installed but not configured
 2019-07-24 16:50:36+0300 [-] initializing www plugin 'waterfall_view'
 2019-07-24 16:50:36+0300 [-] initializing www plugin 'console_view'
 2019-07-24 16:50:36+0300 [-] initializing www plugin 'grid_view'
 2019-07-24 16:50:36+0300 [-] NOTE: www plugin 'sitenav' is installed but not configured
 2019-07-24 16:50:36+0300 [-] BuildbotSite starting on 80
 2019-07-24 16:50:36+0300 [-] Starting factory <buildbot.www.service.BuildbotSite object at 0x7fe31c2657b8>
 2019-07-24 16:50:36+0300 [-] adding 3 new workers, removing 0
 2019-07-24 16:50:36+0300 [-] PBServerFactory starting on 4000
 2019-07-24 16:50:36+0300 [-] Starting factory <twisted.spread.pb.PBServerFactory object at 0x7fe31c147470>
 2019-07-24 16:50:37+0300 [-] BuildMaster is running
 2019-07-24 16:50:37+0300 [-] buildbotNetUsageData: sending {'installid': 'b6193b126b96689351d2fe95787c5a03fc0879f9', 'versions': {'Python': '3.6.8', 'Buildbot': '2.3.1', 'Twisted': '19.2.1'}, 'platform': {'platform': 'Linux-4.15.18-10- pve-x86_64-with-centos-7.6.1810-Core', 'system': 'Linux', 'machine': 'x86_64', 'processor': 'x86_64', 'python_implementation': 'CPython', 'version': '#1 SMP PVE 4.15.18-32', 'distro': 'centos:7'}, 'plugins': {'buildbot/worker/base/Worker': 3, 'buildbot/config/BuilderConfig': 3, 'buildbot/schedulers/basic/SingleBranchScheduler': 1, 'buildbot/reporters/mail/MailNotifier': 1, 'buildbot/changes/gitpoller/GitPoller': 1, 'buildbot/steps/worker/MakeDirectory': 1, 'buildbot/steps/source/git/Git': 3, 'buildbot/steps/shell/ShellCommand': 9, 'buildbot/steps/package/rpm/rpmbuild/RpmBuild': 1}, 'db': 'sqlite', 'mq': 'simple', 'www_plugins': ['waterfall_view', 'console_view', 'grid_view']}
 2019-07-24 16:50:37+0300 [Broker,0,127.0.0.1] worker 'CentOS' attaching from IPv4Address(type='TCP', host='127.0.0.1', port=37332)
 2019-07-24 16:50:37+0300 [Broker,0,127.0.0.1] Got workerinfo from 'CentOS'
 2019-07-24 16:50:37+0300 [-] bot attached
 2019-07-24 16:50:37+0300 [Broker,0,127.0.0.1] Worker CentOS attached to Rac-GUI-RPM-builder
 2019-07-24 16:50:37+0300 [-] buildbotNetUsageData: buildbot.net said: ok
 2019-07-24 16:50:39+0300 [Broker,1,192.168.55.15] worker 'Windows-10' attaching from IPv4Address(type='TCP', host='192.168.5.145', port=49831)
 2019-07-24 16:50:39+0300 [Broker,1,192.168.55.15] Got workerinfo from 'Windows-10'
 2019-07-24 16:50:40+0300 [-] bot attached
 2019-07-24 16:50:40+0300 [Broker,1,192.168.55.15] Worker Windows-10 attached to Rac-GUI-WIN-builder
 2019-07-24 16:50:41+0300 [Broker,2,192.168.55.99] worker 'Debian-10' attaching from IPv4Address(type='TCP', host='192.168.5.9', port=44430)
 2019-07-24 16:50:41+0300 [Broker,2,192.168.55.99] Got workerinfo from 'Debian-10'
 2019-07-24 16:50:41+0300 [-] bot attached
 2019-07-24 16:50:41+0300 [Broker,2,192.168.55.99] Worker Debian-10 attached to Rac-GUI-DEB-builder

Time je postupak postavljanja dovršen. Možete vidjeti trenutni status putem web sučelja. Gdje također možete vidjeti greške u izradi, izbaciti zamrznuti proces ako je nešto pošlo krivo itd.

Odmah nakon lansiranja, naše vrijedne radnike možete pregledati kroz izbornik “Builds” -> “Workers”

Nakon što se završi prvi proces izgradnje (tj. promjene Git repozitorija), status procesa će se pojaviti na prvoj stranici.

Ako kliknete mišem na željenu liniju, otvorit će vam se stranica s trenutnim stanjem ovog procesa, gdje možete vidjeti što se događa, koje greške itd.

Cijela konfiguracija čarobnjaka može se pronaći ovdje

import os, re
from buildbot.plugins import steps, util, schedulers, worker, changes, reporters
c= BuildmasterConfig ={}
c['workers'] = [ worker.Worker('CentOS', '123456'), worker.Worker('Debian-10', '123456'), worker.Worker('Windows-10', '123456')]
c['protocols'] = {'pb': {'port': 4000}} 
c['change_source'] = []
c['change_source'].append(changes.GitPoller(
repourl = 'https://bitbucket.org/svk28/rac-gui.git',
project = 'Rac-GUI',
branches = True,
pollInterval = 600
))
# служба запуска сборки
c['schedulers'] = []
c['schedulers'].append(schedulers.SingleBranchScheduler(
name="Rac-GUI-schedulers",
change_filter=util.ChangeFilter(branch='master'),
builderNames=["Rac-GUI-RPM-builder", "Rac-GUI-DEB-builder", "Rac-GUI-WIN-builder"],
properties = {'owner': 'admin'}
))
@util.renderer
def get_name_version_release(props):
prog_name = "rac-gui"
prog_version = "1.0.3"
prog_release = "3"
return {
"prog_name": prog_name
#"prog_version": prog_version,
#"prog_release": prog_release
}
rac_gui_build_RPM = util.BuildFactory()
rac_gui_build_RPM.addStep(steps.Git(
repourl = 'https://bitbucket.org/svk28/rac-gui.git',
workdir = 'rac-gui',
haltOnFailure = True,
submodules = True,
mode='full',
progress = True)
)
# Добавим определение версии из основного файла
rac_gui_build_RPM.addStep(
steps.SetPropertyFromCommand(
command="grep version ../rac-gui/rac_gui.tcl | grep -oE 'b[0-9]{1,2}.[0-9]{1,2}.[0-9]{1,2}b'", property="version"
)
)
# Добавим определение релиза из основного файла
rac_gui_build_RPM.addStep(
steps.SetPropertyFromCommand(
command="grep release ../rac-gui/rac_gui.tcl | grep -oE 'b[0-9]{1,3}b'", property="release"
)
)
rac_gui_build_RPM.addStep(steps.ShellCommand(
command=["tar", "czf", util.Interpolate("/home/buildbot/rpmbuild/SOURCES/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"), "../rac-gui"]))
rac_gui_build_RPM.addStep(steps.ShellCommand(
command=["cp", "../rac-gui/rac_gui.spec", "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"]))
rac_gui_build_RPM.addStep(steps.ShellCommand(
command=["sed", "-i", util.Interpolate("s/.*Version:.*/Version:t%(prop:version)s/"), "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"]))
rac_gui_build_RPM.addStep(steps.ShellCommand(
command=["sed", "-i", util.Interpolate("s/.*Release:.*/Release:t%(prop:release)s/"), "/home/buildbot/rpmbuild/SPECS/rac_gui.spec"]))
rac_gui_build_RPM.addStep(steps.RpmBuild(
specfile="/home/buildbot/rpmbuild/SPECS/rac_gui.spec",
dist='.el5',
topdir='/home/buildbot/rpmbuild',
builddir='/home/buildbot/rpmbuild/build',
rpmdir='/home/buildbot/rpmbuild/RPMS',
sourcedir='/home/buildbot/rpmbuild/SOURCES'
))
# Скопируем файл на мастер
rac_gui_build_RPM.addStep(
steps.FileUpload(
workersrc=util.Interpolate("/home/buildbot/rpmbuild/RPMS/noarch/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm"),
masterdest=util.Interpolate("/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm")
)
)
rac_gui_build_RPM.addStep(
steps.FileUpload(
workersrc=util.Interpolate("/home/buildbot/rpmbuild/SOURCES/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"),
masterdest=util.Interpolate("/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz")
)
)
rac_gui_build_RPM.addStep(
steps.MasterShellCommand(
command=["/usr/local/bin/deploy-ftp.tcl",
util.Interpolate("--local-file=/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm"),
util.Interpolate("--remote-file=uploads/rac-gui/rac-gui-%(prop:version)s-%(prop:release)s.noarch.rpm")]
)
)
rac_gui_build_RPM.addStep(
steps.MasterShellCommand(
command=["/usr/local/bin/deploy-ftp.tcl",
util.Interpolate("--local-file=/home/buildbot/builds/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz"),
util.Interpolate("--remote-file=uploads/rac-gui/rac-gui-%(prop:version)s-%(prop:release)s.tar.gz")]
)
)
####################################
##      DEB 
####################################
rac_gui_build_DEB = util.BuildFactory()
rac_gui_build_DEB.addStep(steps.Git(
repourl = 'https://bitbucket.org/svk28/rac-gui.git',
haltOnFailure = True,
submodules = True,
mode='full',
workdir='build',
progress = True)
)
# Добавим определение версии из основного файла
rac_gui_build_DEB.addStep(
steps.SetPropertyFromCommand(
command="grep version rac_gui.tcl | grep -oE 'b[0-9]{1,2}.[0-9]{1,2}.[0-9]{1,2}b'", property="version"
)
)
# Добавим определение релиза из основного файла
rac_gui_build_DEB.addStep(
steps.SetPropertyFromCommand(
command="grep release rac_gui.tcl | grep -oE 'b[0-9]{1,3}b'", property="release"
)
)
# Переименуем запускаемый файл
rac_gui_build_DEB.addStep(steps.ShellCommand(
command=["mv", "rac_gui.tcl", "racgui"]))
# Поменяем пути к библиотекам
rac_gui_build_DEB.addStep(steps.ShellCommand(
command=["sed", "-i", "s+^set dir(lib)+set dir(lib) /usr/share/rac-gui/lib ;#+g", "racgui"]))
# Поменяем пути к файлам
rac_gui_build_DEB.addStep(steps.ShellCommand(
command=["sed", "-i", "s+[pwd]+/usr/share/rac-gui+g", "racgui"]))
# заархивируем исходники
rac_gui_build_DEB.addStep(steps.ShellCommand(
command=["tar", "czf", util.Interpolate("../rac-gui_%(prop:version)s.orig.tar.gz"), "."]))
# Соберём пакет
rac_gui_build_DEB.addStep(steps.ShellCommand(
command=["dpkg-buildpackage"]))
# Скопируем файл на мастер
rac_gui_build_DEB.addStep(
steps.FileUpload(
workersrc=util.Interpolate("../rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb"),
masterdest=util.Interpolate("/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb")
)
)
rac_gui_build_DEB.addStep(
steps.MasterShellCommand(
command=["/usr/local/bin/deploy-ftp.tcl",
util.Interpolate("--local-file=/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb"),
util.Interpolate("--remote-file=uploads/rac-gui/rac-gui_%(prop:version)s-%(prop:release)s_amd64.deb")]
)
)
############################################
#       WIN
############################################
rac_gui_build_WIN = util.BuildFactory()
rac_gui_build_WIN.addStep(steps.Git(
repourl = 'https://bitbucket.org/svk28/rac-gui.git',
haltOnFailure = True,
submodules = True,
mode='full',
workdir='build',
progress = True)
)
# Добавим определение версии из основного файла
rac_gui_build_WIN.addStep(
steps.SetPropertyFromCommand(
command="powershell -command "((Get-Content .rac_gui.tcl | Select-String -Pattern 'version:') -split 's')[2]",
property="version"
)
)
# Добавим определение релиза из основного файла
rac_gui_build_WIN.addStep(
steps.SetPropertyFromCommand(
command="powershell -command "((Get-Content .rac_gui.tcl | Select-String -Pattern 'release:') -split 's')[2]",
property="release"
)
)
# Создадим запускаемый файл
rac_gui_build_WIN.addStep(steps.ShellCommand(
command=["c:binfreewrap.exe", "rac_gui.tcl"]))
# запакуем то, что получилось
rac_gui_build_WIN.addStep(steps.ShellCommand(
command=["c:Program Files7-zip7z.exe", "a", "-r", util.Interpolate("..rac-gui_%(prop:version)s-%(prop:release)s.win.zip"), "..build"]))
# скопируем на мастер
rac_gui_build_WIN.addStep(
steps.FileUpload(
workersrc=util.Interpolate("..rac-gui_%(prop:version)s-%(prop:release)s.win.zip"),
masterdest=util.Interpolate("/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s.win.zip")
)
)
# Скопируем файл на хостинг
rac_gui_build_WIN.addStep(
steps.MasterShellCommand(
command=["/usr/local/bin/deploy-ftp.tcl",
util.Interpolate("--local-file=/home/buildbot/builds/rac-gui_%(prop:version)s-%(prop:release)s.win.zip"),
util.Interpolate("--remote-file=uploads/rac-gui/rac-gui_%(prop:version)s-%(prop:release)s.win.zip")]
)
)
c['builders'] = [
util.BuilderConfig(name="Rac-GUI-RPM-builder", workername='CentOS', factory=rac_gui_build_RPM),
util.BuilderConfig(name="Rac-GUI-DEB-builder", workername='Debian-10', factory=rac_gui_build_DEB),
util.BuilderConfig(name="Rac-GUI-WIN-builder", workername='Windows-10', factory=rac_gui_build_WIN),
]
c['services'] = []
template=u'''
<h4>Build status: {{ summary }}</h4>
<p> Worker used: {{ workername }}</p>
{% for step in build['steps'] %}
<p> {{ step['name'] }}: {{ step['result'] }}</p>
{% endfor %}
<p><b> -- The Buildbot</b></p>
'''
mailNotifier = reporters.MailNotifier(fromaddr="[email protected]",
sendToInterestedUsers=False,
mode=('all'),
extraRecipients=["[email protected]"],
relayhost="mail.domain.local",
smtpPort=587,
smtpUser="[email protected]",
smtpPassword="**********",
messageFormatter=reporters.MessageFormatter(
template=template, template_type='html',
wantProperties=True, wantSteps=True))
c['services'].append(mailNotifier)
c['title'] = "The process of bulding"
c['titleURL'] = "http://vm-srv-build1:80/"
c['buildbotURL'] = "http://vm-srv-build1/"
c['www'] = dict(port=80,
plugins=dict(waterfall_view={}, console_view={}, grid_view={}))
c['db'] = {
'db_url' : "sqlite:///state.sqlite"
}

Materijali

U pripremi članka korišteni su sljedeći materijali:

Izvor: www.habr.com

Buildbot u primjerima

Instalacija

GNU / Linux

MS Windows

Postavljanje čarobnjaka

Materijali

Dodajte komentar Odustani odgovor