(Image by from
Hi!
My name is Evgeny Cherkin, I am a programmer for a development team in a mining company Polymetal.
Starting any large project, you start to think: “What software is better to use to maintain it?”. An IT project goes through a series of stages before releasing the next version. It is good when the chain of these stages is automated. In itself, the automated process of releasing a new version of an IT project is called Continuous Integration. buildbot turned out to be a good helper for us, realizing this process.
In this article, I decided to present an overview of the possibilities buildbot. What is this software capable of? How to approach him and how to build a normal, EFFECTIVE WORKING RELATIONSHIP with him? You can apply our experience on your own by creating a working assembly and testing service for your project on your machine.
Content
Content
1. Why BuildBot?
Earlier on habr-e, I met articles about the implementation Continuous Integration using buildbot. For example, seemed to me the most informative. There is another example - . These articles can be edited , afterward, in English. The coupe makes a good starting point. After reading these articles, you will surely want something on buildbot сделать.
Stop! Has anyone actually used it in their projects? It turns out yes applied it in their tasks. Can be found use buildbot and in the google codes archives.
So what is the logic of people using buildbot? After all, there are other tools: cruise control и Jenkins. I will answer like this. For most tasks Jenkins and the truth will suffice. In its turn, buildbot - more adaptive, while tasks are solved there as simply as in Jenkins. You choose. But since we are looking for a tool for a developing target project, why not choose one that will allow, starting from simple steps, to get a build system that has interactivity and a unique interface.
For those whose target project is written in python, the question arises: “Why not choose an integration system that has a clear interface in terms of the language used in the project?”. And now it's time to present the benefits buildbot.
So, our "instrumental quartet". For myself, I have identified four features buildbot:
- It is an open source framework under the GPL license.
- This is the use of python as a tool for configuring and describing the required actions.
- This is the ability to receive a response from the machine on which the assembly takes place
- Finally, these are the minimum requirements for a Host. Deployment requires python and twisted and does not require a VM and java machine.
2. Concept led by BuildMaster
Central to the task distribution architecture is BuildMaster. It is a service that:
- tracks changes in the project source tree
- sends commands to be executed by the Worker service to build the project and test it
- notifies users about the results of their actions
BuildMaster configured via file master.cfg. This file is in the root BuildMaster. Later I will show how this root is created. The file itself master.cfg contains python - a script that uses calls buildbot.
The next most important object buildbot It has a name Worker. This service can be run on another host with a different OS, or maybe on where BuildMaster. It can also exist in a specially prepared virtual environment with its own packages and variables. These virtual environments can be prepared using python utilities like verticalenv, venv.
BuildMaster broadcast commands to everyone Worker-y, and he, in turn, performs them. That is, it turns out that the process of building and testing a project can go on Worker-e under Windows and on another Worker under linux.
Checkout project source code occurs on each Workereh.
3. Installation
So let's go. I will be using Ubuntu 18.04 as the host. On it I will place one BuildMaster-a and one Worker-a. But first you need to install python3.7:
sudo apt-get update
sudo apt-get install python3.7
For those who need python3.7.2 instead of 3.7.1, you can do the following:
sudo apt-get update
sudo apt-get software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get install python3.7
sudo ln -fs /usr/bin/python3.7 /usr/bin/python3
pip3 install --upgrade pip
The next step is to set tweeted и buildbot, as well as packages that allow you to use additional functionality buildbot-and.
/*Все что под sudo будет установленно для всех пользователей в директорию /usr/local/lib/python3.7/dist-packages*/
#На хосте который производит мониторинг Worker-ов
sudo pip install twisted #Библиотека twisted
sudo pip install buildbot #BuildMaster
#Дополнительный функционал
pip install pysqlite3 #Устанавливаем базу sqllite в учебных целях
pip install jinja2 #framework наподобие django, для web и для почтовых рассыллок
pip install autobahn #Web cокеты для связи BuildMaster->Worker
pip install sqlalchemy sqlalchemy-migrate #Для отображения схемы базы данных
#Для Web отображения BuildBot-a
pip install buildbot-www buildbot-grid-view buildbot-console-view buildbot-waterfall-view
pip install python-dateutil #Отображение дат в web
#На стороне хоста который непосредственно осуществляет сборку и тестирование
pip install buildbot-worker #Worker
#Дополнительный функционал
sudo pip install virtualenv #Виртуальная среда
4. First steps
Time to create BuildMaster. It will be in our folder. /home/habr/master.
mkdir master
buildbot create-master master # Собственно сдесь и создаемNext step. Let's create Worker. It will be in our folder. /home/habr/worker.
mkdir worker
buildbot-worker create-worker --umask=0o22 --keepalive=60 worker localhost:4000 yourWorkerName password
When you run Worker, then by default it will create in /home/habr/worker folder with the name of the project, which is specified in master.cfg. And in the folder with the name of the project, he will create a directory build, and then it will do checkout. working directory for Worker-a will become a directory /home/habr/yourProject/build.
"Golden Key
And now what I wrote the previous paragraph for: a script that Master will require Worker-a do remote in this directory will not be executed because the script does not have permission to run. To fix the situation, you need a key --umask=0o22, which prohibits writing to this directory, but leaves the launch rights. And that's all we need.
BuildMaster и Worker establish a connection with each other. It happens that it breaks and Worker waiting for a response from BuildMaster-A. If there is no response, the connection is restarted. Key --keepalive=60 just needed in order to indicate the time after which connect reboots.
5. Configuration. Step by step recipe
Configuration BuildMaster is carried out on the side of the machine where we executed the command create-master. In our case, this is the directory /home/habr/master. Configuration file master.cfg does not yet exist, but the command itself has already created the file master.cmg.sample. You need to rename it to master.cfg.sample в master.cfg
mv master.cfg.sample master.cfgLet's open this master.cfg. And let's see what it consists of. And after that we will try to make our configuration file.
master.cfg
c['change_source'] = []
c['change_source'].append(changes.GitPoller(
'git://github.com/buildbot/hello-world.git',
workdir='gitpoller-workdir', branch='master',
pollInterval=300))
c['schedulers'] = []
c['schedulers'].append(schedulers.SingleBranchScheduler(
name="all",
change_filter=util.ChangeFilter(branch='master'),
treeStableTimer=None,
builderNames=["runtests"]))
c['schedulers'].append(schedulers.ForceScheduler(
name="force",
builderNames=["runtests"]))
factory = util.BuildFactory()
factory.addStep(steps.Git(repourl='git://github.com/buildbot/hello-world.git', mode='incremental'))
factory.addStep(steps.ShellCommand(command=["trial", "hello"],
env={"PYTHONPATH": "."}))
c['builders'] = []
c['builders'].append(
util.BuilderConfig(name="runtests",
workernames=["example-worker"],
factory=factory))
c['services'] = []
c['title'] = "Hello World CI"
c['titleURL'] = "https://buildbot.github.io/hello-world/"
c['buildbotURL'] = "http://localhost:8010/"
c['www'] = dict(port=8010,
plugins=dict(waterfall_view={}, console_view={}, grid_view={}))
c['db'] = {
'db_url' : "sqlite:///state.sqlite",
}
5.1 BuildmasterConfig
c = BuildmasterConfig = {} BuildmasterConfig — base dictionary of the configuration file. It must be included in the configuration file. For ease of use, its alias is introduced in the configuration code "vs". Titles в c["keyFromDist"] are fixed elements to interact with BuildMaster. Under each key, the corresponding object is substituted as a value.
5.2 workers
c['workers'] = [worker.Worker("example-worker", "pass")]This time we point BuildMaster-y list of Worker-ov. Myself Worker we created , indicating you-worker-name и Password. Now they must be specified instead example worker и pass .
5.3 change_source
c['change_source'] = []
c['change_source'].append(changes.GitPoller(
'git://github.com/buildbot/hello-world.git',
workdir='gitpoller-workdir', branch='master',
pollInterval=300))
By key change_source dictionary c we get access to the list where you want to put the object that queries the repository with the source code of the project. The example uses a Git repository that is polled at regular intervals.
The first argument is the path to your repository.
workdir represents the path to the folder where on the side Worker-a relative to the path /home/habr/worker/yourProject/build will git store the local version of the repository.
branch contains a specific branch in the repository to follow.
pollInterval contains the number of seconds after which BuildMaster will poll the repository for changes.
There are several methods to track changes in the project repository.
The simplest method is Polls, which implies that BuildMaster periodically polls the server with the repository. If c reflected the changes in the repository, then BuildMaster with some delay will create the inner object Change and send it to the event handler Scheduler, which will launch the steps for building and testing the project on Worker-e. These steps will include Update repository. Exactly on Worker-e will create a local copy of the repository. The details of this process will be covered below in the next two sections. ( и ).
An even more elegant method of tracking changes to a repository is to directly send messages from the server that hosts it to BuildMaster-y about changing the source codes of the project. In this case, as soon as the developer makes c, the server with the project repository will send a message BuildMaster-y. And that, in turn, will intercept it by creating an object PBChangeSource. This object will then be passed to Scheduler, which activates the steps for building the project and testing it. An important part of this method is this work with hook-server scripts in the repository. In script hook-a, responsible for processing actions when c-e, you need to call the utility sendchange and specify the network address BuildMaster-A. You need to specify the network port that will listen PBChangeSource. PBChangeSource, by the way, is part of BuildMaster-A. This method will require the right admin-a on the server where the project repository is located. First you need to make a backup of the repository.
5.4 schedulers
c['schedulers'] = []
c['schedulers'].append(schedulers.SingleBranchScheduler(
name="all",
change_filter=util.ChangeFilter(branch='master'),
treeStableTimer=None,
builderNames=["runtests"]))
c['schedulers'].append(schedulers.ForceScheduler(
name="force",
builderNames=["runtests"]))
schedulers - this is an element that acts as a trigger that starts the entire chain of assembly and testing of the project.
The changes that have been committed change_source, transformed in the course of work buildbot-a to object Change and now every Sheduler based on them, it builds requests to start the project build process. However, it also determines when these requests are passed on to the queue. An object Builder keeps a queue of requests and tracks the state of the current assembly on a separate Worker-e. Builder also exists on BuildMaster-e and on Worker-e. He sends from BuildMaster-and on Worker-a already specific build - a series of steps to be followed.
We see that in the current example of such schedulers 2 are created. Moreover, each has its own type.
SingleBranchScheduler is one of the most popular scheduling classes. It watches one branch and triggers on a committed change in it. When he sees the changes, he can postpone sending the build request (postpone for the period specified in the special parameter treeStableTimer) AT name specifies the name of the schedule that will be displayed in buildbot-web interface. IN ChangeFilter a filter is set, passing through which changes in the branch induce the schedule to send a build request. IN builderNames the name is indicated builder-a, which we will set a little later. The name in our case will be the same as the project name: yourProject.
ForceScheduler a very simple thing. This type of schedule is triggered by mouse click through buildbot-web interface. The parameters are the same as in SingleBranchScheduler.
PS #3. Suddenly come in handy
periodic - this is a schedule that works with a certain fixed time interval. The call looks like this
from buildbot.plugins import schedulers
nightly = schedulers.Periodic(name="daily",
builderNames=["full-solaris"],
periodicBuildTimer=24*60*60)
c['schedulers'] = [nightly]
5.5 BuildFactory
factory = util.BuildFactory()
factory.addStep(steps.Git(repourl='git://github.com/buildbot/hello-world.git', mode='incremental'))
factory.addStep(steps.ShellCommand(command=["trial", "hello"],
env={"PYTHONPATH": "."}))
periodicBuildTimer specifies the time of this periodicity in seconds.
BuildFactory creates a specific build, which then builder refers to Worker. In BuildFactory indicates the steps to be taken Worker-y. Steps are added by calling a method addStep
The first added step in this example is git clean -d -f -f –xThen git checkout. These actions are included in the parameter method, which is not explicitly specified but implies a default value fresh... Parameter mode='incremental' says that the files are from the directory where the chechout, while those missing in the repository remain untouched.
The second added step is a script call trial with parameter Hello on the side Worker-a from directory /home/habr/worker/yourProject/build with environment variable PATHONPATH=… Thus, you can write your own scripts and execute them on the side Worker-a through step util.ShellCommand. These scripts can be put directly into the repository. Then at chechout-e they will fall into /home/habr/worker/yourProject/build. However, then there are two "buts":
- Worker must be created with a key so that it does not block execution rights after checkout-and.
- RџSЂRё git push-e of these scripts must specify a property exacutableso that later chechout-e did not lose permission to execute the Git script.
5.6 builders
c['builders'] = []
c['builders'].append(util.BuilderConfig(name="runtests",
workernames=["example-worker"],
factory=factory))
About what is Builder was told . Now I will tell you in more detail about how to create it. BuilderConfig is a constructor builder. Such constructors in c['builders'] more than one can be specified as this is a list of objects builder type. Now let's rewrite the example from buildbotbringing it closer to our problem.
c['builders'] = []
c['builders'].append(util.BuilderConfig(name="yourProject",
workernames=["yourWorkerName"],
factory=factory))
Now let's talk about the settings. BuilderConfig.
name sets the name builder-a. Here we named it yourProject... This means that on Worker-e this path will be created /home/habr/worker/yourProject/build. Sheduler looks for builder just by that name.
workernames contains a sheet Worker-ov. Each of which must be added to c['workers'].
factory - specific buildwith which is associated builder. It will send an object build on Worker to complete all the steps included in this build-and.
6. Example of own configuration
Here is the sample project architecture that I propose to implement via buildbot
.
As a version control system, we will use svn. The repository itself will be located in some cloud. Here is the address of this cloud . In the cloud under svn there is an account username: user, passwd: Password. Scripts that are steps build-a will also be in the branch svn, in a separate folder buildbot/worker_linux. These scripts are in the repository with the saved property executable.
BuildMaster и Worker run on the same host project.host .BuildMaster stores its files in a folder /home/habr/master. Worker same stores in the following path /home/habr/worker. Process communication BuildMaster-a i Worker-a is conducted through port 4000 according to the protocol buildbot-a, that is 'pb' protocol.
The target project is entirely written in python. The task is to track its changes, create an executable file, generate documentation, and conduct testing. In case of failure, all developers need to send a message to the mail that there is an unsuccessful action.
web display buildbot we will connect to port 80 for project.host. Apatch is not required. As part of the library Twisted there is already a web server, buildbot uses it.
To store internal information for buildbot we will use sqlite.
Mailing list needs a host smtp.your.domain - it is allowed to send letters from the mail [email protected] without authentication. Also on the host'smtp ' Protocol is being heard at post 1025.
There are two people involved in the process: admin и user. admin administers buildbot. user is the person making c-s.
Exacutable file is generated via pyinstaller. Documentation is generated via doxygen.
For this architecture, I wrote this master.cfg:
master.cfg
import os, re
from buildbot.plugins import steps, util, schedulers, worker, changes, reporters
c= BuildmasterConfig ={}
c['workers'] = [ worker.Worker('yourWorkerName', 'password') ]
c['protocols'] = {'pb': {'port': 4000}}
svn_poller = changes.SVNPoller(repourl="https://svn.host/svn/yourProject/trunk",
svnuser="user",
svnpasswd="password",
pollinterval=60,
split_file=util.svn.split_file_alwaystrunk
)
c['change_source'] = svn_poller
hourlyscheduler = schedulers.SingleBranchScheduler(
name="your-project-schedulers",
change_filter=util.ChangeFilter(branch=None),
builderNames=["yourProject"],
properties = {'owner': 'admin'}
)
c['schedulers'] = [hourlyscheduler]
checkout = steps.SVN(repourl='https://svn.host/svn/yourProject/trunk',
mode='full',
method='fresh',
username="user",
password="password",
haltOnFailure=True)
projectHost_build = util.BuildFactory()
cleanProject = steps.ShellCommand(name="Clean",
command=["buildbot/worker_linux/pyinstaller_project", "clean"]
)
buildProject = steps.ShellCommand(name="Build",
command=["buildbot/worker_linux/pyinstaller_project", "build"]
)
doxyProject = steps.ShellCommand(name="Update Docs",
command=["buildbot/worker_linux/gendoc", []]
)
testProject = steps.ShellCommand(name="Tests",
command=["python","tests/utest.py"],
env={'PYTHONPATH': '.'}
)
projectHost_build.addStep(checkout)
projectHost_build.addStep(cleanProject)
projectHost_build.addStep(buildProject)
projectHost_build.addStep(doxyProject)
projectHost_build.addStep(testProject)
c['builders'] = [
util.BuilderConfig(name="yourProject", workername='yourWorkerName', factory=projectHost_build)
]
template_html=u'''
<h4>Статус построенного релиза: {{ summary }}</h4>
<p>Используемый сервис для постраения: {{ workername }}</p>
<p>Проект: {{ projects }}</p>
<p>Для того что бы посмотреть интерфейс управления пройдите по ссылке: {{ buildbot_url }}</p>
<p>Для того что бы посмотреть результат сборки пройдите по ссылке: {{ build_url }}</p>
<p>Используя WinSCP можно подключиться к серверу c ip:xxx.xx.xxx.xx. Войдя под habr/password, забрать собранный executable файл с директории ~/worker/yourProject/build/dist.</p>
<p><b>Построение было произведено через Buildbot</b></p>
'''
sendMessageToAll = reporters.MailNotifier(fromaddr="[email protected]",
sendToInterestedUsers=True,
lookup="your.domain",
relayhost="smtp.your.domain",
smtpPort=1025,
mode="warnings",
extraRecipients=['[email protected]'],
messageFormatter=reporters.MessageFormatter(
template=template_html,
template_type='html',
wantProperties=True,
wantSteps=True)
)
c['services'] = [sendMessageToAll]
c['title'] = "The process of bulding"
c['titleURL'] = "http://project.host:80/"
c['buildbotURL'] = "http://project.host"
c['www'] = dict(port=80,
plugins=dict(waterfall_view={}, console_view={}, grid_view={}))
c['db'] = {
'db_url' : "sqlite:///state.sqlite"
}
First you need BuildMaster-a i Worker-a. Then paste this file master.cfg в /home/habr/master.
The next step is to start the service BuildMaster-A
sudo buildbot start /home/habr/master
Then start the service Worker-a
buildbot-worker start /home/habr/worker
Ready! Now buildbot will track changes and act on c-y in svnby following the build and test steps of a project with the above architecture.
Below I will describe some of the features of the above master.cfg.
6.1 On the way to your master.cfg
While writing my master.cfg a lot of errors will be made, so reading the log file will be required. It is stored as BuildMaster-ec absolute path /home/habr/master/twistd.log, and on the side Worker-a with absolute path /home/habr/worker/twistd.log. As you read the error and fix it, you will need to restart the service BuildMaster-a. Here's how it's done:
sudo buildbot stop /home/habr/master
sudo buildbot upgrade-master /home/habr/master
sudo buildbot start /home/habr/master
6.2 Working with svn
svn_poller = changes.SVNPoller(repourl="https://svn.host/svn/yourProject/trunk",
svnuser="user",
svnpasswd="password",
pollinterval=60,
split_file=util.svn.split_file_alwaystrunk
)
c['change_source'] = svn_poller
hourlyscheduler = schedulers.SingleBranchScheduler(
name="your-project-schedulers",
change_filter=util.ChangeFilter(branch=None),
builderNames=["yourProject"],
properties = {'owner': 'admin'}
)
c['schedulers'] = [hourlyscheduler]
checkout = steps.SVN(repourl='https://svn.host/svn/yourProject/trunk',
mode='full',
method='fresh',
username="user",
password="password",
haltOnFailure=True)
To start, let's take a look at svn_poller. It's still the same interface, regularly polling the repository once a minute. In this case svn_poller refers only to the branch trunk. Mysterious parameter split_file=util.svn.split_file_alwaystrunk sets the rules: how to split the folder structure svn on branches. He also offers them relative paths. In its turn split_file_alwaystrunk simplifies the process by saying that the repository only trunk.
В Schedulers is indicated ChangeFilterwho sees none and associates a branch with it trunk by a given association through split_file_alwaystrunk. Responding to changes in trunk, Launches builder with name yourProject.
properties needed here so that the admin receives mailings from the build and test results as the owner of the process.
Step build-a checkout able to do a complete removal of any files lying in the local version of the repository Worker-A. A then do complete svn update. Mode set via parameter mode=full, method=fresh... Parameter haltOnTailure says that if svn update will be executed with an error, then the entire build and test process should be suspended, since further actions do not make sense.
6.3 Letter to you: reporters are authorized to report
reporters is an email notification service.
template_html=u'''
<h4>Статус построенного релиза: {{ summary }}</h4>
<p>Используемый сервис для постраения: {{ workername }}</p>
<p>Проект: {{ projects }}</p>
<p>Для того что бы посмотреть интерфейс управления пройдите по ссылке: {{ buildbot_url }}</p>
<p>Для того что бы посмотреть результат сборки пройдите по ссылке: {{ build_url }}</p>
<p>Используя WinSCP можно подключиться к серверу c ip:xxx.xx.xxx.xx. Войдя под habr/password, забрать собранный executable файл с директории ~/worker/yourProject/build/dist.</p>
<p><b>Построение было произведено через Buildbot</b></p>
'''
sendMessageToAll = reporters.MailNotifier(fromaddr="[email protected]",
sendToInterestedUsers=True,
lookup="your.domain",
relayhost="smtp.your.domain",
smtpPort=1025,
mode="warnings",
extraRecipients=['[email protected]'],
messageFormatter=reporters.MessageFormatter(
template=template_html,
template_type='html',
wantProperties=True,
wantSteps=True)
)
c['services'] = [sendMessageToAll]
He can send messages .
MailNotifier uses mail to send notifications.
template_html sets the text template for the mailing. HTML is used to create the markup. It's engine modified (can be compared to d). buildbot has a set of variables whose values are substituted into the template during the formation of the message text. These variables are enclosed in {{ double curly braces }}. For example, summary displays the status of completed operations, i.e. success or failure. A projects will bring out yourProject. So, with the help of control commands in jinja2, variables buildbot-a and python string formatters, you can create a quite informative message.
MailNotifier contains the following arguments.
fromaddr - the address from which the mailing will be sent to everyone.
sendToInterestedUsers=True sends a message to the owner and user who made the c.
lookup — a suffix to be added to the usernames receiving the mailing list. So admin how the user will receive the mailing at the address [email protected].
relayhost specifies the name of the host where the server is open smtp, smptPort sets the port number that is listening smtp server.
mode="warning" says that the mailing should be done only if there is at least one step build-a that ended with a failure or warning status. In case of success, mailing is not required.
extra Recipients contains a list of persons to whom the mailing should be made, in addition to the owner and the person who carried out the c.
messageFormatter is an object specifying the message format, its template, and a set of variables available from jinja2. Options such as wantProperties=True и wantSteps=True define this set of available variables.
with['services']=[sendMessageToAll] provides a list of services, among which will be our reporter.
We did it! Congratulations
We created our own configuration and saw the functionality that buildbot. This, I think, is enough to understand whether this tool is needed to create your project. Is he interested in you? Will it be useful to you? Is he comfortable to work with? Then I write this article not in vain.
And further. I would like the professional community using buildbot, became wider, the manuals were translated, and there were even more examples.
Thank you all for your attention. Good luck.
Source: habr.com