GitHub Actions as CI/CD for Static Generator Site and GitHub Pages

Having gone through a bit, Habr was surprised that very few articles were published on the topic of GitHub's (beta-) features - Actions.

It would seem that such understatement can be explained by the fact that the functionality is still in testing, albeit “beta”. But it is a useful feature of the beta that allows you to use this tool in private repositories. It is about working with this technology that I will tell in this article.

Prehistory

If you start in order, it is probably worth mentioning the moment that in the process of finding a quick, convenient, easy and free option for hosting a personal site “About me”, I had to spend several nights and go through many articles.

Someone chooses hosting, someone chooses a cloud server, and those who don’t want to understand the work, interaction and payment for all this like uploading static sites to the repository, since now this can be done both on GitHub and GitLab .

Of course, this is everyone's personal choice.

My final choice was GitHub Pages.

About Pages

Who doesn’t know gh-pages - this is such an option for storing documentation in the form of a site and it is provided free of charge, and in addition to documentation, it is also proposed to store personal sites. This functionality is provided by GitHub to all users and is available in the repository settings.

A branch is used for the project repository gh-pages, for a custom site - a separate repository with the name username.github.io with site sources master branch.

More details can be seen in documentation, but I will only note that GitHub, with amazing generosity, allows everyone to link their own domain to such a site, simply by adding a file CNAME with the domain name and setting the DNS of your domain provider to the GitHub servers.

I am sure that there are many articles on how to deploy such a site here, so this is not about that further.

Occurrence of a problem

The problem was that when using a static generator, there is a need to write additional scripts and use libraries to simplify the process of generating pages and loading them into the repository. Simply, if you store the sources in a separate private repository, then every time you make any change on the site, it was necessary to deploy the local environment for the subsequent generation of static pages and publication in the main repository of the site.

There is an abundance static generators and they all have the same problem. These actions take too much time and effort, and as a result, stop working on the site, especially after several migrations from OS to OS or incidents with data loss on hard drives (this was the case in my case).

Just recently, either in a pop-up notification on the site, or in the newsletter from GitHub, a newly built-in CI / CD was noticed, which allowed these actions to be carried out with minimal effort.

About static page generators

I will not focus on this subparagraph, but I will share a couple of theses that I came to during the selection and use of these:

1) choose a generator for your programming language, or one that was as clear as possible. I came up with this idea at a time when I myself had to add some functionality for the site to work, put down crutches for its greater stability and automation. In addition, this is a good reason to write additional functionality in the form of plugins yourself;

2) on which particular generator to stop is a personal choice, but it should be borne in mind that for an initial immersion in the work of the GitHub Pages functionality, you must first set yourself Jekyll. Fortunately, it allows you to generate a site from source directly in the repository (I will repeat this with my choice).

My choice of generator is based on the first point. Pelican which is written in Python easily replaced Jekyll, which is foreign to me (used for almost a year). As a result, even the creation and editing of articles, the robot on the site gives additional experience in a language that is interesting to me.

Formulation of the problem

The main task will be to write such a script (actually a configuration file) that would automatically generate static pages from a private repository. The solution will involve the functionality of the virtual environment. The script itself will add ready-made pages to the public repository.

Solution tools

Tools that we will use to solve the problem:

GitHub Actions;
Python3.7;
Pelican;
git
GitHub Pages.

The solution

In total, having got acquainted a little with the documentation and having figured out how scripts for Actions are written, it became clear that this mechanism will completely solve the problem that has arisen. At the time of writing, to use this functionality, you must subscribe for beta testing!

GitHub Actions as CI/CD for Static Generator Site and GitHub Pages
Description of the new functionality by Github itself

The writing of the Actions script begins with the creation of a named file in the folder .github and its subfolder workflows. You can do this both manually and from the editor in the Actions tab on the repository page.

GitHub Actions as CI/CD for Static Generator Site and GitHub Pages
An example of an empty script form

Briefly comment on the form

name: CI    # название скрипта: будет отображаться во вкладке Actions

on: [push]  # действие, по которому запускается данный скрипт

jobs:       # роботы, которые будут выполняться
  build:    # сборка, которая..

    runs-on: ubuntu-latest      # ..будет запущена на основе этого образа

    steps:              # шаги которые будут проделаны после запуска образа
    - uses: actions/checkout@v1     # переход в самую актуальную ветку
    - name: Run a one-line script   # имя работы номер 1
      run: echo Hello, world!       # суть работы номер 1 (bash-команда записана в одну строку)
    - name: Run a multi-line script   # имя работы номер 2
      run: |                    # суть работы номер 2 (многострочная)
        echo Add other actions to build,
        echo test, and deploy your project.

Let's write our own based on the template:

0) You can leave the name and "CI". It's a matter of taste here.

1) Next, you need to select the action / trigger that will lead to the launch of the script, in our case this is the usual push of a new commit to the repository.

on:
  push

2) The image on the basis of which the script will be launched will also be left from the example, since Ubuntu It's quite satisfactory in terms of the required functionality. Looking at available tools it becomes clear that it can be any necessary or just a convenient image (or a docker container based on it).

  build:
    runs-on: ubuntu-latest

3) In the steps, we will first set up the environment to prepare for the main work.

3.1) go to the branch we need (standard step checkout):

- uses: actions/checkout@v1

3.2) install Python:

    - name: Set up Python
      uses: actions/setup-python@v1
      with:
        python-version: 3.7

3.3) install the dependencies of our generator:

    - name: Install dependencies
      run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

3.4) create a directory in which the site pages will be generated:

   - name: Make output folder
      run: mkdir output

4) In order for the work on the site to be consistent, namely, not to delete previous changes and to be able to add changes to the site repository without conflicts, the next step is to clone the site repository each time:

   - name: Clone master branch
      run: git clone "https://${{ secrets.ACCESS_TOKEN }}@github.com/${GITHUB_ACTOR}/${GITHUB_ACTOR}.github.io.git" --branch master --single-branch ./output

This step calls the system variables:

variable GITHUB_ACTOR GitHub installs itself, and this is the username that caused this script to run;
variable secrets.ACCESS_TOKEN it's generated Github management token, we can pass it as an environment variable by setting it in the tab Secrets our repository settings. Please note that during the generation the token will be provided to us once, there will be no more access to it. As well as the values of the Secrets items.

5) Let's move on to generating our pages:

   - name: Generate static pages
      run: pelican content -o output -s publishconf.py

The parameters passed to the generator are responsible for the directory where the generated files will be sent (-o output) and the configuration file that we use to generate (-s publishconf.py; you can read about the approach to separating the local config and the config for publication in the Pelican documentation).

Let me remind you that we have in the folder output site repository already cloned.

6) Set up git and index our changed files:

    - name: Set git config and add changes
      run: |
          git config --global user.email "${GITHUB_ACTOR}@https://users.noreply.github.com/"
          git config --global user.name "${GITHUB_ACTOR}"
          git add --all
      working-directory: ./output

In this paragraph, an already known variable is used, and the working directory is indicated in which the commands from this step will be launched. The command to change to the working directory would otherwise look like − cd output.

7) Generate a commit message, commit the changes and push them to the repository. So that the commit is not wasted, and accordingly does not give an error in bash (the output result is not 0) — first we will check whether it is necessary to commit and push something at all. For this we use the command git diff-index --quiet --cached HEAD -- which at the exit to the terminal will give 0 if there are no changes from the previous version of the site, and 1 there are such changes. Then we process the result of this command. Thus, in the information about the execution of the script, we will write down useful information about the state of the site at this stage, instead of automatically crashing and sending us a report about the crash of the script.

We also carry out these actions in our directory with ready-made pages.

   - name: Push and send notification
      run: |
          COMMIT_MESSAGE="Update pages on $(date +'%Y-%m-%d %H:%M:%S')"
          git diff-index --quiet --cached HEAD -- && echo "No changes!" && exit 0 || echo $COMMIT_MESSAGE
          # Only if repo have changes
          git commit -m "${COMMIT_MESSAGE}"
          git push https://${{ secrets.ACCESS_TOKEN }}@github.com/${GITHUB_ACTOR}/${GITHUB_ACTOR}.github.io.git master
      working-directory: ./output

Experience the Power of Effective Results

As a result, such a script allows you not to think about creating static pages. By adding changes directly to a private repository, whether it's working with git from any system or creating a file through GitHub's web interface, Actions does it all. In case of an unexpected fall of the script, a notification will be sent to the mail.

Full code

I'll leave my working version, in it, in the last step, sending a notification that a commit has been launched to the main repository is added.

The Secrets described above are used where the bot token and the user ID to which the message needs to be sent are added.

name: Push content to the user's GitHub pages repository

on:
  push

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v1
    - name: Set up Python
      uses: actions/setup-python@v1
      with:
        python-version: 3.7
    - name: Install dependencies
      run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
    - name: Make output folder
      run: mkdir output
    - name: Clone master branch
      run: git clone "https://${{ secrets.ACCESS_TOKEN }}@github.com/${GITHUB_ACTOR}/${GITHUB_ACTOR}.github.io.git" --branch master --single-branch ./output
    - name: Generate static pages
      run: pelican content -o output -s publishconf.py
    - name: Set git config and add changes
      run: |
          git config --global user.email "${GITHUB_ACTOR}@https://users.noreply.github.com/"
          git config --global user.name "${GITHUB_ACTOR}"
          git add --all
      working-directory: ./output
    - name: Push and send notification
      run: |
          COMMIT_MESSAGE="Update pages on $(date +'%Y-%m-%d %H:%M:%S')"
          git diff-index --quiet --cached HEAD -- && echo "No changes!" && exit 0 || echo $COMMIT_MESSAGE
          git commit -m "${COMMIT_MESSAGE}"
          git push https://${{ secrets.ACCESS_TOKEN }}@github.com/${GITHUB_ACTOR}/${GITHUB_ACTOR}.github.io.git master
          curl "https://api.telegram.org/bot${{ secrets.BOT_TOKEN }}/sendMessage?text=$COMMIT_MESSAGE %0ALook at ${GITHUB_ACTOR}.github.io %0ARepository%3A github.com/${GITHUB_ACTOR}/${GITHUB_ACTOR}.github.io&chat_id=${{ secrets.ADMIN_ID }}"
      working-directory: ./output

Screenshots

GitHub Actions as CI/CD for Static Generator Site and GitHub Pages
The result of one of the launches displayed in the Actions tab of the source repository