How to connect GitLab and Pantheon and optimize workflows Drupal и WordPress

Our guest, developer tool creator Pantheon, talks about how to automate deployments. WordPress using GitLab CI/CD.

В Pantheon I work in developer relations, so I'm always looking for new ways to help developers. WordPress и Drupal Solve automation problems in workflows. To do this, I love experimenting with new tools and combining them to achieve greater efficiency.

I often see developers struggle with a single staging server.

So-so fun - waiting for your turn to use an intermediate server or sending clients a URL marked: “Look here, but don’t look here yet.”

multidev environments - one of the cool Pantheon tools - solve this problem, because with them you can create environments for Git branches on demand. Each multidev environment has its own URL and database, so developers can work, check quality, and get approval without stepping on each other's heels.

But Pantheon doesn't have tools for version control or continuous integration and deployment (CI/CD). But it is a flexible platform with which you can integrate any tools.

I also noticed that teams use some tools for development, and others for build and deployment.

For example, they have different tools for version control and CI/CD. You have to fiddle around and switch between tools to edit code and diagnose problems.

For GitLab there is a complete set of development tools: version control, tickets, merge requests, best-in-class CI / CD pipeline, container registry and stuff like that. I have yet to come across an application that has so much to manage the development workflow.

I love automation, so I learned how to connect Pantheon to GitLab so that commits to the master branch on GitLab are deployed to the master development environment in Pantheon. GitLab merge requests can also create and deploy code to multidev environments in Pantheon.

In this guide, I'll show you how to set up a connection between GitLab and Pantheon and optimize your workflow. WordPress и Drupal.

Of course it is possible, mirror a GitLab repository, but we will do everything with pens to delve into GitLab CI and in the future use this tool not only for deployment.

Introduction

For this post, you need to understand that Pantheon breaks down each site into three elements: code, database, and files.

The code includes CMS files such as the core, plugins, and themes. WordPressThese files are managed in Git repositorieshosted by Pantheon, meaning we can deploy code from GitLab to Pantheon with Git.
Files in Pantheon are called media files, that is, pictures for the site. They are usually uploaded by users and Git ignores them.

Create a free account, learn more about Pantheon Workflow or sign up for a demo at pantheon.io.

Assumptions

My Pantheon and GitLab project is called pantheon-gitlab-blog-demoThe project name must be unique. We'll be working with a website here. WordPressYou can take it too Drupal, but some things will need to be changed.

I will use Git command lineand you can work in GUI, if you want to.

We create a project

To start, we create GitLab project (we will return to this later).

Now we are creating a website WordPress on PantheonThen we install it. WordPress for the website dashboard.

If your hands are itching to change something, for example, remove and add plugins, be patient. The site is not yet connected to GitLab, and we want all code changes to go through GitLab.

When will we install it? WordPress, return to the Pantheon website dashboard and change the development mode to Git.

Initial commit on GitLab

Now you need to transfer the initial code WordPress From the Pantheon website to GitLab. To do this, we clone the code from the Pantheon website's Git repository locally and then push it to the GitLab repository.

To make it easier and safer, add SSH key to Pantheon and we will not enter the password every time we clone the Pantheon Git repository. At the same time already add SSH key to GitLab.

To do this, we clone the Pantheon site locally by copying the command from the Clone with Git field on the site dashboard.

If you need help, read the documentation getting started with Git for Pantheon.

Now let's change git remote originto point to GitLab instead of Pantheon. It can be done командой git remote.

Let's go to the GitLab project and copy the repository URL from the Clone dropdown on the project details page. Let's select the Clone with SSH option, because we have already configured the SSH key.

By default git remote for a local copy of the code repository − origin. This can be changed from git remote set-url origin [URL репозитория GitLab], where instead of brackets we enter the actual URL.

Finally, we launch git push origin master --forceto send the code WordPress from the Pantheon site on GitLab.

The --force option is only needed once. Then in teams git push GitLab won't have it.

Set up credentials and variables

Remember how we added an SSH key locally to log in to Pantheon and GitLab? The SSH token can be used to authorize GitLab and Pantheon.

GitLab has great documentation. Let's see See the section on SSH keys when using the Docker executor in the document on using SSH keys with GitLab CI/CD.

We will now complete the first two steps: create a new SSH key pair locally with ssh-keygen and add the private key as a variable to the project.

Then we will set SSH_PRIVATE_KEY How GitLab CI/CD environment variable in the project settings.
In the third and fourth steps, we will create a file .gitlab-ci.yml with content like this:

before_script:
  # See https://docs.gitlab.com/ee/ci/ssh_keys/README.html
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add - > /dev/null
  - mkdir -p $HOME/.ssh && echo "StrictHostKeyChecking no" >> "$HOME/.ssh/config"
  - git config --global user.email "$GITLAB_USER_EMAIL"
  - git config --global user.name "Gitlab CI"

Until we commit the file .gitlab-ci.yml, then you will need to add something else to it.

Now do the fifth step and add the public key you created in the first step to the services you need access to in the build environment.

In our case, we want to access Pantheon from GitLab. Follow the instructions in the Pantheon document on adding an SSH key to Pantheon and do this step.

Remember: closed SSH is in GitLab, open is in Pantheon.

Let's set up a few more environment variables. The first one is called PANTHEON_SITE. Its value is the name of the Pantheon site on your machine.

The name on the machine is listed at the end of the Clone with Git command. You have already cloned the site locally, so this will be the directory name of the local repository.

Next, set the environment variable PANTHEON_GIT_URL. This is the URL of the Git repository for the Pantheon site that we have already used.

We enter only the URL of the SSH repository, without git clone and the name of the site on the machine at the end.

Phew. That's done, now we can finish our file .gitlab-ci.yml.

Create a deployment task

What we'll be doing with GitLab CI at first is very similar to what we've done with Git repositories in the past. But this time, let's add the Pantheon repository as a second Git remote source, and then push the code from GitLab to Pantheon.

To do this, set stage deploy и task deploy:dev, because we will deploy to the development environment on Pantheon. As a result, the file .gitlab-ci.yml will look like this:

stages:
- deploy

before_script:
  # See https://docs.gitlab.com/ee/ci/ssh_keys/README.html
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add - > /dev/null
  - mkdir -p $HOME/.ssh && echo "StrictHostKeyChecking no" >> "$HOME/.ssh/config"
  - git config --global user.email "$GITLAB_USER_EMAIL"
  - git config --global user.name "Gitlab CI"

deploy:dev:
  stage: deploy
  environment:
    name: dev
    url: https://dev-$PANTHEON_SITE.pantheonsite.io/
  script:
    - git remote add pantheon $PANTHEON_GIT_URL
    - git push pantheon master --force
  only:
    - master

Variables SSH_PRIVATE_KEY, PANTHEON_SITE и PANTHEON_GIT_URL should look familiar - we set up these environment variables before. With these variables we will be able to use the values ​​in the file .gitlab-ci.yml many times, and you only need to update them in one place.

Finally, add, commit and push the file .gitlab-ci.yml on Gitlab.

Checking the deployment

If we did everything right, the task deploy:dev will run successfully in GitLab CI/CD and push the commit .gitlab-ci.yml at the Pantheon. Let's get a look.

Submitting merge request branches to Pantheon

Here we will use my favorite Pantheon feature − multidevwhere you can create additional Pantheon environments for Git branches on request.

Access to multidev is limited, so this section can be omitted. But if you have access, you can seriously improve performance by setting up automatic creation of multidev environments on Pantheon from GitLab merge requests.

First, let's make a new Git branch locally with git checkout -b multidev-support. Now let's change something in .gitlab-ci.yml.

I like to include the merge request number in the Pantheon environment name. For example, the first merge request is mr-1, the second - mr-2 etc.

The merge request is changing so we need to dynamically determine the names of the Pantheon branches. On GitLab, it's easy - you need to use predefined environment variables.

We can take $CI_MERGE_REQUEST_IIDto specify the merge request number. Let's apply all of this along with the global environment variables we specified earlier and add a new deploy:multidev task at the end of the file .gitlab-ci.yml.

deploy:multidev:
  stage: deploy
  environment:
    name: multidev/mr-$CI_MERGE_REQUEST_IID
    url: https://mr-$CI_MERGE_REQUEST_IID-$PANTHEON_SITE.pantheonsite.io/
  script:
    # Checkout the merge request source branch
    - git checkout $CI_COMMIT_REF_NAME
    # Add the Pantheon git repository as an additional remote
    - git remote add pantheon $PANTHEON_GIT_URL
    # Push the merge request source branch to Pantheon
    - git push pantheon $CI_COMMIT_REF_NAME:mr-$CI_MERGE_REQUEST_IID --force
  only:
    - merge_requests

It will be similar to our task deploy:dev, only the branch goes to Pantheon, not to master.

We have added and committed the updated file .gitlab-ci.yml, and now push a new branch to GitLab with git push -u origin multidev-support.

Now let's create a new merge request from the branch multidev-supportby clicking Create merge request.

After creating a merge request, we look at how the CI / CD task is performed deploy:multidev.

Look - a new branch has been sent to Pantheon. But if we go to the multidev section on the Pantheon site dashboard, we won’t see the new environment there

Let's take a look at the Git Branches section.

As a result, our branch mr-1 made it to the Pantheon. Create an environment from a branch mr-1.

We have created a multidev environment, now let's go back to GitLab and take a look at the section Operations > Environments. We will see entries for dev и mr-1.

This is because we have added an entry environment With name name и url into CI/CD tasks. If we click on the open environment icon, we will navigate to the URL of the multidev environment on Pantheon.

Automate the creation of multidev

In principle, you can stop here and just remember to create a multidev environment for each merge request, but this process can be automated.

Pantheon has a command line tool Terminuswhere you can work with the platform automatically. Terminus allows you to create multidev environments from the command line - perfect for GitLab CI.

We need a new merge request to test this. Create a new branch with git checkout -b auto-multidev-creation.

To use Terminus in GitLab CI/CD tasks, you need a machine token to authenticate with Terminus and a Terminus container image.

Creating a Pantheon machine token, save it in a safe place and add it as a global environment variable in GitLab with the name PANTHEON_MACHINE_TOKEN.

If you forgot how to add GitLab environment variables, go back to where we defined PANTHEON_SITE.

Create a Dockerfile with Terminus

If you don't use Docker or don't like files Dockerfiletake my image registry.gitlab.com/ataylorme/pantheon-gitlab-blog-demo:latest and skip this section.

GitLab has a container registry, where we can build and host a Dockerfile for our project. Let's create a Dockerfile with Terminus to work with Pantheon.

Terminus is a PHP command line tool, so let's start with the PHP image. I install Terminus via Composer, so I'll use official Docker Composer image. We create Dockerfile in the local repository directory with the following content:

# Use the official Composer image as a parent image
FROM composer:1.8

# Update/upgrade apk
RUN apk update
RUN apk upgrade

# Make the Terminus directory
RUN mkdir -p /usr/local/share/terminus

# Install Terminus 2.x with Composer
RUN /usr/bin/env COMPOSER_BIN_DIR=/usr/local/bin composer -n --working-dir=/usr/local/share/terminus require pantheon-systems/terminus:"^2"

Follow the instructions for building and sending images from the section Build and push images в container registry documentationto assemble an image from Dockerfile and submit it to GitLab.

Opening the section registry in the GitLab project. If everything went according to plan, our image will be there. Write down a link to the image tag - we need it for the file .gitlab-ci.yml.

Section script in the task deploy:multidev starts to grow, so let's move it to a separate file. Create a new file private/multidev-deploy.sh:

#!/bin/bash

# Store the mr- environment name
export PANTHEON_ENV=mr-$CI_MERGE_REQUEST_IID

# Authenticate with Terminus
terminus auth:login --machine-token=$PANTHEON_MACHINE_TOKEN

# Checkout the merge request source branch
git checkout $CI_COMMIT_REF_NAME

# Add the Pantheon Git repository as an additional remote
git remote add pantheon $PANTHEON_GIT_URL

# Push the merge request source branch to Pantheon
git push pantheon $CI_COMMIT_REF_NAME:$PANTHEON_ENV --force

# Create a function for determining if a multidev exists
TERMINUS_DOES_MULTIDEV_EXIST()
{
    # Stash a list of Pantheon multidev environments
    PANTHEON_MULTIDEV_LIST="$(terminus multidev:list ${PANTHEON_SITE} --format=list --field=id)"

    while read -r multiDev; do
        if [[ "${multiDev}" == "$1" ]]
        then
            return 0;
        fi
    done <<< "$PANTHEON_MULTIDEV_LIST"

    return 1;
}

# If the mutltidev doesn't exist
if ! TERMINUS_DOES_MULTIDEV_EXIST $PANTHEON_ENV
then
    # Create it with Terminus
    echo "No multidev for $PANTHEON_ENV found, creating one..."
    terminus multidev:create $PANTHEON_SITE.dev $PANTHEON_ENV
else
    echo "The multidev $PANTHEON_ENV already exists, skipping creating it..."
fi

The script is in a private directory and does not allow web access to Pantheon. We have a script for our multidev logic. Let's update the section now deploy:multidev File .gitlab-ci.ymlto make it look like this:

deploy:multidev:
  stage: deploy
  environment:
    name: multidev/mr-$CI_MERGE_REQUEST_IID
    url: https://mr-$CI_MERGE_REQUEST_IID-$PANTHEON_SITE.pantheonsite.io/
  script:
    # Run the multidev deploy script
    - "/bin/bash ./private/multidev-deploy.sh"
  only:
    - merge_requests

We need to make sure that our tasks are executed in the created custom image, so let's add a definition image with registry URL in .gitlab-ci.yml. As a result, we have such a file .gitlab-ci.yml:

image: registry.gitlab.com/ataylorme/pantheon-gitlab-blog-demo:latest

stages:
- deploy

before_script:
  # See https://docs.gitlab.com/ee/ci/ssh_keys/README.html
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add - > /dev/null
  - mkdir -p $HOME/.ssh && echo "StrictHostKeyChecking no" >> "$HOME/.ssh/config"
  - git config --global user.email "$GITLAB_USER_EMAIL"
  - git config --global user.name "Gitlab CI"

deploy:dev:
  stage: deploy
  environment:
    name: dev
    url: https://dev-$PANTHEON_SITE.pantheonsite.io/
  script:
    - git remote add pantheon $PANTHEON_GIT_URL
    - git push pantheon master --force
  only:
    - master

deploy:multidev:
  stage: deploy
  environment:
    name: multidev/mr-$CI_MERGE_REQUEST_IID
    url: https://mr-$CI_MERGE_REQUEST_IID-$PANTHEON_SITE.pantheonsite.io/
  script:
    # Run the multidev deploy script
    - "/bin/bash ./private/multidev-deploy.sh"
  only:
    - merge_requests

Add, commit and send private/multidev-deploy.sh и .gitlab-ci.yml. Now we return to GitLab and wait for the CI / CD task to complete. Be patient: multidev can take several minutes to create.

Then we go to look at the multidev list on Pantheon. O miracle! multidev environment mr-2 already here.

Conclusion

My team had a lot more fun when we started opening merge requests and creating environments automatically.

With the powerful tools of GitLab and Pantheon, you can connect GitLab to Pantheon automatically.

Since we are using GitLab CI/CD, our workflow will have room to grow. Here are a couple of ideas to get you started:

• Add a build step.
• Add automated testing.
• Add a task to ensure code standards are followed.
• Add dynamic application security testing.

Write what you think about GitLab, Pantheon and automation.

PS Did you know that Terminus, Pantheon's command line tool, can be extended via plugins?

We at Pantheon did a good job on version 2 of our plugin for Terminus build tools with GitLab support. If you don't want to mess around with per-project setup, try this plugin and help us test the v2 beta. For the Terminus team build:project:create you only need a Pantheon token and a GitLab token. It will deploy one of the sample projects with Composer and automated testing, create a new project in GitLab, a new Pantheon site, and connect them using environment variables and SSH keys.

About the Developer

Andrew Taylor creates tools for developers in Pantheon.

Source: habr.com