How to Connect GitLab and Pantheon and Optimize Workflows for Drupal and WordPress

Our guest, a developer tools creator from Pantheon, explains how to automate WordPress deployments using GitLab CI/CD.

In Pantheon I work in developer relations, so I'm always looking for new ways to help WordPress and Drupal developers solve automation issues in their workflows. I love experimenting with new tools and combining them for effective work.

I often see developers struggling with a single staging server.

It’s not much fun waiting for your turn to use the staging server or sending clients a URL with a note: 'Watch here, but don’t look here yet.'

Multidev environments — one of Pantheon's cool tools — solve this issue as they allow for environments to be created on demand for Git branches. Each multidev environment has its own URL and database, enabling developers to work, check quality, and get approvals without stepping on each other's toes.

However, Pantheon lacks version control tools or continuous integration and deployment (CI/CD). On the upside, it's a flexible platform that can integrate with any tools.

I've also noticed that teams use different tools for development and for building and deploying.

For example, they have different tools for version control and CI/CD. This means a lot of fiddling and switching between tools to edit code and troubleshoot issues.

At GitLab there's a comprehensive suite of development tools available: for version control, tickets, merge requests, a best-in-class CI/CD pipeline, a container registry, and so on. I haven't encountered applications that have as many tools for managing the development workflow.

I love automation, so I explored how to connect Pantheon to GitLab, allowing commits to the main branch in GitLab to deploy to the main development environment in Pantheon. Also, merge requests on GitLab can create and deploy code to multidev environments in Pantheon.

In this guide, I will explain how to set up the connection between GitLab and Pantheon and optimize the WordPress and Drupal workflow.

Of course, you can mirror the GitLab repository, but we’ll do everything manually to dig deeper into. GitLab CI and in the future, use this tool not only for deployment.

Introduction

For this post, it is important to understand that Pantheon divides each site into three components: code, database, and files.

The code includes CMS files, such as the core, plugins, and WordPress themes. These files are managed in a Git repository, hosted on Pantheon, which means we can deploy code from GitLab to Pantheon using Git.
The files in Pantheon refer to media files, such as images for the site. They are usually uploaded by users, and Git ignores them.

Create a free account, learn more about the Pantheon workflow or sign up for a demo at pantheon.io.

Assumptions

My project on Pantheon and GitLab is called pantheon-gitlab-blog-demo. The project name must be unique. Here we will work with a WordPress site. Drupal can also be used, but some changes will be necessary.

I will be using the Git command line, but you can work in the graphical interface, if you prefer.

Creating the project

First, we create a GitLab project (we'll come back to this).

Now we create a WordPress site on Pantheon. Then we install WordPress for the site's dashboard.

If you're eager to change something, such as removing and adding plugins, hold off. The site is not yet connected to GitLab, and we want all code changes to go through GitLab.

Once we install WordPress, we return to the Pantheon site dashboard and switch the development mode to Git.

Initial commit to GitLab

Now we need to transfer the initial WordPress code from the Pantheon site to GitLab. For that, we clone the code from the Pantheon site's Git repository locally, and then push it to the GitLab repository.

To make it easier and safer, we'll add the SSH key to Pantheon and won't have to enter the password every time we clone the Pantheon Git repository. At the same time, we’ll add the SSH key to GitLab as well..

To do this, we clone the Pantheon site locally, copying the command from the Clone with Git field on the site dashboard.

If you need help, read the documentation on getting started with Git for Pantheon..

Now we will change git remote origin, to point to GitLab instead of Pantheon. This can be done using the git remote command..

Let's go to the GitLab project and copy the repository URL from the Clone dropdown on the project details page. We will choose the Clone with SSH option, since we have already set up the SSH key.

the net/http git remote for the local copy of the code repository – origin. This can be changed with git remote set-url origin [GitLab repository URL], where instead of brackets, the actual URL is entered.

Finally, we run git push origin master --force, to push WordPress code from Pantheon to GitLab.

The --force parameter is needed only once. After that, it won't be in the commands git push on GitLab.

Setting up credentials and variables

Remember how we locally added an SSH key to authenticate with Pantheon and GitLab? An SSH token can be used to authenticate with GitLab and Pantheon.

GitLab has excellent documentation. Let's take a look at the SSH keys section when using the Docker executor in the SSH keys usage document with GitLab CI/CD.

Now we will perform the first two steps: we will create a new pair of SSH keys locally with ssh-keygen and add the private key as a variable in the project.

Then we will set SSH_PRIVATE_KEY as as a GitLab CI/CD environment variable in the project settings.
In steps three and four, we will create a file .gitlab-ci.yml with the following content:

before_script:
  # See https://docs.gitlab.com/ee/ci/ssh_keys/README.html
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add - > /dev/null
  - mkdir -p $HOME/.ssh && echo "StrictHostKeyChecking no" >> "$HOME/.ssh/config"
  - git config --global user.email "$GITLAB_USER_EMAIL"
  - git config --global user.name "Gitlab CI"

For now, let's not commit the file .gitlab-ci.yml, as it will need a few more additions.

Now we perform step five and add the public key we created in the first step to the services you need access to in the build environment.

In our case, we want to access Pantheon from GitLab. Follow the instructions in the Pantheon document on adding an SSH key to Pantheon and complete this step.

Remember: the private SSH key goes in GitLab, the public key goes in Pantheon.

Let's set up a few more environment variables. The first is called PANTHEON_SITE. Its value is the name of the Pantheon site on your machine.

The name on the machine is specified at the end of the Clone with Git command. Since you have already cloned the site locally, this will be the name of the local repository directory.

Next, let's set the environment variable PANTHEON_GIT_URL. This is the Git repository URL for the Pantheon site that we have already used.

We enter only the SSH repository URL, without git clone and the site name on the machine at the end.

Phew. We did that, now we can finish our file .gitlab-ci.yml.

Create a deployment task

What we will initially do with GitLab CI is very similar to what we did with Git repositories before. But this time we will add the Pantheon repository as a second remote Git source and then push the code from GitLab to Pantheon.

To do this, we will set up the stage deploy and the job deploy:dev, since we will be deploying to the development environment on Pantheon. As a result, the file .gitlab-ci.yml will look like this:

stages:
- deploy

before_script:
  # See https://docs.gitlab.com/ee/ci/ssh_keys/README.html
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add > /dev/null
  - mkdir -p $HOME/.ssh && echo "StrictHostKeyChecking no" >> "$HOME/.ssh/config"
  - git config --global user.email "$GITLAB_USER_EMAIL"
  - git config --global user.name "Gitlab CI"

deploy:dev:
  stage: deploy
  environment:
    name: dev
    url: https://dev-$PANTHEON_SITE.pantheonsite.io/
  script:
    - git remote add pantheon $PANTHEON_GIT_URL
    - git push pantheon master --force
  only:
    - master

Variables SSH_PRIVATE_KEY, PANTHEON_SITE and PANTHEON_GIT_URL should look familiar — we set up these environment variables earlier. With these variables, we can use the values in the file .gitlab-ci.yml many times, and updating them will only need to be done in one place.

Finally, we'll add, commit, and send the file .gitlab-ci.yml to GitLab.

Checking the deployment

If we did everything correctly, the job deploy:dev will successfully run in GitLab CI/CD and push the commit .gitlab-ci.yml to Pantheon. Let's take a look.

Sending merge request branches to Pantheon

Here we will use my favorite Pantheon feature — multidev, where you can create additional Pantheon environments for Git branches on demand.

Access to multidev is limited, so this section can be skipped. But if you have access, you can significantly boost productivity by setting up automatic multidev environments on Pantheon from GitLab merge requests.

First, we will create a new Git branch locally using git checkout -b multidev-support. Now let's modify something again in .gitlab-ci.yml.

I like to include the merge request number in the name of the Pantheon environment. For example, the first merge request is mr-1, the second is mr-2 and so on.

The merge request changes, so we need to dynamically determine the names of Pantheon branches. On GitLab, it's simple — we just need to use predefined environment variables.

We can take $CI_MERGE_REQUEST_IID, to specify the merge request number. Let's apply all of this along with the global environment variables we defined earlier and add a new task deploy:multidev at the end of the file .gitlab-ci.yml.

deploy:multidev:
  stage: deploy
  environment:
    name: multidev/mr-$CI_MERGE_REQUEST_IID
    url: https://mr-$CI_MERGE_REQUEST_IID-$PANTHEON_SITE.pantheonsite.io/
  script:
    # Checkout the merge request source branch
    - git checkout $CI_COMMIT_REF_NAME
    # Add the Pantheon git repository as an additional remote
    - git remote add pantheon $PANTHEON_GIT_URL
    # Push the merge request source branch to Pantheon
    - git push pantheon $CI_COMMIT_REF_NAME:mr-$CI_MERGE_REQUEST_IID --force
  only:
    - merge_requests

It will look like our task deploy:dev, only the branch is pushed to Pantheon, not to master.

We added and committed the updated file .gitlab-ci.yml, and now let's push the new branch to GitLab with git push -u origin multidev-support.

Now let's create a new merge request from the branch multidev-support, by clicking Create merge request.

After creating the merge request, we can see how the CI/CD task is progressing deploy:multidev.

Look — a new branch is sent to Pantheon. But if we go to the multidev section on the Pantheon site dashboard, we won't see the new environment there

Let's take a look at the Git Branches section.

As a result, our branch mr-1 has reached Pantheon. Let's create an environment from the branch mr-1.

We created a multidev environment, and now let's go back to GitLab and check the Operations > Environments. We will see entries for dev and mr-1.

This is because we added an entry environment named name and url in the CI/CD tasks. If we click the open environment icon, we will go to the multidev environment URL on Pantheon.

Let's automate the creation of multidev

In principle, we could stop here and just remember to create a multidev environment for each merge request, but this process can be automated.

Pantheon has a command-line tool Terminus, where you can work with the platform automatically. In Terminus, we can create multidev environments from the command line — ideal for GitLab CI.

We need a new merge request to test this. Let's create a new branch using git checkout -b auto-multidev-creation.

To use Terminus in GitLab CI/CD tasks, a machine token is needed for authentication in Terminus and a container image with Terminus.

Creating a Pantheon machine token, save it in a secure place and add it as a global environment variable in GitLab named PANTHEON_MACHINE_TOKEN.

If you forgot how to add GitLab environment variables, go back to where we defined PANTHEON_SITE.

Creating Dockerfile with Terminus

If you're not using Docker or don't like files Dockerfile, grab my image registry.gitlab.com/ataylorme/pantheon-gitlab-blog-demo:latest and skip this section.

GitLab has a container registry, where you can build and host Dockerfile for our project. Let's create a Dockerfile with Terminus to work with Pantheon.

Terminus is a command-line tool in PHP, so let's start with a PHP image. I install Terminus via Composer, so I'll base it on the official Docker Composer image. We create Dockerfile in the local repository directory with the following content:

# Use the official Composer image as a parent image
FROM composer:1.8

# Update/upgrade apk
RUN apk update
RUN apk upgrade

# Make the Terminus directory
RUN mkdir -p /usr/local/share/terminus

# Install Terminus 2.x with Composer
RUN /usr/bin/env COMPOSER_BIN_DIR=/usr/local/bin composer -n --working-dir=/usr/local/share/terminus require pantheon-systems/terminus:"^2"

Follow the instructions for building and pushing images from the Build and push images downward API support (simultaneously with this in container registry documentation, to build the image from Dockerfile and push it to GitLab.

Open the section Registry in the GitLab project. If everything went as planned, our image will be there. Write down the image tag link — we need it for the file .gitlab-ci.yml.

Partition script in the task deploy:multidev starts to grow, so let's move it to a separate file. We create a new file private/multidev-deploy.sh:

#!/bin/bash

# Store the mr- environment name
export PANTHEON_ENV=mr-$CI_MERGE_REQUEST_IID

# Authenticate with Terminus
terminus auth:login --machine-token=$PANTHEON_MACHINE_TOKEN

# Checkout the merge request source branch
git checkout $CI_COMMIT_REF_NAME

# Add the Pantheon Git repository as an additional remote
git remote add pantheon $PANTHEON_GIT_URL

# Push the merge request source branch to Pantheon
git push pantheon $CI_COMMIT_REF_NAME:$PANTHEON_ENV --force

# Create a function for determining if a multidev exists
TERMINUS_DOES_MULTIDEV_EXIST()
{
    # Stash a list of Pantheon multidev environments
    PANTHEON_MULTIDEV_LIST="$(terminus multidev:list ${PANTHEON_SITE} --format=list --field=id)"

    while read -r multiDev; do
        if [[ "${multiDev}" == "$1" ]]
        then
            return 0;
        fi
    done <<< "$PANTHEON_MULTIDEV_LIST"

    return 1;
}

# If the mutltidev doesn't exist
if ! TERMINUS_DOES_MULTIDEV_EXIST $PANTHEON_ENV
then
    # Create it with Terminus
    echo "No multidev for $PANTHEON_ENV found, creating one..."
    terminus multidev:create $PANTHEON_SITE.dev $PANTHEON_ENV
else
    echo "The multidev $PANTHEON_ENV already exists, skipping creating it..."
fi

The script is located in a private directory and does not allow web access on Pantheon. We have a script for our multidev logic. Now let's update the section deploy:multidev file .gitlab-ci.yml, so it looks like this:

deploy:multidev:
  stage: deploy
  environment:
    name: multidev/mr-$CI_MERGE_REQUEST_IID
    url: https://mr-$CI_MERGE_REQUEST_IID-$PANTHEON_SITE.pantheonsite.io/
  script:
    # Run the multidev deploy script
    - "/bin/bash ./private/multidev-deploy.sh"
  only:
    - merge_requests

We need to ensure that our tasks run in the created custom image, so let's add the definition image with the registry URL in .gitlab-ci.yml. As a result, we have the following file .gitlab-ci.yml:

image: registry.gitlab.com/ataylorme/pantheon-gitlab-blog-demo:latest

stages:
- deploy

before_script:
  # See https://docs.gitlab.com/ee/ci/ssh_keys/README.html
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d 'r' | ssh-add - > /dev/null
  - mkdir -p $HOME/.ssh && echo "StrictHostKeyChecking no" >> "$HOME/.ssh/config"
  - git config --global user.email "$GITLAB_USER_EMAIL"
  - git config --global user.name "Gitlab CI"

deploy:dev:
  stage: deploy
  environment:
    name: dev
    url: https://dev-$PANTHEON_SITE.pantheonsite.io/
  script:
    - git remote add pantheon $PANTHEON_GIT_URL
    - git push pantheon master --force
  only:
    - master

deploy:multidev:
  stage: deploy
  environment:
    name: multidev/mr-$CI_MERGE_REQUEST_IID
    url: https://mr-$CI_MERGE_REQUEST_IID-$PANTHEON_SITE.pantheonsite.io/
  script:
    # Run the multidev deploy script
    - "/bin/bash ./private/multidev-deploy.sh"
  only:
    - merge_requests

Adding, committing, and pushing private/multidev-deploy.sh and .gitlab-ci.yml. Now let's return to GitLab and wait for the CI/CD job to complete. Please hold on: the multidev may take a few minutes to create.

Then we go to check the multidev list on Pantheon. Oh miracle! The multidev environment mr-2 is already here.

Conclusion

My team became much more productive when we started opening merge requests and creating environments automatically.

With powerful tools like GitLab and Pantheon, you can connect GitLab to Pantheon automatically.

Since we are using GitLab CI/CD, our workflow has room to grow. Here are a couple of ideas to get started:

• Add a build step.
• Add automated testing.
• Add a task to ensure coding standards compliance.
• Add dynamic application security testing.

Let us know what you think about GitLab, Pantheon, and automation.

P.S. Did you know that Terminus, the Pantheon command line tool, can be extended through plugins?

We've been hard at work on version 2 of our Terminus build tools plugin with GitLab support. If you don’t want to bother with setup for each project, try this plugin and help us test beta v2. For the Terminus team build:project:create only a Pantheon token and a GitLab token are needed. It will deploy one of the project examples with Composer and automated testing, create a new project in GitLab, a new site on Pantheon, and connect them using environment variables and SSH keys.

About the Author

Andrew Taylor creates tools for developers at Pantheon.

Source: habr.com