Drupal Gitlab CI - Running contrib tests locally

Gitlab CI is now the preferred default for running tests in contrib in Drupal. This is great! Gitlab gives us a lot of new control over how tests are run. While moving project update bot to gitlab I wanted local developement really badly.

Drupal chose to create a gitlab templates repository to get control on how Gitlab is run for contrib testing. Great! There is pretty good guide available on drupal.org to get you up and running easily. The basics are really simple though.

Getting started with Drupal’s Gitlab

To get started you just include the a .gitlab-ci.yml based on the template. See below for the important part.

include: 
  ################
  # DrupalCI includes: 
  # As long as you include this, any future includes added by the Drupal Association will be accessible to your pipelines automatically.
  # View these include files at https://git.drupalcode.org/project/gitlab_templates/
  ################
  - project: $_GITLAB_TEMPLATES_REPO
    ref: $_GITLAB_TEMPLATES_REF
    file:
      - '/includes/include.drupalci.main.yml'
      - '/includes/include.drupalci.variables.yml'
      - '/includes/include.drupalci.workflows.yml'

So in short is just tells gitlab where to look for some templates and include those into your project. Easy stuff!

The (not so fun) testing loop

But what if you want to customize, or test your gitlab workflow? This almost always ends up the the following loop:

Commit .gitlab-ci.yml
Wait for run (to fail)
Adjust .gitlab-ci.ml
Repeat
…
Profit.

This is not really optimal, you keep waiting for gitlab to trigger and rub your stuff. But fear not, there is a way to get Gitlab CI running locally.

The deprecated Gitlab runner

When searching for a way to run locally I started at Gitlab ofcourse. I did find the Gitlab runner documentation. Which seemed fine, until i noticed it didn’t support includes and apparently a lot more.

It seems like Gitlab doesn’t want to support local runners, perhaps in the future, it’s a well requested feature.

Our saviour? firecow/gitlab-ci-local

So while scouring the internet I found a little gem: firecow/gitlab-ci-local.

This repository aims to bring gitlab-ci to your local machine. After interacting with the maintainer a few times after running into some problems I feel pretty confident this will be well supported.

So how to use is? Basically the easiest way is to use NPM to install the package globally.

npm install -g gitlab-ci-local

You do need to make sure you have docker installed. Its a fully docker based solution.

Playing nicely with templates

As a test I’ve used the ‘keycdn’ module, which is also used by the Gitlab team to test the templates. You’d recon you can just clone the repository and go right? Not entirely.

bjorn@1WNLK63:~/projects/keycdn$ gitlab-ci-local
Project include could not be fetched { project: , ref: HEAD, file: includes/include.drupalci.main.yml }
Error: Command failed with exit code 2: git archive --remote=ssh://[email protected]:22/.git HEAD includes/include.drupalci.main.yml | tar -f - -xC .gitlab-ci-local/includes/git.drupal.org//HEAD/
remote:
remote: ========================================================================
remote:
remote: The namespace you were looking for could not be found.
remote:
remote: ========================================================================
remote:
fatal: the remote end hung up unexpectedly
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

The error here is pretty logical. In a normal environment the variables are set globally by gitlab and point to the right gitlab repository. We need to set those also for ourselves. We can do that in multiple ways, but the easiest is probably making your own little shell script.

I’ve been using this one:

~/.local/bin/drupal-ci-local

#!/bin/bash

gitlab-ci-local \
  --remote-variables [email protected]:project/gitlab_templates=includes/include.drupalci.variables.yml=1.0.x \
  --variable="_GITLAB_TEMPLATES_REPO=project/gitlab_templates" \
  "$@"

This script does nothing more than making sure we use the correct variables to run our setup. If you put this is an directory that can be executed, like ~/.local/bin or /usr/local/bin you can use it anywhere in your shell.

So for example:

$ git clone [email protected]:project/keycdn
$ cd keycdn
$ drupal-ci-local composer # runs the composer step
$ drupal-ci-local # runs all steps!

What’s really cool about this is the fact that all the changes that the gitlab makes to your project. So should you run into something really weird, you can actually debug it! Yay!

Running chromedrivertests

There is are still some quirks, when using chromedriver will need some changes to know where to go. There are multiple containers:

The main container is called: “build”
The database is called: “database”
The chromedriver is called: “chrome”

The current implementation for Drupal CI kinda cheats. It makes all services available on localhost on the runner. This is not default behaviour and uses a featureflag. The proper implementation would be to use the container names to get things up and running. This should be possible, but will need some work to make sure testing url’s and chromedriver urls are correct.

RTFM – there is more!

Really really really read the readme on the repository: firecow/gitlab-ci-local. There is loads of other things you can do like global variables and such. Lovely!

Conclusion

You can run Gitlab locally pretty easy. There will be quircks, but lets see if we can fix those together 🙂

Issue #3365438 on drupal.org.

Drupal Gitlab CI – Running contrib tests locally