Continuous Integration With GitLab CI And Docker Using A Raspberry Pi

I know I’ve mentioned this before, but I’ve recently started to get serious about my content production and deployment approach on The Polyglot Developer. My goal is to be able to write my tutorials in Markdown, push to GitLab, and have it automatically deployed as a Docker container on my production server. Being able to automate things and take advantage of Docker will definitely improve my productivity in the long term.

So I’ve been playing around with tools on the subject, more specifically GitLab, because that is what I’m using to save a history of the blog. GitLab is a source code repository, but also a whole lot more given its ability to do continuous integration, continuous deployment, and work with Docker directly.

We’re going to take a look at installing GitLab and Docker on a Raspberry Pi, then configuring a GitLab CI Runner to take control of our continuous integration process every time we push some code. While it might sound easy, there are some certain things that aren’t so obvious in the setup and configuration.

Before we get too invested, I want to point out what is hopefully the obvious. The Raspberry Pi, while awesome, is not a very powerful computer. GitLab, GitLab Runner, and Docker all have requirements which push the Raspberry Pi to its limits and beyond. For this reason none of the tasks that we do will be fast, but regardless it makes a great way to test a setup before choosing a more stable production solution.

Installing GitLab on a Raspberry Pi

There are numerous methods for installing GitLab on a Raspberry Pi, but if you’re using Raspbian like I am, there is an official image available to us.

Before running the commands that follow, pick a hostname for your Raspberry Pi that you’re satisfied with. By default, the hostname will be raspberrypi, but mine for example, was changed to hyrule. This will determine how we configure GitLab and how we connect to it from a browser. For example, I intend to connect to mine by navigating to http://hyrule.local in the browser. Remember, you probably won’t have a domain name attached to your Raspberry Pi like you would a production server.

With your hostname figured out, execute the following commands on your Raspberry Pi:

sudo apt-get install curl openssh-server ca-certificates apt-transport-https
curl https://packages.gitlab.com/gpg.key | sudo apt-key add -

The above commands will make sure we have the appropriate software installed and ready to go. By software, I mean things necessary to install GitLab. The documentation recommends we install Postfix so we can send emails, but we’re going to skip that step.

Next, execute the following command:

sudo curl -sS https://packages.gitlab.com/install/repositories/gitlab/raspberry-pi2/script.deb.sh | sudo bash=

The above command will add the GitLab package repository so that way we can download GitLab using an apt-get command. With the repository available, we can finalize the installation by executing the following:

sudo EXTERNAL_URL="http://hyrule.local" apt-get install gitlab-ce

Notice in the above command I am defining my local hostname as the external URL. This configuration will be used internally in GitLab and will be stored in the configuration files. Make sure to swap hyrule.local with that of your actual Raspberry Pi.

We’re not quite done yet. The documentation would lead you to believe that you can go to http://hyrule.local to create a root level account, but we didn’t install Postfix so there is an extra step we have to do. By default, any account created, even root, will require an email confirmation. We need to disable this functionality so that accounts don’t need email confirmation.

On the Raspberry Pi, open the /etc/gitlab/gitlab.rb file, which is the main configuration file for GitLab, and find the following line:

gitlab_rails['gitlab_email_enabled'] = false

Make sure that the line is set to false and that it is not commented with a hash symbol. Before we attempt to create our account, we need to reconfigure and restart the GitLab instance. From the command line, execute the following:

sudo gitlab-ctl reconfigure
sudo gitlab-ctl restart

When the restart finishes and GitLab is available, navigate to http://hyrule.local in your web browser. Remember, your hostname might not match mine. The first time you launch GitLab in the browser, you’ll be prompted to create a password for the root user.

Congratulations, GitLab is now functional and you can start pushing your source code repositories to it!

Installing Docker for Container Deployments and Management on a Raspberry Pi

Being able to push source code to your private GitLab instance is only half the battle. The end-goal is to have a runner that can do post-push tasks. There are numerous ways to work with GitLab Runners, but we’re going to focus on the Docker approach. For this reason, we need to get Docker installed on our Raspberry Pi.

On the Raspberry Pi, execute the following command:

curl -sSL https://get.docker.com | sh

The above command will download and install the Docker runtime. More information around configuring Docker on a Raspberry Pi beyond the basics can be found in a previous tutorial I wrote titled, Deploying Docker Containers on a Raspberry Pi Device.

You’ll need to use sudo when working with Docker unless you start creating groups and introducing risks in your setup. It is a minute detail that won’t affect us going forward.

Configuring a GitLab CI Runner with Docker for Continuous Integration Pipelines

Like previously mentioned, there are many different methods towards installing and configuring a GitLab Runner. In our example we’re going to install the GitLab Runner on the host and have it create Docker containers every time it needs to operate.

On the Raspberry Pi, execute the following to install the GitLab Runner:

curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner

With the GitLab Runner installed, we need to register it for use in our pipeline. Per the GitLab documentation, the recommended way to register a GitLab Runner is to use the docker-in-docker executor like so:

sudo gitlab-runner register -n \
    --url http://hyrule.local/ \
    --registration-token REGISTRATION_TOKEN \
    --executor docker \
    --description "Docker Runner" \
    --docker-image "docker:stable" \
    --docker-privileged

Notice that we’re using our hostname URL in the above command. You’ll also need to find the REGISTRATION_TOKEN within the GitLab administration dashboard.

We’re not quite done yet, and this is where I had the most struggle. So hyrule is my Raspberry Pi hostname and given most Mac and Linux configurations, it can be accessed over the network as hyrule.local. However, our GitLab Runner will be spinning up a Docker container and that container will try to access hyrule.local which will make no sense because Docker containers are on their own network and cannot immediately access things on the host. Not to mention, the container won’t have Bonjour and the necessary software to access hostnames anyways.

To get beyond this, we actually need to define a mapping for our runner that says if hyrule.local is not found, redirect to a different host or IP address instead. In this circumstance, we’ll use an IP address instead.

Open the /etc/gitlab-runner/config.toml file and you’ll find something that looks like the following:

[[runners]]
    name = "Docker Runner"
    url = "http://192.168.1.40"
    token = "JTv8KRdaHeLkbEGpQRqi"
    executor = "docker"
    [runners.docker]
        tls_verify = false
        image = "docker:stable"
        privileged = true
        disable_entrypoint_overwrite = false
        oom_kill_disable = false
        disable_cache = false
        volumes = ["/cache"]
        shm_size = 0
        extra_hosts = ["hyrule.local:192.168.1.40"]

Take note of a few things that I’ve changed in the above. If for some reason the registration didn’t like your http://hyrule.local url, you might have had to use an IP. The IP listed is that of my Raspberry Pi. I’ve also added an extra_hosts array which has my mapping between the hostname and my IP address.

After you make the change, execute the following:

sudo gitlab-runner restart

Now when we actually start a job, it won’t fail with an error like the following:

Cloning repository...
Cloning into '/builds/root/test'...
fatal: unable to access 'http://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@hyrule.local/root/test.git/': Could not resolve host: hyrule.local
/bin/bash: line 62: cd: /builds/root/test: No such file or directory

It took me so long to figure out what was going on with the above error. In the end it made total sense that my container wouldn’t know what hyrule.local is and that I’d have to come up with a different solution. Of course on a production ready server, you probably won’t have this problem because you’ll have a valid domain name.

In theory, GitLab and the GitLab Runner is properly configured and ready to act as part of the Docker CI pipeline.

Testing the Continuous Integration Setup on the Raspberry Pi

So how can we make sure that everything is working correctly? We can create a new project and push our source code to it. On your computer, not the Raspberry Pi, create the following files:

Dockerfile
.gitlab-ci.yml

The Dockerfile file is going to be what we want to build. In theory you could have anything and no Dockerfile file. The .gitlab-ci.yml file is where all of our continuous integration logic goes.

Open the Dockerfile file and include the following:

FROM alpine:latest
RUN apk add -U git

The above blueprint will use an Alpine Linux image and install Git. It doesn’t do a whole lot, but at least it does something. Open the project’s .gitlab-ci.yml file and include the following:

image: docker:latest

services:
  - docker:dind

build:
  stage: build
  script:
    - docker build -t test .

We’re saying that we want to use the docker:dind service with our GitLab Runner and when the script is ran, it will only build our Docker image. Remember, the build script doesn’t have to be Docker related. We could be running tests in a Node.js application if we wanted.

To give credit where credit is deserved, both the Dockerfile file and the .gitlab-ci.yml files were taken from the GitLab documentation.

Try pushing your source code and look at the jobs within the GitLab dashboard. You should see your continuous integration process start. While not the focus here, we could add a deployment pipeline to this as well.

Conclusion

You just saw how to install GitLab, Docker, and a GitLab Runner on a Raspberry Pi. With everything combined, we have an incredibly inefficient source code repository and continuous integration (CI) service. Like I said previously, the Raspberry Pi is severely underpowered for this kind of work, but at least it is a cool example and useful for experimenting.

If you get 502 errors in GitLab, you may need to increase the swap memory. The recommendations say your computer should have 4GB of memory, but you can get away with adding 1GB of swap and be fine.

Nic Raboy

Nic Raboy is an advocate of modern web and mobile development technologies. He has experience in C#, JavaScript, Golang and a variety of frameworks such as Angular, NativeScript, and Unity. Nic writes about his development experiences related to making web and mobile development easier to understand.