Create A Raspberry Pi Cluster With Docker Swarm And Raspbian Linux

Back when I was first learning about Docker, I had tried to use it on one of my Raspberry Pi devices. This was when Docker had first become available officially for Raspbian. When I was playing around with Docker and my Raspberry Pi, I had written a tutorial for deploying containers, or more specifically a Ghost container. This was a single container on a single device or node. Since then I started playing around with Docker Swarm and decided to create a Raspberry Pi cluster.

We’re going to see how to take several Raspberry Pi devices, cluster them using Docker Swarm, and deploying containers to Swarm with replica copies.

If you don’t already have a few Raspberry Pi units, you can get them for pretty cheap on Amazon. Before getting too far into this guide, you should have already flashed your devices with Raspbian Linux. I’ll be using Raspbian Lite, but Pixel should work out fine. If you’re not sure how to do this, check out my tutorial on flashing Raspbian.

Now you have to decide if you plan on having a local Docker client on your local computer, or strictly the Raspberry Pi units. Having a local client will allow you to use Docker Machine to control your remote Docker hosts without first using SSH. You have to make this decision because the configuration will be slightly different.

Provisioning the Raspberry Pi Nodes with Docker Machine

If you’re like me and want to be able to control the Swarm via your Mac or Windows computer, you’re going to want to be able to connect via Docker Machine. As of writing this, Docker Machine doesn’t play nice with Raspbian Linux out of the box. By doesn’t play nice, I mean you’ll get errors when you try to provision your node.

On each Raspberry Pi node you’ll need to edit the /etc/os-release file. In this file, swap out ID=raspbian to look like ID=debian as seen below:

PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)"
NAME="Raspbian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

If you’re not comfortable doing this one change yourself, you can use the script found here, published by Dieter Reuter.

Before we use our local Docker client to provision the nodes, we need to configure how authentication happens. Docker Machine requires an account with passwordless sudoer privileges. For this example, the root account works, just make sure you add your public SSH key.

From your local computer execute the following command:

docker-machine create \
--driver generic \
--generic-ip-address 192.168.1.28 \
--generic-ssh-key /Users/nraboy/.ssh/rsa \
--engine-install-url https://get.docker.com \
--engine-storage-driver overlay2 \
goomba

In the above command we are using the Generic driver for Docker and specifying the IP address of our Raspberry Pi node. We are also specifying the private key location for the key that matches what was added to the node. Because we need a special version of Docker, we need to specify the install URL as well as the special storage driver to be used. I am calling my node goomba, which is also going to be the host name.

This command will take a bit of time, but when it has completed, you should be able to see it in your list of Docker Machines by executing the following command:

docker-machine ls

Make sure you proceed to following the same steps for any other Raspberry Pi that you wish to provision. Once complete, you have three ways of controlling Docker on the nodes.

Using Docker Machine to SSH into the Raspberry Pi

With Docker Machine, you can SSH into any of your listed nodes. From the Docker Shell, execute the following command:

docker-machine ssh goomba

The above command will SSH me into the node that I’ve called goomba and if you haven’t figured it out, that is a Super Mario reference.

Choosing a Default Docker Machine

For me, SSH kind of defeats the point of having a configured Docker Machine. Instead, you can actually set a default machine and any future Docker commands run from your local client will actually be run against the particular machine. You can choose a default machine by executing the following:

eval $(docker-machine env goomba)

If you list your Docker Machines, you’ll see that now goomba is the default. This makes it convenient when managing containers remotely. For more information on managing remote machines, check out this previous tutorial I wrote on the topic.

Using Standard SSH to Gain Access to the Raspberry Pi

Finally, just because we’ve provisioned the Raspberry Pi nodes with Docker Machine, doesn’t mean we have to use Docker Machine in the long term. You can still SSH into any of your nodes as if Docker didn’t exist.

For example, the following should still work:

ssh root@goomba.local

Of course, the above assumes your host name matches mine and it is on your local network. Otherwise, just use the IP address and you’ll be fine.

Provisioning the Raspberry Pi Nodes without Docker Machine

So let’s say you have no desire to have Docker running locally on your computer. You don’t need Docker installed to work with Docker remotely.

As explained in my previous Raspberry Pi with Docker tutorial, you can SSH into the Raspberry Pi and execute the following command:

curl -sSL https://get.docker.com | sh

While you can’t control locally, you can still create clusters or stand-alone nodes.

Creating a Docker Swarm Raspberry Pi Cluster

As of right now your Raspberry Pi units should have Docker installed and ready to go. Now we’re going to initialize a Swarm and add worker or manager nodes to the Swarm.

Connect to one of your nodes and execute the following command:

docker swarm init --advertise-addr 192.168.1.28

Make sure the --advertise-addr matches the IP address of the currently connected-to node. The above command should only be executed on one node and when it completes, it should show something like this:

docker swarm join \
--token SWMTKN-1-50782k633d63x3l654zfy64u9ztw9m2x3d0w0cbc11669sdukj-2y3mt0tbioch0fw21seg7uuk5 \
192.168.1.28:2377

Of course use whatever you get as a result, but it should be executed on every Raspberry Pi that you wish to be a worker node in the Swarm.

You can see all nodes in your cluster, by executing the following from a manager node:

docker node ls

So how about installing a service on this Docker Swarm?

Remember, these are Raspberry Pi units that use the ARM architecture. You’ll have to use images that are designed to be run on ARM, or create your own image. For help creating your own images, check out a previous tutorial I wrote.

From the manager node, execute the following to deploy Apache:

docker service create --replicas 5 -p 80:80 --name apache httpd

The above command will deploy Apache for ARM with five replica copies dispersed evenly across the cluster. Try to load a node in your web browser to check it out. Try loading a node that doesn’t have Apache on it and it should still load which is part of the beauty of a Docker Swarm.

For more information on what you can do with a Docker Swarm, check out my previous tutorial, Creating a Cluster of Microservice Containers with Docker Swarm.

Conclusion

You just saw how to provision a Raspberry Pi cluster with Docker Engine to be used in a Docker Swarm. While it would probably be more useful to use a remote web service like Digital Ocean or Linode, at least we know it is possible on these cheap Internet of Things (IoT) devices.

If you don’t already have Raspberry Pi units, they can be found on Amazon for pretty cheap. Definitely a fun toy to play around with when learning new technologies.

For more information on using Docker with a Raspberry Pi, check out my tutorial Deploying Docker Containers on a Raspberry Pi Device and for more information on Docker Swarm clustering, check out my tutorial Creating a Cluster of Microservice Containers with Docker Swarm.

Nic Raboy

Nic Raboy is an advocate of modern web and mobile development technologies. He has experience in C#, JavaScript, Golang and a variety of frameworks such as Angular, NativeScript, and Unity. Nic writes about his development experiences related to making web and mobile development easier to understand.