Difficulty: Basic Requirements: Docker, Ansible, Vagrant, Virtualbox

Hodwy, my dear reader, this time I want to show you a very simple method to deploy a whole docker swarm during your coffee break. Using that powerfull hammer of Ansible we will proceed, step-by-step, with a fully automated installation and deploy of docker in swarm mode on a quantity of servers (you choose how many)

What you are thinking right now, I guess, is: “yet another ansible role to perform the same old things?“

Yes.

The idea comes from two strict requirements:

  • quickly deploy on-demand new swarm nodes, to horizontally scaling the cluster
  • conform to prior used standard tools (ansible)

The former is not as simple as “just install docker and forget”, it should be also configured in a proper way. By instance: using a devicemapper volume to achieve production ready performance, choosing the right logger configuration, adding internal registry or choose custom labels to identify the nodes.

Docker Swarm

In case you don’t know, docker has a builtin capability to create cluster of nodes orchestrating containers alltogheter. This option is called “Swarm mode”, it is available from version 1.12.0 and it has interesting features (see Swarm mode overview) plus it’s deadly-easy to make it works.

Enter Ansible

In the world of automation tools Ansible is one of the best, in my opinion. It has only 2 requirements (if you don’t pretend to orchestrate exotic things): python and a ssh connection to target server using a public key authentication (example here) User defined libraries called roles can easly expand the basic playbooks, strucured as a self contained and reusable code.

The Task:

Our target is to deploy a full HA swarm with 3 master and 3 worker nodes. Just because I am not an huge fun of cloud providers AND to show you how this ansible roles can perform well also on premise, we need to use vagrant and virtualbox. A this address GitHub - mbovo/vagrantfiles: Various vagrant files you can find a simple and powerful Vagrantfile ready to go.

Please note: you need Virtualbox and vangrant installed Note: this will spawn 6 virtual machines using 13 Gb of ram, feel free to edit Vagrantfile to resize your cluster at will

git clone https://github.com/mbovo/vagrantfiles
cd vagrantfiles/centos7
vagrant up

Ok, it’s time for you first coffee break.

After vm creation you can start using the ansible role found here: https://github.com/mbovo/ansible_stuffs.git

cd ../../
git clone https://github.com/mbovo/ansible_stuffs.git
cd ansible_stuffs

As you can see there are various directories and roles, for our interests all the magic is performed by roles/docker. How can a generic role knows about our cluster configuration? Using hosts file, of course! Lets take a look to hosts file inside the directory, it is a simple “ini-style” configuration file with one target host per line, grouped under differents names and with some variables after the ip address (the key=value pairs)

[swarm:children]
managers
workers

[managers]
# master=true is the first node where the swarm is created, MUST be unique
# adv_addr is the advertise address with port
192.168.60.10 master=true adv_addr=192.168.60.10:2377
192.168.60.20 manager=true adv_addr=192.168.60.10:2377
192.168.60.30 manager=true adv_addr=192.168.60.10:2377

[workers]
192.168.60.40 worker=true adv_addr=192.168.60.10:2377
192.168.60.50 worker=true adv_addr=192.168.60.10:2377
192.168.60.60 worker=true adv_addr=192.168.60.10:2377

The first line tell ansible to use the group called swarm as a “metagroup” with two children groups: managers and workers The managers group is a list of three hosts, our swarm managers ip address. As you can see in the comment, only one of them must have the variable master set to true. This is not special at all, but will be the first node where the swarm is initialized. The variable _advaddr is used to explicitly set where the swarm must listen on, in case you have (and yes, vagrant boxes has) multiple network addresses. This should be meaningful only on master but I’m too lazy to fix that (improvments are always welcome)

Deploy all the Swarm!

With one simple line of ansible-playbook we will perform the installation of docker on all nodes at once: We are asking to configure devicemapper on lvm as storage backend using /dev/sdb as phisical volume (don’t worry it has been created for you by the special Vagrantfile )

ansible-playbook -i hosts playbooks/docker.yml -e "myhosts=swarm action=install docker_use_lvm=true docker_pvnane=/dev/sdb" -u root

Ok, it’s time for another coffee.

Now we have our six nodes with docker installed and configured, but the swarm is not ready yet. In order to configure docker engines in swarm mode we shall use the action=swarm of the same docker role

ansible-playbook -i hosts playbooks/docker.yml -e "myhosts=swarm action=swarm" -u root

See? easy and fun!

I want Moar!

What about joining new workers to a cluster already up&running? Add a new entry on our hosts file:

[swarm:children]
managers
workers

[managers]
# master=true is the first node where the swarm is created, MUST be unique
# adv_addr is the advertise address with port
192.168.60.10 master=true adv_addr=192.168.60.10:2377
192.168.60.20 manager=true adv_addr=192.168.60.10:2377
192.168.60.30 manager=true adv_addr=192.168.60.10:2377

[workers]
192.168.60.40 worker=true adv_addr=192.168.60.10:2377
192.168.60.50 worker=true adv_addr=192.168.60.10:2377
192.168.60.60 worker=true adv_addr=192.168.60.10:2377
192.168.60.70 worker=true

And ask ansible to perform the job:

ansible-playbook -i hsots playbooks/docker.yaml -e "myhosts=192.168.60.70,192.168.60.10 action=swarm" -u root

Please note we are adding node 192.168.60.70 to the swarm but we need to list also the node marked as “master” in order to retrieve the join tokens used by the docker daemon docker swarm join

Conclusions

We have just scraped the surface but with this ansible role we can deploy full swarms, add or remove nodes, update docker engines all with just one command line. Hope was someway usefull, if you have bug/pull request please contribute to it!