Using Jenkins to build and test a Docker Compose cluster (via Rails, Elasticsearch, and RSpec)
In this post I’ll share a working example that uses Jenkins to build and test your Docker Compose cluster. For this example I created a simple Rails project that connects to Postgresql and Elasticsearch. I used RSpec to test the connectivity of the container services, and then configured Jenkins to build the project and run the test suite.
I installed my Docker dependencies via Brew on OSX.
$ brew list --versions | grep -i docker
docker 17.03.0
docker-compose 1.11.2
docker-machine 0.10.0
docker-machine-nfs 0.4.1
Rails Project
# rvm files
echo docker-jenkins > .ruby-gemset
echo ruby-2.4.0 > .ruby-version
cd .
gem install rails
rails -v
Rails 5.0.2
# new project with postgresql connection
rails new . --api -d postgresq
# init database
rake db:create && rake db:migrate
Add gems, edit file: Gemfile
group :development, :test do
gem 'rspec-rails'
end
gem 'elasticsearch-model'
gem 'elasticsearch-rails'
Execute bundle install
to install the new gems.
Executed rails g rspec:install
to generate the default RSpec configuration.
Enabled Elasticsearch logging integration. Edit file: config/application.rb
, added:
require 'elasticsearch/rails/instrumentation'
Updated database configuration to use environment variables for connectivity. edit file: config/database.yml
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
host: <%= ENV.fetch('POSTGRESHOST', 'localhost') %>
password: <%= ENV.fetch('POSTGRESPASS', 'postgres') %>
username: <%= ENV.fetch('POSTGRESUSER', 'postgres') %>
Add a migration to create a table for a Person model (first and last name), new file: db/migrate/20170315211521_create_people.rb
class CreatePeople < ActiveRecord::Migration[5.0]
def change
create_table :people do |t|
t.string :first_name
t.string :last_name
t.timestamps
end
end
end
Created the Person model with Elasticsearch integration, new file: app/models/person.rb
class Person < ApplicationRecord
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
end
Added an RSpec model test file for the Person model. I purposely did not mock the backend to ensure the services are connecting properly. new file: spec/models/person_spec.rb
require 'rails_helper'
RSpec.describe Person, type: :model do
describe 'elasticsearch' do
before do
Person.destroy_all
end
after do
person.destroy
end
let(:first_name) { "first #{DateTime.now.to_i}" }
let(:last_name) { "last #{DateTime.now.to_i}" }
let!(:person) { Person.create!(first_name: first_name, last_name: last_name) }
let(:get_params) do
{
index: Person.index_name, id: person.id
}
end
let!(:document) { Person.__elasticsearch__.client.get(get_params) }
it 'indexes model data' do
expect(document['_source']['id']).to eq(person.id)
expect(document['_source']['first_name']).to eq(first_name)
expect(document['_source']['last_name']).to eq(last_name)
end
end
end
Docker Integration
Add a simple Dockerfile for the Rails container that adds netcat to test for service connectivity, new file: Dockerfile
FROM ruby:2.4.0
RUN apt-get update -qq && apt-get install -y netcat
ENV APP_HOME /rails
WORKDIR $APP_HOME
Created the Docker compose file to define the containers for Rails, Elasticsearch, and Postgresql, new file: docker-compose.yml
version: '2'
services:
elasticsearch:
image: elasticsearch:2.4.4
ports:
- '9200:9200'
- '9300:9300'
volumes:
- elasticsearch:/usr/share/elasticsearch/data
postgres:
environment:
- POSTGRES_PASSWORD=postgres
- POSTGRES_USER=postgres
image: postgres:9.6.2
ports:
- '5432:5432'
volumes:
- postgres:/var/lib/postgresql/data
rails:
build:
context: .
dockerfile: Dockerfile
command: bin/docker-start-rails
depends_on:
- elasticsearch
- postgres
environment:
- ELASTICSEARCH_URL=http://elasticsearch:9200
- POSTGRESHOST=postgres
- POSTGRESPASS=postgres
- POSTGRESUSER=postgres
links:
- elasticsearch
- postgres
ports:
- '3000:3000'
volumes:
- .:/rails
- bundle:/usr/local/bundle
volumes:
bundle: {}
elasticsearch: {}
postgres: {}
Added a shell script to start rails, new file: bin/docker-start-rails
#!/bin/sh
rm tmp/pids/server.pid
bundle check || bundle install
# wait for postgresql
until nc -vz postgres 5432 2>/dev/null; do
echo "Postgresql is not ready, sleeping."
sleep 1
done
# wait for elasticsearch
until nc -vz elasticsearch 9200 2>/dev/null; do
echo "Elasticsearch is not ready, sleeping."
sleep 1
done
rake db:create
rake db:migrate
rails s -b 0.0.0.0
I added a simple Ping controller which will be used by Jenkins to ensure the Rails container is up before starting the RSpec test suite. new file: app/controllers/api/pings_controller.rb
class Api::PingsController < ApplicationController
def show
render json: 'pong'
end
end
And the corresponding route for the ping request, edit file: config/routes.rb
, added:
Rails.application.routes.draw do
namespace :api do
resource :ping, only: :show
end
end
At this point the application can be functionally tested via: docker-compose build && docker-compose up
Jenkins Integration
I added a bin script as an entry point for Jenkins. The script defines a docker machine called “jenkins”, ensures it is created, running, and active. It stops and removes any existing containers. It then builds the Docker compose cluster, waits for Rails to respond to the ping request, and executes the RSpec test suite. new file: bin/jenkins
#!/bin/sh
set -x
DOCKER_MACHINE=jenkins
EXISTS=$(docker-machine ls --filter name=$DOCKER_MACHINE -q)
if [ "$EXISTS" = "" ]; then
docker-machine create -d virtualbox $DOCKER_MACHINE
eval $(docker-machine env $DOCKER_MACHINE)
fi
STATE=$(docker-machine ls --filter name=$DOCKER_MACHINE | awk '{print $4}' | tail -1)
if [ "$STATE" = "Stopped" ]; then
docker-machine start $DOCKER_MACHINE
eval $(docker-machine env $DOCKER_MACHINE)
fi
ACTIVE=$(docker-machine ls --filter name=$DOCKER_MACHINE | awk '{print $2}' | tail -1)
if [ "$ACTIVE" = "-" ]; then
eval $(docker-machine env $DOCKER_MACHINE)
fi
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
docker-compose build
docker-compose up -d
IP=$(docker-machine ip $DOCKER_MACHINE)
# wait for rails
while true
do
response=$(curl http://$IP:3000/api/ping 2>/dev/null)
if [ "$response" = "pong" ]; then
break
fi
sleep 1
done
RAILS_CONTAINER=$(docker ps -a | grep -i 3000 | awk '{print $NF}')
docker exec -i $RAILS_CONTAINER rspec
RSPEC_RESULT=$?
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
exit $RSPEC_RESULT
I installed Jenkins locally via brew:
brew install jenkins
brew list --versions | grep -i jenkins
jenkins 2.50
The following images shows the settings I used for the Jenkins Docker Compose project. I integrated with my Github repo and set Jenkins to poll every 15 minutes for changes. I added a build step to timeout after 10 minutes, and execute the following:
#!/bin/bash -l
bin/jenkins