Faster CI - how we reduced our build time by 25%
We’ve been using Travis CI to run tests and automate deployments on our projects for the last few years and have been very happy with the service they provide. Our data visualisation tool, Dataseed, has used Travis from the start and over the years we’ve built up a large amount of tests for the font-end and back-end code. Having a comprehensive test suite is great for catching regressions and doing refactors but the length of time the build and tests were taking had become a real pain point.
In this post I’m going to look at the steps we took to reduce the time it took for the whole process to run, so that changes became visible on our staging/UAT site quicker.
Status Quo
Our .travis.yml
initially looked like this:
language: python
python: 2.7
env:
global:
- ES_URL=https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.3.deb
- HEROKU_BRANCH=dev
- HEROKU_REPO=git@heroku.com:dataseed-stage.git
- secure: <HEROKU DEPLOYMENT KEY>
services:
# Start redis as Travis service
- redis-server
install:
# Install ElasticSearch
- sudo apt-get purge elasticsearch
- wget --output-document=/tmp/elasticsearch.deb $ES_URL
- sudo dpkg --install /tmp/elasticsearch.deb
- sudo service elasticsearch start
# Install Python dependencies
- sudo apt-get -qq install python-dev libpq-dev libevent-dev libxml2-dev libxslt-dev
- pip install --requirement=$TRAVIS_BUILD_DIR/requirements.txt
# Install JS dependencies
- npm install
- bower install
script:
# Run Python tests
- python $TRAVIS_BUILD_DIR/test.py
# Run JS tests
- gulp test
after_success:
# Submit code coverage information to Coveralls
- coveralls
# Only deploy from $HEROKU_BRANCH
# When Travis builds a pull request $TRAVIS_BRANCH is set to the base branch
# (the branch we want to merge into). The second condition below prevents
# from deploying pull requests with base branch dev.
- if [ $TRAVIS_BRANCH != $HEROKU_BRANCH ] || [ $TRAVIS_PULL_REQUEST != false ]; then exit 0; fi
# Install Heroku gem
- wget -qO- https://toolbelt.heroku.com/install-ubuntu.sh | sh
# Configure Git/SSH
- git remote add heroku $HEROKU_REPO
- git fetch --unshallow
- echo "Host heroku.com" >> ~/.ssh/config
- echo " StrictHostKeyChecking no" >> ~/.ssh/config
- echo " CheckHostIP no" >> ~/.ssh/config
- echo " UserKnownHostsFile=/dev/null" >> ~/.ssh/config
# Add Heroku SSH keys
- heroku keys:clear
- yes | heroku keys:add
# Deploy to Heroku (always "master" branch)
- yes | git push --force heroku $TRAVIS_BRANCH:master
As you can see, the steps that are taken are:
- Install a specific version of ElasticSearch
- Install Python dependencies
- Install Javascript dependencies
- Run back-end tests
- Run front-end tests
- Deploy to Heroku
Below I’ll go through the various changes we made and the rationale behind each one.
APT Packages
In order to use Travis’s container-based infrastructure we needed to eliminate all uses of sudo
. As you can see above, it was only used for installing additional packages with apt which we were able to replace with the Travis apt add-on:
sudo: false
cache:
apt: true
addons:
apt:
sources:
- elasticsearch-1.7
packages:
- elasticsearch
- libpq-dev
- libevent-dev
- libxml2-dev
- libxslt1-dev
- python-dev
Note that we had to add the elasticsearch-1.7
source so that we could use a specific version (1.7) of ElasticSearch. You can see what non-default packages are available, and request new ones, in the apt-source-whitelist Github repository.
By switching to containers we gained the following:
- Faster start-up
- Increased CPU from 1.5 to 2 virtual cores
- Increased memory from 3GB to 4GB
- Better network throughput
Javascript Dependencies
We used Travis’s caching to cache downloads of packages from npm and Bower. Unfortunately this only saved us a couple of seconds as most of the time spent by these package managers is moving the packages into place.
cache:
directories:
- $HOME/.cache/bower
- $HOME/.npm
Python Dependencies
Aside from the tests themselves, the slowest part of the process was installing the Python dependencies from PyPI. By using Python wheels in combination with Travis’s caching we managed to reduce the median time for this step from 4 minutes to 15 seconds!
cache:
directories:
- $HOME/.cache/pip
install:
- pip install -U pip wheel
- pip install --requirement=$TRAVIS_BUILD_DIR/requirements.txt
Back-End Tests
As you’d expect, the slowest part of the process has always been running the tests themselves. The simplest way to speed the tests up was to run them in parallel, which we used the nose multiprocess plugin for. The test.py
script used to run the test suite was updated as follows:
from nose import run
from nose.plugins.multiprocess import MultiProcess
run(argv=[
'tests',
'--processes=2',
'--with-coverage',
'--cover-package=dataseed',
])
plugins=[MultiProcess()])
Heroku Deploy
When we first wrote our Travis configuration integrated Heroku deployment wasn’t available from Travis. Now that this is provided by Travis we can remove our custom code to deploy via Git/SSH and in the process save a few seconds.
deploy:
provider: heroku
app: <HEROKU APP NAME>
on: dev
api_key:
secure: <HEROKU API TOKEN>
This simply deploys to our Heroku staging app when commits are made to the dev
branch (and all tests have passed). To add the encrypted API key we used the following command:
travis encrypt $(heroku auth:token) --add deploy.api_key
Conclusion
After putting everything together, our final configuration was as follows:
language: python
python: 2.7
sudo: false
cache:
apt: true
directories:
- $HOME/.cache/pip
- $HOME/.cache/bower
- $HOME/.npm
addons:
apt:
sources:
- elasticsearch-1.7
packages:
- elasticsearch
- libpq-dev
- libevent-dev
- libxml2-dev
- libxslt1-dev
- python-dev
services:
# Start Redis and ElasticSearch as Travis service
- redis-server
- elasticsearch
install:
# Install Python dependencies
- pip install -U pip wheel
- pip install --requirement=$TRAVIS_BUILD_DIR/requirements.txt
# Install JS dependencies
- npm install
- bower install
script:
# Run Python tests
- python $TRAVIS_BUILD_DIR/test.py
# Run JS tests
- gulp test
after_success:
# Submit code coverage information to Coveralls
- coveralls
deploy:
provider: heroku
app: <HEROKU APP NAME>
on: dev
api_key:
secure: <HEROKU API TOKEN>
In total we reduced the time from commit to deploy by 25%, which was much appreciated by developers and QA! Reducing friction in this way makes for a quicker development and testing cycle which ultimately gets new functionality into the hands of our users faster than before.