Run RSpec in parallel on Github Actions using the parallel_tests gem

Introduction

In this article, we will see how to run RSpec in parallel on Github Actions using the parallel_tests gem.

We will also take advantage of the Gitub Actions matrix feature to run our tests on multiple runners at the same time, gaining even more time.

Prerequisites

I assume you have a Rails app with RSpec already configured.

Why run RSpec in parallel?

Running RSpec in parallel can save you a lot of time. The more tests you have, the more time you will save. This is particularly valuable on a CI environment where you and your team will run your tests many times a day.

How to run RSpec in parallel

• Add the parallel_tests gem to your Gemfile:

group :test do
  gem 'parallel_tests'
end

• Run bundle install
• Setup multiple databases to be used by parallel_tests:

For this, we can modify the config/database.yml file to be configurable through environment variables:

test:
  <<: *default
  database: my_app_test<%= ENV['TEST_ENV_NUMBER'] %>
  username: <%= ENV['DB_USER'] %>
  ...

This will create multiple databases, each of them will have a different name, based on the TEST_ENV_NUMBER environment variable, which will be set by parallel_tests.

• Create multiple databases:

$ rake parallel:create

• Migrate the databases:

$ rake parallel:setup

• Everything is ready to run RSpec in parallel:

$ rake parallel:spec

How to run RSpec in parallel on Github Actions

Now, we have a setup that will work locally, and we can advantage of the cores of our machine to run our tests in parallel.

But, what about Github Actions? How can we run our tests in parallel on Github Actions?

We can use the same strategy on Gihub Actions, to run our tests in parallel, here’s a sample workflow:

name: Pull request
on:
  pull_request:
  push:
    branches:
      - master
jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:14
        ports:
          - 5432:5432
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
      redis:
        image: redis
        ports:
        - 6379:6379
        options: --entrypoint redis-server
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v3
        with:
          node-version: 16

      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 2.7.7

      - uses: actions/cache@v3
        with:
          path: vendor/bundle
          key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
          restore-keys: |
            ${{ runner.os }}-gems-

      - name: Install dependent libraries
        run: sudo apt-get install libpq-dev libvips unzip

      - name: Bundle install
        run: |
          bundle config path vendor/bundle
          bundle install --jobs 4 --retry 3

      - name: Setup Database
        run: |
          bundle exec rake parallel:create
          bundle exec rake parallel:setup
        env:
          RAILS_ENV: test
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres

      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn cache dir)"

      - uses: actions/cache@v3
        id: yarn-cache
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-

      - uses: nanasess/setup-chromedriver@v2
      - name: Run RSpec
        id: rspec-test
        run : |
          yarn
          bundle exec rails webpacker:compile
          bundle exec rake parallel:spec

This workflow is a fully functional workflow that will run your RSpect tests in parallel on Github Actions, including specs that rely on Postgresql.

How to run RSpec in parallel on Github Actions using the matrix feature

While this is great and will save you a lot of time, we can do even better.

Github runners, by default, have 2 cores only, so our setup would speed up our tests, but not as much as what we can do on our local machines where we have 8 or more cores available.

To go even further, we can use the Github Actions matrix feature to run our tests on multiple runners at the same time. With this, we can have X runners, each of them running specs in parallel using the parallel_tests gem.

In our example below, we’ll use 4 runners, so we’ll have a total parallelism of 8. According to the Github Actions docs, we can have up to 256 runners, so we can have a total parallelism of 512, that’s plenty!

Here’s the updated workflow that will support multiple runners and will run our tests in parallel on each runner:

name: Pull request
on:
  pull_request:
  push:
    branches:
      - master
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        # Set N number of parallel jobs you want to run tests on.
        # Use higher number if you have slow tests to split them on more parallel jobs.
        # Remember to update ci_node_index below to 0..N-1
        ci_node_total: [4]
        # set N-1 indexes for parallel jobs
        # When you run 2 parallel jobs then first job will have index 0, the second job will have index 1 etc
        ci_node_index: [0, 1, 2, 3]
    services:
      postgres:
        image: postgres:14
        ports:
          - 5432:5432
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
      redis:
        image: redis
        ports:
        - 6379:6379
        options: --entrypoint redis-server
    steps:
      - uses: actions/checkout@v1
      - uses: actions/setup-node@v3
        with:
          node-version: 16

      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 2.7.7

      - uses: actions/cache@v3
        with:
          path: vendor/bundle
          key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
          restore-keys: |
            ${{ runner.os }}-gems-

      - name: Install dependent libraries
        run: sudo apt-get install libpq-dev libvips unzip

      - name: Bundle install
        run: |
          bundle config path vendor/bundle
          bundle install --jobs 4 --retry 3

      - name: Setup Database
        run: |
          bundle exec rake parallel:create
          bundle exec rake parallel:setup
        env:
          RAILS_ENV: test
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres

      - name: Get yarn cache directory path
        id: yarn-cache-dir-path
        run: echo "::set-output name=dir::$(yarn cache dir)"

      - uses: actions/cache@v3
        id: yarn-cache
        with:
          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
          restore-keys: |
            ${{ runner.os }}-yarn-

      - uses: nanasess/setup-chromedriver@v2
      - name: Run RSpec
        id: rspec-test
        run : |
          yarn
          bundle exec rails webpacker:compile
          ./bin/ci_spec

So, what’s new here?

We added the strategy section, and we defined 2 variables:

• ci_node_total: the total number of runners we want to use
• ci_node_index: the index of the current runner

This will create a matrix of 4 runners, each of them will have a different ci_node_index value, from 0 to 3.

We also added a new step, ./bin/ci_spec, which is a script that will run our tests in parallel using the parallel_tests gem.

This script is based on the parallel_tests docs and adapted to work with Github Actions:

#!/usr/bin/env ruby

# This script is used to run tests in parallel in Github Actions.
# parallel_test test -n 8 --only-group 1,2

CI_RUNNER_PROCESS_COUNT = 2

parallel_tests_is_one_indexed = 1
node_index = ENV['CI_NODE_INDEX'].to_i

groups = "#{node_index * CI_RUNNER_PROCESS_COUNT + parallel_tests_is_one_indexed },#{node_index * CI_RUNNER_PROCESS_COUNT + parallel_tests_is_one_indexed + 1}"

total_parallelism = CI_RUNNER_PROCESS_COUNT * ENV['CI_NODE_TOTAL'].to_i

exec "bundle exec parallel_test ./spec -t rspec -n #{total_parallelism} --only-group #{groups}"

This script will run our tests in parallel using the parallel_tests gem, and will make sure that each runner will run the appropriate specs. What’s nice about this script is that it will work with any number of runners, and it will split the specs evenly between them, as parallel_tests will do the hard work for us, by splitting the specs into groups based on filesize by default.

Conclusion

Using the parallel_tests gem, we can run our tests in parallel on Github Actions, and we can take advantage of the Github Actions matrix feature to run our tests on multiple runners at the same time, gaining even more time.