Split your RSpec build into N equal parts with rspec-parts.
We've been subscribed to the Travis Pro service for our CI process for a while. We pay for the Startup plan, which gives us 5 concurrent executors at any given time. Over the last two years, we've slowly been pecking away at converting our minitest suite to RSpec. Within the last month, we finally bit the bullet and devoted some developer time to finishing that task. Since we moved from three test suites (minitest, RSpec, and Jasmine) to just two, we had some concurrency available that was not being fully utilized for a single build.
You can see our impressively long 1 hr and 3 minute build below.
Amazing, right? Our goal then was to take our exceptionally long RSpec build and split it into four evenly split concurrent jobs, thereby using all five executors for one build, allowing us to deploy much quicker.
Let's make it faster!
At first we initialized a
Rake::FileList on the entirety of the
spec/ directory. The
Rake::FileList class is
particularly useful, allowing us to easily manipulate lists of files.
Avdi Grimm has a good blog post that dives into the details of the class, and covers it better than I will here.
Rake::FileList returns an enumerable of all files within the
spec/ directory, which we can then split into N even
#in_groups. Let's run that on Travis now.
As you can see, it's pretty lopsided. The entire directory for
finder/ specs, which are considerably faster than the rest got grouped
together and ended up running in eight minutes. Another build that contained the entirety of the
features/ directory took over
20 minutes to complete.
So, how do we try to split this more evenly?
We can assume that each file within a subdirectory of
spec/ will run in about the same amount of time
spec/models/b_spec.rb will be similar in run times relative to one another, as will
So, we'll iterate over each subdirectory and break them into even groups. Since we want to split into four jobs, we'll take the
models/ directory, split it into four even groups.
Then we'll take the
views/ directory, and split that into four groups. Similarly with
controllers/, and every other subdirectory within
Those groups will then be added back to a master
Rake::FileList which will then be run by rspec.
Simple enough premise, but let's see how this method fairs:
As you can see, the job times are more evenly distributed than the last build, and we've reduced the time we need to wait for a single build to finish from over an hour and change to just under 20 minutes.
So, how did we do it?
The important part of our
.travis.yml file looks like this now:
env: - TEST_SUITE=rspec RSPEC_PART=1 RSPEC_GROUPS=4 - TEST_SUITE=rspec RSPEC_PART=2 RSPEC_GROUPS=4 - TEST_SUITE=rspec RSPEC_PART=3 RSPEC_GROUPS=4 - TEST_SUITE=rspec RSPEC_PART=4 RSPEC_GROUPS=4 - TEST_SUITE=jasmine script: "script/ci-travis-$TEST_SUITE"
Each element in the
env array represents one build, so you can see how we split the RSpec builds into four parts.
script/ci-travis-rspec file is an executable script which looks like this:
#!/bin/bash xvfb-run -e /dev/stdout bundle exec rake spec:part[$RSPEC_PART,$RSPEC_GROUPS]
And that's it!
The essence of that script lies in the
rake spec:part command. It's using the
which we developed here at Wanelo. It takes two arguments, the first being the part to run, the second being the number of
groups to split the suite into. To run build one of two you'd run
rake spec:part[1,2], for build two
If we wanted to upgrade our Travis CI plan to give us 10 concurrent executors, we could simply modify the command to
rake spec:part[1,9], still accouting for our one Jasmine build.
The rspec-parts gem is available on github and is pretty well documented as to how to use it in the README.
If you've got any questions on how to use it, feel free to poke us in the comments or post an issue on github.