Michael MacDonald

Testing Sphinx with Cucumber

In Rails, Testing on September 2, 2009 at 8:00 am

For awhile, all of my Cucumber features that involved search were marked as TODO. The search worked but I had no integration tests for it because the default setup of Cucumber uses the transactional fixtures setting to run each scenario inside of a transaction to ensure that the database starts in the same known state for each scenario. Unfortunately ThinkSphinx won’t work with this setup. Thankfully, Brandon from opensoul.org wrote up a great post on how to modify your Cucumber setup to get it working.

Further to Brandon’s post, here’s how I’ve incorporated running search features with Cucumber…

What I wanted was to keep using the standard transaction approach for my normal Cucumber features but to switch to using database cleaner for sphinx features. Now, I haven’t been able to figure out how to turn on/off the use of transactional fixtures in Cucumber.

To enable it you use:


But how do you turn it off?

My current solution is to pull this out of the main env.rb file and turn it on in a webrat.rb support file. This meant I needed to set up my Cucumber profiles to selectively load the appropriate support files (in cucumber.yml):

default: -t ~@sphinx -r features/support/env.rb -r features/support/webrat.rb -r features/step_definitions features
sphinx: -t @sphinx -r features/support/env.rb -r features/support/sphinx.rb -r features/step_definitions features

I then mark all sphinx features/scenarios with the cucumber tag @sphinx. Now I can run my sphinx features using:

cucumber -p sphinx

Finally, switching back and forth between development and testing was causing me grief in starting and stopping the search daemon. The simplest solution was to assign a different port for testing (in sphinx.yml):

  port: 3312
  port: 3313

Update (30/9/2009):

When I upgraded to the newest version of Cucumber (0.3.104) and ran script/generate cucumber I discovered this change in the env.rb:

# Whether or not to run each scenario within a database transaction.
# If you leave this to true, you can turn off traqnsactions on a
# per-scenario basis, simply tagging it with @no-txn
Cucumber::Rails::World.use_transactional_fixtures = true

This would eliminate the need to extract the use_transactional_fixtures setting from the env.rb. Definitely worth revisiting my strategy above now.

Update (15/10/2009):

By upgrading to the latest version of Cucumber (0.4.0) I have been able to undo most of the workaround mentioned in this post. I re-generated the env.rb file using ruby script/generate cucumber and I removed my cucumber profiles. If I want to run only sphinx scenarios I simply use cucumber -t @sphinx or to run non-sphinx scenarios I use cucumber -t ~@sphinx. I created a db_cleaner.rb file and I modified my sphinx.rb file removing all the database cleaner references. Finally, I add the @no-txn tag to all my scenarios which are already tagged with @sphinx. Separation of the database cleaner behaviour from the @sphinx tag means that the @no-txn stuff can be applied elsewhere as required eg selenium features.

Update (19/12/2009):

Be aware that the behaviour of tags changed with cucumber 0.4.3 to allow logical ORing and ANDing of tags. If you use tags in your profile definitions you might find you get different behaviour when you pass the @sphinx tag. My default profile is configured to exclude all scenarios and features marked with the @selenium tag. So when I run cucumber -t @sphinx cucumber will run all scenarios and features marked without @selenium OR with @sphinx. This results in all of my non-selenium scenarios running, not just the sphinx related ones. I still haven’t figured out a suitable fix for this however I just wanted to point this out.

If you don’t use profiles or don’t have tags specified in your profile then the above technique will continue to work as described above.

Update (02/06/2010):

As for version 1.3.2, Thinking Sphinx now provides a test helper for use with Cucumber. I simply add the following lines inside my features/support/sphinx.rb file:

require 'cucumber/thinking_sphinx/external_world'

Now I can remove all the startup/setup code I had previously (as show in Brandon’s original post). I also no longer need an at_exit block since that is handled by the ThinkingSphinx::Test.start_with_autostop. Now my indexing step is simply:


I have also found it necessary to add a sleep(0.25) after my indexing step to ensure that Sphinx has enough time to perform the reindex before Cucumber moves onto the next step.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: