Michael MacDonald

Test Business Behaviour, Don’t Rely on UI

In Testing on January 26, 2011 at 1:15 am

I often see Cucumber features written like this:

Given I a have an account with email: "user@test.com" and password: "password"
When I go to the log in page
And I fill in "Email" with "user@test.com"
And I fill in "Password" with "password"
And I press "Log in"
Then I should see "Logged in successfully."
And I should not see "Log in"

It works and does the job but I feel that it isn’t enough. It’s fragile in several ways.

First, if any of your form field or button labels change or your flash message changes, you’ll need to update your feature even though the behaviour hasn’t actually changed! I totally understand why we write features like this – it’s convenient, it’s quick and easy.

Second, have we actually verified that the user has logged in? All we’ve really done is verify that we see a particular flash message and that we no longer see a piece of text on the page. We haven’t actually verified the behaviour under test here. Again, it’s convenience but it’s one that can cause you trouble.

I’ve had a feature like the above where the flash text I was looking for actually appeared in a login error message. For example:

Then I should see "successfully logged in"

My success flash message was “#{user.name} successfully logged in” and my feature passed. But it was a false positive. In fact the user had not successfully logged in at all. My error message was “#{user.name} was not successfully logged in” so when the login failed my step would still pass because the string is found on the page.

Okay, that’s easy to fix but it showed me just how easy it is for a feature to be fragile and too dependent on content and UI elements. Any rearrangement or inclusion of new content on a page can have a dramatic effect on your cucumber features. Worst case scenario is that you actually break functionality but your tests return a false positive.

To take it a step further, what if you had only implemented the flash handling and not the actual logging in of the user yet. You would get a passing feature.

My point is that the verification is not strong enough. You are reliant on an artefact of the logging in process. And this artefact is not part of the business rule.

The business rule here is that when a user logs in with the correct credentials they are successfully logged in. It is ancillary whether they see a flash message telling them they are logged in or if they can’t see the login button anymore. What you need to check is that the user is actually logged in.

So replace the last two lines in the example with:

Then I should be logged in
And I should be on the dashboard page

In your step definition you should use the logic that your system uses (or will use) to check if a user is logged in. For example:

Then /^I should be logged in$/ do
  # get your user first
  user.should be_logged_in

To take it further, you can remove the other content and UI references to make your feature more robust:

Given I a have an account with email: "user@test.com" and password: "password"
When I go to the log in page
And I fill in the email field with "user@test.com"
And I fill in the password field with "password"
And I press the login button
Then I should be logged in
And I should be on the dashboard page

I admit that this is more work so, like most things, you need to decide what’s the most appropriate for your context. If nothing else, give some extra thought to your Then steps – are they thorough enough? are they verifying the primary business behaviour?

  1. […] This post was mentioned on Twitter by Michael MacDonald, Arndt Touby. Arndt Touby said: @fpauser: Test Business Behaviour, Don’t Rely on UI « Ruby Flare http://bit.ly/dYN6eu […]

  2. It doesn’t matter to your user that some code somewhere says he’s logged in. What matters to him is the functionality that logging in provides, like being able to see the dashboard page. In my opinion, the “I should be logged in” step is irrelevant from the user’s point-of-view.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: