Sharing BDD specifications between testers and developers using StoryQ

From what I’ve seen, teams that decide to follow an agile way of working quite quickly manage to adopt the frequently cited agile practices, apart from one; the multi-skilled team¹. As a result, rather than working in a way where anyone can (and is encouraged to) do anything to help the project succeed, many projects still have what they refer to as “developers” and “testers”. Don’t get me wrong, from what I’ve seen this isn’t an agile adoption killer, but it does bring with it some challenges in terms of ownership / responsibility (call it what you will), around some of the newer agile practices where things are still evolving², like Behaviour Driven Development (BDD).

When it comes to adoption, BDD can lead to some interesting discussions due its wide reaching influence. As a practice, BDD can help agree the scope with the customer, act as requirements for developers and help testers verify the solution. It is this cross cutting nature that can leave multiple members of the team thinking it is their practice to champion and adopt. Individuals from any corner of the team could easily find themselves at a conference (supposedly focused on their specialism) listening to a talk on BDD, in some instances blissfully unaware that other members of their team have received a similar presentation on BDD, just with a subtly different spin.

So if we focus solely on the test creation part of BDD, who is responsible for creating the automated test to prove that a feature has been built to the agreed specification?  My opinion is that both developers and testers have their part to play, each with their own unique skills to contribute, but to date I haven’t seen a truly integrated effort between a group of developers and a group of testers to implement BDD, at least not one where everyone helps both define and verify specifications. What follows below is a way of working that I plan to try when I get the chance. If you have tried something similar I would love to hear from you.

Regardless of who ultimately creates the automated test, every BDD cycle needs to start with a specification. A simple example is included below.

In order to keep my financial information secure

As a registered user

I want to be asked for a username and password when I visit my online banking website

With scenario – “Incorrect password”

Given a registered user

When they enter an incorrect password

Then they should be informed that they have not been granted access

Once we have a specification, we need to represent that specification in a tool that can ultimately make it “executable”. I’ve opted to use StoryQ⁴ (shown below is an abstract class I have written to represent my specification).

    [TestClass]
    abstract public class UserLoginSpec
    {
        [TestMethod]
        public void LoginAsAnInvalidUser()
        {
            new Story("User Login")
            .InOrderTo("Keep my personal financial information secure")
            .AsA("Registered User")
            .IWant("To be asked for a username and password when I visit my online banking website")
            .WithScenario("Incorect password")
            .Given(ARegisteredUser)
            .When(TheyEnterAnIncorrectPassword)
            .Then(TheyShouldSeeAnErrorMessageStatingTheyHaveNotBeenGrantedAccess)
            .Execute();
        }

        abstract public void ARegisteredUser();
        abstract public void TheyEnterAnIncorrectPassword();
        abstract public void TheyShouldSeeAnErrorMessageStatingTheyHaveNotBeenGrantedAccess();
    }

Code Sample 1 – UserLogin Specification

Anyone who has used StoryQ, or any other BBD framework for that matter, will know that we now need to write some methods to wire our specification to our solution. For arguments sake, let’s assume we’re creating a Microsoft ASP.NET MVC application. Who would be the best person in the team to implement these methods – a developer or a tester?

In this situation, I can’t help thinking, why not both? Why not let developers utilise their expertise to implement our specification methods in a way that interacts with our solution below the UI (mocking where necessary), whilst leaving the “testers” to use their expertise to implement our specification methods in a way that interacts with our solution above the UI? Both sets of methods could implement the same specification, so the project only has to maintain one version of what the application should do (a single version of the truth).

I’ve included below an example of what this would look like in code. Notice how both test classes inherit from the same specification class and override each method to create their own version of the test (above and below the UI). I’ve used the sample ASP.NET MVC application as the basis for my example. In case you’re wondering, the above UI version of the test is using the Selenium³ API to provide UI level automation.

    public class UserLogin_BelowUI : UserLoginSpec
    {
        AccountController accountController;
        LogOnModel logOnModel;

        public override void ARegisteredUser()
        {
            logOnModel = new LogOnModel()
            {
                UserName = "someUser",
                Password = "badPassword",
                RememberMe = false
            };
        }

        public override void TheyEnterAnIncorrectPassword()
        {
            accountController = new AccountController()
            {
                FormsService = new MockFormsAuthenticationService(),
                MembershipService = new MockMembershipService()
            };

            accountController.LogOn(logOnModel, null);
        }

        public override void TheyShouldSeeAnErrorMessageStatingTheyHaveNotBeenGrantedAccess()
        {
            Assert.AreEqual("incorrect password", accountController.ModelState[""].Errors[0].ErrorMessage);
        }
    }

Code Sample 2 – UserLogin Below UI Test

    public class UserLogin_AboveUI : UserLoginSpec
    {
        UserData testUserData;

        public override void ARegisteredUser()
        {
            testUserData = TestUserFactory.GetUserWithIncorrectPassword();
        }

        public override void TheyEnterAnIncorrectPassword()
        {
            browser.Click("link=Log On");

            browser.Type("UserName", testUserData.Username);
            browser.Type("Password", testUserData.Password);

            browser.Click("//input[@value='Log On']");
        }

        public override void TheyShouldSeeAnErrorMessageStatingTheyHaveNotBeenGrantedAccess()
        {
            Assert.IsTrue(browser.IsTextPresent("incorrect password"));
        }
    }

Code Sample 3 – UserLogin Above UI Test

.

UserLogin Above UI Test and UserLogin Below UI Test in Visual Studio Test Explorer

But is this duplication? Will a project benefit from having a test that checks a specification below the UI and another test that checks the same specification above the UI?  The table below contains examples of different quality related questions a project could ask themselves and the degree to which each type of test can help answer that question (the table is incomplete; many more examples could be added). I have colour coded the cells in the table to show my preferred type of test for a particular question (those coloured green being my preference). As you can see, tests that are good at answering one question are not so good, or completely incapable, of answering others. This is why I believe a joint effort between testers and developers to check each specification above and below the UI is worth considering.

Quality question	Would an ‘Above UI’ test answer this question?	Would a ‘Below UI’ test answer this question?
Does the AJAX alter the DOM in the correct way?	Yes	No
Are the UI elements mapped to the model correctly?	Yes	No
Does my website work on multiple browsers?	Yes	No
Is the business logic correct?	Yes (but slow)	Yes (fast)
Has my latest check-in broken the build?	Yes (but slow)	Yes (fast)
Does my error handling work as expected?	Yes (but hard)	Yes (easy)

 

There is, of course, nothing to say that every specification must be tested both above and below the UI. If a particular specification doesn’t lend itself to being tested either above or below the UI, the team can make the conscious choice to create a single test in this instance, possibly adding an attribute to the other test class (or specification class) to signal that a test has been consciously skipped rather than forgotten. On the topic of tracking and reporting, a team could also use some simple reflection to keep track of what specifications were being checked above the UI, below the UI or both. With something like this is place, any member of the team could add a specification and those responsible for creating tests would quickly realise there was a gap to fill. Even the results of exploratory testing or good old fashion bug hunts could be incorporated in this way.

There is also nothing to say that a UI-level test has to do everything above the UI. If the “Given” part of the test is particularly difficult to reproduce above the UI then that part of the test could be performed below the UI. For example, a test could directly manipulate the database to setup a particular system state, before performing an action and checking the expected result above the UI.

References

1. http://antonymarcano.com/blog/2011/05/updated-lessons-learned-in-close-quarters-battle/

The link above is to an article written by Antony Marcano in which he describes a multi-skilled team.

2 http://www.bcs.org/upload/pdf/tester-dec10.pdf

The link above is to an article written by Gojko Adzic in the December 2010 edition of The Tester in which he describes the on-going work to consolidate the practices and the language around BDD (or whatever it ends up being called!)

3 http://storyq.codeplex.com

More information about StoryQ

4 http://seleniumhq.org

More information about Selenium