First impressions of Behat – BDD for PHP

After my ill-prepared talk the other night at the Behat London User Group meetup I thought I’d extend my usage of Behat into this blog post…

A Bit of Background

About 2 years ago, my team at the BBC began to adopt behavioural driven development (BDD).  BBC Worldwide’s Chief Technical Architect (and super-smart-dude), Julian Everett inspired the team to give BDD a go.  It took a while for the business analysts and product owners to pick up the Gherkin syntax and even longer for the PHP developers to pick up Cucumber, Capybara and a degree of Ruby.  It wasn’t until we started writing tests and running them everyday did I realise the real benefit of having acceptance criteria and functional tests, opposed to relying solely on unit tests and manual “gorilla” testing.

Over the past year, I’ve been building a website as a side-project in my spare time.  This project is approx 50% JavaScript and 50% PHP.  About 3 weeks ago I started to get worried about the quality of the code particularly with inconsistent behaviour in edge-case scenarios.  The testability of the PHP and JavaScript was pretty straight forward – PHPUnit and Qunit.   My major concern was around the JavaScript unit tests and the gap between Html/browser/user and the JS unit test coverage and how the JS communicated with the PHP (ajax/api) endpoints.

Naturally I began to think BDD functional tests and then thought about the huge amount of effort required in wiring up Ruby, Cucumber, Capybara, Gems, Bundle etc.  From my own experience at the BBC, it’s not as straight-forward and easy as everyone says, so I began to think of alternatives!

Enter Behat….

That’s where Behat enters the scene!  In my opinion, writing step definitions in PHP just makes sense when you’re a PHP developer.  Secondly, Behat is really-really-really easy to install, even with a browser driver like Sahi or Selenium 2 webdriver – boom!

So…. 3 weeks ago I installed Behat and Selenium 2 on my Windows 7 PC and started writing Gherkin acceptance criteria and Behat step definitions immediately.  Since then, I’ve written 282 scenarios (just fixed the last failure!):

282 scenarios (282 passed)
2117 steps (2117 passed)
115m55.183s

Here are some lessons I’ve learnt along the way…..

Lessons Learnt

In no particular order, here is a list of things I’ve learnt about Behat so-far, which might help others who are new to Behat too.

1.  Getting YAML Config Parameters in Context

Below is an illustration in how to access your YAML config settings in your context (FeatureContext.php)…

behat.yml:

default:
  context:
    parameters:
      javascript_session: selenium
      browser: firefox
      base_url: http://localhost/public/
      secure_base_url: http://safe.localhost/public/

FeatureContext.php:

protected $_baseUrl;
protected $_secureBaseUrl;

public function __construct(array $parameters)
    {
        $this->_baseUrl       = $parameters['base_url'];
        $this->_secureBaseUrl = $parameters['secure_base_url'];
    }

2.  Hooks

Similar to xUnit’s setUp and tearDown, Behat has Hooks that can be executed before and after the suite, feature and/or scenarios are run.  Below is an example of logging a user out before each scenario.

/**
 * @BeforeScenario
 */
public function logoutUser(ScenarioEvent $event)
{
     try {
         $event->getContext()->getSession()->visit($this->_baseUrl.'bdd-user-logout');
     } catch (Exception $e) {
         return;
     }
}

It’s important to point out that you’ll need to add an PHP 5.3 alias to your context (FeatureContext.php), for example:

use Behat\Behat\Event\ScenarioEvent

Note, depending on which hook you’re using, you’ll need to alias a different file. Read about hooks in the documentation.

3.  Tags

I’ve started to group scenarios and features into a number of categories (tags):

  • wip – Not completed yet.  Shouldn’t be executed on continuous integration (CI) box;
  • smoke-test – A broad selection of key tests that can be run quickly to make sure you having broken anything;
  • long – Indicates that the scenario will take some time to run;
  • bug – Known bug.  Don’t execute ever.  Similar to Skip.

An example of a scenario with multiple tags:

@wip @long
Scenario:  Log newly created user in
    Given I am unauthenticated
    When I fill........

Firstly, it’s important to remember that your full suite of tests should be executed before you release to live.  It’s also good practice to run your test suite periodically throughout the day (perhaps midday and midnight).

Now, why create tags?  After only 3 weeks of writing tests, they are already taking 2 hours to execute the whole suite.  There are times when you want to run a sample of test to ensure key parts of the system are running.  These tests are called smoke-tests and are usually around 5-10min in duration, compared with the full suite that exceeds 2 hrs.  It’s particularly useful to run smoke-tests immediately after a release to the live environment (in parallel with the full suite of tests).

Here is an example of tags in a behat.yml file:

default:
  filters:
    tags: '~@bug&&~@wip'
ci:                    # run all 'ready' tests
  filters:
    tags: '~@bug&&~@wip'
wip:                   # run tests that are still in dev
  filters:
    tags: '@wip&&~@bug'
smoke:                 # run key tests (fast test)
  filters:
    tags: '@smoke-test,@smoke'
no-long:               # dont run long tests
  filters:
    tags: '@~bug&&~@long'

In the above example, you’ll see tags separated with “&&” (and) instead of “,” (or), this is because there is a tag-exclusion “~” character, therefore the statement should be an AND instead of an OR.

An example of executing tags on the command line:

behat --tags="@wip" --tags="~@bug"

Or by running the executing tags via a profile (above):

behat --profile=smoke

4.  Does Element Exist or Not?

Here’s a quick snippet of code that throws an exception when an element does not exist on the page:

    public function iShouldSeeLoadingMessage()
    {
        $page = $this->getSession()->getPage();
        $el = $page->find('css', '#app-loading');
        if ($el === null) {
            throw new exception('Missing loading message');
        }
    }

5.  You Must Wait for Selenium, dude!

Selenium doesn’t wait for the page to load or for a JavaScript action to complete itself. You must place a wait after each action to ensure that the content is loaded into the page or the JavaScript action has finished running. For example, with the following acceptance criteria:

Given I click on "signup button"
Then I should see the "signup form" on the page

Let’s suppose that when you click on the signup button, an Ajax call is made to load a signup form into the page dynamically (this probably takes at least 1 sec). What we’d need to do is add a wait before we check to see if the form is actually in the page.

Here is the step definition for “Given I click on “#signup-button”“:

public function iClickon($field)
{
    $this->clickLink($field);
    $this->wait(2000);   // wait 2sec
}

It’s difficult to judge how long the wait should be. Personally, I’ve found that depending on what the action is, it can very from 0.5s to 10s. Getting the right wait time can significantly reduce the time your tests take to execute.

6.  Triggering JavaScript Events on Form Input

Using Selenium 2, when I fill in a form input with some text, the JavaScript event triggers are not fired. Below is a selection of ways I’ve found to manually trigger the JavaScript events. Trigger jQueryUI datepicker and other custom jQuery plugins:

$this->getSession()->wait(500, '$("#'.$field.'").trigger("keyup")');
$this->getSession()->wait(500, '$("#'.$field.'").trigger("keydown")');

Trigger jQueryUI autocomplete and jQueryUI datepicker (select date):

$el = $this->getSession()->getPage()->find('css', '#username-input');
$el->mouseOver();
$el->click();

Another possible way to trigger an event is to click on the label of a form input:

$el = $this->getSession()->getPage()->find('css', 'label[for=username-input]');
$el->click();

7.  Use the Context to Store Data

One really cool feature with Behat is the “context” and being able to persist data between steps.

Acceptance Criteria:

Given I am authenticated
When I visit "basic details form" page
Then the "username" field should contain "{{username}}" value

The step definition of the first line “Given I am authenticated” creates a new user on the website (by using a magic BDD-only PHP endpoint against a non-live database). The step definition:

protected $_username;

public function iAmAuthenticated()
{
    $this->visit($this->_secureBaseUrl.'bdd-create-user.php');
    $css = $this->getSession()->getPage()->find('css', '#bdd-username');
    if ($css === null) {
        throw new exception('BDD username missing');
    }
    $this->_username = $css->getText();
}

In the above, when Behat visits “bdd-create-user.php” a randomly generated user is created and a light-weight Html page is returned with a “#bdd-username” element containing the newly created username. That username is then stored in “$this->_username”, making it available to other steps in the scenario.

The step definition of the third line “Then the “username” field should contain “{{username}}” value” has the following step definition:

public function theFieldShouldContainValue($field, $value)
{
    // This line has been simplified for this example:
    $value = str_replace('{{username}}', $this->_username, $value);
    // Run in-built assertion:
    $this->assertFieldContains($field, $value);
}

The above scenario illustrates that creating a user in the first step and storing the newly created user’s username, we can use it later on in the scenario to ensure a form contains the user’s username.

Summary

Overall I’ve been really impressed with Behat.  I would never have imagined that Behat and Mink were going to even compare to Cucumber and Capybara, but it does!!

Behat is really easy to use; the numbers speak for themselves:  for my project, I’ve managed to knock out almost 300 scenarios within 3 weeks (in my spare time).  Many of the scenarios require very complex step definitions due to the use of complex JavaScript plugins that I’m using.  I think this can be attributed to the simple nature of Behat and Mink (plus Selenium).

About the Author
Brett is the Lead Web Developer at BBC.com working on a number of products, such as the BBC International Homepage, News, Sport, Travel and the back-end work on the iPhone and iPad applications.

Advertisements
Posted in Behat - BDD for PHP, Behavioural Driven Development (BDD), jQuery, PHP, Test Driven Development (TDD)
7 comments on “First impressions of Behat – BDD for PHP
  1. jzalas says:

    It was a good ill-prepared talk 😉 thanks for sharing!

    Just my three cents:

    in general It’s better to avoid unconditional wait. You might end up with a suite which takes too long to run.

    wait() method accepts javascript code as a second argument. You can use it to wait until script evaluates to true or it times out. With ajax requests I usually look for changes in DOM. In most cases wait() will take less time than a specified timeout.

    Also, Selenium driver has a waitForPageToLoad method. Might be that Selenium2 has something similar.

  2. brettscott says:

    Thanks Jakub 😉 You’re definitely right about the wait() methods increasing run times. When I get a second, I’ll give your suggestion a go by using the second parameter on the wait() method to see if the task I’m waiting for has completed, using JavaScript. Thanks for the idea!

    I would try to use the waitForPageToLoad() method or equivalent, but because my app is JavaScript heavy (and minimal page loads), in my situation I don’t think this method would be appropriate.

    Thanks for your suggestions mate!

    Brett.

  3. You could try out the Sahi driver. Sahi automatically waits for AJAX and page loads.

  4. lmb says:

    I’m running into the same problem with selenium not triggering javascript events on a dynamic form. It’ll work when I have the browser in focus, but only when I use the workarounds you’ve described above. Then when I try to run the tests with xvfb via jenkins, or even if I just don’t have the browser in focus when I’m running them locally, the dynamic part of the form doesn’t load, and my tests fail. Will webdriver just not work for this? Is the Sahi driver the only solution? Or am I just missing something all together?

    • brettscott says:

      Hi lmb,
      Personally, I haven’t been in the situation where a browser window has come out of focus. My understanding is that the browser window must be in focus in order to apply changes to the form or to test the window contents using Selenium – or any driver for that matter. You must assign your desired window back into focus. Are you seeing this problem because you’re running your tests in parallel or because of a popup window?
      Brett.

  5. Thanks for a great article!
    Usage of tags inspired me to add them in my own scenarios.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: