First impressions of Behat – BDD for PHP

After my ill-prepared talk the other night at the Behat London User Group meetup I thought I’d extend my usage of Behat into this blog post…

A Bit of Background

About 2 years ago, my team at the BBC began to adopt behavioural driven development (BDD).  BBC Worldwide’s Chief Technical Architect (and super-smart-dude), Julian Everett inspired the team to give BDD a go.  It took a while for the business analysts and product owners to pick up the Gherkin syntax and even longer for the PHP developers to pick up Cucumber, Capybara and a degree of Ruby.  It wasn’t until we started writing tests and running them everyday did I realise the real benefit of having acceptance criteria and functional tests, opposed to relying solely on unit tests and manual “gorilla” testing.

Over the past year, I’ve been building a website as a side-project in my spare time.  This project is approx 50% JavaScript and 50% PHP.  About 3 weeks ago I started to get worried about the quality of the code particularly with inconsistent behaviour in edge-case scenarios.  The testability of the PHP and JavaScript was pretty straight forward – PHPUnit and Qunit.   My major concern was around the JavaScript unit tests and the gap between Html/browser/user and the JS unit test coverage and how the JS communicated with the PHP (ajax/api) endpoints.

Naturally I began to think BDD functional tests and then thought about the huge amount of effort required in wiring up Ruby, Cucumber, Capybara, Gems, Bundle etc.  From my own experience at the BBC, it’s not as straight-forward and easy as everyone says, so I began to think of alternatives!

Enter Behat….

That’s where Behat enters the scene!  In my opinion, writing step definitions in PHP just makes sense when you’re a PHP developer.  Secondly, Behat is really-really-really easy to install, even with a browser driver like Sahi or Selenium 2 webdriver – boom!

So…. 3 weeks ago I installed Behat and Selenium 2 on my Windows 7 PC and started writing Gherkin acceptance criteria and Behat step definitions immediately.  Since then, I’ve written 282 scenarios (just fixed the last failure!):

282 scenarios (282 passed)
2117 steps (2117 passed)
115m55.183s

Here are some lessons I’ve learnt along the way…..

Lessons Learnt

In no particular order, here is a list of things I’ve learnt about Behat so-far, which might help others who are new to Behat too.

1.  Getting YAML Config Parameters in Context

Below is an illustration in how to access your YAML config settings in your context (FeatureContext.php)…

behat.yml:

default:
  context:
    parameters:
      javascript_session: selenium
      browser: firefox
      base_url: http://localhost/public/
      secure_base_url: http://safe.localhost/public/

FeatureContext.php:

protected $_baseUrl;
protected $_secureBaseUrl;

public function __construct(array $parameters)
    {
        $this->_baseUrl       = $parameters['base_url'];
        $this->_secureBaseUrl = $parameters['secure_base_url'];
    }

2.  Hooks

Similar to xUnit’s setUp and tearDown, Behat has Hooks that can be executed before and after the suite, feature and/or scenarios are run.  Below is an example of logging a user out before each scenario.

/**
 * @BeforeScenario
 */
public function logoutUser(ScenarioEvent $event)
{
     try {
         $event->getContext()->getSession()->visit($this->_baseUrl.'bdd-user-logout');
     } catch (Exception $e) {
         return;
     }
}

It’s important to point out that you’ll need to add an PHP 5.3 alias to your context (FeatureContext.php), for example:

use Behat\Behat\Event\ScenarioEvent

Note, depending on which hook you’re using, you’ll need to alias a different file. Read about hooks in the documentation.

3.  Tags

I’ve started to group scenarios and features into a number of categories (tags):

  • wip – Not completed yet.  Shouldn’t be executed on continuous integration (CI) box;
  • smoke-test – A broad selection of key tests that can be run quickly to make sure you having broken anything;
  • long – Indicates that the scenario will take some time to run;
  • bug – Known bug.  Don’t execute ever.  Similar to Skip.

An example of a scenario with multiple tags:

@wip @long
Scenario:  Log newly created user in
    Given I am unauthenticated
    When I fill........

Firstly, it’s important to remember that your full suite of tests should be executed before you release to live.  It’s also good practice to run your test suite periodically throughout the day (perhaps midday and midnight).

Now, why create tags?  After only 3 weeks of writing tests, they are already taking 2 hours to execute the whole suite.  There are times when you want to run a sample of test to ensure key parts of the system are running.  These tests are called smoke-tests and are usually around 5-10min in duration, compared with the full suite that exceeds 2 hrs.  It’s particularly useful to run smoke-tests immediately after a release to the live environment (in parallel with the full suite of tests).

Here is an example of tags in a behat.yml file:

default:
  filters:
    tags: '~@bug&&~@wip'
ci:                    # run all 'ready' tests
  filters:
    tags: '~@bug&&~@wip'
wip:                   # run tests that are still in dev
  filters:
    tags: '@wip&&~@bug'
smoke:                 # run key tests (fast test)
  filters:
    tags: '@smoke-test,@smoke'
no-long:               # dont run long tests
  filters:
    tags: '@~bug&&~@long'

In the above example, you’ll see tags separated with “&&” (and) instead of “,” (or), this is because there is a tag-exclusion “~” character, therefore the statement should be an AND instead of an OR.

An example of executing tags on the command line:

behat --tags="@wip" --tags="~@bug"

Or by running the executing tags via a profile (above):

behat --profile=smoke

4.  Does Element Exist or Not?

Here’s a quick snippet of code that throws an exception when an element does not exist on the page:

    public function iShouldSeeLoadingMessage()
    {
        $page = $this->getSession()->getPage();
        $el = $page->find('css', '#app-loading');
        if ($el === null) {
            throw new exception('Missing loading message');
        }
    }

5.  You Must Wait for Selenium, dude!

Selenium doesn’t wait for the page to load or for a JavaScript action to complete itself. You must place a wait after each action to ensure that the content is loaded into the page or the JavaScript action has finished running. For example, with the following acceptance criteria:

Given I click on "signup button"
Then I should see the "signup form" on the page

Let’s suppose that when you click on the signup button, an Ajax call is made to load a signup form into the page dynamically (this probably takes at least 1 sec). What we’d need to do is add a wait before we check to see if the form is actually in the page.

Here is the step definition for “Given I click on “#signup-button”“:

public function iClickon($field)
{
    $this->clickLink($field);
    $this->wait(2000);   // wait 2sec
}

It’s difficult to judge how long the wait should be. Personally, I’ve found that depending on what the action is, it can very from 0.5s to 10s. Getting the right wait time can significantly reduce the time your tests take to execute.

6.  Triggering JavaScript Events on Form Input

Using Selenium 2, when I fill in a form input with some text, the JavaScript event triggers are not fired. Below is a selection of ways I’ve found to manually trigger the JavaScript events. Trigger jQueryUI datepicker and other custom jQuery plugins:

$this->getSession()->wait(500, '$("#'.$field.'").trigger("keyup")');
$this->getSession()->wait(500, '$("#'.$field.'").trigger("keydown")');

Trigger jQueryUI autocomplete and jQueryUI datepicker (select date):

$el = $this->getSession()->getPage()->find('css', '#username-input');
$el->mouseOver();
$el->click();

Another possible way to trigger an event is to click on the label of a form input:

$el = $this->getSession()->getPage()->find('css', 'label[for=username-input]');
$el->click();

7.  Use the Context to Store Data

One really cool feature with Behat is the “context” and being able to persist data between steps.

Acceptance Criteria:

Given I am authenticated
When I visit "basic details form" page
Then the "username" field should contain "{{username}}" value

The step definition of the first line “Given I am authenticated” creates a new user on the website (by using a magic BDD-only PHP endpoint against a non-live database). The step definition:

protected $_username;

public function iAmAuthenticated()
{
    $this->visit($this->_secureBaseUrl.'bdd-create-user.php');
    $css = $this->getSession()->getPage()->find('css', '#bdd-username');
    if ($css === null) {
        throw new exception('BDD username missing');
    }
    $this->_username = $css->getText();
}

In the above, when Behat visits “bdd-create-user.php” a randomly generated user is created and a light-weight Html page is returned with a “#bdd-username” element containing the newly created username. That username is then stored in “$this->_username”, making it available to other steps in the scenario.

The step definition of the third line “Then the “username” field should contain “{{username}}” value” has the following step definition:

public function theFieldShouldContainValue($field, $value)
{
    // This line has been simplified for this example:
    $value = str_replace('{{username}}', $this->_username, $value);
    // Run in-built assertion:
    $this->assertFieldContains($field, $value);
}

The above scenario illustrates that creating a user in the first step and storing the newly created user’s username, we can use it later on in the scenario to ensure a form contains the user’s username.

Summary

Overall I’ve been really impressed with Behat.  I would never have imagined that Behat and Mink were going to even compare to Cucumber and Capybara, but it does!!

Behat is really easy to use; the numbers speak for themselves:  for my project, I’ve managed to knock out almost 300 scenarios within 3 weeks (in my spare time).  Many of the scenarios require very complex step definitions due to the use of complex JavaScript plugins that I’m using.  I think this can be attributed to the simple nature of Behat and Mink (plus Selenium).

About the Author
Brett is the Lead Web Developer at BBC.com working on a number of products, such as the BBC International Homepage, News, Sport, Travel and the back-end work on the iPhone and iPad applications.

Advertisements
Posted in Behat - BDD for PHP, Behavioural Driven Development (BDD), jQuery, PHP, Test Driven Development (TDD)

Lucene Number Range Search – Integers & Floats (with Zend_Search_Lucene)

I came across an interesting problem this afternoon which took me about an hour to unravel.   Things aren’t quite as straight-forward as one would expect when using Lucene’s range search in Zend Framework against a number, integer or float.

Ignoring for a moment that I’m using the less performant Zend_Search_Lucene instead of Solr, let me illustrate the problem.

The code below simply adds a field ‘price’ with the value ‘9.99’ ($9.99) to a document in a given Lucene index.

$doc = new Zend_Search_Lucene_Document();
$doc->addField(Zend_Search_Lucene_Field::Text('price', '9.99'));
$this->index->addDocument($doc);

Using Lucene’s Index Toolkit (Luke), a query to find the above document might look like:

price:9.99

Which will return any document with a field ‘price’ of value ‘9.99’.
Read more ›

Posted in PHP, Zend Framework, Zend_Search_Lucene

Internationalization and Localization with PHP – Zend_Locale, Zend_Translate and Zend_Date

I’ve been working on a project in my spare time.  As part of the initial analysis I decided that this project would be suitable for a wider, multi-lingual, international audience but probably in 2 or 3 years time.   I then had to make the difficult decision on whether to do the work now and waste the time on a new feature I’ll never actually use, or wait until I actually need the feature and then have perform a major refactor of the whole project.

My real issue was that I’d never created a truly international website (multi-lingual, multiple date formats) and without knowing how to implement it, I’d have to go off and find out!  I was very surprised to find that making a web application internationalised and localised with Zend Framework is actually very straight-forward.

Read more ›

Posted in Internationalisation, PHP, Zend Framework, Zend_Locale, Zend_Translate

Considerations When Building an Authentication System (Part 3: Database)

Please read Part 1 and Part 2 of this series as it will give you some background thoughts.

Read more ›

Posted in PHP, Zend Framework

Considerations When Building an Authentication System (Part 2: Cookies)

Please read Part 1 as it will give you some background thoughts.

Part 1 of this instalment covered the use of Sessions (cookies and data) and how to use them securely. This part will cover cookies and the importance cookies play in authentication systems.

Read more ›

Posted in PHP, Zend Framework, Zend_Filter, Zend_Session

Considerations When Building an Authentication System (Part 1: Session)

This is a 3 part series.

Background
Over the past couple of months, I’ve been implementing an authentication system for a Zend Framework project I’m working on in my spare time.  This post is going to outline considerations when building an authentication system for web applications built with Apache HTTPD, PHP and Zend Framework.

Authentication is a lot more complex than a simple username and password in a database, a form and some business logic to confirm that the credentials match.  When building an authentication system for your website, you’ll also need to consider browser persistence (remember me), session hijacking, database security and SSL/HTTPS and what impact these things have across multiple domains and servers.

Read more ›

Posted in PHP, Zend Framework, Zend_Filter, Zend_Session