After my ill-prepared talk the other night at the Behat London User Group meetup I thought I’d extend my usage of Behat into this blog post…
A Bit of Background
About 2 years ago, my team at the BBC began to adopt behavioural driven development (BDD). BBC Worldwide’s Chief Technical Architect (and super-smart-dude), Julian Everett inspired the team to give BDD a go. It took a while for the business analysts and product owners to pick up the Gherkin syntax and even longer for the PHP developers to pick up Cucumber, Capybara and a degree of Ruby. It wasn’t until we started writing tests and running them everyday did I realise the real benefit of having acceptance criteria and functional tests, opposed to relying solely on unit tests and manual “gorilla” testing.
Over the past year, I’ve been building a website as a side-project in my spare time. This project is approx 50% JavaScript and 50% PHP. About 3 weeks ago I started to get worried about the quality of the code particularly with inconsistent behaviour in edge-case scenarios. The testability of the PHP and JavaScript was pretty straight forward – PHPUnit and Qunit. My major concern was around the JavaScript unit tests and the gap between Html/browser/user and the JS unit test coverage and how the JS communicated with the PHP (ajax/api) endpoints.
Naturally I began to think BDD functional tests and then thought about the huge amount of effort required in wiring up Ruby, Cucumber, Capybara, Gems, Bundle etc. From my own experience at the BBC, it’s not as straight-forward and easy as everyone says, so I began to think of alternatives!
Enter Behat….
That’s where Behat enters the scene! In my opinion, writing step definitions in PHP just makes sense when you’re a PHP developer. Secondly, Behat is really-really-really easy to install, even with a browser driver like Sahi or Selenium 2 webdriver – boom!
So…. 3 weeks ago I installed Behat and Selenium 2 on my Windows 7 PC and started writing Gherkin acceptance criteria and Behat step definitions immediately. Since then, I’ve written 282 scenarios (just fixed the last failure!):
282 scenarios (282 passed) 2117 steps (2117 passed) 115m55.183s
Here are some lessons I’ve learnt along the way…..
Lessons Learnt
In no particular order, here is a list of things I’ve learnt about Behat so-far, which might help others who are new to Behat too.
1. Getting YAML Config Parameters in Context
Below is an illustration in how to access your YAML config settings in your context (FeatureContext.php)…
behat.yml:
default:
context:
parameters:
javascript_session: selenium
browser: firefox
base_url: http://localhost/public/
secure_base_url: http://safe.localhost/public/
FeatureContext.php:
protected $_baseUrl;
protected $_secureBaseUrl;
public function __construct(array $parameters)
{
$this->_baseUrl = $parameters['base_url'];
$this->_secureBaseUrl = $parameters['secure_base_url'];
}
2. Hooks
Similar to xUnit’s setUp and tearDown, Behat has Hooks that can be executed before and after the suite, feature and/or scenarios are run. Below is an example of logging a user out before each scenario.
/**
* @BeforeScenario
*/
public function logoutUser(ScenarioEvent $event)
{
try {
$event->getContext()->getSession()->visit($this->_baseUrl.'bdd-user-logout');
} catch (Exception $e) {
return;
}
}
It’s important to point out that you’ll need to add an PHP 5.3 alias to your context (FeatureContext.php), for example:
use Behat\Behat\Event\ScenarioEvent
Note, depending on which hook you’re using, you’ll need to alias a different file. Read about hooks in the documentation.
3. Tags
I’ve started to group scenarios and features into a number of categories (tags):
- wip – Not completed yet. Shouldn’t be executed on continuous integration (CI) box;
- smoke-test – A broad selection of key tests that can be run quickly to make sure you having broken anything;
- long – Indicates that the scenario will take some time to run;
- bug – Known bug. Don’t execute ever. Similar to Skip.
An example of a scenario with multiple tags:
@wip @long
Scenario: Log newly created user in
Given I am unauthenticated
When I fill........
Firstly, it’s important to remember that your full suite of tests should be executed before you release to live. It’s also good practice to run your test suite periodically throughout the day (perhaps midday and midnight).
Now, why create tags? After only 3 weeks of writing tests, they are already taking 2 hours to execute the whole suite. There are times when you want to run a sample of test to ensure key parts of the system are running. These tests are called smoke-tests and are usually around 5-10min in duration, compared with the full suite that exceeds 2 hrs. It’s particularly useful to run smoke-tests immediately after a release to the live environment (in parallel with the full suite of tests).
Here is an example of tags in a behat.yml file:
default:
filters:
tags: '~@bug&&~@wip'
ci: # run all 'ready' tests
filters:
tags: '~@bug&&~@wip'
wip: # run tests that are still in dev
filters:
tags: '@wip&&~@bug'
smoke: # run key tests (fast test)
filters:
tags: '@smoke-test,@smoke'
no-long: # dont run long tests
filters:
tags: '@~bug&&~@long'
In the above example, you’ll see tags separated with “&&” (and) instead of “,” (or), this is because there is a tag-exclusion “~” character, therefore the statement should be an AND instead of an OR.
An example of executing tags on the command line:
behat --tags="@wip" --tags="~@bug"
Or by running the executing tags via a profile (above):
behat --profile=smoke
4. Does Element Exist or Not?
Here’s a quick snippet of code that throws an exception when an element does not exist on the page:
public function iShouldSeeLoadingMessage()
{
$page = $this->getSession()->getPage();
$el = $page->find('css', '#app-loading');
if ($el === null) {
throw new exception('Missing loading message');
}
}
5. You Must Wait for Selenium, dude!
Selenium doesn’t wait for the page to load or for a JavaScript action to complete itself. You must place a wait after each action to ensure that the content is loaded into the page or the JavaScript action has finished running. For example, with the following acceptance criteria:
Given I click on "signup button" Then I should see the "signup form" on the page
Let’s suppose that when you click on the signup button, an Ajax call is made to load a signup form into the page dynamically (this probably takes at least 1 sec). What we’d need to do is add a wait before we check to see if the form is actually in the page.
Here is the step definition for “Given I click on “#signup-button”“:
public function iClickon($field)
{
$this->clickLink($field);
$this->wait(2000); // wait 2sec
}
It’s difficult to judge how long the wait should be. Personally, I’ve found that depending on what the action is, it can very from 0.5s to 10s. Getting the right wait time can significantly reduce the time your tests take to execute.
6. Triggering JavaScript Events on Form Input
Using Selenium 2, when I fill in a form input with some text, the JavaScript event triggers are not fired. Below is a selection of ways I’ve found to manually trigger the JavaScript events. Trigger jQueryUI datepicker and other custom jQuery plugins:
$this->getSession()->wait(500, '$("#'.$field.'").trigger("keyup")');
$this->getSession()->wait(500, '$("#'.$field.'").trigger("keydown")');
Trigger jQueryUI autocomplete and jQueryUI datepicker (select date):
$el = $this->getSession()->getPage()->find('css', '#username-input');
$el->mouseOver();
$el->click();
Another possible way to trigger an event is to click on the label of a form input:
$el = $this->getSession()->getPage()->find('css', 'label[for=username-input]');
$el->click();
7. Use the Context to Store Data
One really cool feature with Behat is the “context” and being able to persist data between steps.
Acceptance Criteria:
Given I am authenticated
When I visit "basic details form" page
Then the "username" field should contain "{{username}}" value
The step definition of the first line “Given I am authenticated” creates a new user on the website (by using a magic BDD-only PHP endpoint against a non-live database). The step definition:
protected $_username;
public function iAmAuthenticated()
{
$this->visit($this->_secureBaseUrl.'bdd-create-user.php');
$css = $this->getSession()->getPage()->find('css', '#bdd-username');
if ($css === null) {
throw new exception('BDD username missing');
}
$this->_username = $css->getText();
}
In the above, when Behat visits “bdd-create-user.php” a randomly generated user is created and a light-weight Html page is returned with a “#bdd-username” element containing the newly created username. That username is then stored in “$this->_username”, making it available to other steps in the scenario.
The step definition of the third line “Then the “username” field should contain “{{username}}” value” has the following step definition:
public function theFieldShouldContainValue($field, $value)
{
// This line has been simplified for this example:
$value = str_replace('{{username}}', $this->_username, $value);
// Run in-built assertion:
$this->assertFieldContains($field, $value);
}
The above scenario illustrates that creating a user in the first step and storing the newly created user’s username, we can use it later on in the scenario to ensure a form contains the user’s username.
Summary
Overall I’ve been really impressed with Behat. I would never have imagined that Behat and Mink were going to even compare to Cucumber and Capybara, but it does!!
Behat is really easy to use; the numbers speak for themselves: for my project, I’ve managed to knock out almost 300 scenarios within 3 weeks (in my spare time). Many of the scenarios require very complex step definitions due to the use of complex JavaScript plugins that I’m using. I think this can be attributed to the simple nature of Behat and Mink (plus Selenium).
About the Author
Brett is the Lead Web Developer at BBC.com working on a number of products, such as the BBC International Homepage, News, Sport, Travel and the back-end work on the iPhone and iPad applications.
Brett's Blog 