class: title # Acceptance testing with Cucumber.js and Protractor [@wongatech](http://twitter.com/wongatech) \#AngularJSLondon --- # The journey... - Explanation of acceptance testing and BDD - How Cucumber.js and Protractor make it easy to write acceptance tests for Angular apps - Some tips and tricks to make that more enjoyable --- # Acceptance tests are conversations *Developer:* > “I’d like to talk to you about the login **scenario**...” *Product Owner:* > “**Given** I am on the login page > **When** I login > **Then** I should be logged in” ??? - Shared understanding - Edge cases: are these all the outcomes? --- class: bigger # BDD improves TDD > “Programmers wanted to know where to start, what to test and what not to test, > how much to test in one go, what to call their tests, and how to understand > why a test fails.” .right[*— [Dan North](http://dannorth.net/introducing-bdd/)*] ??? - It should do X = what to test, where to start - What can be described in a sentence = how much to test - Behaviour: "Should," Should it? - Tests can be challenged - Failing test: could be a bug could be behaviour has changed - **Acceptance criteria should be executable** --- class: bigger # A “ubiquitous language” for analysis > **Given** some initial context (the givens), > **When** an event occurs, > **Then** some outcome *should* occur. --- # Cucumber.js lets you translate acceptance criteria into JavaScript ```xml Given I am on the login page ``` ```javascript this.Given('I am on the login page', function () { browser.get('#/login'); }); ``` ??? - BDD test framework - JBehave then Cucumber, then Cucumber in other languages - That then proliferated to all the languages ever - Deceptively simple mapping - How does Cucumber work? --- # Feature files ```xml Feature: Login As a customer I want to be able to log in So that I can access restricted features Scenario: Unauthenticated user Given I am on the login page When I login Then I should be logged in Scenario: User who hasn't logged in for ages Given I haven't logged in for 4 score and 7 years And I am on the login page When I login Then I should see the welcome back message ``` ??? - Put our acceptance criteria in features - Organise criteria into "features" and "scenarios" - Feature: login, search, dashboard - Scenarios: login OK, login failed, login whilst already logged in - Don't hide feature files away --- # Step definitions ```javascript this.Given('I am on the login page', function () { browser.get('#/login'); }); ``` ??? - Step definitions in .js files - Can do anything that can be automated - Just chunks of code, not magical - Cucumber.js runs in node.js, many APIs - Protractor integration means it can be used in steps with no setup --- # Step definitions can do anything ```javascript this.Given('I am a new user', function () { // Create some values user = { name: 'Ben' }; }); ``` --- # Step definitions can do anything ```javascript this.Given('I am a new user', function () { // Call a factory to create a user user = mockDataFactory.createNewUser(); }); ``` --- # Step definitions can do anything ```javascript this.Given('I am a user with a loan', function () { // Trigger a Jenkins job which generates a user, then get the result jenkins.job.build('generate-user-with-loan', function () { var userDataUrl = 'http://jenkins/job/generate-user-with-loan/lastBuild/artifact/userData.json/*view*'; request.get({url: userDataUrl, json: true}, function (userData) { user = userData; done(); }); }); }); ``` --- # Sidebar: Testing asynchronous code with Cucumber.js ``` this.When('I do a thing', function (done) { doAsynchronousThing(function () { done(); }); }; ``` ``` this.When('I do a thing', function (done) { doThingThatReturnsPromise().then(function () { done(); }); }; ``` .smaller[Note: Steps returning promises will be supported in Cucumber.js 0.5. [[1]](https://github.com/cucumber/cucumber-js/pull/193)] ??? - Protractor steps return promises --- class: feature-eg .left-2-5[ ```xml Given I am on the login page ``` .feature-eg-2[ ```xml When I login ``` ] .feature-eg-3[ ```xml Then I should be logged in ``` ] ] .right-3-5[ ``` this.Given('I am on the login page', function (done) { browser.get('#/login').then(function () { done(); }); }); this.When('I login', function (done) { var user = 'ben', pass = 'pass'; element(by.css('.email')) .sendKeys(user) .then(function() { return element(by.css('.password')) .sendKeys(pass) }); .then(function() { return element(by.tagName('button')).click(); }); .then(function() { done(); }); }); this.Then('I should be logged in', function (done) { browser.getCurrentUrl().then(function (url) { assert.equal(url, '#/login'); done(); }); }); ``` ] ??? - Duplication - Fragile to page structure --- class: smaller # A page object is an API for a page > “When you write tests against a web page, you need to refer to elements within > that web page in order to click links and determine what's displayed. > “However, if you write tests that manipulate the HTML elements directly your > tests will be brittle to changes in the UI. > “A page object wraps an HTML page, or fragment, with an application-specific > API, allowing you to manipulate page elements without digging around in the > HTML.” .right[*— [Martin Fowler](http://martinfowler.com/bliki/PageObject.html)*] --- # An example page object ``` // instantiate var page = new LoginPage(); // load the page page.get(); // do things (login in this case) page.login('ben', 'pass'); // query the page (check if page has validation errors) if (page.hasErrors()) { console.log('✘'); }; ``` ??? - `page.get`: we do it this way so we can instantiate page objects in multiple steps --- class: long-code-snippet .left-3-5[ ```javascript var LoginPage = function () { var url = '#/login', emailElem = element(by.css('.email')), passwordElem = element(by.css('.password')), submitElem = element(by.css('button')), invalidElem = element(by.css('.ng-invalid')); this.get = function () { return browser.get(url); }; this.login = function (email, password) { emailElem.sendKeys(email); passwordElem.sendKeys(password); return submitElem.click(); }; this.hasErrors = function () { return invalidElem.count().then(function (count) { return count > 0; }) }; }; ``` ] .right-2-5.po-regions.image-frame[ ![Login page](ivylogin.png) ] --- class: long-code-snippet .left-3-5[ ```javascript var LoginPage = function () { var url = '#/login', emailElem = element(by.css('.email')), passwordElem = element(by.css('.password')), submitElem = element(by.css('button')), invalidElem = element(by.css('.ng-invalid')); this.get = function () { return browser.get(url); }; this.login = function (email, password) { emailElem.sendKeys(email); passwordElem.sendKeys(password); return submitElem.click(); }; this.hasErrors = function () { return invalidElem.count().then(function (count) { return count > 0; }) }; }; ``` ] .right-2-5.po-regions.image-frame[ ![Login page](ivylogin.png) .po-region-email[emailElem] .po-region-password[passwordElem] .po-region-submit[submitElem] ] ??? - Strings, booleans, etc - Actions: login, search, save, update - Assertions: `getEmail()` returns string value - Step definitions become trivial in many cases --- # Using page objects in step definitions ```javascript this.Given('I am on the login page', function (done) { new LoginPage() .get() .then(function () { done(); }); }); this.When('I login', function (done) { new LoginPage() .login('ben', 'password') .then(function () { done(); }); }); this.Then('I should be logged in', function (done) { new LoginPage() .hasErrors().should.become(false) .then(function () { done(); }); }); ``` --- class: section # Patterns --- # Keep acceptance criteria high-level **Bad**: Given I am a user with permission 55 **Good**: Given I am user who can delete posts Test shouldn't need to change when implementation does. --- # Don't drive the UI to setup state, use factories **Bad**: ``` element(by.css(.register)).click().then(function () { // fill in registration form // login // make some profile changes // you can see where this is going right? }) ``` **Good**: ``` user = mockDataFactory.createAuthenticatedUser(); ``` --- # Avoid using parameters, tables, etc, in criteria **Bad**: ```xml When I login with username "ben" and password "pass" ``` **Good**: ```xml When I login with valid credentials ``` ??? - Comes down to how technical your team is - If the PO can write the file, you're doing it right - Distinction between business readable and business writable - Downside: more step definitions --- # Never say click **Bad**: ```xml When I select "Summary" from the "Report Type" dropdown ``` **Good**: ```xml When I choose the summary report type ``` ??? - Test shouldn't break when we change the UI to radio buttons --- # Reuse steps (don't constrain your language) **Bad**: ``` this.When(/^I change "([^"]*)" to (-?\d+) (\w*) from (?:now|today)$/, function (placeholder, value, units, done) { selectDateFromNow(placeholder, value, units).then(done); }); ``` **Good**: ``` var loginSuccessfully = function (done) { // ... }; this.When('I login successfully', loginSuccessfully); this.When('I login with valid credentials', loginSuccessfully); ``` ??? - When I change "Next payday" to 30 days from today - Constrained language - Reuse step implementations - Step factories - Watch out for silent failure on duplicate steps --- # Group by application domain not actor **Bad**: - All unauthenticated user tests in one feature - All admin user tests in one feature **Good**: - All dashboard tests in one feature --- class: long-code-snippet # Stop once you've hit an assertion **Bad**: ```xml When I go to the registration page And I complete the form and submit it Then I should see the 2nd registration page When I complete the form and submit it ... ``` **Good**: ```xml Scenario: Registration page 1 When I go to the registration page When I complete the form and submit it Then I should see the 2nd registration page Scenario: Registration page 2 Given I'm on the 2nd registration page When I complete the form and submit it And submit ``` ??? - Easier to read - Easier to debug - Assertions about individual page, not mid-journey - Angular makes this possible - Can jump into any page - 1:1 mapping between pages and models so don't need state updates --- # [Chai](http://chaijs.com/)/[Chai as Promised](https://github.com/domenic/chai-as-promised/) for assertions ``` promise.should.be.fulfilled; promise.should.be.rejected; promise.should.become('foo'); promise.should.become('foo').notify(done); ``` --- # Reuse `element()`s **Bad**: ``` this.When('I click submit', function () { return element(by.css('.submit')).click(); }) ``` **Good**: ``` var submitElem = element(by.css('.submit')); this.When('I click submit', function () { return submitElem.click(); }) ``` --- class: section # Pyramids and Pipelines --- # Test pyramid .center[![Test pyramid](pyramid.png)] ??? - Acceptance testing != integration testing - Acceptance is often UI, integration is UI, but the 2 are different - Mocks: too many permutations, would be slow and unstable if integration - Get view of every page on every device - 1000:100:10 ratio --- # Pipeline .pipeline[ .pipeline-stage[Build] .pipeline-stage[Unit] .pipeline-stage[Acceptance] .pipeline-stage[Integration] .pipeline-stage[Release] ] ??? - The pipeline: when we use mock data, when we don't --- class: smaller # The journey's end - Acceptance tests are conversations - BDD gives you a common language - Cucumber.js lets you translate acceptance criteria into JavaScript - You can call Protractor (or anything) from your steps - Page objects hide page details from the step definitions - We use mocks for acceptance tests and they sit between unit and integration tests in our pipeline --- # Thanks! If this sounds interesting, come and work with us! Follow [@wongatech](http://twitter.com/wongatech). --- class: smaller # Further reading - http://dannorth.net/introducing-bdd/ - http://lizkeogh.com/behaviour-driven-development/ - http://www.slideshare.net/lunivore/behavior-driven-development-11754474 - http://www.infoq.com/news/2014/04/cucumberjs-bdd-biezemans - http://martinfowler.com/bliki/PageObject.html - http://martinfowler.com/bliki/TestPyramid.html - http://talks.codegram.com/bdd-best-practices-and-antipatterns#/intro