by Adam Brett

Outside-in TDD with PHP

Just as TDD is generally accepted to be the best way to write your automated tests, outside-in is generally the best way to do TDD.

Outside-in starts with the requirements, and works from customer tests, which test the presence of functionality, down the pyramid and into programmer tests, which test correctness, ending with unit tests on the bottom.

In this post, I'll look at outside-in TDD with PHP, the methodology, and frameworks. Whilst the methodology here can be applied to other languages, the specifics are written for PHP.

With all of the code and tests below, we're going to apply these rules:

  • You are not allowed to write any production code unless it is to make a failing unit test pass.
  • You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
  • You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

These three rules are known as the Gold Rules of TDD, or the Micro Cycle of TDD (the cycle being Red, Green, Refactor, Repeat, or put another way: Make it work, Make it right, Make it fast), and are essential to successfully practicing TDD, and especially outside-in TDD.

If you've ever spoken to anyone that says TDD doesn't work, insists that writing tests after code is better (or the more recent version: TDD is Dead), it's probably because they don't know about, or don't know how to effectively apply these rules.

Getting Started

Let's start by creating a new project, we'll create a simple "Hello World" application.

Create a new directory to hold your hello-world application:

mkdir -p ~/Projects/php--hello-world
cd ~/Projects/php--hello-world

Now we want to initialize a new composer project (make sure you have composer installed on your path).

echo {} > composer.json

We also need some directories to hold your source and tests:

mkdir -p {src,tests/{unit,integrated,functional}}

You should end up with a directory structure something like this:

.
├── composer.json
├── src
└── tests
  ├── functional
  ├── integrated
  └── unit

Now we require some frameworks and libraries to help us develop.

All dependencies for an application, including build dependencies (except language compilers or runtimes) should be bundled with your application via a package manager. This allows you to get up and running on a project, or switch between projects using multiple languages or frameworks in a consistent manner, without having to worry about versions of packages on your local machine, and later will allow us to run isolated and reproducable builds in docker containers. Fortunately for us, composer is great at this.

Functional Tests

Outside-in starts with functional tests, so let's grab ourselves a BDD framework. Functional/Feature/System/E2E tests are the top of the pyramid, which means they are high level, slow, and we only want just enough of them. They don't have to be written in the BDD style, but that is the current prevailing trend.

I like Behat, and it's the most popular BDD framwork for PHP, so that's what I'll use:

composer require --dev behat/behat

Once composer finishes, and you are returned to your prompt, run vendor/bin/behat --version to make sure it's properly installed. You should see something like this:

behat version 3.1.0

As an aside, vendor/bin is the correct location for vendor binaries. You might be tempted to move this to a bin directory in the root of your project, composer even has a config setting to make this the default, but please don't. That's for your application binaries, not vendors.

Behat comes with a handy --init command, which will create all of the files you need to get started with it. Unfortunately for us, it expects your functional tests to be in a features directory in the root of your project, so before we initialize anything, we need to create a config file.

Create a behat.yml in the root of your project, then open it up in your favourite editor and add:

default:
  autoload:
    '': tests/functional/bootstrap
  suites:
    default:
      paths: [ tests/functional ]

Now drop back to your terminal and run vendor/bin/behat --init:

+d tests/functional/bootstrap - place your context classes here
+f tests/functional/bootstrap/FeatureContext.php - place your definitions, transformations and hooks here

All done!

Let's try running behat and see what happens:

$ vendor/bin/behat
No scenarios
No steps
0m0.00s (7.57Mb)

Nice, time to add our first scenario. Create a file at tests/functional/hello-world.feature and open it in your favourite editor. Drop in the following:

Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

Cool. Let's run behat again:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

No scenarios
No steps
0m0.01s (7.68Mb)

Good, still not broken, time to add a scenario and see what happens:

Scenario: Viewing the "Hello World" message
  Given I am on the root context
  Then I should see "Hello World"

Now run behat:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context
    Then I should see "Hello World"

1 scenario (1 undefined)
2 steps (2 undefined)
0m0.01s (7.81Mb)

--- FeatureContext has missing steps. Define them with these snippets:

    /**
     * @Given I am on the root context
     */
    public function iAmOnTheRootContext()
    {
        throw new PendingException();
    }

    /**
     * @Then I should see :arg1
     */
    public function iShouldSee($arg1)
    {
        throw new PendingException();
    }

Excellent. Behat has run our scenario and said it doesn't know how to handle the steps we added. We need to fill in the details to tell it what to do, so let's do that.

Copy and paste the example functions it's given you in to tests/functional/bootstrap/FeatureContext.php, and then run behat again:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context            # FeatureContext::iAmOnTheRootContext()
      TODO: write pending definition
    Then I should see "Hello World"           # FeatureContext::iShouldSee()

1 scenario (1 pending)
2 steps (1 pending, 1 skipped)
0m0.01s (7.92Mb)

If you see an error here, it's probably because Behat hasn't imported PendingException, and you need to add: use Behat\Behat\Tester\Exception\PendingException; to the use statements at the to of your FeatureContext.php.

Now it's time to start filling in the content of our steps. As our functional tests are Customer Tests, and are designed to test for the presence of functionality (rather than the correctness which would be Programmer Tests), we'll want to run them end-to-end. This means spinning up a web server and a browser and running tests against that.

Driving a browser with Behat is best done with Mink, so drop back to the terminal and install it:

composer require --dev behat/mink

Mink on its own doesn't do much, it's just an interface, to make it work, we need a driver. For most people, this will mean selenium, but it doesn't have to. If your application is simple enough you can use a headless browser, which is what I'm going to do here in order to keep this simpler. Drop back to your terminal and install the mink goutte driver:

composer require --dev behat/mink-goutte-driver

Now run your behat tests again:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context            # FeatureContext::iAmOnTheRootContext()
      TODO: write pending definition
    Then I should see "Hello World"           # FeatureContext::iShouldSee()

1 scenario (1 pending)
2 steps (1 pending, 1 skipped)
0m0.01s (8.15Mb)

You may wonder why we keep running our tests after every tiny change, even if it has little chance of affecting our tests (we don't even have an application at the moment). This is a core tennant of Test Driven Development & the Golden Rules, and one of its biggest benefits: Run your tests after every change, no matter how small, and if your tests were working before, but aren't now, you know exactly what caused it.

Now mink is installed, we need to add it to our behat tests. Open up FeatureContext.php again and add the following to the constructor:

$this->driver = new \Behat\Mink\Driver\GoutteDriver();
$this->session = new \Behat\Mink\Session($this->driver);

$this->session->start();

If $driver and $session don't exist as class properties, add them as protected:

protected $driver;
protected $session;

Now run your tests again:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context            # FeatureContext::iAmOnTheRootContext()
      TODO: write pending definition
    Then I should see "Hello World"           # FeatureContext::iShouldSee()

1 scenario (1 pending)
2 steps (1 pending, 1 skipped)
0m0.02s (8.80Mb)

No change is a good thing, that's what we want. We can now start using the driver to fill in our step definitions.

As Test driven development isn't about writing the entire test up front. We need to write the smallest possible part, then run our tests, and if they pass, write the next smallest part, if they fail, it's time to write some code.

We'll start with iAmOnTheRootContext, as that's where our scenario starts. Update it to match the following:

public function iAmOnTheRootContext()
{
    $this->session->visit('http://localhost:8000/');
}

This is as simple as we can possibly make it, so let's run our tests again:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context            # FeatureContext::iAmOnTheRootContext()
      cURL error 7: Failed to connect to localhost port 8000: Connection refused (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) (GuzzleHttp\Exception\ConnectException)
    Then I should see "Hello World"           # FeatureContext::iShouldSee()

--- Failed scenarios:

    tests/functional/hello-world.feature:6

1 scenario (1 failed)
2 steps (1 failed, 1 skipped)
0m0.03s (9.61Mb)

Excellent. Now we have a failure. As is customary with outside-in TDD, we won't move any further on our functional tests, we have a failure, it's time to move down a level.

Integrated Tests

Integrated tests shouldn't be confused with integration tests. There is a subtle difference, in that integrated tests are effectively end-to-end tests without the server and browser overhead.

They often duplicate what we see with our functional tests. The reason we have them is because they will run faster than our functional tests, which we only want to be running occasionally, and are closer to programmer tests in that they have knowledge of data and frameworks and so on. They still aren't there to prove the correctness of the program, and we still only want to be validating functionality with them, just at a slightly lower level.

Because they have knowledge of the framework and data, this is the point at which we need to add a framework. Whilst we can use behat for our integrated tests, I don't like to, so I'm going to install PHPUnit, too:

composer require --dev silex/silex
composer require --dev phpunit/phpunit

Time to write our first integrated test. Create a new file at tests/integrated/HelloWorldTest.php, and add the following:

<?php

namespace App\Tests;

class HelloWorldTest extends \PHPUnit_Framework_TestCase
{

}

Remember, we're writing the minimum amount possible, then running our tests, this looks pretty minimal to me, so let's run them:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

W                                                                   1 / 1 (100%)

Time: 23 ms, Memory: 4.00MB

There was 1 warning:

1) Warning
No tests found in class "App\Tests\HelloWorldTest".

WARNINGS!
Tests: 1, Assertions: 0, Warnings: 1.

Nice. As this is an integrated test, we need to make a minor change, the Silex version of \PHPUnit_Framework_TestCase is \Silex\WebTestCase;, so import it and update the parent, like so:

<?php

namespace App\Tests;

use Silex\WebTestCase;

class HelloWorldTest extends WebTestCase
{

}

Now, run your tests again:

$ vendor/bin/phpunit tests/integrated/
PHP Fatal error:  Class App\Tests\HelloWorldTest contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (Silex\WebTestCase::createApplication) in tests/integrated/HelloWorldTest.php on line 7
PHP Stack trace:
PHP   1. {main}() vendor/phpunit/phpunit/phpunit:0
PHP   2. PHPUnit_TextUI_Command::main() vendor/phpunit/phpunit/phpunit:47
PHP   3. PHPUnit_TextUI_Command->run() vendor/phpunit/phpunit/src/TextUI/Command.php:113
PHP   4. PHPUnit_Runner_BaseTestRunner->getTest() vendor/phpunit/phpunit/src/TextUI/Command.php:135
PHP   5. PHPUnit_Framework_TestSuite->addTestFiles() vendor/phpunit/phpunit/src/Runner/BaseTestRunner.php:59
PHP   6. PHPUnit_Framework_TestSuite->addTestFile() vendor/phpunit/phpunit/src/Framework/TestSuite.php:413
PHP   7. PHPUnit_Util_Fileloader::checkAndLoad() vendor/phpunit/phpunit/src/Framework/TestSuite.php:339
PHP   8. PHPUnit_Util_Fileloader::load() vendor/phpunit/phpunit/src/Util/Fileloader.php:38
PHP   9. include_once() vendor/phpunit/phpunit/src/Util/Fileloader.php:56

A failure! Great. With TDD, as well as writing the minimum amount possible to make our tests fail, we also write the minimum amount possible to make them pass. This error is complaining about a missing abstract method, so let's add it:

<?php

namespace App\Tests;

use Silex\WebTestCase;

class HelloWorldTest extends WebTestCase
{
    public function createApplication()
    {

    }
}

I've not given it a method body, because this may be all that's required to make my tests pass. Let's run them and find out:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

W                                                                   1 / 1 (100%)

Time: 24 ms, Memory: 4.00MB

There was 1 warning:

1) Warning
No tests found in class "App\Tests\HelloWorldTest".

WARNINGS!
Tests: 1, Assertions: 0, Warnings: 1.

Yup, nice. Now we can start adding our tests:

/**
 * @test
 */
public function it_should_display_hello_world_on_the_homepage()
{

}

At this point, we need a way dispatch routes on our application, so we need to import it. The correct place to do this is createApplication that we added above, so let's try that:

<?php

namespace App\Tests;

use Silex\WebTestCase;
use App\App;

class HelloWorldTest extends WebTestCase
{
    public function createApplication()
    {
        return new App();
    }

    /**
     * @test
     */
    public function it_should_display_hello_world_on_the_homepage()
    {

    }
}

Now run your tests:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 25 ms, Memory: 4.00MB

There was 1 error:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
Error: Class 'App\App' not found

tests/integrated/HelloWorldTest.php:12
vendor/silex/silex/src/Silex/WebTestCase.php:39

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Another error, awesome! Time to write some code. The error Error: Class 'App\App' not found is being thrown because we haven't created an application class yet, so let's do that and see if it solves our problem.

Open up src/App.php in your editor, and add the following:

<?php

namespace App;

class App
{

}

This may seem too basic, but it is basic for good reason, if this is the simplest change possible, then if something goes wrong, we only have one place to debug. If we tried to add more at this point, we would be unsure about what was causing any errors:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 22 ms, Memory: 4.00MB

There was 1 error:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
Error: Class 'App\App' not found

tests/integrated/HelloWorldTest.php:12
vendor/silex/silex/src/Silex/WebTestCase.php:39

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

We're still getting the same error, even though we've created our class. There doesn't seem to be anything wrong with the class' code, so it's probably because we haven't created an autoloader yet, so PHP doesn't know how to find our new class.

Open up composer.json, and add an autoload block, pointing at our src directory:

{
    "require-dev": {
        "behat/behat": "^3.1",
        "behat/mink": "^1.7",
        "behat/mink-goutte-driver": "^1.2",
        "silex/silex": "^2.0",
        "phpunit/phpunit": "^5.4"
    },
    "autoload": {
      "psr-4": {
        "App\\": "src/"
      }
    }
}

You can run your tests again now, if you like, but they will still be broken, as we need to re-generate composer's autoload files. To do that, run composer dump-autoload, then run your tests again:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 23 ms, Memory: 4.00MB

OK (1 test, 0 assertions)

Success! But our functional tests are still broken, so time to add some more to our integrated test:

The WebTestCase provided by Silex gives us some convenience methods for working with our integrated application, the details of those aren't massively important here, so update your test to match this:

/**
 * @test
 */
public function it_should_display_hello_world_on_the_homepage()
{
    $client = $this->createClient();
    $client->request('GET', '/');
}

This is the smallest amount we can write, so let's run our tests again:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 35 ms, Memory: 4.00MB

There was 1 error:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
TypeError: Argument 1 passed to Symfony\Component\HttpKernel\Client::__construct() must be an instance of Symfony\Component\HttpKernel\HttpKernelInterface, instance of App\App given, called in vendor/silex/silex/src/Silex/WebTestCase.php on line 62

vendor/symfony/http-kernel/Client.php:41
vendor/silex/silex/src/Silex/WebTestCase.php:62
tests/integrated/HelloWorldTest.php:20

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Ok, an error. It doesn't appear to be with our test, so we probably need to write some code here. It mentions that App\App doesn't match the interface it's expecting. That's probably because our app isn't extending the Silex\Application class, and it's expecting it to. Let's change that:

<?php

namespace App;

use Silex\Application as SilexApplication;

class App extends SilexApplication
{

}

I've imported this class as SilexApplication because I like to read extends and implements as "is-a", so "App is-a SilexApplication" reads better than "App is-a[n] Application", which is how it would have read without renaming the import. This technically isn't the minimum amount of code, but it's pretty close, and is much more readable, so I'll allow it. Let's try running our tests again:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 35 ms, Memory: 6.00MB

OK (1 test, 0 assertions)

It passes, cool! But our functional tests are still failing. If our integrated tests are able to load '/', why can't they? Let's add an assertion to make sure it's returning ok:

/**
 * @test
 */
public function it_should_display_hello_world_on_the_homepage()
{
    $client = $this->createClient();
    $client->request('GET', '/');
    $this->assertTrue($client->getResponse()->isOk());
}

Now our tests:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 36 ms, Memory: 6.00MB

There was 1 failure:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
Failed asserting that false is true.

tests/integrated/HelloWorldTest.php:22

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Aha! So we're not getting a 200 response. That's probably because we haven't defined any routes. Let's go do that, update src/App.php:

<?php

namespace App;

use Silex\Application as SilexApplication;

class App extends SilexApplication
{
    public function __construct()
    {
        $this->get('/', function () { return ''; });
    }
}

And run tests:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 21 ms, Memory: 4.00MB

There was 1 error:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
InvalidArgumentException: Identifier "controllers" is not defined.

vendor/pimple/pimple/src/Pimple/Container.php:96
vendor/silex/silex/src/Silex/Application.php:145
src/App.php:12
tests/integrated/HelloWorldTest.php:12
vendor/silex/silex/src/Silex/WebTestCase.php:39

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Uhoh, an error, we probably need to call the parent constructor:

<?php

namespace App;

use Silex\Application as SilexApplication;

class App extends SilexApplication
{
    public function __construct()
    {
        parent::__construct();
        $this->get('/', function () { return ''; });
    }
}

Let's try again:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 34 ms, Memory: 6.00MB

OK (1 test, 1 assertion)

Awesome.

Stepping Back

Now our integrated tests are seemingly ok, it's time to try our functional tests again:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context            # FeatureContext::iAmOnTheRootContext()
      cURL error 7: Failed to connect to localhost port 8000: Connection refused (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) (GuzzleHttp\Exception\ConnectException)
    Then I should see "Hello World"           # FeatureContext::iShouldSee()

--- Failed scenarios:

    tests/functional/hello-world.feature:6

1 scenario (1 failed)
2 steps (1 failed, 1 skipped)
0m0.02s (9.74Mb)

They're still failing. That's probably because of the missing webserver component. That's not something our integrated tests are going to pick up on. So let's add that now.

Create a directory called public in the root of your project:

mkdir -p public

Then create an index.php in that directory, and open it in your editor, make it match our tests:

<?php

$app = new App\App();
$app->run();

We need the additonal $app->run() here, because this is called automatically by the Silex\WebTestCase class for our tests. Now we'll try and serve it with the built-in php webserver:

php -S 127.0.0.1:8000 -t public/

Then in a new terminal try to hit the address:

curl -sSL 127.0.0.1:8000

There's no error there, but if we head back to our original terminal, we can see an error:

PHP Fatal error:  Uncaught Error: Class 'App\App' not found in public/index.php:5
Stack trace:
#0 {main}
  thrown in public/index.php on line 5
127.0.0.1:44244 [500]: / - Uncaught Error: Class 'App\App' not found in public/index.php:5
Stack trace:
#0 {main}
  thrown in public/index.php on line 5

This is similar to what we had before, and it was to do with autoloading. It's probably because we haven't included the composer autoload file. PHPUnit does that for us automatically, so let's try that:

<?php

require __DIR__ . '/../vendor/autoload.php';

$app = new App\App();
$app->run();

There's no need to restart the web server, so just try hitting it with curl again:

curl -sSL 127.0.0.1:8000

Still no output, but this time the other terminal shows a green success message:

127.0.0.1:44254 [200]: /

Let's try running our functional tests again now:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context            # FeatureContext::iAmOnTheRootContext()
    Then I should see "Hello World"           # FeatureContext::iShouldSee()
      TODO: write pending definition

1 scenario (1 pending)
2 steps (1 passed, 1 pending)
0m0.03s (9.91Mb)

Nice! Our first scenario passes. Time to step back to our functional tests and fill in our second scenario.

Assertions

It's now time to make iShouldSee fail. Behat doesn't come with its own assertion library, so you need to use a 3rd party one. Hamcrest is a good choice, but in the interest of being lightweight, and as we already have PHPUnit installed, we'll use the one bundled with that.

Import the assertions in the use block at the top of the file:

use PHPUnit_Framework_Assert as Assert;

PHPUnit doesn't use modern namespaces, which is a shame, and that's why it uses underscores, but it is coming in the future.

Now update the iShouldSee step to use our new assertion class:

/**
 * @Then I should see :text
 */
public function iShouldSee($text)
{
    Assert::assertContains($text, $this->session->getPage()->getContent());
}

And let's run our tests:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context            # FeatureContext::iAmOnTheRootContext()
    Then I should see "Hello World"           # FeatureContext::iShouldSee()
      Failed asserting that '' contains "Hello World".

--- Failed scenarios:

    tests/functional/hello-world.feature:6

1 scenario (1 failed)
2 steps (1 passed, 1 failed)
0m0.04s (10.35Mb)

Our functional tests are failing again - and it's a proper failure rather than an error, so let's step back down to our integrated tests.

Unit Tests

I'm going to be a little facetious here, because I want to demonstrate where unit tests come in to the mix, so bear with me. Let's start by updating our integrated test to look for the "Hello World" string:

/**
 * @test
 */
public function it_should_display_hello_world_on_the_homepage()
{
    $client = $this->createClient();
    $this->assertTrue($client->getResponse()->isOk());
    $this->assertContains('Hello World', $client->getResponse()->getContent());
}

Now if we run our tests, they should fail:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 37 ms, Memory: 6.00MB

There was 1 failure:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
Failed asserting that '' contains "Hello World".

tests/integrated/HelloWorldTest.php:23

FAILURES!
Tests: 1, Assertions: 2, Failures: 1.

We could make this pass by updating our route to return the string "Hello World", like so:

<?php

namespace App;

use Silex\Application as SilexApplication;

class App extends SilexApplication
{
    public function __construct()
    {
      parent::__construct();
      $this->get('/', function () { return 'Hello World'; });
    }
}

However, our application would then be complete, and I wouldn't have gotten on to unit tests yet.

X-Paths

Unit tests are programmer tests. They test the correctness of a program, rather than the functionality. With this in mind, a single unit should be a single xpath of a function or method. You need one unit test per xpath in order to have 100% coverage. If your method has no conditional statements, and no parameters, it has an xpath of 1, if you add a parameter or conditional, it's 2, another parameter, 4, and so on. X-Path's grow exponentially with parameters and conditionals, so it's vital that you keep your functions and methods small, otherwise you will end up with an unmanageable number of tests.

The best way to keep your xpath low is to avoid conditionals, and follow a few simple rules.

  1. No more than 4 parameters per method

    If there are any more, wrap them in a ParameterObject, or a ValueObject and that test independently (if necessary).

    If the arguments can't be neatly abstracted into a Domain Concept such as the aforementioned, your method is probably doing some unrelated things and is violating the Single Responsibility Principle, so should be refactored in to multiple methods.

  2. No more than 10 lines per method

    Any more, and you're probably breaking the Single Responsibility Principle, and should break the method down in to smaller methods.

  3. Return early

    Some people argue there should be a single return point in a method - this is bad advice. The arguement is that it's difficult to track down all of the possible return points for a method. This is a strawman argument. Methods should be small enough that multiple return points aren't an issue.

  4. Don't use else

    If statements should be abstracted to their own methods, combined with "return early", there should never be a need to use else, else simply becomes the method's default return value.

There are some additional "rules", that help with good design, and help reduce xpath (and therefore the number of tests you need to write):

  1. Favour composition over inheritance

    Collaborators (e.g. strategy pattern) can be tested independently, and reduce the xpath in the consuming code.

  2. 10 methods per class

    Any more, and you're probably breaking the Single Responsibility Principle, and should break the method down in to smaller classes.

  3. 10 classes per namespace

    Any more, and you're probably breaking the Single Responsibility Principle, and should break the namespace down in to smaller namespaces.

There are of course valid exceptions to all of these rules, but following them most of the time will stand you in good stead.

Delivery Mechanism vs App/Domain Code

So far what we've been testing is the framework code, which is essentially the Delivery Mechanism for your application, taking care of common tasks, such as routing, auth, and view rendering, and should not be confused with your application itself.

With this in mind, our contrived domain is printing "Hello World", so we shouldn't be including that code in our route directly as we did above. Instead, this should be abstracted into our application code, and then unit tested to ensure correctness.

Let's update our route to take this in to account:

<?php

namespace App;

use Silex\Application as SilexApplication;
use App\HelloWorld\Response as HelloWorldResponse;

class App extends SilexApplication
{
    public function __construct()
    {
      parent::__construct();
      $this->get('/', function () { return new HelloWorldResponse(); });
    }
}

Notice how I haven't put this into a namespace called "Domain", or "Responses". Application code should be grouped by Context, not responsibility. You can group framework code by responsibility, because that's the framework's context, but not your application.

Let's run our tests again, and make sure they break:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 60 ms, Memory: 6.00MB

There was 1 error:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
Error: Class 'App\HelloWorld\Response' not found

src/App.php:13
vendor/symfony/http-kernel/HttpKernel.php:148
vendor/symfony/http-kernel/HttpKernel.php:66
vendor/silex/silex/src/Silex/Application.php:496
vendor/symfony/http-kernel/Client.php:79
vendor/symfony/browser-kit/Client.php:315
tests/integrated/HelloWorldTest.php:21

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Good. Now let's create that class:

mkdir -p src/HelloWorld

Now create and open src/HelloWorld/Response.php in your editor, and add the following:

<?php

namespace App\HelloWorld;

class Response
{

}

Remember, we're writing the minimum possible to make the tests pass. The error that we got was that the class wasn't found. That's because it didn't exist, so now we've created it and nothing more, we'll run our tests again to make sure that error goes away:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 38 ms, Memory: 6.00MB

There was 1 failure:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
Failed asserting that false is true.

tests/integrated/HelloWorldTest.php:22

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

We have a new error now, and this one is a bit cryptic. We should add some descriptions to our assertions to find out what's actually failing:

/**
 * @test
 */
public function it_should_display_hello_world_on_the_homepage()
{
    $client = $this->createClient();
    $crawler = $client->request('GET', '/');

    $this->assertTrue(
      $client->getResponse()->isOk(),
      'Response should be ok'
    );

    $this->assertContains(
      'Hello World',
      $client->getResponse()->getContent(),
      'Content should contain hello world'
    );
}

Now let's run our tests again:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 49 ms, Memory: 6.00MB

There was 1 failure:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
Response should be ok
Failed asserting that false is true.

tests/integrated/HelloWorldTest.php:25

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

There we go. This demonstrates why it's always a good idea to add a description to your assertions: it's not always obvious what's failing and why. We can see now that it's failing because the response isn't ok. My guess is that's because Silex expects the return value for a route to be a string or Silex Response object, and we're sending a custom object.

Let's update HelloWorld\Response to be castable to a string. But we need to write a test before we write any code; now it's time for our first unit test.

Create a file at tests/unit/HelloWorld/ResponseTest.php, and add the following:

<?php

namespace App\Tests\HelloWorld;

class ResponseTest extends \PHPUnit_Framework_TestCase
{

}

Remember, minimum change required. Now let's run our unit tests to make sure they work:

$ vendor/bin/phpunit tests/unit/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

W                                                                   1 / 1 (100%)

Time: 24 ms, Memory: 4.00MB

There was 1 warning:

1) Warning
No tests found in class "App\Tests\HelloWorld\ResponseTest".

WARNINGS!
Tests: 1, Assertions: 0, Warnings: 1.

That looks reasonable. Let's add a test:

<?php

namespace App\Tests\HelloWorld;

use App\HelloWorld\Response as HelloWorldResponse;

class ResponseTest extends \PHPUnit_Framework_TestCase
{
    /**
     * @test
     */
    public function it_should_be_castable_to_a_string()
    {
        $response = new HelloWorldResponse();
    }
}

That looks like the minimum amount that could fail, let's see:

$ vendor/bin/phpunit tests/unit/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 21 ms, Memory: 4.00MB

OK (1 test, 0 assertions)

Nope, all good. Let's add an assertion then:

<?php

namespace App\Tests\HelloWorld;

use App\HelloWorld\Response as HelloWorldResponse;

class ResponseTest extends \PHPUnit_Framework_TestCase
{
    /**
     * @test
     */
    public function it_should_be_castable_to_a_string()
    {
        $response = new HelloWorldResponse();
        $this->assertTrue(is_string((string) $response), 'Should be a string');
    }
}

Now let's see if it fails:

$ vendor/bin/phpunit tests/unit/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 26 ms, Memory: 4.00MB

There was 1 error:

1) App\Tests\HelloWorld\ResponseTest::it_should_be_castable_to_a_string
Object of class App\HelloWorld\Response could not be converted to string

tests/unit/HelloWorld/ResponseTest.php:15

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

As expected! Let's fix that:

<?php

namespace App\HelloWorld;

class Response
{
    public function __toString()
    {

    }
}

And our tests:

$ vendor/bin/phpunit tests/unit/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 22 ms, Memory: 4.00MB

There was 1 error:

1) App\Tests\HelloWorld\ResponseTest::it_should_be_castable_to_a_string
Method App\HelloWorld\Response::__toString() must return a string value

tests/unit/HelloWorld/ResponseTest.php:15

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.

Ok, looks like we need to actually return something, just adding the method isn't enough, let's fix that:

<?php

namespace App\HelloWorld;

class Response
{
    public function __toString()
    {
       return '';
    }
}

And again:

$ vendor/bin/phpunit tests/unit/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 22 ms, Memory: 4.00MB

OK (1 test, 1 assertion)

Excellent! Now our unit-tests pass, and we've completed the minimum amount of work that should make our integrated tests error go away, let's go back and run them again, to see what happens:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)

Time: 36 ms, Memory: 6.00MB

There was 1 failure:

1) App\Tests\HelloWorldTest::it_should_display_hello_world_on_the_homepage
Content should contain hello world
Failed asserting that '' contains "Hello World".

tests/integrated/HelloWorldTest.php:31

FAILURES!
Tests: 1, Assertions: 2, Failures: 1.

There we go. The error has gone away, but now we have a new error. This seems like a real error, so let's head back to our unit tests and add a new test:

/**
 * @test
 */
public function it_should_cast_to_hello_world()
{
    $response = new HelloWorldResponse();
    $this->assertEquals('Hello World', (string) $response, 'Should be Hello World');
}

And run it:

$ vendor/bin/phpunit tests/unit/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

.F                                                                  2 / 2 (100%)

Time: 24 ms, Memory: 4.00MB

There was 1 failure:

1) App\Tests\HelloWorld\ResponseTest::it_should_cast_to_hello_world
Should be Hello World
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'Hello World'
+''

tests/unit/HelloWorld/ResponseTest.php:24

FAILURES!
Tests: 2, Assertions: 2, Failures: 1.

We were expecting that, let's fix it now:

<?php

namespace App\HelloWorld;

class Response
{
    public function __toString()
    {
       return 'Hello World';
    }
}

And unit tests:

$ vendor/bin/phpunit tests/unit/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

..                                                                  2 / 2 (100%)

Time: 22 ms, Memory: 4.00MB

OK (2 tests, 2 assertions)

Integrated tests:

$ vendor/bin/phpunit tests/integrated/
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 37 ms, Memory: 6.00MB

OK (1 test, 2 assertions)

Functional tests:

$ vendor/bin/behat
Feature: Saying Hello
  In order to create a contrived example
  As a post author
  I need a service that returns "Hello World"

  Scenario: Viewing the "Hello World" message # tests/functional/hello-world.feature:6
    Given I am on the root context            # FeatureContext::iAmOnTheRootContext()
    Then I should see "Hello World"           # FeatureContext::iShouldSee()

1 scenario (1 passed)
2 steps (2 passed)
0m0.04s (10.32Mb)

Great! Let's curl it or hit it in a browser to make sure it works:

$ curl 127.0.0.1:8000
Hello World

Perfect!

Conclusion

This was a long post, so thanks for sticking with me this far. Hopefully you've learnt a thing or two about how to practice outside-in TDD. If you're a PHP developer, you should have all of the tools you need to get started straight away, and if not, the principles outlined here should give you a good grounding in the practice of outside-in TDD and things to look out for in your own language of choice.

For exclusive content, including screen-casts, videos, and early beta access to my projects, subscribe to my email list below.


I love discussion, but not blog comments. If you want to comment on what's written above, head over to twitter.