Test Classes Don’t Work

Test Classes don’t work as a test structure.

It’s worth clarifying what I mean by the test class. I’m
speaking specifically about the following structure of an test:

  • having a test class, that contains the setup and teardown method for test fixtures
  • putting multiple tests in that class
  • having the execution of a test look something like:
    * run setup
    * execute test
    * run teardown

More or less, something like:

class TestMyStuff:

     def setUp(self):
         self.fixture_one = create_fixture()
         self.fixture_two = create_another_fixture()

     def tearDown(self):
         teardown_fixture(self.fixture_one)
         teardown_fixture(self.fixture_two)

     def test_my_stuff(self):
         result = something(self.fixture_one)
         assert result.is_ok

This pattern is prevalent across testing suites, since they follow the
XUnit pattern of test design.

Why Test Classes are the Norm

Removing the setup and teardown from your test fixtures keep things
clean: it makes sense to remove them from you test body. When looking at code,
you only want to look at context that’s relevant to you, otherwise it’s harder
to identify what should be focused on:

def test_my_stuff():
    fixture = create_fixture()

    try:
        result = something(fixture)
        assert result.is_ok
    finally:
        teardown_fixture(fixture)

So, it makes sense to have setup and teardown methods. A lot of the
time, you’ll have common sets of test fixtures, and you want to share
them without explicitly specifying them every time. Most languages
provide object-oriented programming, which allows state that is
accessible by all methods. Classes are a good vessel to give a test
access to a set of test fixtures.

When You Have a Hammer…

The thing about object oriented programming is, it’s almost always a
single inheritance model, and multiple inheritance gets ugly
quickly. It’s not very easy to compose test classes together. In the
context of test classes, why would you ever want to do that?

Test fixtures. Tests depend on a variety of objects, and you don’t
want to have to multiple the setup of the same test fixtures across
multiple classes. Even when you factor it out, it gets messy quick:

class TestA():
    def setUp(self):
        self.fixture_a = create_fixture_a()
        self.fixture_b = create_fixture_b()

    def tearDown(self):
        teardown_fixture(self.fixture_a)
        teardown_fixture(self.fixture_b)

    def test_my_thing(self):
        ...


class TestB():
    def setUp(self):
        self.fixture_b = create_fixture_b()

    def tearDown(self):
        teardown_fixture(self.fixture_b)

    def test_my_other_thing(self):
        ...

class TestB():
    def setUp(self):
        self.fixture_c = create_fixture_b()
        self.fixture_b = create_fixture_c()

    def tearDown(self):
        teardown_fixture(self.fixture_b)

    def test_my_other_other_thing(self):
        ...

At this rate, a test class per test would become necessary, each with
the same code to set up and teardown the exact same fixture.

To avoid this, there needs to be a test system that:

  • has factories for test fixtures
  • as little code as possible to choose the fixtures necessary, and to
    clean them up.

A Better Solution: Dependency Injection

In a more general sense, a test fixtures is a dependency for a
test. If a system existed that handled the teardown and creation of
dependencies, it’s possible to keep the real unique logic alone
in the test body.

Effectively, this is the exact description of a dependency injection
framework
:
specify the dependencies necessary, and the framework handles the
rest.

For Python as an example, py.test has this capability. I declare a common fixture
somewhere, and can consume it implicitly in any test function:

# example copied from the py.test fixture page.
import pytest

@pytest.fixture
def smtp(request):
    import smtplib
    server = smtplib.SMTP("merlinux.eu")
    # addfinalizer can be used to hook into the fixture cleanup process
    request.addfinalizer(lambda: teardown(server))

def test_ehlo(smtp):
    response, msg = smtp.ehlo()
    assert response == 250
    assert 0 # for demo purposes

With pytest, You can even use fixtures while generating other fixtures!

It’s a beautiful concept, and a cleaner example of how test fixtures
could be handled. No more awkward test class container to handle creation
and teardown of fixtures.

As always, thoughts and comment are appreciated.

Author: toumorokoshi

Love to code, love emacs!